This is an entry for the usual Wednesday working meeting GENETIS holds. We are largely carrying on from last week's hackathon hammering out the remaining pieces that need to be fixed. See the previous ELOG post for a list of those.

Alex

Amy pointed out on the Monday call that the proper way to simulate the ANITA antenas is to use one source at a time. I modified the script I had so that it makes two simulations per antenna. The first one will use the power source connected to the "VPol" ridges and the second one uses the one connected to the "HPol" ridges. After checking this, it looks like the asymmetry I had pointed out before fell away. In other words, I think XF only wanted to use a single power source anyway. So now we should be able to produce the VV, HH, VH, and HV data that icemc and PUEOsim need.

Ryan and I also worked a bit on the slopes of the ridges. We experimented with the parameter that affects the curvature, which seems to be able to be too large. But many of the designs still look to have almost no slope and many still caved inward rather than sloping outward. We're still working on figuring out what causes that, but I think it means we need another constraint on distance of the top of the inner ridges from the center. 

We also tried figuring out why the sides of the curved part of the ridges don't stay in the same plane as the straight part of the ridges. I'm confident that they do stay in a single plane (I attached a Mathematica notebook that shows as much by finding that the torsion of the curve is zero; for some reason I couldn't print it to a PDF). We think that we should be able to rotate the curve into the same plane as the straight part of the ridge, but I haven't figured out how yet.

 Nick and I continued working on the HPol antenna. We want to check if there are any other things in Steph's example XF file that are significant to the results of the simulation (versus just being there as a representation of the model). It looks like there are, and Nick is working to figure out what they are by deleting them (in a copy) and generating simulation results for them to compare to the original. Otherwise, the most significant parts (the copper plates) are now being made in a script and just need to be connected to one another.

Dylan

I tried to build root from source again and ran into an error 3 hours into the build. I messaged Will and asked if he knew anything about the error. I then reran the build with Will's suggested change and ran into the same error 3 hours later. I did, however, manage to build pueo by sourcing the installation of root in Will's shared pueo directory. I'm now testing getting a txt output file from the simulation as it turns out my initial solution didn't work.

Update:

I got a working version of root installed. Also, I was able to modify pueoSim to output a veff.txt file. Now, I need to modify where it reads in the gain files, and then modify the icemc specific bash scripts to work for pueosim.

Attendance: Julie, Mitchell, Ryan, Alex P., Leo, Alex M.

Today's Tasks

Mitchell: Worked on Python Project and helped with Ryan's final bash project questions

Ryan: Finished the bash project, and is about to start the Python project. 

Leo: Worked on lectures, which will restart on March 17th.

Julie, Alex P, Alex M: Worked on fixing the job XF submission scripts. Worked on making it so that it saves state after each individual's XF simulation is complete. We tried to parallelize as many of the XF scripts as possible, but if we have more than one it asks us to choose a license for each individual -- ie requires user input each time. Since we haven't found a way to get around it AND since running with 1GPU is super fast (under 5 min per individual), we are going to run them in series.

TO-DO List

1. Scaling test run: Do a test run to see if the scaling (when we also scale the grid spacing) actually affects the run time for XF. (PASSED THIS TASK TO JULIE. WILL DO TOMORROW 3/4)
2. Evolve: Right now, we are using an interactive job with only a CPU running at half scale, and have just edited our software so that we submit jobs for the XF simulations requesting 1GPU (could be 2GPU if we choose to edit it so). 
   1. WHY WE ARE DOING THIS:
      1. Remember that we only need an interactive job when XF initializes and reads in the DNA for each individual. The GUI closes once the simulations start and we no longer need an interactive job. 
      2. We were requesting a 1GPU interactive job and just running XF from the command line; however, OSC has been SUPER busy lately and we haven't been getting them (especially since we need long wall times if we are running the full simulations over many generations within the interactive job). However, we realized that if we submit an interactive job at a CPU (so that XF can allow the GUI to pop up), and then run the simulations by requesting 1-2GPUs we can then request a much smaller wall time and it will be quicker for the simulations to run --- ie we would be requesting a wall time of about 10 minutes using 1-2GPUs per simulation for each job submission versus 5+ hour wall time of an interactive job using 1-2GPUs.
      3. WE HAVE TESTED THIS, AND WE HAVE ERRORS. WE WORKED ON FIXING THEM TODAY, AND HOPE TO FINALIZE THEM TOMORROW (3/4) AND RUN!

3. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized. (ALEX M)

   2. As of 3/3, I have not gotten an update on this. I will check on the status of this tomorrow. 

4. Comment code thoroughly (Everyone needs to help with this)

   1. ​Julie made some progress on this over the weekend. 

      1. Made comments on what each bash script version does, and cleaned up our directories from old code so that it doesn't confuse the others when they start helping 

   2. WAY more needs to still be done. 

5. Get new proposal results for March 12th deadline.

Today's Tasks

Julie: Write bash practice assignment, assist others when they have questions, work on Git lecture. (Will finish bash assignments early next week)

Evelyn, Mitchell, Eliot: Make progress on the "Learn the Command Line" course on Code Academy. (Should be done by the end of the week?)

Leo: Start gathering information for the "physics of antennas" lecture series! Note: Remember that Leo wants to be an educator, and would like us to find opportunities to work on "teaching" items. Most of the students don't have any background on how antennas work. He started learning about them and is going to do a lecture series (likely the first 20 minutes of every meeting) starting in a few weeks! (Should start series by hopefully 2/21)

Alex M: Clean up unused code. Fix hard-coded directories (likely finished & tested by end of day 2/11)

Alex P: Fix hard-coded variables (likely finished & tested by end of day 2/11)

I've tacked on our to-do list in the meantime so we can keep carrying the items still pending along with us. As we complete them, I will drop them off the list! 

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim error fixed (Alex, Julie) 

   1. It seems that if AraSim jobs are left to run while the user is away, it doesn't properly write one or more of the flags, and makes the loop get stuck at AraSim. We must then re-run from that state by manually editing the Save State text file.

   2. We are going to try to get XF to run at the command line so that we can submit it as a job. This would alleviate us using an interactive job. We *hope* this would fix the error since the user wouldn't be "gone" while utilizing a GPU in an interactive job. 

      1. Alex emailed XF to ask a few questions! Stay tuned!

2. Make Hardcoded directories variables.

   1. Xmacros (use sed command) 

   2. All over bash script functions.

   3. AraSim job files (sed command?)

3. Make the following variables in the bash script:

   
   In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

   Grid spacing=0.1
   Frequency= factor of n higher 
   This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.
   
   
   

   In roulette_algirithm.cpp

   Length = 50cm/Multiplier_factor; 
   STD= 15cm/Multiplier_factor 
   Radius = 1.5cm/Multiplier_factor; 
   STD= 0.75cm/Multiplier_factor 
   
   Also the "Multiplier_factor" line 89.
   
   This "Multiplier_factor"has to be the same as the GeoFactor in fitnessFunction_ARA_amy.cpp AND the same as the factor higher in frequency in the xmacro.
   ​NOTE:THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   
   

   In fitnessFunction_ARA_amy.cpp:

   GeoFactor=2 
   This GeoFactor has to be the same as the Multiplier_factor in the roulette algorithm AND the same as the factor higher in frequency in the xmacro
   NOTE: THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   

4. Get rid of all code we are not using anymore.  

   1. Currently using: XF_loop_AraSeed.sh

      1. Look at what items are being called in here, and remove old versions such as XF_loop_Prototype.sh, XF_loop_Functionized.sh, and the related scripts. 

      2. These can temporarily moved to on "old code" directory and we can delete once we finalize the loop. 

5. Turn tournament on in the GA. 

6. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being seeded.

       

Today we continued with the to-do lists items previously posted. I marked all the to-do list items that have been completed with green and a slash. Note that I have also added a new item at the bottom of the to-do list.

Today's Tasks

Julie: Write bash practice assignment (almost done!)

Evelyn, Mitchell, Eliot, Ryan: Make progress on the "Learn the Command Line" course on Code Academy. (Mitchell is done, and the others are close.)

Leo: Start gathering information for the "physics of antennas" lecture series! Note: Remember that Leo wants to be an educator, and would like us to find opportunities to work on "teaching" items. Most of the students don't have any background on how antennas work. He started learning about them and is going to do a lecture series 

Alex M: Make sure antenna scaling is made a variable in the bash script everywhere that it exists, start mass commenting all of the code so it reads as cleanly as the GA does. (Finished making the antenna scaling a variable. needs to be tested, but we waited forever for wall time)

Alex P: Fix hard-coded variables (done but tried to test it. Waited for wall time and never got it), and looked into ways to get around requesting an interactive job. For some reason, we don't need an interactive job to forward things like plots, but we do for the XF UI. We were trying to figure out why this is, and if there's a way to fix this. So far, no luck. 

AMY QUESION: Could we actually consider running on NASA computers? I think Alex P. found a way the NASA computing centers let you adjust for this issue, and it didn't work on OSC. If we end up not finding a solution with OSC, is it at all possible to have XF installed with NASA computing centers? (NOTE: I JUST ASKED ALEX FOR MORE DETAILS AND WILL ADD THEM HERE TOMORROW -- IE WEDNESDAY)

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim error fixed (Alex, Julie) 

   1. It seems that if AraSim jobs are left to run while the user is away, it doesn't properly write one or more of the flags, and makes the loop get stuck at AraSim. We must then re-run from that state by manually editing the Save State text file.

   2. We are going to try to get XF to run at the command line so that we can submit it as a job. This would alleviate us using an interactive job. We *hope* this would fix the error since the user wouldn't be "gone" while utilizing a GPU in an interactive job. 

      1. Alex emailed XF to ask a few questions! Stay tuned!

2. Make Hardcoded directories variables

3. Make the following variables in the bash script:

   
   In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

   Grid spacing=0.1
   Frequency= factor of n higher 
   This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.
   
   
   

   In roulette_algirithm.cpp

   Length = 50cm/Multiplier_factor; 
   STD= 15cm/Multiplier_factor 
   Radius = 1.5cm/Multiplier_factor; 
   STD= 0.75cm/Multiplier_factor 
   
   Also the "Multiplier_factor" line 89.
   
   This "Multiplier_factor"has to be the same as the GeoFactor in fitnessFunction_ARA_amy.cpp AND the same as the factor higher in frequency in the xmacro.
   ​NOTE:THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   
   

   In fitnessFunction_ARA_amy.cpp:

   GeoFactor=2 
   This GeoFactor has to be the same as the Multiplier_factor in the roulette algorithm AND the same as the factor higher in frequency in the xmacro
   NOTE: THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   

4. Get rid of all code we are not using anymore.  

   1. Currently using: XF_loop_AraSeed.sh

      1. Look at what items are being called in here, and remove old versions such as XF_loop_Prototype.sh, XF_loop_Functionized.sh, and the related scripts. 

      2. These can temporarily moved to on "old code" directory and we can delete once we finalize the loop. 

5. Turn tournament on in the GA. 

6. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being seeded.

7. Comment code thoroughly 

   1. The Gen alg is commented so thorougly that it can read like a book. We want this with ALL of our code; it will help all the less familiar people get comfortable. 

Today's Tasks

Julie: Write Python project! I have officially finished the bash course for the students. It can be found here: https://docs.google.com/document/d/1nGdPrwfYJOrO6NBT76Wsh6T0hDmGHo2-23oJ1WyeNcs/edit?usp=sharing. Mitchell is currently trying it out and will give me comments. 

Mitchell: Work through my bash project and give me comments!

Evelyn, Eliot, Ryan: Make progress on the "Learn the Command Line" course on Code Academy. 

Leo: continuing gathering information for the "physics of antennas" lecture series! These will take place every Friday meeting starting next week. 

Alex M: Try to get scaling into the xmacro, comment the code better.

Alex P: Add on the AraSim fixes mentioned below!

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim errors fixed

   1. Explanation of issue (1): Usually 1 of ~100 AraSim jobs never completes. OSC shows "completed", but the flag is never written and the output file is not filled in entirely. The error file shows the following.

      1. Carl believes this is due to the way we are saving our AraSim outputs. Instead of saving the files directly to our desired location, we are going to save them to $TMPDIR and use "pbsdcp" to copy them from $TMPDIR to our desired location. This may alleviate the problem since it will not be writing all the files to our desired location at once. 

DATA_LIKE_OUTPUT=1,2 doesn't work with DETECTOR=0,1,2
DATA_LIKE_OUTPUT controls data-like output into UsefulAtriStationEvent format; without a real station selected (using DETECTOR==3,4), the mapping to the data-like output will not function correctly
There are 1 errors from settings. Check error messages.
SysError in <TFile::Flush>: error flushing file outputs/AraOut.setup.txt.run29.root (Stale file handle)

1. Explanation of issue (2): We keep timing out of the interactive job when AraSim is running (if the user isn't remaining active on their terminal).
   1. OSC suggested that we use a virtual desktop (via ondemand) insead of using an interactive job. We will try this. 
      • Note: We should now be requesting 2GPUs instead of 1GPU. 

1. Make the following variables in the bash script. 

   1. Alex M is working on this. We can't directly make this a variable. We need to take the array of frequencies and multiply it by some value and THEN feed it into the xmacro. He is looking into whether of not he can directly do this in bash. If not, I have written a small python script that does this, and he can write it to that file; however, he can't append it at the end so it's not the ideal way. We are going to continue to try to do it through bash first. 

In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

Grid spacing=0.1
Frequency= factor of n higher 
This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.

1. Turn tournament on in the GA. 

2. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized.

3. Comment code thoroughly

   1. The Gen alg is commented so thorougly that it can read like a book. We want this with ALL of our code; it will help all the less familiar people get comfortable. 

Today's Tasks

Julie: Continue writing Python project! 

Mitchell, Evelyn, Eliot (not here on Tuesdays), Ryan: Work through bash project

Leo: continuing gathering information for the "physics of antennas" lecture series! We start this Friday at 4pm. 

Alex M: Try to get scaling into the xmacro, comment the code better.

Alex P: Add on the AraSim fixes mentioned below! He is testing this now. We are waiting for updates. 

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim errors fixed

   1. Explanation of issue (1): Usually 1 of ~100 AraSim jobs never completes. OSC shows "completed", but the flag is never written and the output file is not filled in entirely. The error file shows the following.

      1. UPDATE: Alex P. has edited our bash job script to save the outputs to $TMPDIR and then to pbscp (submit a "job" to copy them over) them into our correct directory that the bash script needs them in. He is testing this now and will report back. 

      2. Note: we still need to make sure that every time we run that we divide AraSim runs into MANY more jobs (ie further parallelize by dividing into, say, 100 instead of 10). 

DATA_LIKE_OUTPUT=1,2 doesn't work with DETECTOR=0,1,2
DATA_LIKE_OUTPUT controls data-like output into UsefulAtriStationEvent format; without a real station selected (using DETECTOR==3,4), the mapping to the data-like output will not function correctly
There are 1 errors from settings. Check error messages.
SysError in <TFile::Flush>: error flushing file outputs/AraOut.setup.txt.run29.root (Stale file handle)

1. Explanation of issue (2): We keep timing out of the interactive job when AraSim is running (if the user isn't remaining active on their terminal).
   1. OSC suggested that we use a virtual desktop (via ondemand) instead of using an interactive job. We will try this. 
      • Note: We should now be requesting 2GPUs instead of 1GPU. UPDATE: Alex P. requested one, but it took too long and he had to leave before he got it. 

1. Make the following variables in the bash script. 

   1. UPDATE: Alex M. figured out how to pass the scale factor into the xmacros. We used the "tr" command in bash. He will be testing this and can update us at the Thursday meeting. 

In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

Grid spacing=0.1
Frequency= factor of n higher 
This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.

1. Turn tournament on in the GA. 

2. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized.

3. Comment code thoroughly

   1. UPDATE: Alex commented code that utilized the scale factor (Part_E.sh). More still needs to be done. 

Today's Tasks

Julie: I finished the Python assignment. I am making sure my solutions encompass all of the different ways to solve this, and will then start on the C++ project. 

Mitchell, Evelyn, Eliot, Ryan: Work through my bash project.

Leo: At 4 pm today Leo will begin his "physics of antennas" lecture series! 

Alex M: All scaling is corrected to be set at the bash script level. Alex is now going to work on commenting code!

Alex P: Still working on our AraSim edits!

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim errors fixed

   1. Explanation of issue (1): Usually 1 of ~100 AraSim jobs never completes. OSC shows "completed", but the flag is never written and the output file is not filled in entirely. The error file shows the following.

      1. UPDATE: Alex still trying to figure out how to input Car's suggestion. He also needs to add this for the flag files that are being saved. Here is his most recent question:

Alright so I have pbsdcp working with generic files and I now have it working in the AraSim call but my current worry is that it wasn't creating a file unless I had

./AraSim setup.txt $runNum outputs/ a_${num}.txt > $AraSimDir/outputs/a_${num}.txt

So I just did this to have some a_1.txt, before we had the last line writing directly to our antennaPerformance metric (this makes the AraOut file typically I just kept it as this name for testing purporses), now if I get rid of the portion after the > or the part after outputs/ it doesn't seem to make a file that we need. I think I don't understand to get the full output of AraSim without using this. Because I don't understand how using this is and then using pbsdcp is any more efficient.

Now maybe if I had the portion right after the > go directly into TMPDIR instead of using pbsdcp to both scatter to and gather from TMPDIR that would be better

1. Make the following variables in the bash script. 

   1. UPDATE: Alex M. finished editing the scaling factor to be at the bash level for every piece of code that uses it. 

2. Turn tournament on in the GA. 

3. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized.

4. Comment code thoroughly

   1. UPDATE: Alex commented code that utilized the scale factor (Part_E.sh). He is now going to begin commenting the other code to clean it up. We want it to be read as cleanly as our GA. 

Attendance: Julie, Mitchell, Ryan, Evelyn, Eliot, Alex M.

Today's Tasks

Julie: Worked on C++ project, and helping others with the Python project; I also made additions, clarifications, and edits to the Bash project (also updated the solutions and pushed them to gitHub). I still need to do the following:

• Add the instructions from the google doc to the GitHub pages for both the Python and the Bash projects.
• Work on the C++ project!
• Run a test evolution this weekend to make sure AraSim stuff is fixed and that XF job submissions work. 
• EVOLVE

Mitchell: Started Python Project.

Evelyn, Eliot, Ryan: Work through my bash project. 

Leo: Worked on lectures. 

Alex M: Helped make XF job scripts. Needs to do the following:

• Parallelize the AraActualBicone properly. 
• EVOLVE!

Alex P: Needs to do the following:

• Do a test run to see if the scaling (when we also scale the grid spacing) actually affects the run time for XF.

TO-DO List

1. Scaling test run: Do a test run to see if the scaling (when we also scale the grid spacing) actually affects the run time for XF. (ALEX P)
2. Evolve: Right now, we are using an interactive job with only a CPU running at half scale, and have just edited our software so that we submit jobs for the XF simulations requesting 1GPU (could be 2GPU if we choose to edit it so). 
   1. WHY WE ARE DOING THIS:
      1. Remember that we only need an interactive job when XF initializes and reads in the DNA for each individual. The GUI closes once the simulations start and we no longer need an interactive job. 
      2. We were requesting a 1GPU interactive job and just running XF from the command line; however, OSC has been SUPER busy lately and we haven't been getting them (especially since we need long wall times if we are running the full simulations over many generations within the interactive job). However, we realized that if we submit an interactive job at a CPU (so that XF can allow the GUI to pop up), and then run the simulations by requesting 1-2GPUs we can then request a much smaller wall time and it will be quicker for the simulations to run --- ie we would be requesting a wall time of about 10 minutes using 1-2GPUs per simulation for each job submission versus 5+ hour wall time of an interactive job using 1-2GPUs.
      3. WE HAVE NOT TESTED OUR .SH JOB SUBMISSION SCRIPTS! JULIE WILL DO IT THIS WEEKEND!

3. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized. (ALEX M)

4. Comment code thoroughly (Everyone needs to help with this)

5. Get new proposal results for March 12th deadline.

Attendance: Julie, Evelyn, Mitchell, Ryan, Alex M.

Since we have been having issues with the lack of availability of GPUs on OSC, we have come up with a new to-do list:

1. Try to keep running on Pitzer with 1 core instead of 40. Based on first attempts, it is saying this will reduce wait times to ~3 hours (will vary based on usage), and after that 3 hours, human interaction will be needed. So the plan is, someone requests a job in the morning, and when the interactive job starts, the person is responsible for doing the thing which will allow the generation to run. We will aim for one generation per day under this plan, taking turns. Better than no evolving.
2. I have asked Heechang at OSC if we can install the latest version of XF on Ruby and/or Owens. That would be better. There seems to be more available on Owens at least.
3. Our initial attempt to run on Ruby is giving us an error which is probably to do with different inputs required in different versions. Will attempt to pursue this as a last resort, but the older version of XF does require lots more human interaction so until other options are exhausted it's not worth it.

Today's Tasks

Mitchell: Worked on Python Project, learned to evolve (note: Mitchell may leave the group at the end of the semester)

Ryan: Starting the Python project

Evelyn: Finish bash project.

Julie: C++ project, look at Amy's proposal draft. 

Alex M:  Fix XF solver to submit each individual as a separate job (thus we can request a smaller wall time for each job)

TO-DO List

1. Scaling test run: Do a test run to see if the scaling (when we also scale the grid spacing) actually affects the run time for XF. 
   1. We are going to pause on working on this, because we are struggling to get GPU time. We will test this whenever we see OSC less busy (ie if we log on at a lucky time). 
2. Evolve: Try to keep running on Pitzer with 1 core instead of 40. Based on first attempts, it is saying this will reduce wait times to ~3 hours (will vary based on usage), and after that 3 hours, human interaction will be needed. So the plan is, someone requests a job in the morning, and when the interactive job starts, the person is responsible for doing the thing which will allow the generation to run. We will aim for one generation per day under this plan, taking turns. Better than no evolving.

3. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized. (ALEX M)

   2. As of 3/5, I have not gotten an update on this. I will check on the status of this. 

4. Comment code thoroughly (Everyone needs to help with this)

   1. ​Julie made some progress on this over the weekend. 

      1. Made comments on what each bash script version does, and cleaned up our directories from old code so that it doesn't confuse the others when they start helping 

   2. WAY more needs to still be done. 

5. Get new proposal results for March 12th deadline:

   1. ​Julie is looking at what Amy wrote here 

I just wanted to give an update on what I've been doing today for GENETIS. I went ahead and recorded myself running the loop. It should be useful for new people we're onboarding but I also went through a whole generation, which I think is interesting for everyone to understand the timescale of each part of the loop. I'm making a small edit (I have no clue how to edit videos haha) so once I'm done I'll try to send it via email and post it on slack. I also started a long run Amy requested using antennas which are asymmetric in length and opening angle and 50 individuals per generation. Since it has so many individuals it will take a long time to go through each generation. I also had to decrease the number of AraSim jobs (thus increasing the number of neutrinos per job) because otherwise we'd be submitting too many jobs and have a lot of them blocked, holding us back from running. If anyone has problems with me using up too many jobs let me know and I can try to decrease the numer of jobs per individual futher (right now I have 10 jobs per individual with 3000 neutrinos per job, which gives us 500 AraSim jobs per generation).

Attendance: Julie Rolla, Alex M. 

Today Alex and I met to discuss how to move forward given the COVID-19 restrictions. We have resent out a message polling people's schedules to create online working sessions. Here is what we are planning:

Once we collect times for each group, we will have separate meetings for all of these entities. This way it's easier for us to meet without all having different items to discuss/work on. All of these will be zoom meetings moving forward. Meeting info will be announced shortly!

Tasks for "the Loop Group":

1. Make wall time a variable for XF sim.
2. Test current software edits.
   1. Such as item #1 above
   2. Make sure the bottleneck function is working.
      1. Right now it seems to be working; however, we keep hitting a wall time error. If we can make the wall time a variable then we can be sure the bottleneck function is working, and the wall time is truly the error. 
3. Get Ruby working on XF
   1. when I type:
      /usr/local/xfdtd/7.8.1.4/remcom/bin/xfdtd        I get:

      Info: Using latest available version (7.8.1.4)
      Info: Running /usr/local/xfdtd/7.8.1.4/remcom/XFdtd_7.8.1.4/bin/xfdtd 
      Error: Unable to find executable to run.  This command must exist

4. Get Alex's account of Ruby working. 
   1. He received an email and is still unable to log in. He is touching base with them again. 

Tasks for "the Training Group":

1. Julie and Alex need to finish the C++ project! 

The current working draft of the GENETIS paper can be seen here: https://www.overleaf.com/6783528497tvqbjphsgvzn

Additionally, here's our outline for the paper. The section assignments are as follows:

I. Intro (Julie)

II. Types of Algorithms (Suren)

III. XFdtd (Cade)

IV. AraSim -- will be called something else... (Max)

V. Fitness function (Suren)

VI. Results (Julie)

VII. Conclusion (We will see..)

Note that the order of the sections may change.

This is a general update. Most of our recent meetings have been to teach the students to use Git properly, or to answer questions on the training modules. 

Update

1. Git Update: I recently cleaned up our Git and GitHub so that we have a better way to develop this code. Once I cleaned it up the goal is to educate all of the students to properly use Git so we develop code correctly moving forward. The repository had become quite a mess. The students were constantly developing on a branch, and never merging it back to the main branch. This means that we never had a working copy that we could run on while edits were being made in parallel. I attempted to merge the edits to the main branch with the hopes of getting this functioning so we could evolve while working on the bottleneck function; however, it looks like someone with poor knowledge of Git edited the main branch and we had an abundance of merge conflicts. 

What I've done now is reset the repository so that all merge conflicts are resolved and a working copy exists on the main branch. I've also added a .gitignore that will ignore all data files. Git doesn't like these because (1) it hates when tracked files disappear -- ie when we delete trial run data, if someone accidentally pushed it then Git complains-- and (2) it means it we constantly are adding more untracked files (every single new file we make is untracked and then gets tracked, like the flag files). So, I've had it only track useful code. 

The goal once I train the other on Git is to move the place of the data files and make it it's own git repository (once we start doing fewer trial runs so we won't lose important data). If the others learn to check the status of files before pushing them, they can make sure they don't push useless trial data sets, and either (1) delete them, or (2) add them to the .gitignore.

As for now, we successfully have the working, run-able copy on the main branch now. When any more revisions are made (e.g. the talk of adding the wall time variable), we can branch and edit on that so we still run. 

2. Evolution/bottleneck update: We got the loop working to the point that it can run all the way through a generation without fatal problems. The only potential problem we can see right now is if we need to go backward in the savestate. We're working on making that not a problem--the solution is to better organize the output files and data, which we were interested in anyway. Right now we are running with 10 individuals in each generation. Alex got through 2 generations without a problem, so he'll keep running but now we're trying to make some of these more minor fixes so we may have to stop the run. Alex does have plots (they don't look great, but at least they're being made correctly without error). At the moment we are swimming in GPUs. We haven't had much trouble at all this week with getting them, so that's why we were able to get 2 generations done between around midnight last night and 2ish today; not sure how long that will persist, but Pitzer has had relatively little usage whenever we've looked lately.

Comments on the plots: We're discussing this now, we don't think the individuals should really be color coded/separate on the legend because it is not like they follow one individual across multiple generations; however, we do need to label the dotted line. Another thing we were considering is decreasing the penalty factor for when the cones are too big for the hole. If we get something with a much much better Veff but it's only slightly larger than the hole, it might be worth it to help that one stick around a little bit more.

3. Wall time Variable Update: Also, last week Carl and Alex talked about the variable wall time again. We decided the best idea is to start by implementing a variable wall time based on the cylindrical volume that he mentioned before, but in the future, we might also be able to check the time of each GPU job and then plot it against the dimensions of the cone/cylindrical volume to get a better estimate of the wall time for each job

Meeting Attendance: Julie, Alex M., Alex P, Eliot, Leo, Evelyn, Mitchell, Ryan. 

Today is the deadline for confirming our attendance at the virtual APS meeting. Alex M. will be confirming and we will be orienting ourselves so that we can have some results. Here is our to-do list for today:
 

1. Create a method for automatically adding our plots to dropbox each day. 
   • The way we are going to do it is by adding the command: mail -s "Test Subject" someemailaddress@gmail.com < testScript
     • We will create a random email to use for this that will only be receiving emails from this command. 
     • We are trying to find and set up a client that as soon as this random email address receives an email, it uploads it to the Dropbox here. 
   • This is still being worked on. 
2. Start developing a database of old antenna runs and parameters, and pull data from that for any repeat dimensions. 
   • This should speed up some of our run time. 
   • We want to create a database of old antenna dimensions and their associated UAN files. That way, if we repeat an antenna geometry very closely, instead of re-submitting a job to produce the gain patterns, we can pull from one that has already been created.
   • This should limit:
     • The number of jobs we have to submit.
     • The number of GPU hours/minutes we have to request and wait for.
   • These should probably still be run through AraSim, however, since the energy could change from run to run.
3. Keep running and collecting data. 
   • Do we want to run and have data for 20ish generations with 10 individuals for APS?
     • We have already made some progress on this so we could likely make more generations for APS if we keep going. 
   • Do we want to instead switch to doing something like 20 individuals even if we get fewer generations completed?

Notes apropos developing our database for antenna info. This is how we are going to do it, and what we need to develop in our code:

1. Add all info to csv files.
   • The csv files will have columns: length, radius, theta, uan file directory and name (ie /file/location/database#/).
   •  # will be the row that this matches the L, R, Theta within the csv file
   • Inside /file/location/database#/ will be all the uan files for each frequency for these dimensions. 
     • We need an "if/else" statement in the bash script that tells the code to look in this csv file (read it in). 
       • IF the produced L, R, Theta match any of those that currently exist, save the 4th column (file location) to a variable (say we call it ExistingUan=/file/location/database#/). 
         • Then cd ExistingUan
         • Make a loop that:
           • does CP /file/location/database#/gen_individual_freq.uan BiconeEvolutionOSC/BiconeEvolution/current_antenna_evolution/XF_Loop/Evolutionary_Loop/Run_Outputs/$RunName/ (ie to the file name and location that XFintoAra.py reads. This will make sure that it has the data as if we actually completed the XF simulations for the antennas that had repeat sizes from our past runs (I mean past by runs we may have EVER done -- not past generations). 
             • We need to copy ALL of the frequency uan files for that individual into Run_Outputs/$RunName 
       • Else fill another line in that csv file with the L, R, Theta of this individual so we can keep it for later. 
         • mkdir the new directory /file/location/database#/
         • Add the name of this new directory (ie database#) to the 4th column on the csv file next to the correlating L, R, Theta.  
         • Then submit the XF job for this individual. 
         • Once it finished the XFSolver job and produces the .uan files (for each frequency), do a loop that:
           • cp individual_freq.uan /file/location/database#/gen_individual_freq.uan
             • Note that I wrote to cp individual_freq.uan instead of gen_individual_freq.uan. Note that XF produces it as individual_freq.uan, and it is changed to gen_individual_freq.uan when we move it into the Run_Outputs/$RunName directory.
             • So you can do it either (1) after it moves it to Run_Outputs/$RunName with the name gen_individual_freq.uan, or (2) before it moves the file with the name individual_freq.uan. 
2. We need to make a new directory that holds all of these database# directories and their uan files, and the csv file the "if/else" statement checks. 
   • We need to add this new directory to the .gitignore!

Note that Pitzer usage over the weekend was much better! Maybe weekends are our best time to run? Today (Monday) has been a little bit worse. 

Over the past few days the following items have been accomplished:

• We got the jobs to stop canceling on OSC.
  • Issue was the destination command for the error and output files for the GPU jobs inside the GPU job script; we fixed it by taking it out.
    • This issue was causing jobs to fail. They are no longer failing, and the loop is working again.
• Set up a way to be able to automatically upload both the L, R, Theta plots and the Fitness Score plot to dropbox as we work.
  • Set up an email that automatically uploads to dropbox under the folder GP_Antennas/Updates/DailyFitnessScoreImages when you send an attachment with that email.
  • We added this command into the loop:
    mail -s "Subject" dropbox.2dwp1o@zapiermail.com < FScorePlot2D.png
    • It automatically has it upload an image when the loop finishes a generation. It's currently so that the file will be named whatever the subject of the email is so we can control that by whatever is after the -s command. It's currently set to not overwrite.
  • Still need to add the Veff plot to dropbox. 
• Made gitpush.sh and gitpushforce.sh
  • Each time one of us pushes changes to GitHub, it makes a permssion lock on the directories. From now on, after looking at what files are changed (using git status. Remember to make sure to add any data files missed to the .gitignore)users should do:
    • git add .
    • git commit -m "add message" or git commit --amend --no-edit
    • Then run: ./gitpush.sh if you made a full new commit, or use ./gitpushforce.sh if you amended this edit to the previous commit. They both exist in: /fs/project/PAS0654/BiconeEvolutionOSC
      • These files run the git push or git push -f commands and then immediately run a chmod to change permissions after. 
• Alex is starting the APS slides!
  • He should be sharing them shortly. I told him to look at Suren's from last year for guidance. 
• We have started working on adding the feature that pulls non-unique (ie repeat parameter) individuals from an antenna database. 
  • It's currently being developed on the Antenna_Database branch in git. 
  • Alex has started writing c++ code that reads in the csv file and appends unique items to it. 
  • Comments from Amy we need to remember: 
    • Doing "==" on doubles is bad!
    • We want this "likeliness" to be between 1 and 10% (start with 10%). 
      • Remember: is this difference between L stored and new L a fraction of the wavelength?

The students have been meeting to work out the summer schedule. We are dividing into to groups:

(1) Paper writing: Julie, Alex M., Alex. P

• We need Amy to start an overleaf draft (she has the premium version that allows you to share with more than 1 person)

  • https://www.overleaf.com/5612932179jzykrkvztqnb

• In tomorrow's collaboration meeting (Now Fridays at 3pm) we are going to request discussion on an outline

(2) Adding wall time variable structure: Eliot, Evelyn, Leo (Alex P. will help oversee)

• In order to prepare group (2) Alex M. has been going over the edits he started and is discussing his code. We also are taking some time to catch them up on the current code -- ie how it runs, what all of the pieces involved do -- so that they can know how to test the code after edits are made, and where each essential piece may lie. We will be going over this for the next few days in good detail. 
• Group (2) will then be coming up with a scheduled time each day that they will be coworking on zoom. This will work the same as if they were sitting in an office together. They'll announce this time to everyone once it is set.

Alex P, Alex M, and I are going to push to rapidly finish this antenna database so that the others can create a new branch to work on the wall time variable with. This will leave us very busy for the next few days. 

We met today for a long time and pushed to catch up. The following is what we have done:

(1) Worked for a bit on our current tasks. They will work on these current tasks before they start on their summer projects. 

• Antenna database (Julie, Alex P., Alex M.)
• Worked on a paper outline (Julie, Alex M.)
• Worked on wall time variable (Eliot, Evelyn, Leo)

       Note I also have to: (1) Fix merge conflict and make sure students are good about only editing the dev branch moving forward. (2) Get branches and GitHub set up for both the asymmetric bicone and the non-linear bicone. 

(2) Went over the details of how the loop works as a group.

• Went over how each piece works.
• Went over how to run it. 
• Went over best ways to test new edits. 

(3) Went over summer tasks.

• Eliot: Add parameters to bicone (ie make bicone asymmetric)
• Julie & Alex M.: Paper, and helping others
• Alex P.: Frequency database for AraSim (to speed up AraSim)
• Evelyn & Leo: Make non-linear bicone (wiggly sides) 

GENETIS Loop Work: Fixing Consol Errors By Julie Rolla

1. Python programs are not running: I think either (1) the user isn't module loading python, or (2) they are loading the wrong version of Python (remember there's a discrepancy for Python2, and Python3). 

Solution: I added a line "module load python/3.6-conda5.2" in our araenv.sh script to address the issue. All of the users are sourcing the araenv.sh script in their .bashrc when they login. It's in the directory: /fs/project/PAS0654/BiconeEvolutionOSC. I also added a line in the XF_Loop.sh script (at the top in the "Initialization of Variables" section) that sources araenv.sh just in case the user hasn't put that in their .bashrc. The line reads: source /fs/project/PAS0654/BiconeEvolutionOSC/araenv.sh

Error message fixed:

 0 
Data successfully read. Data successfully written.
Fitness scores successfully written.
Fitness function concluded.
rm: cannot remove 'runData.csv': No such file or directory
Unable to parse the pattern
^CTraceback (most recent call last):
  File "LRPlot.py", line 99, in <module>
    plotLR(generations, lengths, radii, g.numGens, g.destination)
  File "LRPlot.py", line 45, in plotLR
    plt.show()
  File "/usr/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 143, in show
    _show(*args, **kw)
  File "/usr/lib64/python2.7/site-packages/matplotlib/backend_bases.py", line 108, in __call__
    self.mainloop()
  File "/usr/lib64/python2.7/site-packages/matplotlib/backends/backend_gtk.py", line 83, in mainloop
    gtk.main()
KeyboardInterrupt

2. rm runData.csv error: The following error was printed to consol

rm: cannot remove 'runData.csv': No such file or directory

I found these line in section (E) of the loop.

cd "$WorkingDir"
rm runData.csv
python gensData.py 0
cd Antenna_Performance_Metric
python LRPlot.py "$WorkingDir" "$WorkingDir"/Run_Outputs/$RunName 1 $NPOP
cd ..
# Note: gensData.py floats around in the main dir until it is moved to         

 I think this error is fine. This is how it works: (Step 1) removes runData.csv, if it exists. (Step 2) Runs gensData.py, which creates a new runData.csv. We wanted to delete it before the .py program was run at the start of the run, because gensData.py appends data to runData.csv, if it already exists. (alternatively if runData.csv doesn't exists it creates it-- which is what we want in this case). That's also why we don't delete it later in the "looping part" of XF_Scripts.sh (part G). We want it to append at this point, so we only delete it in the very beginning to make sure it doesn't append to an old run.

3. cp: cannot stat 'Antenna_Performance_Metric/1.uan': No such file or directory (similarly for 2.uan, 3.uan...etc): This error exists just as the fitness score is being calculated -- ie part E. Here is the print:

cp: cannot stat 'Antenna_Performance_Metric/1.uan': No such file or directory
cp: cannot stat 'Antenna_Performance_Metric/2.uan': No such file or directory
Congrats on getting a fitness score!
  File "FScorePlot.py", line 1
    -*- coding: utf-8 -*-
    ^
IndentationError: unexpected indent

Note that we have a few errors here. Right now I am directly addressing the error with the "cp" command. It looks like these files are actually being saved as gen_individual#.uan. For example 0_1.uan. The current line in part E says:

for i in `seq 1 $NPOP`

do

    #Remove if plotting software doesnt need                                                                                                                                                             

    #cp data/$i.uan ${i}uan.csv                                                                                                                                                                          

    cp Antenna_Performance_Metric/$i.uan "$WorkingDir"/Run_Outputs/$RunName/${i}.uan

done

However, this is not how these files are being saved. I corrected the first iteration of the loop (ie Part E for generation 0) by editing the line to say:

for i in `seq 1 $NPOP`

do

    #Remove if plotting software doesnt need                                                                                                                                                             

    #cp data/$i.uan ${i}uan.csv                                                                                                                                                                          

    cp Antenna_Performance_Metric/0_$i.uan "$WorkingDir"/Run_Outputs/$RunName/0_${i}.uan

done

And for the looping part of XF_Loop.sh (ie part G) it has been edited to say:

for i in `seq 1 $NPOP`

        do

    # Gens data used to create a .csv file for the uan file for gain plotting                                                                                                                            

    # cp Antenna_Performance_Metric/$i.uan ${i}uan.csv                                                                                                                                                   

                cp Antenna_Performance_Metric/${gen}_${i}.uan "$WorkingDir"/Run_Outputs/$RunName/${gen}_${i}.uan

        done

4. Made Number of Neutrinos thrown (NNU) in AraSim's setup.txt file a variable: I've added the following line in part D -- as well as the looping section in part G. 

These lines replace the number of neutrinos thrown in our setup.txt AraSim file with what ever number you assigned NNT at the top of XF_Loop.sh in the "variables" section. "setup_dummy.txt" is a copy of setup.txt that has  NNU=num_nnu instead of a number. NNU is the number of neutrinos thrown (as seen in setup.txt). This "sed" line finds every instance of num_nnu in setup_dummy.txt and replaces it with $NNT (the number you assigned NNT in the variable section at the top of XF_Loop.sh). It then pipes this information into setup.txt (and overwrites the last setup.txt file allowing the number of neutrinos thrown to be as variable changed at the top of this script instead of manually changing it in setup.txt each time. Command works the following way: sed "s/oldword/newwordReplacingOldword/" path/to/filewiththisword.txt > path/to/fileWeAreOverwriting.txt. Both setup.txt and setup_dummy.txt can be found in the directory: /fs/project/PAS0654/BiconeEvolutionOSC/AraSim/

cd "$AraSimExec"

for i in `seq 1 $NPOP`

do

#This next line replaces the number of neutrinos thrown in our setup.txt AraSim file with what ever number you assigned NNT at the top of this program. setup_dummy.txt is a copy of setup.txt that has \

NNU=num_nnu (NNU is the number of neutrinos thrown. This line finds every instance of num_nnu in setup_dummy.txt and replaces it with $NNT (the number you assigned NNT above). It then pipes this infor\

mation into setup.txt (and overwrites the last setup.txt file allowing the number of neutrinos thrown to be as variable changed at the top of this script instead of manually changing it in setup.txt e\

ach time. Command works the following way: sed "s/oldword/newwordReplacingOldword/" path/to/filewiththisword.txt > path/to/fileWeAreOverwriting.txt                                                      

        sed "s/num_nnu/$NNT/" /fs/project/PAS0654/BiconeEvolutionOSC/AraSim/setup_dummy.txt > /fs/project/PAS0654/BiconeEvolutionOSC/AraSim/setup.txt

        #We will want to call a job here to do what this AraSim call is doing so it can run in parallel                                                                                                  

        cd $WorkingDir

        qsub -v num=$i AraSimCall.sh

        rm outputs/*.root

done

5. rm cannot remove 'simulation_PEC.xmacro': No such file or directory error appearing in generations after the first (ie not seen in gen 0, but seen in gen 1+).

Statistics initialized.
Tournament parents selected.
Tournament complete.
rm: cannot remove 'simulation_PEC.xmacro': No such file or directory

I found that it tries to remove it twice. See here:

########  Loop (G)  ######################################################################################                                                                                                                                                                                                                                                                                        

#     1. Does steps A-F for each generation                                                                                                                                                                                                    

###########################################################################################################                                                                                              

for gen in `seq 1 $TotalGens`

do

# used for fixed number of generations                                                                                                                                                                   

#gen=0; while [ `cat highfive.txt` -eq 0 ]; do (( gen++ ))                                                                                                                                               

# use for runs until convergence                                                                                                                                                                         

#should the line below be double indented?                                                                                                                                                               

        read -p "Starting generation ${gen}. Press any key to continue... " -n1 -s

#Part A                                                                                                                                                                                                  

        cd $XmacrosDir

        rm simulation_PEC.xmacro                                               

        cd "$WorkingDir"

        ./roulette_algorithm.exe cont $NPOP

        cp generationDNA.csv Run_Outputs/$RunName/${gen}_generationDNA.csv

    ##chmod -R 777 $WorkingDir                                                                                                                                                                           

#we can make a function that does this with the flag $gen                                                                                                                                                

#doing this in part A would mean setting $gen=0                                                                                                                                                          

#Part B                                                                                                                                                                                                  

                # First, remove the old .xmacro files                                                                                                                                                    

        #when do that, we end up making the files only readable; we should just overwrite them                                                                                                           

        #alternatively, we can just set them as rwe when the script makes them                                                                                                                           

        cd $XmacrosDir

        #I'm commenting out the next two lines. They are written to below (around lines 154-170)                                                                                                         

        #If we keep them this way then the files have restricted permissions                                                                                                                             

        rm output.xmacro

        rm simulation_PEC.xmacro

I have commented the first removal of simulation_PEC.xmacro. If there are any issues with it running, we can go ahead and comment it back in; however, I suspect this is not a necessary line to have added in. A run has not been performed since these edits. So, once it has been tested, we can delete the comments out:

#Part A                                                                                                                                                                                                  

        #I think these next two lines can be deleted. Its repeated again just below...11/30/19 comment by Julie  #cd $XmacrosDir #rm simulation_PEC.xmacro                                               

        cd "$WorkingDir"

        ./roulette_algorithm.exe cont $NPOP

        cp generationDNA.csv Run_Outputs/$RunName/${gen}_generationDNA.csv

6. Python Error from FScorePlot.py: The error can be seen in the line below.        

Congrats on getting a fitness score!

  File "FScorePlot.py", line 1

    -*- coding: utf-8 -*-

    ^

IndentationError: unexpected indent

Congrats on getting some nice plots!

I've commented out this strange line at the start of this python program:   -*- coding: utf-8 -*-. This fixed the immediate error, but threw another one when I tried to run it outside of the loop with the command 

python FScorePlot.py /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/Julie_11_30/ /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/Julie_11_30/ 2

The next error it threw can be seen below. 

Traceback (most recent call last):

  File "FScorePlot.py", line 79, in <module>

    fScores.append(fScorei[2:])

AttributeError: 'numpy.ndarray' object has no attribute 'append'

            This is Alex P. I edited the append() and changed it to np.append(fScores, fScorei[2:]) on line 79 in FScorePlot.py

7. rm: cannot remove 'outputs/*.root': No such file or directory: This error comes up right after the AraSim job is submitted in part D. 

1011007.pitzer-batch.ten.osc.edu
rm: cannot remove 'outputs/*.root': No such file or directory
Waiting for AraSim jobs to finish...

It's definitely coming from the lines:

for i in `seq 1 $NPOP`
do        #We will want to call a job here to do what this AraSim call is doing so it can run in parallel
        cd $WorkingDir
        qsub -v num=$i AraSimCall.sh        rm outputs/*.rootdone

We are currently in $WorkingDir at this point-- which is: WorkingDir= /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop. There is not directory called "outputs" in this location. This is why we are getting an error.Can someone explain what .root files we are trying to remove, and why? I cannot proceed unless someone makes this clear to me, as I do not know what files we are looking to remove (and where they exist/when they are created).

Immediately after that, I get the following error:

Waiting for AraSim jobs to finish...
rm: remove write-protected regular file 'AraSimFlags/1.txt'? y
rm: remove write-protected regular file 'AraSimFlags/2.txt'? y
mv: cannot stat '*.root': No such file or directory

It looks like it is coming from the following line:

cd "$WorkingDir"/Antenna_Performance_Metric
mv *.root "$WorkingDir/Run_Outputs/$RunName/RootFilesGen0/"

However-- again-- it looks like it's trying to move .root files from the directory: /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Antenna_Performance_Metric and put them into: /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/$RunName/RootFilesGen0/

It can't move these, because they do not exist where it is trying to take them from. Similarly, the question in bold above would help me with this a lot. From what I can see, we are only utilizing an .txt file output from Arasim. Does Arasim output this to .txt, or are we somehow converting from a .root to a .txt? 

 This error has not yet been corrected!!!!       

Cade's comment here:

The .root files live in BiconeEvolutionOSC/AraSim/outputs/ folder. Thus, we were looking at the wrong directory. They were being removed because it is a giant file (GB size) and are created everytime AraSim runs. It just overwrites itself everytime though so it doesnt add up too much. We can have them deleted later if needed. It's our choice on whether we take it out completely, or redirect to delete the .root files in the correct directory.

8. New errors found after running with these corrections: These need to be fixed!

cp: cannot stat ‘Antenna_Performance_Metric/0_1.uan’: No such file or directory

cp: cannot stat ‘Antenna_Performance_Metric/0_2.uan’: No such file or directory

Congrats on getting a fitness score!

Traceback (most recent call last):

  File "FScorePlot.py", line 86, in <module>

    Gen = np.zeros(fScores.shape[0]*fScores.shape[1])

I found the cp error. It looks like I was incorrect in #3 about this cp command for the .uan files. I moved all of the currently existing .uan files into a new directory in

 /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Antenna_Performance_Metric/UAN

With all of them out of the Antenna_Performance_Metric directory, I did a run from scratch to see what it creates. Here's the outcome: So on generation 1, for individual one its going

1_1.uan
1_2.uan
1_3.uan

ie-- > individual_frequencyStep.uan

So once it hits all 60 frequencies for individual 1, it does all 60 for individual 2 by saying:
2_1.uan
2_2.uan
2_3.uan

...etc.This isn't great because it never actually cares about generation number. In section E (so generation 0) we currently have it as:

for i in `seq 1 $NPOP`
do
    #Remove if plotting software doesnt need                                                                                                                                                             
    #cp data/$i.uan ${i}uan.csv                                                                                                                                                                          
    cp Antenna_Performance_Metric/0_$i.uan "$WorkingDir"/Run_Outputs/$RunName/0_${i}.uan
done

and in section G of the looping part (gens 1+) we have:

    for i in `seq 1 $NPOP`
        do
    # Gens data used to create a .csv file for the uan file for gain plotting                                                                                                                            
    # cp Antenna_Performance_Metric/$i.uan ${i}uan.csv                                                                                                                                                   
                cp Antenna_Performance_Metric/${gen}_${i}.uan "$WorkingDir"/Run_Outputs/$RunName/${gen}_${i}.uan
      done

This doesn't seem like it's being output from XF the way we thought it is. I think the easiest way to do this is to edit the xmacros to have it output like:

generation_individual_frequency.uan

and thus

cp Antenna_Performance_Metric/${gen}_${i}_${frequency}.uan

 If so, we'd have to make a new variable in the bash script for $frequency to go from 1-60, and have it loop moving all of those. I will be looking into this with Cade tomorrow 12/3/19 (since he is the xmacro script expert). Stay tuned!

Phew....thanks for sticking with me on this long update.