One of our foundational questions tied to the optimization of the GA has been "How many individuals should we simulate". Up to now, our minds were made up for us by the speed of arasim being great enough that the time cost of simulating individuals was great enough that the improvements made from having more were not enough to justify the slowdown. However, with the upgrade to the faster, more recent version of arasim, I decided to re-examine this. This was also spurred on by the fact that the last time I ran this test we were testing GA performance by final generation metrics rather than by how many generations it took to reach a benchmark. So in one of my optimization tests, I tracked this data. 

To start, using the same run proportions, using a .5 chi-squared benchmark, the average time across all 89 run types used in this run was 25.4 generations for 50 individuals as compared to 8.3 generations running the same test for 100. Furthermore, the minimum number of generations for 50 individuals was 4.8 while using 100 individuals yielded 2.4. So on average running 100 individuals was about 3 times fast at reaching this benchmark than with 50. And when comparing the best result regardless of run type, 100 individuals was still 2 times quicker than the min for 50 individuals. Finally, the run that yielded an average of 2.4 generations for 100 individuals took an average of 29.2 generations with 50 individuals or roughly 12 times the generations. 

For the test we will discuss, we ran 89 different run types that all used 60% rank, 20% roulette, and 20% tournament selection respectively. These test had the following ranges:

6-18% of individuals through reproduction (steps of 3%)

64-88% of individuals through crossover (steps of 12%)

0-10% mutation rate (steps of 5%)

1-5% sigma on mutation (steps of 1%)

These tests also used our fitness scores with simulated error of .1 to imitate arasim's behavior and as such we used the chi-squared value to evaluate these scores as there is no error on those values. 

Comparing this same test with a tighter chi-squared benchmark of .25, we see similar results. On average 50 individuals took 37.1 generations to reach this point while 100 individuals took 16.0 generations. Similarly, the minimums amount of gens for 50 individuals was 15.4 while 100 individuals was 5. Finally, the corresponding run for the 5 generation min with 100 individuals took 41.8 generations with 50 individuals. These correspond to speed up's of 2.3, 3.08, and 8.36 respectively. 

This data implies that on average, independent of run type, we should expect to have to use 2-3 times fewer generations while running 100 individuals than we would running 50 individuals but we could see up to 8-12 times fewer generations to reach benchmarks. Another data set using a different set of selection methods was also tested for this and again yielded similar results, though overall the runs from the first batch were better across both 50 and 100 individuals and so those results are likely to be more indicative of the parameters we use in a true run. 

The data being examined in these results can be found here: https://docs.google.com/spreadsheets/d/1GlfnjQSO6VI8MuUGYTUcLkjwDZU98nyFFysgTTfVFOE/edit?usp=sharing  

Work in progress. Update on H-pol XF design. The first plot is the gain pattern with ferrite rod sets (blue) vs. with nothing but the copper plates. 
The second plot is the gain pattern with everything but the ferrite rod sets vs. with nothing but the copper plates.
The ferrite rod sets function to narrow the gain pattern.

How To Update PueoSim For GENETIS:

First, whoever updates pueoSim needs access to pueoBuilder, pueoSim, and niceMC on GitHub (ask Will for permissions).

Once you do, go into the peuoSim directory at /fs/ess/PAS1960/buildingPueoSim/

and source set_env.sh

`source set_env.sh`

Then, we want to make copies of the files we are currently modifying for the GENETIS loop.

For PueoSim v1.1.0 these are:

1. nicemc/src/EventGenerator.cc  # Modified to create custom output root files to calculate errors

2. pueosim/test/simulatePueo.cc # Modified to stop the simulation of the LF antenna which is not needed for GENETIS

3. pueosim/src/Seavey.cc # Modified to read in custom gain files

4. pueosim/src/pueo.cc # Modified to read in custom gain files

5. pueosim/include/Seavey.hh # Modifies to read in custom gain files

Currently, I store copies of these in the `/fs/ess/PAS1960/buildingPueoSim/backups` directory in case somebody accidentally overwrites the files in pueoBuilder. 

Once you’ve made the copies, you can run `./pueoBuilder.sh` from the `/fs/ess/PAS1960/buildingPueoSim/pueoBuiler` directory. This will rebuild pueoSim and niceMC, pulling the latest updates from GitHub. 

You may need to delete the pueoSim and niceMC directories in order for the builder to pull the latest version from GitHub. Or, if it’s being really stubborn, move the whole pueoBuilder directory to a temporary location and run the builder from scratch with 

`git clone git@github.com:PUEOCollaboration/pueoBuilder` and then `./pueoBuilder.sh`

Then, you will need to reference the copies of the changed files to make changes to the new version of pueoSim. Hope this doesn’t cause too much of a headache, and when you’re done, return to the /fs/ess/PAS1960/buildingPueoSim/pueoBuiler directory. 

Then you simply type

`make install`

then

`make`

And now, pueoSim should be ready to run!

EventGeneratior.cc Changes and Rationale:

PueoSim’s default output ROOT files are very large and therefore time-consuming to parse through to get the information we need to calculate effective volume and errors. So, we want to create a custom ROOT file with only the variables we want, greatly increasing the speed of the analysis.

To do this, we want to create a new TFile with corresponding TTree and TBranches that will store the loop.positionWeight, loop.directionWeight, and neutrino.path.weight. Then, we want to fill the Tree when a detector is triggered and write the results to the file at the end of the loop.

Sample code for initializing the TFile:

simulatePueo.cc Changes and Rationale:

By default, pueoSim v1.1.0 runs a simulation for the normal antenna and a low-frequency (LF) antenna. As GENETIS is evolving for the main antenna, we are not interested in using computation time to simulate the LF antenna. So, we comment out the lines that initialize the LF detector in this file.

Seavey.cc, Seavey.hh, and pueo.cc Changes and Rationale:

As of pueoSim v1.1.0, the simulation software only has the ability to read in one antenna. This is a problem, as we want it to be able to read in files for any antenna we make. So, we need to change these scripts to be able to read in whatever gain files we want.

The convention the loop uses is to input gain files in the format of 

vv_0_Toyon{runNum}

hh_0_Toyon{runNum}

…

In the ./pueoBuilder/components/pueosim/data/antennas/simulated directory

So, the main change is getting the runNum variable into Seavey.cc file where the gains are read. Currently, we define a global variable at the stary of pueo.cc and pass it into where Seavey is called. This means we have to add runNum as a parameter in Seavey.cc as well as Seavey.hh. Finally, we change the name of the read-in files to be in the above format AFTER dividing runNum by 1000 (this is because pueoSim uses the run number as a random number generating seed, so we divide runNum by 1000 to be able to read in the same gain patterns for multiple seeds of the same individual).

Note: We used to change pueoSim to output a veff.csv file. We now handle this calculation by analyzing ROOT files, so this change is no longer necessary. 

Attached is a .txt file you can find in the Evolutionary_Loop directory as Loop_Error_List.txt. It's a list of the current errors we sometimes experience in the loop, along with how to fix them if you encounter them while running. If people encounter errors that aren't in the list, let everyone know in the #genstudents chat on slack and update the file with the error message and when it was encountered (what state the loop was in) and the possible cause and solution if you know it. 

We want to automate tests in pull requests in GitHub to run unit tests on individual functions to ensure they return the expected results. This will help prevent bugged code from being pushed to working branches. We will add a list of unit tests to this ELOG as we design them.

Here are a collection of antenna gain patterns with the antenna power source located at various heights at 500 MHz. The heights are indicated in the name of the images and were located at 3cm, 6cm, 12cm, and 24cm from the base. One thing to note is that the power source becomes longer as a function of height (since it's connected to the ridges, which slope back). 

The 3D gain plots attached show the gain becoming more uniform until we get to the last one, where it seems to reinvert and be much stronger in the upward direction than below. I'll update this post with some polar slices for a more complete comparison between them.

EDIT: 07/17/23

I've made a few adjustments and added plots of them.

• First, I separated the ridges. Previously, the ridges touched at their inner corners. This electrically connected them and may be why we saw very high peaks in the first set of impedance plots. You can see the plots named Separated_Impedance_<x>_cm.png.
  • This appears to have lowered the peaks considerably and also shifted them to lower frequencies. The peaks are still high when the source is placed high.
  • We seem to be developing another peak at higher frequencies, although this also existed in the previous tests for the high sources.
• After this, I added a waveguide and positioned the source between the extensions of the ridges into the waveguide. I've also added a picture that shows the waveguide in the geometry. 
  • The sources here are positioned from 1 cm below the ridges to 8 cm below the ridges. The naming convention is the same as above, except it includes "Waveguide" in the name.
  • The peaks are considerably lower here and seem consistent as we place sources further down. They are all below 500 Ohms, compared to as much as 4000 Ohms in the previous tests.

EDIT: 08/10/23

I've been trying more tweaks to the design to get rid of the peaks we see in the plots below. I'm having trouble actually getting the impedance to match close to the 50 Ohm value we expect from the cable. The best case situations seem to come from when the ridges are still touching, which we don't want. Here are the properties I've been changing:

• Ridge positions (gene in GA)
  • We don't want the ridges to touch, but I can't get them to be very far from each other anyway. The minimum wavelength in our bandwidth would be 20 cm, but that's close to the size of the bottom of the antenna for when we make it large, so the separation is still small.
  • When the ridges do touch is actually when I get the best matching,
• Bottom size (gene in GA)
  • I've been changing the size of the bottom of the antenna. This is something that can change substantially between individuals in the GA, so even if I find a good value for this, the impedance it gives wouldn't be the same for all individuals we see in an evolution.
  • This does seem to move the location of the impedance peak (in frequency) but I can't seem to get it fully out of band.
• Waveguide length
  • I've been changing the length of the waveguide between 3 cm and 10 cm. We said that somewhere in the middle should be the most reasonable (the example antenna we saw looked to have a waveguide of ~5 cm long). It seems like the source position is more important.
• Source position
  • I've been moving the position of the power source inside the waveguide. This seems to move around the peak and can lower its height, but I haven't been able to get peaks lower than ~400 Ohms, except in cases where the ridges are in contact.

I've added a picture of the best impedance I managed to get, but this involved an antenna with a small base and ridges touching. My next step is to just try a grid search by making many antennas automatically and simulating them all to look for any that have a nice impedance in our band. 

As of 01/30/2020, the proposal was submitted! The project description and summary are attached below. 

Bicone Loop Status Update: 

The loop currently runs; however, it does seem that if AraSim jobs are left to run while the user is away, it doesn't properly write one or more of the flags, and makes the loop get stuck at AraSim. We must then re-run from that state by manually editing the Save State text file. 

The loop currently has the following additions implemented.

• Functionalized loop (XF_Loop_AraSeed.sh)
  • Cleaner bash script which runs smaller bash scripts, instead of one massive script. 
  • Submits multiple AraSim jobs in parallel for each individual to increase speed. 
    • Ie if 100k neutrinos are thrown per individual ($NNT=100000, $Seed=10) we can, say, break it up into a "seed" of 10 -- which then submits 10 jobs with NNT=10k for each individual. 
  • Still needs this to be fixed: The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being seeded. It is just submitting one job the following way.
    • If NNT=100000 and Seed=10 (ie 10 jobs with 10k neutrinos), it will only run AraActualBicone as 1 run with 10k neutrino number (ie NNT/seed=NNT_New).
• Save State 
  • After every "part" -- ie part a, part b, etc -- of the loop, we save state by writing a series of numbers to a text file. 

    • ${gen}, ${state}, {indiv}

  • If the loop is killed for any reason during the process, the text file is read in, and the loop can continue forward from the last completed item/section. 

  • This saves the state after every individual runs through XF. 

• Antenna sizing

  • The larger the antenna, the longer XF takes to run. A true-to-size ARA antenna takes anywhere from 1.5 to 2 hours per individual. 

  • What we've done: Allowed us to scale the antenna by a factor of "x".

    • If we make the gaussian centered around something, say, 5 times smaller than the ARA antenna size, we then make the frequencies 5 times larger. This should have the same effect. 

    • This also means we've had to scale these sizes back up when plotting. 

    • Still needs this to be fixed: all of this has been hardcoded in! We need to make these variables in the MANY places we need to adjust for this scaling. See the to-do list below for details. 

Moving forward, I will be doing the following:

1. Getting our less coding-savvy teammates familiar with bash, python, and c++. This will be crucial for us when we start adding more parameters to our GA. 
2. Mastering XF (along with Alex, and Alex). 
3. Getting the team to clean up some messy coding done during the proposal hustle. 
4. Fixing some errors. 
5. Teach the ENTIRE team git. 

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim error fixed (Alex, Julie) 

   1. It seems that if AraSim jobs are left to run while the user is away, it doesn't properly write one or more of the flags, and makes the loop get stuck at AraSim. We must then re-run from that state by manually editing the Save State text file.

   2. We are going to try to get XF to run at the command line so that we can submit it as a job. This would alleviate us using an interactive job. We *hope* this would fix the error since the user wouldn't be "gone" while utilizing a GPU in an interactive job. 

      1. Alex emailed XF to ask a few questions! Stay tuned!

2. Make Hardcoded directories variables.

   1. Xmacros (use sed command) 

   2. All over bash script functions.

   3. AraSim job files (sed command?)

3. Make the following variables in the bash script:

   
   In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

   Grid spacing=0.1
   Frequency= factor of n higher 
   This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.
   
   
   

   In roulette_algirithm.cpp

   Length = 50cm/Multiplier_factor; 
   STD= 15cm/Multiplier_factor 
   Radius = 1.5cm/Multiplier_factor; 
   STD= 0.75cm/Multiplier_factor 
   
   Also the "Multiplier_factor" line 89.
   
   This "Multiplier_factor"has to be the same as the GeoFactor in fitnessFunction_ARA_amy.cpp AND the same as the factor higher in frequency in the xmacro.
   ​NOTE:THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   
   

   In fitnessFunction_ARA_amy.cpp:

   GeoFactor=2 
   This GeoFactor has to be the same as the Multiplier_factor in the roulette algorithm AND the same as the factor higher in frequency in the xmacro
   NOTE: THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   

4. Get rid of all code we are not using anymore.  

   1. Currently using: XF_loop_AraSeed.sh

      1. Look at what items are being called in here, and remove old versions such as XF_loop_Prototype.sh, XF_loop_Functionized.sh, and the related scripts. 

      2. These can temporarily moved to on "old code" directory and we can delete once we finalize the loop. 

5. Turn tournament on in the GA. 

6. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being seeded.

Atttendance: Julie, Max, Cade, Suren, Evelyn, Sophie

Today's tasks:

Max works on AraSim section (needs to do more research and reading)

Julie, Suren, Sophie, Evelyn work on plotting software to plot gain patterns of indiviudals

Cade preps impuse XF code to be implemented

To-Do list:

I. Divide up sections on paper, and work on them. See http://radiorm.physics.ohio-state.edu/elog/GENETIS/1 for section assignments. 

II. Implement impuse code. 

• This will require us to change XFintoARA.py so that it takes less output files. 
• Is there anything else different with the output files?

III. Finish Evelyn and Sophie's plotting software.

• This means also puting it onto Nutau, and adding in lines to the bash script (both to move and save the images, and to run the code!)

Here are things we can do if we are waiting for Remcom:

I. Change sides of bicone to be sinusoid instead of lines.

II. Take known bicone parameters and put it through AraSim to check if it's working properly.

• This also means we should make plots with it to triple check. 

III. Test GA and make it maximize other paramaters to see how it does. 

This is an entry for the usual Wednesday working meeting GENETIS holds. We are largely carrying on from last week's hackathon hammering out the remaining pieces that need to be fixed. See the previous ELOG post for a list of those.

Alex

Amy pointed out on the Monday call that the proper way to simulate the ANITA antenas is to use one source at a time. I modified the script I had so that it makes two simulations per antenna. The first one will use the power source connected to the "VPol" ridges and the second one uses the one connected to the "HPol" ridges. After checking this, it looks like the asymmetry I had pointed out before fell away. In other words, I think XF only wanted to use a single power source anyway. So now we should be able to produce the VV, HH, VH, and HV data that icemc and PUEOsim need.

Ryan and I also worked a bit on the slopes of the ridges. We experimented with the parameter that affects the curvature, which seems to be able to be too large. But many of the designs still look to have almost no slope and many still caved inward rather than sloping outward. We're still working on figuring out what causes that, but I think it means we need another constraint on distance of the top of the inner ridges from the center. 

We also tried figuring out why the sides of the curved part of the ridges don't stay in the same plane as the straight part of the ridges. I'm confident that they do stay in a single plane (I attached a Mathematica notebook that shows as much by finding that the torsion of the curve is zero; for some reason I couldn't print it to a PDF). We think that we should be able to rotate the curve into the same plane as the straight part of the ridge, but I haven't figured out how yet.

 Nick and I continued working on the HPol antenna. We want to check if there are any other things in Steph's example XF file that are significant to the results of the simulation (versus just being there as a representation of the model). It looks like there are, and Nick is working to figure out what they are by deleting them (in a copy) and generating simulation results for them to compare to the original. Otherwise, the most significant parts (the copper plates) are now being made in a script and just need to be connected to one another.

Dylan

I tried to build root from source again and ran into an error 3 hours into the build. I messaged Will and asked if he knew anything about the error. I then reran the build with Will's suggested change and ran into the same error 3 hours later. I did, however, manage to build pueo by sourcing the installation of root in Will's shared pueo directory. I'm now testing getting a txt output file from the simulation as it turns out my initial solution didn't work.

Update:

I got a working version of root installed. Also, I was able to modify pueoSim to output a veff.txt file. Now, I need to modify where it reads in the gain files, and then modify the icemc specific bash scripts to work for pueosim.

Attendance: Julie, Mitchell, Ryan, Alex P., Leo, Alex M.

Today's Tasks

Mitchell: Worked on Python Project and helped with Ryan's final bash project questions

Ryan: Finished the bash project, and is about to start the Python project. 

Leo: Worked on lectures, which will restart on March 17th.

Julie, Alex P, Alex M: Worked on fixing the job XF submission scripts. Worked on making it so that it saves state after each individual's XF simulation is complete. We tried to parallelize as many of the XF scripts as possible, but if we have more than one it asks us to choose a license for each individual -- ie requires user input each time. Since we haven't found a way to get around it AND since running with 1GPU is super fast (under 5 min per individual), we are going to run them in series.

TO-DO List

1. Scaling test run: Do a test run to see if the scaling (when we also scale the grid spacing) actually affects the run time for XF. (PASSED THIS TASK TO JULIE. WILL DO TOMORROW 3/4)
2. Evolve: Right now, we are using an interactive job with only a CPU running at half scale, and have just edited our software so that we submit jobs for the XF simulations requesting 1GPU (could be 2GPU if we choose to edit it so). 
   1. WHY WE ARE DOING THIS:
      1. Remember that we only need an interactive job when XF initializes and reads in the DNA for each individual. The GUI closes once the simulations start and we no longer need an interactive job. 
      2. We were requesting a 1GPU interactive job and just running XF from the command line; however, OSC has been SUPER busy lately and we haven't been getting them (especially since we need long wall times if we are running the full simulations over many generations within the interactive job). However, we realized that if we submit an interactive job at a CPU (so that XF can allow the GUI to pop up), and then run the simulations by requesting 1-2GPUs we can then request a much smaller wall time and it will be quicker for the simulations to run --- ie we would be requesting a wall time of about 10 minutes using 1-2GPUs per simulation for each job submission versus 5+ hour wall time of an interactive job using 1-2GPUs.
      3. WE HAVE TESTED THIS, AND WE HAVE ERRORS. WE WORKED ON FIXING THEM TODAY, AND HOPE TO FINALIZE THEM TOMORROW (3/4) AND RUN!

3. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized. (ALEX M)

   2. As of 3/3, I have not gotten an update on this. I will check on the status of this tomorrow. 

4. Comment code thoroughly (Everyone needs to help with this)

   1. ​Julie made some progress on this over the weekend. 

      1. Made comments on what each bash script version does, and cleaned up our directories from old code so that it doesn't confuse the others when they start helping 

   2. WAY more needs to still be done. 

5. Get new proposal results for March 12th deadline.

Today's Tasks

Julie: Write bash practice assignment, assist others when they have questions, work on Git lecture. (Will finish bash assignments early next week)

Evelyn, Mitchell, Eliot: Make progress on the "Learn the Command Line" course on Code Academy. (Should be done by the end of the week?)

Leo: Start gathering information for the "physics of antennas" lecture series! Note: Remember that Leo wants to be an educator, and would like us to find opportunities to work on "teaching" items. Most of the students don't have any background on how antennas work. He started learning about them and is going to do a lecture series (likely the first 20 minutes of every meeting) starting in a few weeks! (Should start series by hopefully 2/21)

Alex M: Clean up unused code. Fix hard-coded directories (likely finished & tested by end of day 2/11)

Alex P: Fix hard-coded variables (likely finished & tested by end of day 2/11)

I've tacked on our to-do list in the meantime so we can keep carrying the items still pending along with us. As we complete them, I will drop them off the list! 

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim error fixed (Alex, Julie) 

   1. It seems that if AraSim jobs are left to run while the user is away, it doesn't properly write one or more of the flags, and makes the loop get stuck at AraSim. We must then re-run from that state by manually editing the Save State text file.

   2. We are going to try to get XF to run at the command line so that we can submit it as a job. This would alleviate us using an interactive job. We *hope* this would fix the error since the user wouldn't be "gone" while utilizing a GPU in an interactive job. 

      1. Alex emailed XF to ask a few questions! Stay tuned!

2. Make Hardcoded directories variables.

   1. Xmacros (use sed command) 

   2. All over bash script functions.

   3. AraSim job files (sed command?)

3. Make the following variables in the bash script:

   
   In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

   Grid spacing=0.1
   Frequency= factor of n higher 
   This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.
   
   
   

   In roulette_algirithm.cpp

   Length = 50cm/Multiplier_factor; 
   STD= 15cm/Multiplier_factor 
   Radius = 1.5cm/Multiplier_factor; 
   STD= 0.75cm/Multiplier_factor 
   
   Also the "Multiplier_factor" line 89.
   
   This "Multiplier_factor"has to be the same as the GeoFactor in fitnessFunction_ARA_amy.cpp AND the same as the factor higher in frequency in the xmacro.
   ​NOTE:THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   
   

   In fitnessFunction_ARA_amy.cpp:

   GeoFactor=2 
   This GeoFactor has to be the same as the Multiplier_factor in the roulette algorithm AND the same as the factor higher in frequency in the xmacro
   NOTE: THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   

4. Get rid of all code we are not using anymore.  

   1. Currently using: XF_loop_AraSeed.sh

      1. Look at what items are being called in here, and remove old versions such as XF_loop_Prototype.sh, XF_loop_Functionized.sh, and the related scripts. 

      2. These can temporarily moved to on "old code" directory and we can delete once we finalize the loop. 

5. Turn tournament on in the GA. 

6. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being seeded.

       

Today we continued with the to-do lists items previously posted. I marked all the to-do list items that have been completed with green and a slash. Note that I have also added a new item at the bottom of the to-do list.

Today's Tasks

Julie: Write bash practice assignment (almost done!)

Evelyn, Mitchell, Eliot, Ryan: Make progress on the "Learn the Command Line" course on Code Academy. (Mitchell is done, and the others are close.)

Leo: Start gathering information for the "physics of antennas" lecture series! Note: Remember that Leo wants to be an educator, and would like us to find opportunities to work on "teaching" items. Most of the students don't have any background on how antennas work. He started learning about them and is going to do a lecture series 

Alex M: Make sure antenna scaling is made a variable in the bash script everywhere that it exists, start mass commenting all of the code so it reads as cleanly as the GA does. (Finished making the antenna scaling a variable. needs to be tested, but we waited forever for wall time)

Alex P: Fix hard-coded variables (done but tried to test it. Waited for wall time and never got it), and looked into ways to get around requesting an interactive job. For some reason, we don't need an interactive job to forward things like plots, but we do for the XF UI. We were trying to figure out why this is, and if there's a way to fix this. So far, no luck. 

AMY QUESION: Could we actually consider running on NASA computers? I think Alex P. found a way the NASA computing centers let you adjust for this issue, and it didn't work on OSC. If we end up not finding a solution with OSC, is it at all possible to have XF installed with NASA computing centers? (NOTE: I JUST ASKED ALEX FOR MORE DETAILS AND WILL ADD THEM HERE TOMORROW -- IE WEDNESDAY)

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim error fixed (Alex, Julie) 

   1. It seems that if AraSim jobs are left to run while the user is away, it doesn't properly write one or more of the flags, and makes the loop get stuck at AraSim. We must then re-run from that state by manually editing the Save State text file.

   2. We are going to try to get XF to run at the command line so that we can submit it as a job. This would alleviate us using an interactive job. We *hope* this would fix the error since the user wouldn't be "gone" while utilizing a GPU in an interactive job. 

      1. Alex emailed XF to ask a few questions! Stay tuned!

2. Make Hardcoded directories variables

3. Make the following variables in the bash script:

   
   In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

   Grid spacing=0.1
   Frequency= factor of n higher 
   This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.
   
   
   

   In roulette_algirithm.cpp

   Length = 50cm/Multiplier_factor; 
   STD= 15cm/Multiplier_factor 
   Radius = 1.5cm/Multiplier_factor; 
   STD= 0.75cm/Multiplier_factor 
   
   Also the "Multiplier_factor" line 89.
   
   This "Multiplier_factor"has to be the same as the GeoFactor in fitnessFunction_ARA_amy.cpp AND the same as the factor higher in frequency in the xmacro.
   ​NOTE:THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   
   

   In fitnessFunction_ARA_amy.cpp:

   GeoFactor=2 
   This GeoFactor has to be the same as the Multiplier_factor in the roulette algorithm AND the same as the factor higher in frequency in the xmacro
   NOTE: THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   

4. Get rid of all code we are not using anymore.  

   1. Currently using: XF_loop_AraSeed.sh

      1. Look at what items are being called in here, and remove old versions such as XF_loop_Prototype.sh, XF_loop_Functionized.sh, and the related scripts. 

      2. These can temporarily moved to on "old code" directory and we can delete once we finalize the loop. 

5. Turn tournament on in the GA. 

6. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being seeded.

7. Comment code thoroughly 

   1. The Gen alg is commented so thorougly that it can read like a book. We want this with ALL of our code; it will help all the less familiar people get comfortable. 

Today's Tasks

Julie: Write Python project! I have officially finished the bash course for the students. It can be found here: https://docs.google.com/document/d/1nGdPrwfYJOrO6NBT76Wsh6T0hDmGHo2-23oJ1WyeNcs/edit?usp=sharing. Mitchell is currently trying it out and will give me comments. 

Mitchell: Work through my bash project and give me comments!

Evelyn, Eliot, Ryan: Make progress on the "Learn the Command Line" course on Code Academy. 

Leo: continuing gathering information for the "physics of antennas" lecture series! These will take place every Friday meeting starting next week. 

Alex M: Try to get scaling into the xmacro, comment the code better.

Alex P: Add on the AraSim fixes mentioned below!

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim errors fixed

   1. Explanation of issue (1): Usually 1 of ~100 AraSim jobs never completes. OSC shows "completed", but the flag is never written and the output file is not filled in entirely. The error file shows the following.

      1. Carl believes this is due to the way we are saving our AraSim outputs. Instead of saving the files directly to our desired location, we are going to save them to $TMPDIR and use "pbsdcp" to copy them from $TMPDIR to our desired location. This may alleviate the problem since it will not be writing all the files to our desired location at once. 

DATA_LIKE_OUTPUT=1,2 doesn't work with DETECTOR=0,1,2
DATA_LIKE_OUTPUT controls data-like output into UsefulAtriStationEvent format; without a real station selected (using DETECTOR==3,4), the mapping to the data-like output will not function correctly
There are 1 errors from settings. Check error messages.
SysError in <TFile::Flush>: error flushing file outputs/AraOut.setup.txt.run29.root (Stale file handle)

1. Explanation of issue (2): We keep timing out of the interactive job when AraSim is running (if the user isn't remaining active on their terminal).
   1. OSC suggested that we use a virtual desktop (via ondemand) insead of using an interactive job. We will try this. 
      • Note: We should now be requesting 2GPUs instead of 1GPU. 

1. Make the following variables in the bash script. 

   1. Alex M is working on this. We can't directly make this a variable. We need to take the array of frequencies and multiply it by some value and THEN feed it into the xmacro. He is looking into whether of not he can directly do this in bash. If not, I have written a small python script that does this, and he can write it to that file; however, he can't append it at the end so it's not the ideal way. We are going to continue to try to do it through bash first. 

In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

Grid spacing=0.1
Frequency= factor of n higher 
This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.

1. Turn tournament on in the GA. 

2. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized.

3. Comment code thoroughly

   1. The Gen alg is commented so thorougly that it can read like a book. We want this with ALL of our code; it will help all the less familiar people get comfortable. 

Today's Tasks

Julie: Continue writing Python project! 

Mitchell, Evelyn, Eliot (not here on Tuesdays), Ryan: Work through bash project

Leo: continuing gathering information for the "physics of antennas" lecture series! We start this Friday at 4pm. 

Alex M: Try to get scaling into the xmacro, comment the code better.

Alex P: Add on the AraSim fixes mentioned below! He is testing this now. We are waiting for updates. 

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim errors fixed

   1. Explanation of issue (1): Usually 1 of ~100 AraSim jobs never completes. OSC shows "completed", but the flag is never written and the output file is not filled in entirely. The error file shows the following.

      1. UPDATE: Alex P. has edited our bash job script to save the outputs to $TMPDIR and then to pbscp (submit a "job" to copy them over) them into our correct directory that the bash script needs them in. He is testing this now and will report back. 

      2. Note: we still need to make sure that every time we run that we divide AraSim runs into MANY more jobs (ie further parallelize by dividing into, say, 100 instead of 10). 

DATA_LIKE_OUTPUT=1,2 doesn't work with DETECTOR=0,1,2
DATA_LIKE_OUTPUT controls data-like output into UsefulAtriStationEvent format; without a real station selected (using DETECTOR==3,4), the mapping to the data-like output will not function correctly
There are 1 errors from settings. Check error messages.
SysError in <TFile::Flush>: error flushing file outputs/AraOut.setup.txt.run29.root (Stale file handle)

1. Explanation of issue (2): We keep timing out of the interactive job when AraSim is running (if the user isn't remaining active on their terminal).
   1. OSC suggested that we use a virtual desktop (via ondemand) instead of using an interactive job. We will try this. 
      • Note: We should now be requesting 2GPUs instead of 1GPU. UPDATE: Alex P. requested one, but it took too long and he had to leave before he got it. 

1. Make the following variables in the bash script. 

   1. UPDATE: Alex M. figured out how to pass the scale factor into the xmacros. We used the "tr" command in bash. He will be testing this and can update us at the Thursday meeting. 

In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

Grid spacing=0.1
Frequency= factor of n higher 
This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.

1. Turn tournament on in the GA. 

2. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized.

3. Comment code thoroughly

   1. UPDATE: Alex commented code that utilized the scale factor (Part_E.sh). More still needs to be done. 

Today's Tasks

Julie: I finished the Python assignment. I am making sure my solutions encompass all of the different ways to solve this, and will then start on the C++ project. 

Mitchell, Evelyn, Eliot, Ryan: Work through my bash project.

Leo: At 4 pm today Leo will begin his "physics of antennas" lecture series! 

Alex M: All scaling is corrected to be set at the bash script level. Alex is now going to work on commenting code!

Alex P: Still working on our AraSim edits!

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim errors fixed

   1. Explanation of issue (1): Usually 1 of ~100 AraSim jobs never completes. OSC shows "completed", but the flag is never written and the output file is not filled in entirely. The error file shows the following.

      1. UPDATE: Alex still trying to figure out how to input Car's suggestion. He also needs to add this for the flag files that are being saved. Here is his most recent question:

Alright so I have pbsdcp working with generic files and I now have it working in the AraSim call but my current worry is that it wasn't creating a file unless I had

./AraSim setup.txt $runNum outputs/ a_${num}.txt > $AraSimDir/outputs/a_${num}.txt

So I just did this to have some a_1.txt, before we had the last line writing directly to our antennaPerformance metric (this makes the AraOut file typically I just kept it as this name for testing purporses), now if I get rid of the portion after the > or the part after outputs/ it doesn't seem to make a file that we need. I think I don't understand to get the full output of AraSim without using this. Because I don't understand how using this is and then using pbsdcp is any more efficient.

Now maybe if I had the portion right after the > go directly into TMPDIR instead of using pbsdcp to both scatter to and gather from TMPDIR that would be better

1. Make the following variables in the bash script. 

   1. UPDATE: Alex M. finished editing the scaling factor to be at the bash level for every piece of code that uses it. 

2. Turn tournament on in the GA. 

3. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized.

4. Comment code thoroughly

   1. UPDATE: Alex commented code that utilized the scale factor (Part_E.sh). He is now going to begin commenting the other code to clean it up. We want it to be read as cleanly as our GA. 

Attendance: Julie, Mitchell, Ryan, Evelyn, Eliot, Alex M.

Today's Tasks

Julie: Worked on C++ project, and helping others with the Python project; I also made additions, clarifications, and edits to the Bash project (also updated the solutions and pushed them to gitHub). I still need to do the following:

• Add the instructions from the google doc to the GitHub pages for both the Python and the Bash projects.
• Work on the C++ project!
• Run a test evolution this weekend to make sure AraSim stuff is fixed and that XF job submissions work. 
• EVOLVE

Mitchell: Started Python Project.

Evelyn, Eliot, Ryan: Work through my bash project. 

Leo: Worked on lectures. 

Alex M: Helped make XF job scripts. Needs to do the following:

• Parallelize the AraActualBicone properly. 
• EVOLVE!

Alex P: Needs to do the following:

• Do a test run to see if the scaling (when we also scale the grid spacing) actually affects the run time for XF.

TO-DO List

1. Scaling test run: Do a test run to see if the scaling (when we also scale the grid spacing) actually affects the run time for XF. (ALEX P)
2. Evolve: Right now, we are using an interactive job with only a CPU running at half scale, and have just edited our software so that we submit jobs for the XF simulations requesting 1GPU (could be 2GPU if we choose to edit it so). 
   1. WHY WE ARE DOING THIS:
      1. Remember that we only need an interactive job when XF initializes and reads in the DNA for each individual. The GUI closes once the simulations start and we no longer need an interactive job. 
      2. We were requesting a 1GPU interactive job and just running XF from the command line; however, OSC has been SUPER busy lately and we haven't been getting them (especially since we need long wall times if we are running the full simulations over many generations within the interactive job). However, we realized that if we submit an interactive job at a CPU (so that XF can allow the GUI to pop up), and then run the simulations by requesting 1-2GPUs we can then request a much smaller wall time and it will be quicker for the simulations to run --- ie we would be requesting a wall time of about 10 minutes using 1-2GPUs per simulation for each job submission versus 5+ hour wall time of an interactive job using 1-2GPUs.
      3. WE HAVE NOT TESTED OUR .SH JOB SUBMISSION SCRIPTS! JULIE WILL DO IT THIS WEEKEND!

3. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized. (ALEX M)

4. Comment code thoroughly (Everyone needs to help with this)

5. Get new proposal results for March 12th deadline.

Attendance: Julie, Evelyn, Mitchell, Ryan, Alex M.

Since we have been having issues with the lack of availability of GPUs on OSC, we have come up with a new to-do list:

1. Try to keep running on Pitzer with 1 core instead of 40. Based on first attempts, it is saying this will reduce wait times to ~3 hours (will vary based on usage), and after that 3 hours, human interaction will be needed. So the plan is, someone requests a job in the morning, and when the interactive job starts, the person is responsible for doing the thing which will allow the generation to run. We will aim for one generation per day under this plan, taking turns. Better than no evolving.
2. I have asked Heechang at OSC if we can install the latest version of XF on Ruby and/or Owens. That would be better. There seems to be more available on Owens at least.
3. Our initial attempt to run on Ruby is giving us an error which is probably to do with different inputs required in different versions. Will attempt to pursue this as a last resort, but the older version of XF does require lots more human interaction so until other options are exhausted it's not worth it.

Today's Tasks

Mitchell: Worked on Python Project, learned to evolve (note: Mitchell may leave the group at the end of the semester)

Ryan: Starting the Python project

Evelyn: Finish bash project.

Julie: C++ project, look at Amy's proposal draft. 

Alex M:  Fix XF solver to submit each individual as a separate job (thus we can request a smaller wall time for each job)

TO-DO List

1. Scaling test run: Do a test run to see if the scaling (when we also scale the grid spacing) actually affects the run time for XF. 
   1. We are going to pause on working on this, because we are struggling to get GPU time. We will test this whenever we see OSC less busy (ie if we log on at a lucky time). 
2. Evolve: Try to keep running on Pitzer with 1 core instead of 40. Based on first attempts, it is saying this will reduce wait times to ~3 hours (will vary based on usage), and after that 3 hours, human interaction will be needed. So the plan is, someone requests a job in the morning, and when the interactive job starts, the person is responsible for doing the thing which will allow the generation to run. We will aim for one generation per day under this plan, taking turns. Better than no evolving.

3. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being parallelized. (ALEX M)

   2. As of 3/5, I have not gotten an update on this. I will check on the status of this. 

4. Comment code thoroughly (Everyone needs to help with this)

   1. ​Julie made some progress on this over the weekend. 

      1. Made comments on what each bash script version does, and cleaned up our directories from old code so that it doesn't confuse the others when they start helping 

   2. WAY more needs to still be done. 

5. Get new proposal results for March 12th deadline:

   1. ​Julie is looking at what Amy wrote here 

I just wanted to give an update on what I've been doing today for GENETIS. I went ahead and recorded myself running the loop. It should be useful for new people we're onboarding but I also went through a whole generation, which I think is interesting for everyone to understand the timescale of each part of the loop. I'm making a small edit (I have no clue how to edit videos haha) so once I'm done I'll try to send it via email and post it on slack. I also started a long run Amy requested using antennas which are asymmetric in length and opening angle and 50 individuals per generation. Since it has so many individuals it will take a long time to go through each generation. I also had to decrease the number of AraSim jobs (thus increasing the number of neutrinos per job) because otherwise we'd be submitting too many jobs and have a lot of them blocked, holding us back from running. If anyone has problems with me using up too many jobs let me know and I can try to decrease the numer of jobs per individual futher (right now I have 10 jobs per individual with 3000 neutrinos per job, which gives us 500 AraSim jobs per generation).

Attendance: Julie Rolla, Alex M. 

Today Alex and I met to discuss how to move forward given the COVID-19 restrictions. We have resent out a message polling people's schedules to create online working sessions. Here is what we are planning:

Once we collect times for each group, we will have separate meetings for all of these entities. This way it's easier for us to meet without all having different items to discuss/work on. All of these will be zoom meetings moving forward. Meeting info will be announced shortly!

Tasks for "the Loop Group":

1. Make wall time a variable for XF sim.
2. Test current software edits.
   1. Such as item #1 above
   2. Make sure the bottleneck function is working.
      1. Right now it seems to be working; however, we keep hitting a wall time error. If we can make the wall time a variable then we can be sure the bottleneck function is working, and the wall time is truly the error. 
3. Get Ruby working on XF
   1. when I type:
      /usr/local/xfdtd/7.8.1.4/remcom/bin/xfdtd        I get:

      Info: Using latest available version (7.8.1.4)
      Info: Running /usr/local/xfdtd/7.8.1.4/remcom/XFdtd_7.8.1.4/bin/xfdtd 
      Error: Unable to find executable to run.  This command must exist

4. Get Alex's account of Ruby working. 
   1. He received an email and is still unable to log in. He is touching base with them again. 

Tasks for "the Training Group":

1. Julie and Alex need to finish the C++ project! 

The current working draft of the GENETIS paper can be seen here: https://www.overleaf.com/6783528497tvqbjphsgvzn

Additionally, here's our outline for the paper. The section assignments are as follows:

I. Intro (Julie)

II. Types of Algorithms (Suren)

III. XFdtd (Cade)

IV. AraSim -- will be called something else... (Max)

V. Fitness function (Suren)

VI. Results (Julie)

VII. Conclusion (We will see..)

Note that the order of the sections may change.