This run followed the first symmetric run. It had an asymmetric length, but the opening angle and inner radius were kept symmetric. 

10 individuals were evolved over 17 generations using 100,000 neutrinos. All selection was done through roulette, and all individuals were formed through crossover and the old mutation method (where individual genes had a 40% to be mutated, and mutated genes were chosen from a gaussian based on the mean and standard deviation of that gene's value in the generation).

This is considered to be the first run of "good" data. Prior to this run, we ran with few neutrinos and had errors which needed to be resolved in AraSim and with the way we handled XFdtd's simulation results being passed to AraSim. Those were resolved in the summer before this run.

This was a symmetric run of 10 individuals over 15 generations using 100,000 neutrinos per individual. This preceded the substantial modifications made by Ryan to the GA. All individuals were formed through crossover, but there was a probability of 40% for genes to be mutated. In this case, mutation was a complete change to the gene, though it was based on the average value of that gene in the generation. There was no reproduction or mutation. All individuals were selected through roulette.

Green individuals indicate that there was no penalty applied, while red individuals had a penalty factor multiplied by their effective volumes. The penalty was of the form exp(-(R-7.5)^2), where R is the outer radius, in cm, of the bicone.

This is the run that is discussed in the GENETIS paper submitted to Physical Review D in December of 2021. It was conducted beginning on March 19, 2021. The preprint can be found here. It ran for 31 generations with 50 individuals per generation. Each individual was run through AraSim for 300,000 neutrinos.

The ratio of generative operators was: 72% crossover, 22% immigration, 6% reproduction. 

The ratio of selection operators was: 80% roulette, 20% tournament, 0% rank.

The highest scoring individual had a fitness score of 5.24. However, when it was rerun with many more neutrinos (3*10^7), it had a score of 4.90, an 11% improvement over the ARA bicone.

This run was conducted before the introduction of the automatic run_details.txt generator. Instead, there is a file called Run_Notes.txt (attached) which details some of the errors that were encountered. These errors were remedied during runs by removing the offending AraOut file (which had too high of a fitness score due to a weight being calculated to be greater than 1, which shouldn't be possible) and have since been prevented by modifying the data file AraSim uses to calculate the weights.

We began a new run of the curved sided GA. Previously, we conducted a run just like this in July of 2022 (titled 2022_07_15_Latest_Greatest). In that run, we made substantial updates to the loop, including using the latest version of AraSim. However, we discovered after the run (which had a very flat fitness score growth curve) that we had not modified AraSim to take in gainfiles appropriately (so it was just evaluating the ARA bicone repeatedly). We've fixed this issue and a few more (see the github repository for the issues that exist and have been resolved) and are redoing this run. The run details will look much the same as that one and are visible in the attached file. Here is the location of the directory where the data for this run will be stored: /fs/ess/PAS1960/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/2022_12_29

For a list of changes made in this run compared to previous ones, see this ELOG post on the previous run: http://radiorm.physics.ohio-state.edu/elog/GENETIS/186

We have started a new run. Here is the directory: /fs/ess/PAS1960/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/2022_07_15_Latest_Greatest

This run is a substantial overhaul. It is a curved run using the latest parameters from Ryan's optimization loop. Here is a list of important changes:

1. The newest version of AraSim has been implemented.
2. The minimum length has been decreased to 10 cm per side.
3. The GA is recording the parents and operators used to generate individuals.
4. The number of individuals has been increased to 100 per generation.
5. The number of neutrinos per individual has been increased to 600k.

This run may take longer than previous runs due to the increased number of individuals. There may need to be modifications made to resolve or work around AraSim errors that delay the loop (due to the auto-resubmit function). Find the details of the run attached.

This run was a short run (stopped due to resource limits) that was conducted to test the minimum length constraint that had been applied in the past. Previously, the minimum length was set to be 37.5 cm for each cone; in this run, it was lowered to 10 cm. It ran for 60 generations with 50 individuals per generation, each evaluated with 300,000 neutrinos. To compare to the paper run, this run was performed using the asymmetric algorithm, meaning each individual had 6 genes (length, inner radius, and opening angle for each cone). The initial generation began with identical individuals, with genes matching the genes of the best performing individual mentioned in the GENETIS paper.

The ratio of generative operators is as follows: 72% crossover, 22% immigration, 6% reproduction. Mutation was also used, with 1% of genes produced by crossover mutated by adding a number chosen from a Gaussian centered at 0 with a width of 5% of the gene's value.

The ratio of selection operators was: 20% roulette, 80% tournament, 0% rank, 0% elite.

Attached are the run details, violin plot, and "rainbow" plot.

This run was started on February 2, 2022. It is a full run of 50 generations with 50 individuals per generation using the quadratic version of the loop. This means that each individual is defined by 8 genes (length, inner radius, linear coefficient, and quadratic coefficient for each cone). Attached is the txt file run_details.txt that is automatically generated when the loop is run. Each individual was run for 300,000 neutrinos.

This run used the usual ratio of generative operators: 72% crossover, 22% immigration, and 6% reproduction. It also used the proper mutation function: 1% of genes created through crossover were mutated by adding a number chosen from a Gaussian distribution centered at 0 with a width of 5% of the gene's value (MUTATION WAS NOT USED BUT WAS AVAILABLE).

This run was the first full run in which the rank selection operator was used. The ratio of selection operators was: 0% roulette, 10% tournament, 90% rank, 0% elite. 

This run used the script fitness_check.py to average the scores of identical individuals that appeared across multiple generations. This gives us a more accurate measure of those individuals' scores. It may also explain why this run appears so flat (on the violin plot): fluctuations in the scores should die down as repeated individuals have more accurate scores, and newly generated individuals that perform highly regress to their mean (actual) score when they are reproduced/recreated through crossover. This has led us to question our GA parameters, as Audrey and Autumn demonstrated that roughly 1/4 individuals in the run are repeat individuals.

Here are the changes that have been made in this (and the preceding) run relative to what we have been doing before.

1. The newest version of AraSim has been implemented.
   1. This was done because AraSim has gone through significant changes since 2019 that make it more accurate and makes it run faster.
2. The minimum length has been decreased to 10 cm per side.
   1. This was done because we previously tested shorter lengths for a response to the reviewer when we submitting the paper to Phys Rev D. Previously, we thought that going below 37.5 cm would lead to unreliable results, but we are reassured that that is not the case now.
3. The GA is recording the parents and operators used to generate individuals.
   1. This should allow us to look back and make a "history" of the evolution by seeing where genes come from in each generation.
4. The number of individuals has been increased to 100 per generation.
   1. Ryan demonstrated that a higher number of individuals does lead to a substantial improvement in the speed at which the evolution converges. 
      1. We previously thought that it did not make a big difference. However, that was done by measuring the average maximum fitness score. If we measure how many generations it takes for the evolution to reach a benchmark score, we find that more individuals helps significantly.
5. The number of neutrinos per individual has been increased to 600k.
   1. This was done to make the fitness scores more precise. At 300k neutrinos, we expect an error of around 0.2. This will cut things down closer to 0.1, which may help us decrease the number of outlier measurements without increasing run time too much because of the AraSim speed up.

This run may take longer than previous runs due to the increased number of individuals. There may need to be modifications made to resolve or work around AraSim errors that delay the loop (due to the auto-resubmit function). Find the details of the run attached.

The following is a run summary of an AREA test using 100 individuals per generation, without the linear frequency dependence on the gain, with the frequency fixed (i.e. the same gain pattern is produced for all frequencies). This run was a preliminary test of the AREA algorithm with a larger NPop (100 individuals per generation) after seeing promising results in the previous test that only used 20 individuals per generation. The percentages of selection methods/operators used attempted to mimic the optimal percentages used for the PAEA Bicone Loop, but because the algorithms do not set these in the same manner, these are highly unlikely to be the best percentages to use. The results show a very quick plateau in fitness score with a loss in diversity of solutions after a low number of generations, potentially meaning that a poor breakdown of selection methods/operators was chosen. This result seems to underscore the need for a test loop-style optimization for AREA to determine the best breakdown of selection methods and operators to use for AREA.

Run details:

• Run Type
  • AREA
• Run Date
  • January 18, 2023
• Run Name
  • 20230118breynoldsrun_noLinearDependAREATest_38RCO_2RM_58TCO_2TM_150000NNU_2Seeds
• Why are we doing this run?
  • This test increased the number of individuals to 100 and attempted to mimic the percentages of selection methods and operators found to be optimal for the PAEA loop.
• What is different about this run from the last?
  • The previous test showed slow/minimal evolution after ~12 generations, and it was speculated that this was due to a small NPop of 20 individuals. This test uses an increased NPop of 100 individuals.
• Symmetric, asymmetric, linear, nonlinear (what order):
  • N/A
• Number of individuals (NPOP):
  • 100
• Number of neutrinos thrown in AraSim (NNT):
  • 300,000 Total (150,000 NNU x 2 Seeds)
• Operatiors used (% of each):
  • 96% Cross-Over, 4% Mutation
• Selection methods used (% of each):
  • 40% Roullette, 60% Tournament
• Are we using the database?
  • N/A

Results:

• Summary and comments on results
  • The fitness score quickly plaeaued and diversity of solutions was lost, potentially due to a unsuccessful choice of selection method and opterator percentages.
  • An important future step is to work with Ryan to create a test loop optimization for the AREA algorithm, as it is written differently in how it is passed selection method and operator percentages and requires its own optimization to find the optimal ratios of these. (Work on creating a test loop optimization for AREA is currently underway).

EDIT 4/10/23
Re-uploading gain pattern associated with most fit individual from this run

The following is a run summary of the AREA test without the linear frequency dependence on the gain, with the frequency fixed (i.e. the same gain pattern is produced for all frequencies). This run was a preliminary test of the AREA algorithm after fixing long-standing errors in the algorithm and finding that the initial constaints implemented did not work with the frequency dependence on gain. Results from this run did show a "proof-of-concept" that the AREA algorithm seems to evolve single frequencies in a promising way, however a low number of individuals may have stalled the evolution.

Run details:

• Run Type
  • AREA
• Run Date
  • December 26, 2022
• Run Name
  • 20221226breynoldsrun_noLinearDependAREATest_8RCO_8RM_2TCO_2TM_30000NNU_10Seeds
• Why are we doing this run?
  • We tested the AREA algorithm set to evolve at a single frequency in order to see if it would evolve a gain pattern at that frequency within the required constraints.
• What is different about this run from the last?
  • This run only evolves gain patterns at a single frequency, allowing us to investigate the evolved gain patterns and check that the required constraints on the power conservation are met, without yet fixing the constraints function to work with the linear frequency dependence relationship. This is also the longest test run since the AREA algorithm was fixed and gotten into working order.
• Symmetric, asymmetric, linear, nonlinear (what order):
  • N/A
• Number of individuals (NPOP):
  • 20
• Number of neutrinos thrown in AraSim (NNT):
  • 300,000 Total (30,000 NNU x 10 Seeds)
• Operatiors used (% of each):
  • 50% Cross-Over, 50% Mutation
• Selection methods used (% of each):
  • 80% Roullette, 20% Tournament
• Are we using the database?
  • N/A

Results:

• Summary and comments on results
  • The evolution appeared to be successful in that the most fit gain pattern found (attached) seems to make sense with our understanding of the underlying physics. Additionally, after inspecting the outputs, conservation of power was confirmed to be properly constrained. However, in the attached violin plot, the fitness score plateaued quickly and never evolved further, with the best individual coming in generation 12. This was speculated to be due to the small population used in this test (only 20 total individuals per generation).

The reviewer asked us to run the loop again but for just symmetric antenna designs. We have completed that run and have results ready. We still need to resimulate the top five individuals, which I'll update this post with later. I attached the violin plot (though it should be fixed to not let the legend overlap the data). See this previous entry for the details of the run: http://radiorm.physics.ohio-state.edu/elog/GENETIS/198

Requesting an interactive job:

qsub -I -X -l walltime=1:00:00 -l nodes=1:ppn=40

(ppn=24 makes it quicker FYI)

Note that walltime is the length of time you'll need it. You will likely need it for 4+ hours if you are running the loop... Probably longer. Note that before you enter this command into your command line, you must be logged into OSC.

For submitting a job with GPUs (which we will be doing to speed up XF now as of 1/24/20), use this command:

qsub -I -X -l walltime=1:00:00 -l nodes=1:ppn=40:gpus=1:default      OUTDATED

srun -A PAS0654 -t 1:00:00 -N 1 -n 40      CURRENT

I fixed a bug in the loop, so we started another rank test run. Run details in the attached file.

The bug was searching for the generationDNA.csv file in the wrong place, meaning that it wasn't able to copy it to the run directory. That meant we didn't have a record of the generation data in the usual format. I don't think that this explains the flatness, since the the generationDNA.csv file was still created every generation correctly, so the GA knew where it was. But this test now corrects that problem and tests the usage of the rank selection method.

The Physics of Results Plots have been added to the Pueo Loop. The current version of the plotter is built for pueoSim v1.0 and located in ${WorkingDir}/Antenna_Performance_Metric (Hasn't been pulled into the loop directory yet).

The pueoSim v1.0 IceFinal files were missing information on the RF direction and information needed to see an amplitude spectrum. I asked Will, and he said that the new version of pueoSim (v1.1.0) outputs the needed information.

I created an updated version of the plotter, and have a pull request here: https://github.com/osu-particle-astrophysics/GENETIS_PUEO/pull/40

However, before this can be implemented in the loop, we need a way to get errors from pueoSim v1.1.0.

Currently, there is a version of rootAnalysis.py here that can analyze the new root files and output fitness scores and errors, but instead of taking 20 minutes for a generation, it now takes 20 minutes per individual to run.

This is because the update splits the IceFinal file into IceFinal_allTree, IceFinal_passTree0, and IceFinal_passTree1. IceFinal_allTree and IceFinal_passTree1 are of comparable size to the previous IceFinal file, but the passTree0 final is

about 10-20 times larger, causing a slowdown in calculations. If we calculate without this file, it takes about 1 minute per individual, still a bit slower than before.

Ideas to solve this issue:

Don't use the passTree0 files (I have asked Will what they're for, and if we need them. Hopefully he responds after the Holiday.)

Instead of running through a Python for loop, call a C++ script that creates a CSV file the Python program can easily load in.

Submit a batch job to run the analysis in parallel before combining the outputs in the correct format. (We should maybe do this anyways?)

Continue using pueoSim v1.0 (I think we can still retain the Theta graphs for the incoming neutrinos, but there won't be any RF graphs or the possibility of amplitude spectrum plots) 

Attached is a preview of plots the v1.1 plotter is capable of making.

Add weights.

Here are a list of things we wish XF would do. We are keeping this list in hopes to add it to a proposal with them. 

1. Save pictures of CAD drawings of simulations. 
2. Close/quit XF GUI. App.quit doesn't actually work for us. 
3. Suppress the GUI and run solely through terminal. 

We have the PUEO loop in working order, though there are a few errors in accuracy and efficiency. We started a preliminary run with few individuals and neutrrinos as a large scale test that may still give us some useful data. 

For this run, we are only evolving the inner side length (10cm to 25 cm) and height (10 cm to 50 cm). The walls are slanted at 45 degrees, so the outer length is completely defined by those two. The ridges are kept at 6 cm wide and 4 cm from the walls at the base. The curvature is set to 0,1 and the ridges are set to have the same height as the walls. 

At the moment, we are using 24 individuals and 300,000 neutrinos. I'll update this ELOG post with more details from the run as they emerge.

https://docs.google.com/spreadsheets/d/1vvcmjByKfcns0-tbAjtePB8ZVGsXAKxXCfbMc99weMI/edit?usp=sharing Here is the spreadsheet link for the population test. 

Here are plots I made for the meeting on 9/3/21. These plots represent a comparison of the gain and realized gain for the 23rd generation of the run being discussed in the upcoming paper. Here is a list of the plots

• Gain vs realized gain
  • Polar plots of the best individual from generation 23 
• S11/VSWR plots
  • Shows the S11/VSWR over the bandwidth for the best individual

This ELOG post contains plots I made this week for comparing the antennas as they were evolved in the run being discussed in the upcoming paper with those same antennas when using realized gain instead of gain. These plots are preliminary, in that they should be edited before being placed in a paper (for example, VSWR is not in dBi).

Plots:

• Gain vs Realized gain
  • Polar plot showing the gain and realized gain of the best individual from the run in the paper
• VSWR/S11
  • The VSWR of the best individual in the run over the frequency bandwidth
• Gain differences
  • The difference between the gain and the realized gain for the best individual over the frequency bandwidth

Attached is a pdf of physics plots generated for the 9_50, 13_84, 18_89, 19_96, and 29_87 crazy sides individuals along with another containing information on the amount of triggered events and effective volumes for each individual. The crazy sides individuals were simulated for 3 million events using the same Arasim version as is currently in the loop.