Bellow lies plots testing different scores and comparing them using a chi^2 score.

The functions used are as follows

Gaussian: e^(-2) (Red)

Inverse: 1/(1+(O-E)^2) (Purple)

Algebraic: 1/(1+(chi^2) )^(3/2)) (Green)

Chi: 1/(1+chi^2) (Blue)

Which are plotted here:

Here's a power point I showed today at the GENETIS meeting for the two most recent runs we did. The first run was symmetric and we ended it at 15 generations. The second one is ongoing and is shown up through generation 9 on the power point.

At the main GENETIS meeting yesterday, we collected the last few things that need to be worked out in order to get the PUEO loop in working order. Here's a list

• Skeletonize the xmacros (Alex)  04/10/23 In progress

  • This involves substituting some of the text to be variables that are replaced by the loop for each generation. 

  • Requires some slight modification of Part B1 in the loop.

• We need to modify PUEOsim to read in different gain files (Dylan) Changes made 04/10/23  Testing to be done.

  • Currently, PUEOsim reads in files like vv_0, vh_el, etc. It needs to be modified to take in the gainfiles as arguments and read different ones for each individual.

  • Requiers slight modification of the batch job scripts, but these are relatively small

• Slight modification on the fitness score read in script (Dylan)  Changes made 04/10/23  Testing to be done.

  • Now that PUEOsim is working and has been modified to output txt files with the effective volume, we just need to slightly amend the the fitness score script to read from those files.

• Fix Ezio's comments on XFintoPUEO.py and modify to read in outputs from two files (Jacob) 04/10/23 in progress, further work this week

  • Since we are running XF twice (once for each power source), we need the conversion script to read in from two files and print out to 8 files (vv_0, vh_el, etc)

We are off again with a new team. The new team consists of Alex Machtay, Alex Patton, Cade Sbrocco, Eliot Ferstl, Evelyn Shank, Mitchell Halley, Scott Janse. A spreadsheet of tasks and deadlines can be found here: https://docs.google.com/spreadsheets/d/1CkcdbediFbripdlwPw6UmhAHdAl5IdkL2efaseJkTTM/edit#gid=0

Updates:

We are now running on OSC. We have acquired a version of XF on OSC and have now transferred the loop to OSC. All of the xmacros for XF should be functioning, and we have managed to make the bash script auto import the xmacro scripts to XF when it boots. The loop exists on /fs/projects/PAS0654/BiconeEvolutionOCS. This is a shared copy so that everyone can run off of the same installation. This is important so that we can make sure everyone is pushing edits regularly, and so that everyone is always running with the most up-to-date edits. 

The loop had not yet been tested for it's current status. Alex M. is testing it and we are waiting to hear back. 

Tasks:

• Check continuity of loop
  • Update: Alex M is testing this
• Install Arasim to /fs/projects/PAS0654/BiconeEvolutionOCS to be edited and messed with
  • Update: this has been completed as of 11/5/19
• Test Alex M's code for editing Arasim's Setup.txt variables
• Clean bash script
  • Update: Mitchell is working on this.
• Make version of bash script which writes .job scripts for Arasim
  • This must write a different .job for each individual so it parallelizes Arasim running for each individual
  • The number of .jobs written must depend on $NPOP
    • ie this bash script should be writing .jobs from scratch each time as (1) the number of them we need depends on $NPOP, (2) the names of the input and output files depends on $NPOP. 
      • It should also delete .jobs at the end -- ie when all generations have completed so that we don't have a random number of .jobs floating around. For example, say we run with 10 individuals one time, and it writes 10 .jobs ($NPOP=10), and then the next run we only have $NPOP=5, we will have 5 extra .jobs floating around not being used unless we clean them all at the end line of the run in the bash script. 
  • Update: Work on this has begun by Alex M, Cade, and Evelyn (Alex P has also been messing with Arasim).
• Make XF be submitted as a job
  • Cade has ideas for this.

Students have been posting individual updates here: https://www.dropbox.com/home/GP_Antennas/Updates

Paperclip Antenna (Ryan and Evelyn working on it):

• https://github.com/gp-antennas/Sph._Harmonics_And_Paperclip

Asymmetric Bicone (Leo and Eliot working on it):

• https://github.com/JulieRolla/GENETISBicone

AREA (Ben working on it):

• Github? We need to ask Steph where their past work exists.
• See AREA thesis attached

Symmetric Bicone(Alex M. and Alex P. working on it):

• https://github.com/JulieRolla/GENETISBicone

Attached is a plot containing bar graphs with error bars representing the average number of generations it took for the GA to achieve a chi-squared value of 0.25 (roughly equated to a 0.8 out of a max 1.0 fitness score). Unlike the fitness scores used by the GA, these values do not have simulated error attached to them and are therefore a better measure of how well the GA is optimizing. These results were obtained by running 10 tests in the test loop for each design, population, and error combination and solving for the average number of generations to meet our threshold and the standard deviation of those scores. From this, a few things immediately pop out, I will address the more obvious one later. But essential for this test, we can see that increasing our population size seems to have a more direct impact on the number of generations needed to reach our threshold than decreased error does in both designs. My best guess regarding this is that the GA depends on diversity in its population in order to produce efficient growth, and an increase in the number of individuals contributes to this, allowing the GA to explore more options.

This leads me to the more easily spotted trend; PUEO is much slower than ARA. This presents an anomaly as this is the opposite of what we would expect from this test loop as PUEO has fewer genes (7) to optimize than ARA (8). It is also important to note that no genes are being held constant in this test for either design, so both designs have the full range of designs provided they are within the constraints. With that in mind, my guess as to what causes this phenomenon is that PUEO's constraints are much stronger than ARA's. How this may affect the growth is that it more heavily bounds the possible solutions, which makes it harder for the GA to iterate on designs. It is possible that during a function like a crossover, the only possible combinations for a pair of children are identical to their parents, effectively performing reproduction. This could limit the genetic diversity in a population and therefore cause an increase of generations needed to reach an answer. We could in theory test this by relaxing the constraints done by PUEO and then running the test again to see how it compares.

Finally, You will notice that no AREA designs are shown. This is because, under current conditions, they never reached the threshold within 50 generations. However, Bryan and I think we know what is happening there. AREA has 28 genes, about four times the amount of genes that our other designs use. Given that our current test loop fitness measure is dependent on a chi-squared value given by: SUM | ((observed(gene) - expected(gene))^2) / expected(gene) |, we can see that given more genes, the harder it gets to approach zero. For example, we can imagine in an ARA design if each index of the sum equals 0.1, you would get a total value of 0.8, while AREA would get a value of 2.8, which seems considerably worse, despite each index being the same. Upon further thinking about it, Bryan and I do not think that a chi-squared is the best measure of fit we could be using in this context. Another thing we thought about is that we have negative expected values in some cases. We have skirted around this by using absolute values, but upon reflection believe this to be an indicator of a poor choice in metric. Chi-Squared calculations seem to be a better fit for positive, independent, and normally distributed values, rather than our discrete values provided by our GA. With this in mind, we propose changing to a Normalized Euclidean Distance metric to calculate our fitness scores moving forward. This is given by the calculation: d = sqrt((1/max_genes) * SUM (observed(gene) - expected(gene))^2). This accomplishes a few things. First, it keeps the same 0 -> infinity bounds that our current measure has, allowing our 1/(1+d) fitness score to be bounded between 0 and 1. Second, it forces all indices to be positive so we don't need to worry about negative values in the calculation. Third, this function is weighted by the number of genes present for any given design, making them easier to compare than our current measure. Finally, as our GA is technically performing a search in an N-dimensional space for a location that provides a maximized fitness score, it makes sense to provide it the distance a solution is from that location as a measure of fit in our test loop. We created a branch on the test loop repository to test this and the results are promising as results from the three designs are much more comparable for the most part (though we still see some slowdown we think is contributed to constraints as mentioned above). Though some further input would be appreciated before we begin doing tests like the one we have done in the plot below.

Summary of results for 3million event simulations in AraSim with both GENETIS version and more recent {~11/2021) version.

Using errors for effective volume from the .out files.

Example shown below.

Radius: 3000 [m]
IceVolume : 8.4823e+10
test Veff(ice) : 6.50867e+09 m3sr, 6.50867 km3sr
test Veff(water eq.) : 5.96845e+09 m3sr, 5.96845 km3sr
And Veff(water eq.) error plus : 0.543588 and error minus : 0.543588

 

To act as our comparison to the evolved antennas while plotting, we have done a simulation of pueoSim with 4,000,000 neutrinos for the measured toyon gains found in /fs/ess/PAS1960/buildingPueoSim/pueoBuilder/components/pueoSim/data/antennas/measured

In order to run the jobs, I used the runJobs.sh script found in /fs/ess/PAS1960/buildingPueoSim which submits job runs of 40,000 neutrinos at a time, mirroring what we do in the actual loop.

The resulting data is now located in /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Toyon_Outputs

Veff: 11750.0503856

Plus Error: 89.0051458227

Minus Error:88.9007824246

The fitness score plots have been updated to read this value in as a comparison.

Machtay_20200831_Asym_Length_and_Angle    10 individuals
Machtay_20200911_Symmetric                             10 individuals, fewer neutrinos
Machtay_20200914_Asymmetric_50_Individuals  50 individuals, fewer neutrinos
Machtay_20200929_Asymmetric_test_2               50 individuals, fewer neutrinos, broaden parameter range

Dropbox link: https://www.dropbox.com/home/GP_Antennas/Updates/DailyFitnessScoreImages/Machtay_20200803_Master_Symmetric_Database_Real

Three things to change before doing the next symmetric run (w/o database):

1. Change how the mutation is done. We have at least a partial solution to this--the standard deviations were all 1 due to type errors in c++. We already have a fix for this.

2. The bicone separation distance was too small (0.05 cm). We should increase it to 2-3 cm to represent an actual buildable bicone.

3. We kept the random number seeds in. This isn't a bug, but it's something we need to remember to change before we run this again.