Attached is a plot containing bar graphs with error bars representing the average number of generations it took for the GA to achieve a chi-squared value of 0.25 (roughly equated to a 0.8 out of a max 1.0 fitness score). Unlike the fitness scores used by the GA, these values do not have simulated error attached to them and are therefore a better measure of how well the GA is optimizing. These results were obtained by running 10 tests in the test loop for each design, population, and error combination and solving for the average number of generations to meet our threshold and the standard deviation of those scores. From this, a few things immediately pop out, I will address the more obvious one later. But essential for this test, we can see that increasing our population size seems to have a more direct impact on the number of generations needed to reach our threshold than decreased error does in both designs. My best guess regarding this is that the GA depends on diversity in its population in order to produce efficient growth, and an increase in the number of individuals contributes to this, allowing the GA to explore more options.

This leads me to the more easily spotted trend; PUEO is much slower than ARA. This presents an anomaly as this is the opposite of what we would expect from this test loop as PUEO has fewer genes (7) to optimize than ARA (8). It is also important to note that no genes are being held constant in this test for either design, so both designs have the full range of designs provided they are within the constraints. With that in mind, my guess as to what causes this phenomenon is that PUEO's constraints are much stronger than ARA's. How this may affect the growth is that it more heavily bounds the possible solutions, which makes it harder for the GA to iterate on designs. It is possible that during a function like a crossover, the only possible combinations for a pair of children are identical to their parents, effectively performing reproduction. This could limit the genetic diversity in a population and therefore cause an increase of generations needed to reach an answer. We could in theory test this by relaxing the constraints done by PUEO and then running the test again to see how it compares.

Finally, You will notice that no AREA designs are shown. This is because, under current conditions, they never reached the threshold within 50 generations. However, Bryan and I think we know what is happening there. AREA has 28 genes, about four times the amount of genes that our other designs use. Given that our current test loop fitness measure is dependent on a chi-squared value given by: SUM | ((observed(gene) - expected(gene))^2) / expected(gene) |, we can see that given more genes, the harder it gets to approach zero. For example, we can imagine in an ARA design if each index of the sum equals 0.1, you would get a total value of 0.8, while AREA would get a value of 2.8, which seems considerably worse, despite each index being the same. Upon further thinking about it, Bryan and I do not think that a chi-squared is the best measure of fit we could be using in this context. Another thing we thought about is that we have negative expected values in some cases. We have skirted around this by using absolute values, but upon reflection believe this to be an indicator of a poor choice in metric. Chi-Squared calculations seem to be a better fit for positive, independent, and normally distributed values, rather than our discrete values provided by our GA. With this in mind, we propose changing to a Normalized Euclidean Distance metric to calculate our fitness scores moving forward. This is given by the calculation: d = sqrt((1/max_genes) * SUM (observed(gene) - expected(gene))^2). This accomplishes a few things. First, it keeps the same 0 -> infinity bounds that our current measure has, allowing our 1/(1+d) fitness score to be bounded between 0 and 1. Second, it forces all indices to be positive so we don't need to worry about negative values in the calculation. Third, this function is weighted by the number of genes present for any given design, making them easier to compare than our current measure. Finally, as our GA is technically performing a search in an N-dimensional space for a location that provides a maximized fitness score, it makes sense to provide it the distance a solution is from that location as a measure of fit in our test loop. We created a branch on the test loop repository to test this and the results are promising as results from the three designs are much more comparable for the most part (though we still see some slowdown we think is contributed to constraints as mentioned above). Though some further input would be appreciated before we begin doing tests like the one we have done in the plot below.

How To Update PueoSim For GENETIS:

First, whoever updates pueoSim needs access to pueoBuilder, pueoSim, and niceMC on GitHub (ask Will for permissions).

Once you do, go into the peuoSim directory at /fs/ess/PAS1960/buildingPueoSim/

and source set_env.sh

`source set_env.sh`

Then, we want to make copies of the files we are currently modifying for the GENETIS loop.

For PueoSim v1.1.0 these are:

1. nicemc/src/EventGenerator.cc  # Modified to create custom output root files to calculate errors

2. pueosim/test/simulatePueo.cc # Modified to stop the simulation of the LF antenna which is not needed for GENETIS

3. pueosim/src/Seavey.cc # Modified to read in custom gain files

4. pueosim/src/pueo.cc # Modified to read in custom gain files

5. pueosim/include/Seavey.hh # Modifies to read in custom gain files

Currently, I store copies of these in the `/fs/ess/PAS1960/buildingPueoSim/backups` directory in case somebody accidentally overwrites the files in pueoBuilder. 

Once you’ve made the copies, you can run `./pueoBuilder.sh` from the `/fs/ess/PAS1960/buildingPueoSim/pueoBuiler` directory. This will rebuild pueoSim and niceMC, pulling the latest updates from GitHub. 

You may need to delete the pueoSim and niceMC directories in order for the builder to pull the latest version from GitHub. Or, if it’s being really stubborn, move the whole pueoBuilder directory to a temporary location and run the builder from scratch with 

`git clone git@github.com:PUEOCollaboration/pueoBuilder` and then `./pueoBuilder.sh`

Then, you will need to reference the copies of the changed files to make changes to the new version of pueoSim. Hope this doesn’t cause too much of a headache, and when you’re done, return to the /fs/ess/PAS1960/buildingPueoSim/pueoBuiler directory. 

Then you simply type

`make install`

then

`make`

And now, pueoSim should be ready to run!

EventGeneratior.cc Changes and Rationale:

PueoSim’s default output ROOT files are very large and therefore time-consuming to parse through to get the information we need to calculate effective volume and errors. So, we want to create a custom ROOT file with only the variables we want, greatly increasing the speed of the analysis.

To do this, we want to create a new TFile with corresponding TTree and TBranches that will store the loop.positionWeight, loop.directionWeight, and neutrino.path.weight. Then, we want to fill the Tree when a detector is triggered and write the results to the file at the end of the loop.

Sample code for initializing the TFile:

simulatePueo.cc Changes and Rationale:

By default, pueoSim v1.1.0 runs a simulation for the normal antenna and a low-frequency (LF) antenna. As GENETIS is evolving for the main antenna, we are not interested in using computation time to simulate the LF antenna. So, we comment out the lines that initialize the LF detector in this file.

Seavey.cc, Seavey.hh, and pueo.cc Changes and Rationale:

As of pueoSim v1.1.0, the simulation software only has the ability to read in one antenna. This is a problem, as we want it to be able to read in files for any antenna we make. So, we need to change these scripts to be able to read in whatever gain files we want.

The convention the loop uses is to input gain files in the format of 

vv_0_Toyon{runNum}

hh_0_Toyon{runNum}

…

In the ./pueoBuilder/components/pueosim/data/antennas/simulated directory

So, the main change is getting the runNum variable into Seavey.cc file where the gains are read. Currently, we define a global variable at the stary of pueo.cc and pass it into where Seavey is called. This means we have to add runNum as a parameter in Seavey.cc as well as Seavey.hh. Finally, we change the name of the read-in files to be in the above format AFTER dividing runNum by 1000 (this is because pueoSim uses the run number as a random number generating seed, so we divide runNum by 1000 to be able to read in the same gain patterns for multiple seeds of the same individual).

Note: We used to change pueoSim to output a veff.csv file. We now handle this calculation by analyzing ROOT files, so this change is no longer necessary. 

The Physics of Results Plots have been added to the Pueo Loop. The current version of the plotter is built for pueoSim v1.0 and located in ${WorkingDir}/Antenna_Performance_Metric (Hasn't been pulled into the loop directory yet).

The pueoSim v1.0 IceFinal files were missing information on the RF direction and information needed to see an amplitude spectrum. I asked Will, and he said that the new version of pueoSim (v1.1.0) outputs the needed information.

I created an updated version of the plotter, and have a pull request here: https://github.com/osu-particle-astrophysics/GENETIS_PUEO/pull/40

However, before this can be implemented in the loop, we need a way to get errors from pueoSim v1.1.0.

Currently, there is a version of rootAnalysis.py here that can analyze the new root files and output fitness scores and errors, but instead of taking 20 minutes for a generation, it now takes 20 minutes per individual to run.

This is because the update splits the IceFinal file into IceFinal_allTree, IceFinal_passTree0, and IceFinal_passTree1. IceFinal_allTree and IceFinal_passTree1 are of comparable size to the previous IceFinal file, but the passTree0 final is

about 10-20 times larger, causing a slowdown in calculations. If we calculate without this file, it takes about 1 minute per individual, still a bit slower than before.

Ideas to solve this issue:

Don't use the passTree0 files (I have asked Will what they're for, and if we need them. Hopefully he responds after the Holiday.)

Instead of running through a Python for loop, call a C++ script that creates a CSV file the Python program can easily load in.

Submit a batch job to run the analysis in parallel before combining the outputs in the correct format. (We should maybe do this anyways?)

Continue using pueoSim v1.0 (I think we can still retain the Theta graphs for the incoming neutrinos, but there won't be any RF graphs or the possibility of amplitude spectrum plots) 

Attached is a preview of plots the v1.1 plotter is capable of making.

Add weights.

Here are some preliminary results from testing the effect of error on growth in the GA. For this test, we start with a simulated error of 0.5 because our true fitness score is bounded between 0.0 and 1.0. From here, we simulate doubling the number of neutrinos by reducing the error by root(2), then root(4), and observed the growth on the fitness scores plots. We did these three tests with both 50 and 100 individuals (plots starting with 30 use 50 and starting with 60 use 100). Looking at the plots, we can see that the only plot that shows growth is 100 individuals with halved error (corresponding to 4 times the number of neutrinos). In fact, it took going to root(25) (corresponding to 25 times the number of neutrinos) to observe comparable growth with 50 individuals. This leads us to believe we likely need to increase both neutrinos and individuals to see meaningful growth in our GA. Further tests are required to see how consistent these results are. Another test, though unrealistic, we ran 1000 individuals with the root(25), this test was optimized in around 10 generations, this could be seen as a maximum performance standard.

Here is a list of things we agreed to do before starting a new PUEO run.  Feel free to add notes here as things are done and we will revisit the list next week.

Change range of the height variable to 0.75-1.75m

Start with all bad individuals so that we can be sure and see growth - all at 0.75m?

Make sure we are using the ratios in the GA that we think we are

If we are using reproduction, are we combining results when individuals are resimulated to reduce errors?  I made a program to accomplish this and created a pull request on the PUEO Github (Dylan)

Investigate whether there are obvious places to speed up PUEOSim

Consolidate the GA to one branch

Understand what is happening with cross-pol, make sure we're not gettting Veff=0 when using XF-generated cross-pol response

Edit: Have the xmacros use the realized gain instead of the idealized gain, which allows for the design to build in matching

Here is some backlogged information as well as recent updates to our progress on AREA and its optimizations:

4/13/2023

We concluded our initial test of the AREA optimization loop. While analyzing our results, we noticed that most of our runs never reached our benchmarks for performance (chi-squared scores of .25 and .1) and that those that did contain abnormally high amounts of mutation. From previous runs using the GA for the GENETIS loop, we find that mutation and immigration (Note: AREA does not use immigration) usually play a smaller role in overall growth and are simply there to promote diversity. Crossover on the other hand is what handles the bulk of the growth done by the GA. Given our results ran counter to that, we decided to look further into the matter.

Upon looking at the fitness score files, we could see that mutation-created individuals did not perform well in the best run types, but the crossover ones did contain the bulk of good scores. This lines up with expectations. However, closer inspection revealed that these crossover-derived individuals had extremely similar fitness scores, which could indicate elitism in selection methods leading to a lack of diversity in solutions. We then looked at a run that used only crossover with one selection type (roulette), and found every individual to have identical (and poor) scores that were the same up to around the 4th decimal place. This is a strong case that the selection methods used are too elitist to grow a population properly. 

We dove into the AREA GA to take a look at the functions doing the selection methods, and discovered several issues potentially causing excess elitism in the GA. It seems that the roulette selection does not behave the way we have used in our other GAs. There appears to be no weighting of individuals; rather, a threshold fitness score is collected and only individuals with a fitness score above the threshold are selected. This causes elitism, as only the most fit individuals are likely to pass this threshold. Tournament was found to be similarly troubled, with a bracket size of ⅓ of the population. This is again very elitist, as it is very large, making it very likely that only the best individuals will be selected. Before we try to further optimize the AREA GA, these selection methods should be addressed and fixed.

For roulette, we propose implementing a similar weighting scheme for individuals as is used in the PAEA GAs that gives preference to the most fit individuals, but still allows others to be selected to encourage diversity of solutions.

For crossover, we propose decreasing the bracket size. This would allow a more diverse range of individuals to be selected through “winning” smaller brackets that would have a lower probability of including top performing individuals.

4/20/23

We (Bryan and Ryan) met to work on improving the selection method functions in the AREA algorithm. 

For roulette selection, we added weighting by fitness score, as is done in the other GENETIS GAs, in an effort to prevent it from disproportionately selecting the most fit individuals only to become parents. As a first test, we ran an abbreviated version of the test-loop code, printing the fitness scores of the parents selected by the roulette method. The fix appears to have passed this initial test, as the fitness scores showed more variety and in the case of roulette crossover two unique parents seemed to be selected.

For tournament selection, the only change necessary appeared to be decreasing the bracket size of the tournament(s), which in turn encourages other individuals besides the most fit to be selected as parents, therefore increasing diversity of solutions. We ran out of time to test this change fully, so we will pick up here in our next meeting.

Looking further ahead, we will plan to work on refactoring the AREA algorithm to use the same functions as the other GENETIS GAs wherever possible. In the meantime, this working version can still be used for feature development so that progress is not halted.

5/11/2023

Upon looking at the results of the recent optimizations, our best runs are still taking over 40 generations to reach an optimized score. Given the speed of AraSim, this is still too slow to be considered optimal. Comparing this test loop run to the ones run on by the broader GENETIS GA, our best runs for the AREA GA are worse than the worst runs from those optimizations. As such, we have decided to pause optimizing the AREA GA in favor of adding it to the broader GENETIS one. As of today, we have finished a rough version of the generating function necessary to create new individuals. We have also started progress on the constraining functions and scaling functions that will be necessary to generate the spherical harmonics used to find the gain and phase. Once these functions are complete, we should be able to transplant this GA back into our test loop for AREA and resume optimizations.
 

In this post, I'm going to give step by step instructions on everything you need to do to run the loop. See Julie's thesis for the most recent iteration of this manual. Also refer to entry 123 for a list of some errors we have encountered historically in the loop and how to resolve them, though some fixes may be outdated.

Running the loop

Getting started

Assuming you have followed the instructions from the onboarding tutorials, you should be ready to access OSC through the interactive website. Instead of using ssh to access OSC, the loop requires that you go to this website: https://ondemand.osc.edu/ . Login with your usual OSC credentials. Once there, you will need to request a Lightweight Desktop by clicking on "Interactive Apps" at the top of the page. See the screenshots attached for how to do this. Old documentation may refer to the Lightweight Desktop as a VDI.

You can request up to 24 hours for a Lightweight Desktop, and it should begin almost immediately. After you request it, the website will take you directly to the waiting page. You can navigate to this page yourself by clicking "My Interactive Sessions" at the top of the ondemand page. When the job begins, you will see a blue buttong that says "Launch Lightweight Desktop". Click it and you should have a new tab open that brings you to a remote desktop environment on OSC that you interact with through your browser. 

New runs

If you're starting a new run for the first time (that is, beginning from generation 0), you will need to edit the script that runs the loop. Here is the path:  /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Scripts/Asym_XF_Loop.sh . At the top of the script is a list of variables that can be changed. See the first screenshot attached here.
For a new run, you must change the variable <RunName>, which is the first variable in the list. It should be in quotes and the convention is to start the name off with the date of the run, using YYYY/MM/DD format--this will always keep everything in chronological order when listing directories with <ls>. Other variables are explained in the comments next to them in Asym_XF_Loop.sh and you should change them depending on the type of run you are conducting. 

To begin the loop, type in the command ./Loop_Scripts/Asym_XF_Loop.sh from the Evolutionary_Loop directory.

When the loop begins, you will be asked to press any key. After you hit a key, the loop will run Part A, which executes the GA using the <start> mode. It will then automatically move on to Part B, which creates the antennas in XFdtd. XF will ask you if you want to specify a location for the XF file to be saved. Click <No>. The loop will automatically save the XF directory into the run directory when it finishes with the first part of part B. In principle (that is, barring errors that stall the loop or interruptions due to the VDI job ending), you will never need to manually enter anything to the loop for it to run beyond this point. 

However, if it is your first time using XF, you will need to set the max GHz setting upon startup to 2 instead of the default 10. 

Continuing Runs

After starting a new run, the loop will run automatically. When your Lightweight Desktop job ends, you'll need to get another one repeat the steps above. You will not be prompted for any input. 

Stepping Back

At times, you may want to continue the loop from an earlier point or interrupt the loop and step back. Usually you'll only want to do this within a generation, but it's possible that you will encounter an error or bug at the end of a generation and wish to return back to that point to remedy it before continuing forward in the next generation. This is a very tricky and delicate process that requires a lot of attention to detail. In the future, we'd like to make an option for the loop to do this automatically when given input at the beginning, but implementing that will take a long time and will be very sophisticated.

The state of the loop is recorded in the savestate file located here: /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/saveStates . The file will be named <Run_Name>.savestate.txt . In it, you will see three numbers. The first one represents the current generation of the loop. The second one represents the state of the loop, that is, which part in the generation it is currently on. The third one represents the individual it is processing. In practice, the third number is rarely relevant, and is almost always 1, though it can be used for Part B, where the loop is processing individuals sequentially, so if it was interrupted on individual eight (for example) you can change the number to be 8 so that it will pick up where it left off without needing to redo the first seven individuals.

If you want to step back to an earlier state in a generation, you'll need to interrupt the loop and revert the state by changing the second number in the savestate file. This is NOT all you will have to do, however. Because of the way files are edited and moved around during states, you will need to manually revert files to be in the format or location needed by the earlier state. 

For example, you may need to revert from state 2 (when XF is creating the antenna geometries) to state 1 (when the GA generates new individuals). In this case, you'll need to edit the savestate file to change the second number from a 2 to a 1. Then, you'll need to change the generationDNA.csv file in Generation_Data to use the values from the previous generation, since the GA will have overwritten that file and uses it to determine what individuals are to be selected from. To do this, you'll need to locate the <gen>_generationDNA.csv file from the previous generation in the <RunName> directory in Run_Outputs. This is should be found here:  /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/<RunName>/Generation_Data . 

The above is a relatively simple example of how to step back in the loop. Stepping back from a late state to an early state will be more complicated, though if you're willing to sacrifice run time you can pretty easily follow the above to just restart a generation. Stepping back to a previous generation will be explained in a future update to this post.

Here are the run details for the latest PUEO run we've started. There are still some bugs that need to be worked out, which are detailed on the github. Here is the directory where the data is stored: /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/2023_05_08

This run uses 50 individuals per generation with 400k neutrinos per individual at an energy exponent of 19. Only the inner side length and outer sidelength are being evolved (which evolves the height since the opening angle is fixed). 

Issues With Part E (Dylan):

The Plots were not automatically created for the first generation. After a quick look through, I found that the fitness score and v effective csv files were being moved before they were copied, so the files didn’t end up in the correct place. Additionally, the errorbars.csv file was not being created, so I made a temporary fix to create a file of 0 error bars to allow for plotting the values while I work on getting the real error values. 

Update: I have modified Will’s script for reading the root file outputs, so it can now read them to output a CSV file with the effective volume along with its positive and negative errors. This script does the same and more as the fitnessfunctionPUEO.py, so I will replace it in the loop when there is downtime. Additionally, this script means we don’t need to output a veff.csv file from pueoSim. I will now work on creating a less “janky” version of my pueoSim change for reading in the gain files, so I can get Will to officially add it to pueoSim and we can pull any updates in the future.

Update, Implementing the new script into Part E:

The rootAnalysis.py script is very picky with environment variables. So, first I had to load and then unload a version of python to make sure the default version is loaded. Then, I moved the sourcing of set_plotting_env.sh to later in the script to avoid having an alternate version of root sourced while running (this would cause errors). Then I source the environment for pueosim (path hardcoded, change when loop isn’t running) and loop through the rootAnalysis.py program for each individual in the generation. This will output a ${gen}_fitnessScores.csv and a ${gen}_errorBars.csv in the run directory for plotting script to use. Because it outputs these automatically, I commented out the old methods we used to create these files and removed the temporary error bar fix as well.

Another Update:

We paused the loop. I changed the hardcoded parts, and we pushed the new updates to the github. I will make comments on the changes later today. 

Wednesday:

It seems like the loop is still not creating the csv files or plots for generation 1 or 2. Alex gave me the error message. The python script rootAnalysis.py had an issue with the include Geoid.h line. This was caused by sourcing the wrong things while trying to get the script working. I found that I only changed my local pueoSim environment to include the fftw installation to the $PATH and $LD_LIBRARY_PATH variables. So, I changed /fs/ess/PAS1960/buildingPueoSim/set_env.sh as well. Also, I found that the include statements in rootAnalysis.py were missing the PAS1960 portion of the path. After fixing these, and adding a missing ‘/’ to the output file path, I manually ran the rootAnalysis.py program for the 2023_05_08 run, mimicking the portion of the bash script that runs it in Part_E. It outputted the correct files in the correct format. I then ran the FScorePlot script. The median line for the ViolinPlot seemed wrong, and the FScorePlot was too scrunched with a y start of 0; I will need to change these.

Plotting Update (Dylan):

Ok, plots still aren’t being made, and I’m not running the loop, so I don’t have access to the error messages, so I’ll go through everything one by one. For Veff_Plots, this isn’t being made as there are currently no vEffectives.csv. This isn’t a big deal, as it’s just a copy of FScorePlot, but I added the creation of vEffectives.csv to rootAnalysis.py anyway. Next, I noticed that runData.csv, which is required for a couple of the plots was not being made. I looked, and the version of gensData.py had the old values for the skiprows when reading in the fitnessScores and generationDNA files, something that was overlooked when we merged the version I was developing with the version Alex was developing on. I can’t access the directory or the file, I think this is why it was overlooked in the merge. I created a temporary PUEO version in Antenna_Performance_Metric (I’m also fine with it just living here permanently). 

Also, the csv files are now in the GenerationData/ directory in the run directory? This affects every plotting script and they will not work without changes. I don’t know why this was changed or if this change is permanent?

Next, ok apparently, VariablePlots isn’t even a file in the loop? We really need to use GitHub better. 

I moved my local version of the program into the loop and it ran fine manually.

I am forsaking the Generation_Data directory, so I changed the rainbowPlotter and dataConverter scripts to read in a location to match all the other plotting scripts. They both also ran fine manually. 

Note: The rainbowPlotter script should be able to create a gradient from the minimum fitness score to the maximum fitness score. (currently, the range is hardcoded and this can cause basically a mono-color graph.)

FScorePlot should be good

The bash script was inputting 5 variables into colorplots when the pueo version only takes in 4 (this is also something that was already fixed before the merge.) It also requires the vEffective.csv files. After copying over the fitnessScore.csv files to vEffective.csv files, it ran fine. (The vEffective.csv files will be made automatically by rootAnalysis.py in the future)

I've started a run of the PUEO loop to get some data for evolving the antenna width and height. We are running with 24 individuals, each with 400k neutrinos at an energy exponent of 19. Here is the output directory: /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/2023_05_01_Test_5

It's very important to note that the cross-pol data being used still comes from the data already stored in pueosim. This is because our extremely low cross polarizations were causing us to have no effective volumes. There's likely some issue in what values from XF we are choosing as our crosspols.

We have the PUEO loop in working order, though there are a few errors in accuracy and efficiency. We started a preliminary run with few individuals and neutrrinos as a large scale test that may still give us some useful data. 

For this run, we are only evolving the inner side length (10cm to 25 cm) and height (10 cm to 50 cm). The walls are slanted at 45 degrees, so the outer length is completely defined by those two. The ridges are kept at 6 cm wide and 4 cm from the walls at the base. The curvature is set to 0,1 and the ridges are set to have the same height as the walls. 

At the moment, we are using 24 individuals and 300,000 neutrinos. I'll update this ELOG post with more details from the run as they emerge.

The reviewer asked us to run the loop again but for just symmetric antenna designs. We have completed that run and have results ready. We still need to resimulate the top five individuals, which I'll update this post with later. I attached the violin plot (though it should be fixed to not let the legend overlap the data). See this previous entry for the details of the run: http://radiorm.physics.ohio-state.edu/elog/GENETIS/198

Plots we want to have for PUEO:

FScorePlot2D

Fitness_Scores_RG

VariablePlot (PUEO equivalent of LRT plot)

Veff_Plot (currently the veff is equivalent to the fitness score for PUEO)

Veffectives_RG

Rainbow_Plot 

Violin Plot

avg_freq

gain_vs_freq

polar_plotter

physics of results

Next steps:

Get the Error values out of the pueoSim root output (talking to Will)

Use output uan files from the new two-run XF simulation to test the frequency plots. (/fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/2023_04_21_Test_6)

Implement the physics of results plots into the loop (Talk to Bryan / Dennis)

Get the average Veff/Fitness score from the default gain pattern in pueoSim to compare our results to.

After some work, I've gotten the physics of results plotting script provided by Dennis working to compare the ARASim outputs from an AREA run to a set of reference data for the ARA bicone (provided by Alex). Among the outputs is a histogram of of events triggered vs cos(theta_nu) (attached here). This is the plot that was previously presented in GENETIS talks in the physics of results discussion.

Here I compared the best individual from the AREA results (Blue- "source_2") that I have on hand (only the ARASim output files from the most recent generation ran are currently saved) with the ARA bicone reference data (Red- "source_1"). Both datasets included a total of 150,000 NNU. Note that the AREA run used was a single frequency test, meaning that the radiation pattern was held constant for all frequencies in the band.

Also attached for reference are the gain pattern of the associated individual from the AREA output and the violin plot of the run producing the data.

Work in progress. Update on H-pol XF design. The first plot is the gain pattern with ferrite rod sets (blue) vs. with nothing but the copper plates. 
The second plot is the gain pattern with everything but the ferrite rod sets vs. with nothing but the copper plates.
The ferrite rod sets function to narrow the gain pattern.

At the main GENETIS meeting yesterday, we collected the last few things that need to be worked out in order to get the PUEO loop in working order. Here's a list

• Skeletonize the xmacros (Alex)  04/10/23 In progress

  • This involves substituting some of the text to be variables that are replaced by the loop for each generation. 

  • Requires some slight modification of Part B1 in the loop.

• We need to modify PUEOsim to read in different gain files (Dylan) Changes made 04/10/23  Testing to be done.

  • Currently, PUEOsim reads in files like vv_0, vh_el, etc. It needs to be modified to take in the gainfiles as arguments and read different ones for each individual.

  • Requiers slight modification of the batch job scripts, but these are relatively small

• Slight modification on the fitness score read in script (Dylan)  Changes made 04/10/23  Testing to be done.

  • Now that PUEOsim is working and has been modified to output txt files with the effective volume, we just need to slightly amend the the fitness score script to read from those files.

• Fix Ezio's comments on XFintoPUEO.py and modify to read in outputs from two files (Jacob) 04/10/23 in progress, further work this week

  • Since we are running XF twice (once for each power source), we need the conversion script to read in from two files and print out to 8 files (vv_0, vh_el, etc)

This is an entry for the usual Wednesday working meeting GENETIS holds. We are largely carrying on from last week's hackathon hammering out the remaining pieces that need to be fixed. See the previous ELOG post for a list of those.

Alex

Amy pointed out on the Monday call that the proper way to simulate the ANITA antenas is to use one source at a time. I modified the script I had so that it makes two simulations per antenna. The first one will use the power source connected to the "VPol" ridges and the second one uses the one connected to the "HPol" ridges. After checking this, it looks like the asymmetry I had pointed out before fell away. In other words, I think XF only wanted to use a single power source anyway. So now we should be able to produce the VV, HH, VH, and HV data that icemc and PUEOsim need.

Ryan and I also worked a bit on the slopes of the ridges. We experimented with the parameter that affects the curvature, which seems to be able to be too large. But many of the designs still look to have almost no slope and many still caved inward rather than sloping outward. We're still working on figuring out what causes that, but I think it means we need another constraint on distance of the top of the inner ridges from the center. 

We also tried figuring out why the sides of the curved part of the ridges don't stay in the same plane as the straight part of the ridges. I'm confident that they do stay in a single plane (I attached a Mathematica notebook that shows as much by finding that the torsion of the curve is zero; for some reason I couldn't print it to a PDF). We think that we should be able to rotate the curve into the same plane as the straight part of the ridge, but I haven't figured out how yet.

 Nick and I continued working on the HPol antenna. We want to check if there are any other things in Steph's example XF file that are significant to the results of the simulation (versus just being there as a representation of the model). It looks like there are, and Nick is working to figure out what they are by deleting them (in a copy) and generating simulation results for them to compare to the original. Otherwise, the most significant parts (the copper plates) are now being made in a script and just need to be connected to one another.

Dylan

I tried to build root from source again and ran into an error 3 hours into the build. I messaged Will and asked if he knew anything about the error. I then reran the build with Will's suggested change and ran into the same error 3 hours later. I did, however, manage to build pueo by sourcing the installation of root in Will's shared pueo directory. I'm now testing getting a txt output file from the simulation as it turns out my initial solution didn't work.

Update:

I got a working version of root installed. Also, I was able to modify pueoSim to output a veff.txt file. Now, I need to modify where it reads in the gain files, and then modify the icemc specific bash scripts to work for pueosim.

This entry will serve to summarize what tasks we've completed and still need to complete after last week's GENETIS hackathon.

Alex

My focus was on the XF scripting for making the PUEO antenna. Before last week, we had a script which made the geometry of a PUEO-like antenna. The xmacro script has been modified in the following ways:

• Now includes power sources connecting the ridges that are opposite each other. 
• Generalized to use variables without hardcoding.
• Reads in data from the output files of the PUEO GA.
• Loops over all read in antennas to set up all of the simulations at once.

At this point, the script for setting up the simulations of the PUEO loop is just about ready for use. The only change to be made is to the frequency list. The script for printing out the gain data is also ready, though it only prints out the gain in theta and phi, not the cross polarization.

Here are some outstanding points that need to be hammered out, though they impede the quality and accuracy of a run rather than the actual functioning of the loop.

• The shape of the ridges in the generated antennas from the GA are considerably different from the PUEO antenna.
  • This is likely something that can be fixed by adjusting the constraints--the subspace of the parameter space that leads to PUEO-like ridges may be very small as-is.
  • You can see an example of an antenna from the GA attached (viewed from slightly below). It looks dramatically different. This seems to be due to constraints on the height and width being too small, but the ridges are also clearly misshapen. 
• The curve on the ridges looks like it needs some work.
  • For very PUEO-like antennas, the slopes look reasonable, but they do flare in or out slightly. This is more dramatic for ridges that deviate from PUEO's significantly.
  • My best guess is that the parametric form of the curves is placing the curves in a plane that is different from the rest of the ridge. This should be fixable by somehow constraining the curve to a given plane.
• The simulation still only outputs polarization data for theta and phi, not the cross polarization.
  • It looks like XF naturally gives us the theta and phi polarization. It's not clear how to get the cross polarization gain (which we think means the gain when the signal is at 45 degrees).
  • One idea is to redo the simulation but with the antenna rotated 45 degrees in the azimuth. But I'm not certain that that iwll work.
  • Amy indicated that the correct way to do this is by simulating the antenna with just one power source at a time.
    • Leaving the VPol source would produce the VV and VH data and leaving the HPol would produce the HH and HV data (HV and VH could be flipped!).
• Currently, the antennas have a fixed opening angle. We'd like to add this as a parameter, though at the moment it might be ok to forego that.  
• We need to make sure that the power sources aren't too close together (fixed!).

To do for this week:

1. Modify the script to make two simulations for each antenna, each with only one power source.

2. Adjust the constriants on the parameters to fix the curvature.

3. When we begin a run, we will want to just evolve the height and the inner side length (these determine both the inner side length and the outer side length).

Dylan and Jacob

Dylan and Jacob focused on getting PUEOsim/icemc working, which will be the simulation software needed for the PUEO loop. Here's a summary from Dylan:

• Plotting and bash scripts should be mostly good to go.
• Currently have 4/5 of the PUEO prerequisites for pueoBuilder, and the GENETIS-specific code is ready to implement and test when it's built. I’m trying to build the correct root version one more time before today’s meeting, and if it fails again I’m going to ask Will.
• The main thing left is to get the correct version of root installed, which while annoying, shouldn't take too long

To do for this week:
1. Once PUEOsim is working, run it with the usual settings to get the default performance. It will need to be submitted as a job array (ask Alex for help with this if you need; ask Will for advice on how many neutrinos to run).

Today was day two of the GENETIS hackathon. See the previous entry for details on day one. 

Alex

I continued working on the XF scripting for the PUEO horn antenna. Amy showed me an actual horn antenna from ANITA in Lisa's office so I could get the dimensions and see how the loads (power sources in XF) were offset. Offsetting the power sources led to fixing the asyemmtry problem from yesterday. I tested offsetting the loads by various heights and found that they all give the same gain patterns except for when they overlap, proving that they were interfering with each other when they crossed.

There is still, however, an asymmetry between the x and y axes (but the gain patterns are now symmetric across either of them). This is more reasonable, though I'm not positive that it should be this way. I attached an image of the gain pattern to show this asymmetry. Note that it is still at 83 MHz, which is well outside the PUEO bandwidth, but higher frequency gain patterns show the same thing (but they have more nodes so they're harder to look at). I attached images from the x-axis, y-axis, and above of the gain pattern. One way to check this will be to compare to the data stored in PUEOsim/icemc and see if they demonstrate similar asymmetries.

My next step is to test the constraints we've decided on using the script. I'll try out a few different values of the parameters and then if everything is working I'll use the GA Ryan and Dylan wrote to generate many individuals and have XF make the models for them. We don't necessarily need to see the simulation outputs from those XF runs, we just need to see if we run into error. Beyond that, we need to figure out how to extract the proper gain data from XF to create the same format that PUEOsim and icemc use. 

Amy also mentioned that we should add the opening angle of the horn antenna as a gene. This may be a little tricky but should be doable. Once we are set with the current XF script and changing the data formatting to match what PUEOsim and icemc need, I'll work on adding that as a gene.

Dylan and Jacob

We tried to get IceMC running and able to read in gain files -  we failed. Something was wrong with my make files and Jacob’s c++ compiler, so neither of us got a working version of IceMC. We then switched gears to focus on pueoSim. I created a branch on the nicemc github to output a veff.txt in the format of IceMC, so we can reuse the already-made code. Jacob and I also have been trying to install pueoSim, but we’ve been running into many errors and issues from the “C++ Gods”. For the next steps, I will finish the veffective branch and create a pull request. Then, I will work on a branch to make it so pueoSim can read in a variable path  to the gain files.

Ryan

Tuesday: Created a new branch on the PEUO repository, version0.0,  and placed the repository in the same location as the Ara one for familiarity. I also added an alias to this directory onto the genstudents channel if people would like to add the path to their .bashrc's. I then went through the the Evolutionary_Loop directory and cleaned up any residual Ara based files that were not needed to run this loop. This will make navigating and troubleshooting easier as we finish building. This branch was then commited and pushed to the github and submitted a pull request.

Today was the first day of the GENETIS Hackathon. We are spending several days this week specifically tackling the PUEO project to get it into full form (or close to it). Here is a progress update after day 1.

Alex

I am working on the xmacro script for the ANITA antenna. Before today, we had a working xmacro script to make the geometry of the ANITA horn antenna. Now I am trying to modify it to make it 1) more true to the actual ANITA antenna (in terms of which parts are electrically connected) and 2) add in the power source and sensors so that we can get simulation results. I've attached the xmacro script here along with a picture of the geometry and some gain patterns. There are four ridges in the antenna. They are all connected by the outer walls of the antenna. However, only the ones directly across from each other are connected by a load (in this case, a power source, since we simulate as a radiator). So we have two power sources, one connected each "ridge bicone". 

In making this, I have generated some gain patterns. I want to test a few cases to see if the results coming from XF seem reasonable, though I don't know what I should expect the gain pattern to look like when I've done things correctly. Here are the options I'll compare:

1. Just use one "ridge bicone".
2. Use two "ridge bicones" with the walls but just one power source.
3. Use two "ridge bicones" without the walls and two power sources.
4. Use the full form, with two "ridge bicones", walls, and two power sources.

Case (4) is the one I expect to be correct. However, when I simulate that, I get a slightly asymmetric gain pattern compared to what I expect (I expect symmetry across the "ridge bicones"). You can see from the attached image that there is a strange preference in one corner. My guess is that this has to do with which ridges are set to be ground and which are set to 1V (the potential). Feel free to ask me for more details about these figures/scripts on slack and I'll fill them in on here. I'll clean up the script and add in full test cases later this week.

The image of the antenna I attached is a bird's eye view. The green lines represent the power sources I've connected between the ridges. They do appear to overlap, but my understanding is that they are not actually affecting each other (they should just be a representation of the power sources). 

The image of a gain pattern (which is 3D and also a bird's eye view) is for the case of the antenna with both power sources and the walls remaining (the antenna is still shown too, to demonstrate the asymmetry of the gain pattern). This is well outside of the frequency range ANITA cares about (83 MHz), and I haven't looked closely at the higher frequencies yet, but I still am surprised by the asymmetry. I'll update tomorrow with more gain patterns to compare.

Dylan and Jacob

Dylan and Jacob are working on the bash scripts for the PUEO loop. They are taking the parts of the bicone loop that are specific to ARA/bicones and changing them to be specific to the PUEO loop. 

All the bash scripting outside of implementing the new software we’re developing should now be complete (we just need to debug when we do test runs). The new GA is implemented into the PUEO loop. We tried and failed to build a version of pueoSim on our own. Will has provided us with a precompiled version and instructions for installing it on our users as well as access to the necessary pueoBuilder repo. Jacob and Dylan are scheduled to go through the installation with him on Thursday at 4pm in his office, M2024.

We have also found that peuoSim currently only outputs root files. So, we will need to create a conversion script or modify pueoSim for when we implement it into the loop. Currently, using IceMC looks to be the easiest way to get a test run going. For the next steps, we have a plan for making IceMC read in a variable file path name to avoid recompiles and issues with parallelization (Alex has given us the path to look at how ARA does it). Then, we will work on installing pueoSim and testing its current viability.

Bryan

Upon checking the existing plotting scripts used for previous GENETIS work, it appears that they can be used for PUEO with minimal changes. In order to test the plotting scripts, sample output data from the PUEO loop is needed, or at least test data that mimics the form of the output for the PUEO loop. After more progress is made with installing an MC package and the PUEO loop can be run to produce test outputs, the plotting scripts can be tested and implemented in the bash script.

Will sent me some info on how to install PUEOsim. I attached a markdown file he sent for how to install PUEOsim. He also sent the path to pre-compiled installation of PUEOsim. Here's the path to the pre-compiled installation: /users/PAS0654/wluszczak/PUEO_shared

Here is his full message, which details that you must also run set_env.sh before running PUEOsim:

a pre-compiled version is located at /users/PAS0654/wluszczak/PUEO_shared . Inside that directory, there's a set_env.sh script that you will need to source before running anything (hopefully this works, though there's a high probability you might run into permissions issues with the dependencies).

To run PUEOsim, use the following (see here for instructions: https://github.com/PUEOCollaboration/pueoSim):

./simulatePueo -i {inputFile} -o {outputDirectory} -r {runNumber} -n {numberOfNeutrinos} -e {energyExponent}

The outputs will be root files, but it should be possible (either modifying our own version directly or with a script) to print out the effective volume to a .txt to read.