This entry will serve to summarize what tasks we've completed and still need to complete after last week's GENETIS hackathon.

Alex

My focus was on the XF scripting for making the PUEO antenna. Before last week, we had a script which made the geometry of a PUEO-like antenna. The xmacro script has been modified in the following ways:

• Now includes power sources connecting the ridges that are opposite each other. 
• Generalized to use variables without hardcoding.
• Reads in data from the output files of the PUEO GA.
• Loops over all read in antennas to set up all of the simulations at once.

At this point, the script for setting up the simulations of the PUEO loop is just about ready for use. The only change to be made is to the frequency list. The script for printing out the gain data is also ready, though it only prints out the gain in theta and phi, not the cross polarization.

Here are some outstanding points that need to be hammered out, though they impede the quality and accuracy of a run rather than the actual functioning of the loop.

• The shape of the ridges in the generated antennas from the GA are considerably different from the PUEO antenna.
  • This is likely something that can be fixed by adjusting the constraints--the subspace of the parameter space that leads to PUEO-like ridges may be very small as-is.
  • You can see an example of an antenna from the GA attached (viewed from slightly below). It looks dramatically different. This seems to be due to constraints on the height and width being too small, but the ridges are also clearly misshapen. 
• The curve on the ridges looks like it needs some work.
  • For very PUEO-like antennas, the slopes look reasonable, but they do flare in or out slightly. This is more dramatic for ridges that deviate from PUEO's significantly.
  • My best guess is that the parametric form of the curves is placing the curves in a plane that is different from the rest of the ridge. This should be fixable by somehow constraining the curve to a given plane.
• The simulation still only outputs polarization data for theta and phi, not the cross polarization.
  • It looks like XF naturally gives us the theta and phi polarization. It's not clear how to get the cross polarization gain (which we think means the gain when the signal is at 45 degrees).
  • One idea is to redo the simulation but with the antenna rotated 45 degrees in the azimuth. But I'm not certain that that iwll work.
  • Amy indicated that the correct way to do this is by simulating the antenna with just one power source at a time.
    • Leaving the VPol source would produce the VV and VH data and leaving the HPol would produce the HH and HV data (HV and VH could be flipped!).
• Currently, the antennas have a fixed opening angle. We'd like to add this as a parameter, though at the moment it might be ok to forego that.  
• We need to make sure that the power sources aren't too close together (fixed!).

To do for this week:

1. Modify the script to make two simulations for each antenna, each with only one power source.

2. Adjust the constriants on the parameters to fix the curvature.

3. When we begin a run, we will want to just evolve the height and the inner side length (these determine both the inner side length and the outer side length).

Dylan and Jacob

Dylan and Jacob focused on getting PUEOsim/icemc working, which will be the simulation software needed for the PUEO loop. Here's a summary from Dylan:

• Plotting and bash scripts should be mostly good to go.
• Currently have 4/5 of the PUEO prerequisites for pueoBuilder, and the GENETIS-specific code is ready to implement and test when it's built. I’m trying to build the correct root version one more time before today’s meeting, and if it fails again I’m going to ask Will.
• The main thing left is to get the correct version of root installed, which while annoying, shouldn't take too long

To do for this week:
1. Once PUEOsim is working, run it with the usual settings to get the default performance. It will need to be submitted as a job array (ask Alex for help with this if you need; ask Will for advice on how many neutrinos to run).

Input File Format for pueoSim (Also ICEMC)

Frequency Range: From 200 MHz to 1500 MHz incrementing by 10 MHz steps

There are 8 different files that are required for pueoSim. They are:

vv_0: Max Gain at each Frequency, Vertical Polarization

hh_0: Max Gain at each Frequency, Horizontal Polarization

vh_0: Max Gain at each Frequency, Vertical to Horizontal Polarization

hv_0: Max Gain at each Frequency, Horizontal to Vertical Polarization

vv_el: Gain at Theta Angles [5, 10, 20, 30, 45, 90] for each Frequency, Vertical Polarization

vv_az: Gain at Phi Angles [5, 10, 20, 30, 45, 90] for each Frequency, Vertical Polarization

hh_el: Gain at Theta Angles [5, 10, 20, 30, 45, 90] for each Frequency, Horizontal Polarization

hh_az: Gain at Phi Angles [5, 10, 20, 30, 45, 90] for each Frequency, Horizontal Polarization

Currently XF doesn't output all the information we need to create all these files. The only files able to be made from current XF outputs are vv_0, vv_el, and vv_az. Once we have the correct XF Outputs, it shouldn't be too much of a hassle to fix the current translation code.

Format for each of the files are the same. One column is Frequency (Hz) and the other is Gain. Attached are example files vv_0 and vv_el to visually see the format, though the frequency range is the current ARA range and not the final PUEO range as I am using ARA data to make these files. The files are named vv_0 (no .txt or .csv or any extension) and vv_el

Link to Google Doc with this information: https://docs.google.com/document/d/1iRUF6hIEyQfMK0LL21caRuHPgXYP30ZdkSkFv_-Y8R0/edit

How To Update PueoSim For GENETIS:

First, whoever updates pueoSim needs access to pueoBuilder, pueoSim, and niceMC on GitHub (ask Will for permissions).

Once you do, go into the peuoSim directory at /fs/ess/PAS1960/buildingPueoSim/

and source set_env.sh

`source set_env.sh`

Then, we want to make copies of the files we are currently modifying for the GENETIS loop.

For PueoSim v1.1.0 these are:

1. nicemc/src/EventGenerator.cc  # Modified to create custom output root files to calculate errors

2. pueosim/test/simulatePueo.cc # Modified to stop the simulation of the LF antenna which is not needed for GENETIS

3. pueosim/src/Seavey.cc # Modified to read in custom gain files

4. pueosim/src/pueo.cc # Modified to read in custom gain files

5. pueosim/include/Seavey.hh # Modifies to read in custom gain files

Currently, I store copies of these in the `/fs/ess/PAS1960/buildingPueoSim/backups` directory in case somebody accidentally overwrites the files in pueoBuilder. 

Once you’ve made the copies, you can run `./pueoBuilder.sh` from the `/fs/ess/PAS1960/buildingPueoSim/pueoBuiler` directory. This will rebuild pueoSim and niceMC, pulling the latest updates from GitHub. 

You may need to delete the pueoSim and niceMC directories in order for the builder to pull the latest version from GitHub. Or, if it’s being really stubborn, move the whole pueoBuilder directory to a temporary location and run the builder from scratch with 

`git clone git@github.com:PUEOCollaboration/pueoBuilder` and then `./pueoBuilder.sh`

Then, you will need to reference the copies of the changed files to make changes to the new version of pueoSim. Hope this doesn’t cause too much of a headache, and when you’re done, return to the /fs/ess/PAS1960/buildingPueoSim/pueoBuiler directory. 

Then you simply type

`make install`

then

`make`

And now, pueoSim should be ready to run!

EventGeneratior.cc Changes and Rationale:

PueoSim’s default output ROOT files are very large and therefore time-consuming to parse through to get the information we need to calculate effective volume and errors. So, we want to create a custom ROOT file with only the variables we want, greatly increasing the speed of the analysis.

To do this, we want to create a new TFile with corresponding TTree and TBranches that will store the loop.positionWeight, loop.directionWeight, and neutrino.path.weight. Then, we want to fill the Tree when a detector is triggered and write the results to the file at the end of the loop.

Sample code for initializing the TFile:

simulatePueo.cc Changes and Rationale:

By default, pueoSim v1.1.0 runs a simulation for the normal antenna and a low-frequency (LF) antenna. As GENETIS is evolving for the main antenna, we are not interested in using computation time to simulate the LF antenna. So, we comment out the lines that initialize the LF detector in this file.

Seavey.cc, Seavey.hh, and pueo.cc Changes and Rationale:

As of pueoSim v1.1.0, the simulation software only has the ability to read in one antenna. This is a problem, as we want it to be able to read in files for any antenna we make. So, we need to change these scripts to be able to read in whatever gain files we want.

The convention the loop uses is to input gain files in the format of 

vv_0_Toyon{runNum}

hh_0_Toyon{runNum}

…

In the ./pueoBuilder/components/pueosim/data/antennas/simulated directory

So, the main change is getting the runNum variable into Seavey.cc file where the gains are read. Currently, we define a global variable at the stary of pueo.cc and pass it into where Seavey is called. This means we have to add runNum as a parameter in Seavey.cc as well as Seavey.hh. Finally, we change the name of the read-in files to be in the above format AFTER dividing runNum by 1000 (this is because pueoSim uses the run number as a random number generating seed, so we divide runNum by 1000 to be able to read in the same gain patterns for multiple seeds of the same individual).

Note: We used to change pueoSim to output a veff.csv file. We now handle this calculation by analyzing ROOT files, so this change is no longer necessary. 

This is a multigenerational tracing of our second-best individual's parents and children:

The second-best individual in this evolution was Bicone 22 from generation 40. This individual is, in fact, a fascinating case as we shall see. But to start the story of this individual we will go back to generation 38 in order to demonstrate some of the peculiarities. 

In generation 38 there were two bicones of no renown,  Bicone 11 and Bicone 17. Bicone 11 was a fairly average individual that was ranked 23rd with a fitness score of 4.72016. Its DNA was {*6.16084, ***79.663, 0.0015434, -0.107765} for side one , and {0.308809, 39.6742, -0.0084247, 0.40629} for side two. One day, by chance it met Bicone 17, another average bicone ranked 24th with a score of 4.71877 and DNA {**2.32499, 79.663, -0.00224948, 0.192602} for side one, and {0.308809, 39.6742, -0.0084247, 0.40629} for side two. These two individuals eventually became the parents of two antennas: Bicone 16 and Bicone 17 of generation 39.

Bicone 16 eventually grew to have been ranked 3rd in fitness score of 4.95323. It’s DNA ended up being a complete balance of its parents sharing side one with Bicone 38.11{2.32499, 79.663, -0.00224948, 0.192602} and side two with bicone 38.17 {0.308809, 39.6742, -0.0084247, 0.40629}. Bicone 16 was an individual with high aspirations and hoped to be reproduced. But alas, it was not meant to be. But bicone 16 came upon some amazing luck, it was selected with itself for crossover. This meant that bicone 16 was able to provide two identical surrogates to survive into the next generation. This is where this Bicone fulfilled its full potential. 

The twin bicones were named Bicone 22 and Bicone 23 in generation 40. Being clones, they shared all their DNA with Bicone 39.16. However, due to some circumstances, they had slightly different fitness scores. Bicone 23 managed a very respectable 5.0117 fitness score and was ranked 2nd in the generation. But not to be outdone Bicone 22 managed to score a 5.17014 and ended up being the second-best performing individual of all time. Being so successful, the two bicones ended up producing 8 children between the two groups.

Bicone 23 was the first to crossover and had 2 children with its partner. These were Bicones 4 and 5. Bicone 4 was ranked 39th with a fitness score of 4.60251 and still shared the DNA of its second side with its grandparent Bicone 38.17 as well as most of its first side with Bicone 38.11 {2.32499, 79.663, -0.00213879, 0.192602} {0.308809, 39.9608, -0.0084247, 0.40629}. Bicone 5 on the other hand, was ranked 14th with a fitness score of 4.80535 and it still shared a lot of DNA with its grandparents {6.16084,53.0851,-0.00224948,0.0534469} {0.966617,39.6742,-0.0084247,0.40629}.

Bicone 22 had 6 children of its own with various other Bicones. These were; Bicone 8, ranked 11th with a fitness Score of 4.81784 and DNA {2.32499, 75.9855, -0.00224948, 0.192602} {0.966617, 39.6742, -0.00320023, 0.213833}; Bicone 9, ranked 7th with a fitness score of 4.88966 and DNA {6.10508, 79.663, -0.000594616, 0.0351901} {0.308809, 42.4246, -0.0084247, 0.40629}; Bicone 12, ranked 12th with a fitness score of 4.81705 and DNA {6.42695, 75.9855, 0.0015434, -0.107765} {0.308809, 39.6742, -0.00320023, 0.213833}; Bicone 13, ranked 2nd with a fitness score of 5.0344 and DNA {2.32499, 79.663, -0.00224948, 0.192602} {0.966617, 42.4246, -0.0084247, 0.40629}; Bicone 34 ranked 3rd with a fitness score of 4.99864 and DNA {0.66148, 73.5522, -0.000594616, 0.00582814} {0.966617, 39.6742, -0.0084247, 0.40629}; and finally Bicone 35, ranked 22nd with fitness score 4.74955 and DNA {2.32499, 79.663, -0.00224948, 0.192602} {0.308809, 42.4246, -0.0084247, 0.40629}

Bellow, I have attached the rainbow plot with the parameters occupied by individual 4 in gen 41 which was again ranked 39th in that generation. From this, we can see that while in its own generation it was a poor performer, overall it was upper middle of the pack. However, because of the density of other better performing antennas in this region, it is hard to distinguish which genes in this antenna are contributing the most to the drop in fitness score compared to its siblings and parents. 


 

*Gene originating from Bicone 38.11

**Gene originating from Bicone 38.17

***Gene originating from Bicone 38.11 and 38.17 that is shared between the two.

Today's Summer 2020 daily update:

As a note, today OSC was down so productivity was more limited

We are almost to where we can start the physical building of the antenna! 

I've attached all the information I currently have regarding the building project. Some of it is messy work notes and some is well-structured.

I’ve attached the following files for the GENETIS building project:

• Building Dump.txt 
  • My working notes that I used while trying to simulate the antenna in XFdtd (very messy) 
• Building Dump of Useful Materials.txt
  • List of materials that I found regarding the building project like slides, elogs, etc.
• Simulating Building Model.txt
  • A writeup I made describing my process for simulating the antenna in XFdtd 
• Done with change materials.zip
  • Solidworks model of antenna

I also made a slide deck that contains the directory locations + has graphs HERE.

Here are some scholarships available to physics majors I've been lucky enough to receive throughout undergrad that allowed me to work on this project unpaid without needing supplemental income from a separate job or loans. 

1. OSU Arts and Sciences Merit Scholarship Pooled Application 

• 500 word personal statement
• 1 letter of recommendation from a faculty member (Amy wrote mine)
• Varying amounts awarded (I got the $3,300 David and Velva Zarley Scholarship)
• Can be used for any education related expensed (tution, rent, food, ...)

2. James L. Smith Scholarship for Physics Majors

• 3 short essays (~200 words)
• Probably varying amounts (I got $4,000)
• Can be used for any education related expensed (tution, rent, food, ...)

3. Physics Class Awards 

• No application, chosen by department (just get good grades, honors might help. I didn't really talk to any of my professors or go to office hours, so milage might vary there)
• Freshman: $50, Sophomore: $250, Junior: $500, Senior: $?
• Can be used for any education related expensed (tution, rent, food, ...)

4. OSU Scholarship Universe

• A couple ~500 word essay
• Very few awarded, I've applied twice: $1,000 the first time and completely ghosted the second
• Can be used for any education related expensed (tution, rent, food, ...)

5. Licking County Foundation (If anyone is from licking county, I HIGHLY RECOMMEND APPLYNG TO THIS. I think there are similar ones for other counties) 

• ~500 word essay
• Varying awards (I've applied 3 times and gotten $2,375, $5,000, and $7,500) 
• Can be used for any education related expensed (tution, rent, food, ...)

Here is a link to a google drive with all my winning scholarship essays:

https://drive.google.com/drive/folders/1qJKpBf4by9wlReU5l_XjxjVsDIfXR4Jc

Copy this slide deck to use when presenting on run updates!

https://docs.google.com/presentation/d/1Tk-B1QbFTP_5pQZovfn0_wbTswZTmOrJURpi374xJWo/edit?usp=sharing

# AraSim CSE Spring 2024 Work

## Goals 
The main goal was to get a working multithreaded version of the AraSim codebase working. Doing this, the hope was to learn how to multithread the code and get it in a good place to hopefully also integrate GPU's at a later date.

## Where is it at currently? 
A bulk of the work done was to functionalize the Connect_Interaction_Detector_V2 to allow for multithreading and to cleanup codebase. 

### Going through code added/changed
Helper functions for multiple parts of code. They went through and split it up into multiple parts. Putting line numbers when needed

Part 1: Clearning Antenna Data
Part 2: Determine gain channel
Part 3: solve ray tracing (Not Done)
Part 4: Process Ray Tracing Solution (Not Done)
Part 5: Calculate Signal Factors (Not Done)
Part 6: Calculate Antenna Gain Factors (Not Done)
Part 7: Process Frequency Domain Signal
Part 8A: Process Neutrino Events. Lines 908 - 1208 (Not Done)
Part 8B: Process Arbitrary Events. Lines 1209 - 1475 
Part 8C: Process Simple Pulser Simulation. Lines 1478 - 1747 (Not Done)
Part 8D: Process PVA Pulser Simulation. Lines 1751 - 2110 (Not Done)
Part 8E: Process Calpulser Event. Lines 2113 - 2445
Part 9A: Process Noise. Lines 2593 - 2955
Part 9B: Process Trigger and Mimic Waveforms. Lines 2994 - 3624

## What still needs to be done? 
- Multithreading still isn't working, multiple threads are writing data to the same place causing the program to crash. This need to be resolved to at least have a working prototype. 

- I believe for multithreading we need to mark explicitly where file/data writing is happening to be able to adjust to make thread-safe

- Double checking that new functions are passing variables in the correct way. The CSE students had this has a slight fear. 

- Some helper functions are still not completed (3-6, 8A, 8C, 8D) 

- Current completed parts of code are all in separate branches and need to be merged after double checking that variable passing is correct

We are beginning a new run with several improvements to the ARA VPol loop to try to evolve antennas that are optimally matched just by their geometry. To do this, we are evolving with the RealizedGain instead of just Gain in the Xmacros (thus taking into account impedance mismatch). We also have the speed up that parallelizes the AraSim jobs on each run. Attached is the run_details.txt file, but the GA parameters are subject to change. Here is what they are to begin:

We have a population size of 50 individuals per generation.

Selection operators (NUMBER selected by each):

• Roulette: 10
• Tournamente: 10
• Rank: 30

Genetic Operators (NUMBER generated by each):

• Reproduction: 6
• Crossover: 36
• Immigration: 8
• Mutation: 2
  • Sigma: 5%

Attached is the license agreement that each person should sign to be able to use XF on OSC.  You can sign it, send it to Amy, and she will return it to OSC with her signature on it.

Slides shown at experts building meeting Aug. 2nd, 2023.

Previously, we have tried to straighten the sides of the best-evolved curved antenna in elog 229. However, there were potential issues with how closely this line resembled the curve of the antenna.

So, I attempted to create another straightened sides antenna using a linear regression model to find the best fitting line for the curve to create an antenna.

I made a Python notebook to separate the equations for each curve into 1000 discrete points. Then I ran a linear regression model to fit a curve of the points 1 - n, and a second curve of the n - 1000 points, looping from n = 2 to n = 999.

These results are from the combined output with the least squared error compared to the original.

Pictured is a plot showing the two sides of the curved bicone in red and blue, with the best fitting lines for each in black as well as a model in XF.

Results:

The antenna has a fitness score of 3.80627 with an error of 0.0759725.

This is much lower than the 5.71 of the curved antenna and 5.11 of the other attempt at straightening.

For the next attempt, we could consider constraining the endpoints to be the same as the original antenna to conserve the radius, and/or adding an extra line to fit the curve.

I've also attached a picture of what my notebook found for fitting 3 lines to the curve (not modeled or tested).

Professor Chen recommended using 3 sides and constraining the outer radii of the cones to match the original curved design.

Path to linear regression notebook: /users/PAS1960/dylanwells1629/developing/notebook.ipynb

Path to XF project: /users/PAS1960/dylanwells1629/straightened.xf

I've been building in XF and have ran into some issues but currently have the following completed:

- Struts are placed, though they are not evenly spaced and the same. I have contacted XF to try and figure out how to get rotation/patterns to do what we need

- Circuit components and circuit board are in the xf file, though the components are not on the circuit board (I'm not totally sure how to put them on, that's the next step). The circuit components should be of spec, I looked up the part numbers from the previous ELOG post and grabbed the values from the manufacturer.

Here are the things that have not been completed/need answered:

- How to evenly space the struts connecting the shape (as stated previously, I've contacted XF to figure out how to do this easily. Im just bad at modeling in XF)

- Add circuit components to circuit board

- Add voltage and ground connections to the circuit board

- On the circuit diagram, figure out J1 and J2 connections. Meaning, which one connects to the signal from the cable and which connects to the other cone that is used as ground?

To see current antenna build go to: " /users/PAS1977/jacobweiler/GENETIS/XFzone/straightened_antenna_for_Jack.xf "

Attached are pictures in case there are issues pulling up model in XF. :)

Standard Loop Architecture:

Complete an evolutionary step FOR EACH antenna before continuing on with the next step.

Steps:

1. Run the Genetic Algorithm for the entire population.

2. Run the XF radio simulation for the entire population.

3. Run the neutrino simulation software for the entire population.

4. Run root analysis and plots for the entire population.

However, due to constraints on the amount XF keys we have, we can only run 4-5 XF simulations at a time. 

So, when the first antennas finish their XF simulations, their outputs will simply sit there until EVERY other antenna finished their XF simulations.

New Proposed Architecture:

Complete necessary evolutionary steps as a population, but string together those that don't rely on data from other antennas.

Steps:

1. Run the Genetic Algorithm for the entire population

2. Run the XF radio simulation for each antenna

      - When an XF simulation finishes for an antenna, submit the pueoSim / root analysis jobs for that antenna

3. Run the plots for the entire population

This will allow us to complete most of our pueoSim computation while the XF portion of the loop is still running, cutting down the time between the final XF job finishing and the final pueoSim job finishing.

Test run of this new architecture with 100 antennas, 7,840,000 neutrinos per antenna.

After the final XF job finished, the pueoSim simulations and analysis of outputted root files were completed in 954 seconds,

Comparison Between the Two Architectures (both using job optimization from Elog 232)

Notes on Chart:

Source of data located in /users/PAS1960/dylanwells1629/buildingPueoSim/testingouts/times.txt and /fs/ess/PAS1960/HornEvolutionTestingOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/2023_07_24_test5/time.txt respectively)

Time from standard uses the time one of the 250 jobs running in parallel took in my testing of parallelizing processes inside of pueoSim jobs: 250 jobs * (40 * 40000 neutrinos per job) / 100 individuals = 4,000,000 neutrinos per individual)

The New Proposed time includes time spent analyzing the outputted root files to find fitness scores and errors, which would have taken around 100 seconds * population size for its number of neutrinos and files per individual (/fs/ess/PAS1960/HornEvolutionTestingOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/2023_07_23_test5/time.txt for data on this number)

So, this new architecture can provide more improvements for the amount and speed of neutrino simulation in the loop on top of the methods discussed in Elog 232.

This architecture could also be applied to see improvements in the Bicone and Hpol loops which are both affected by the limited number of XF keys.

Additional Notes:

1. For this new architecture test, each antenna uses 49 jobs for neutrino simulation instead of 2.5 previously. (49 pueoSim + 1 XF for 50 jobs per antenna, 5,000 jobs per generation)

2. The time for each antenna to submit, queue, and finish its neutrino simulation jobs must be less than the length of the XF job, or the extra time will accumulate for each antenna, losing much of the time benefits. (As long as it is less, the time spent on just pueoSim should be invariant under an increase in population)

3. The number of core hours spent on pueoSim jobs will be roughly the same for the same number of neutrinos as each job is shorter (except for a small contribution of the job overhead taking a higher percentage of total time for shorter jobs)

4. Initially I had thought that maybe queuing pueoSim jobs while running the XF jobs could slow down the queue for the XF jobs. So, I made the loop wait to submit the batch pueoSim jobs until we had space for all of them to be active with the ~250 max jobs per user.

Additionally, while observing my many tests, I didn't notice any correlation between the number of CPU peuoSim jobs in the queue and the number of GPU XF jobs out of the queue.

5. Branch I'm developing this on is here

6. The total time for the test generation was 15.4 hours, which is slightly longer than the ~14 hours from the 2023_05_08 run. However, this test also used double the population size, larger values for the range of antenna heights (on average about 3x taller), and 20x more neutrinos simulated per antenna. So, the actual speed is better than it first looks.

Types of runs I need (May need some other people to help me run these):

1. Optimization runs

   1. Non-error

      1. Search over combinations of selection methods and genetic operators

         1. Fix Sigma at 10%.

      2. Do this for 10 runs for each 

      3. Re-run the best run for 100 tests to see if the results agree with 10 test results

      4. This will be the main talking point

      5. Save the plot for the best combination (proxy and metric)

   2. Error

      1. Re-run the best run for 100 tests with 3 different error amounts (0.1, 0.2, 0.3)

      2. We will compare these results to the non-error run to talk about how errors may affect consistency/speed. 

      3. Save an example plot of an error test for each error (both metric and proxy score)

   3. Population

      1. Re-run the best run for 100 tests with 3 different population sizes (100, 500, 1000)

      2. We will compare these results to the non-error run to talk about how population size affects consistency/speed. 

      3. Save an example plot of a population test for each size (just proxy score)

2. Demonstration runs (single runs, fitness score, no error population 100)

   1. Crossover only

   2. Mutation only

   3. Reproduction only

   4. Injection only