Here are the run details for the latest PUEO run we've started. There are still some bugs that need to be worked out, which are detailed on the github. Here is the directory where the data is stored: /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/2023_05_08
This run uses 50 individuals per generation with 400k neutrinos per individual at an energy exponent of 19. Only the inner side length and outer sidelength are being evolved (which evolves the height since the opening angle is fixed).
Issues With Part E (Dylan):
The Plots were not automatically created for the first generation. After a quick look through, I found that the fitness score and v effective csv files were being moved before they were copied, so the files didn’t end up in the correct place. Additionally, the errorbars.csv file was not being created, so I made a temporary fix to create a file of 0 error bars to allow for plotting the values while I work on getting the real error values.
Update: I have modified Will’s script for reading the root file outputs, so it can now read them to output a CSV file with the effective volume along with its positive and negative errors. This script does the same and more as the fitnessfunctionPUEO.py, so I will replace it in the loop when there is downtime. Additionally, this script means we don’t need to output a veff.csv file from pueoSim. I will now work on creating a less “janky” version of my pueoSim change for reading in the gain files, so I can get Will to officially add it to pueoSim and we can pull any updates in the future.
Update, Implementing the new script into Part E:
The rootAnalysis.py script is very picky with environment variables. So, first I had to load and then unload a version of python to make sure the default version is loaded. Then, I moved the sourcing of set_plotting_env.sh to later in the script to avoid having an alternate version of root sourced while running (this would cause errors). Then I source the environment for pueosim (path hardcoded, change when loop isn’t running) and loop through the rootAnalysis.py program for each individual in the generation. This will output a ${gen}_fitnessScores.csv and a ${gen}_errorBars.csv in the run directory for plotting script to use. Because it outputs these automatically, I commented out the old methods we used to create these files and removed the temporary error bar fix as well.
Another Update:
We paused the loop. I changed the hardcoded parts, and we pushed the new updates to the github. I will make comments on the changes later today.
Wednesday:
It seems like the loop is still not creating the csv files or plots for generation 1 or 2. Alex gave me the error message. The python script rootAnalysis.py had an issue with the include Geoid.h line. This was caused by sourcing the wrong things while trying to get the script working. I found that I only changed my local pueoSim environment to include the fftw installation to the $PATH and $LD_LIBRARY_PATH variables. So, I changed /fs/ess/PAS1960/buildingPueoSim/set_env.sh as well. Also, I found that the include statements in rootAnalysis.py were missing the PAS1960 portion of the path. After fixing these, and adding a missing ‘/’ to the output file path, I manually ran the rootAnalysis.py program for the 2023_05_08 run, mimicking the portion of the bash script that runs it in Part_E. It outputted the correct files in the correct format. I then ran the FScorePlot script. The median line for the ViolinPlot seemed wrong, and the FScorePlot was too scrunched with a y start of 0; I will need to change these.
Plotting Update (Dylan):
Ok, plots still aren’t being made, and I’m not running the loop, so I don’t have access to the error messages, so I’ll go through everything one by one. For Veff_Plots, this isn’t being made as there are currently no vEffectives.csv. This isn’t a big deal, as it’s just a copy of FScorePlot, but I added the creation of vEffectives.csv to rootAnalysis.py anyway. Next, I noticed that runData.csv, which is required for a couple of the plots was not being made. I looked, and the version of gensData.py had the old values for the skiprows when reading in the fitnessScores and generationDNA files, something that was overlooked when we merged the version I was developing with the version Alex was developing on. I can’t access the directory or the file, I think this is why it was overlooked in the merge. I created a temporary PUEO version in Antenna_Performance_Metric (I’m also fine with it just living here permanently).
Also, the csv files are now in the GenerationData/ directory in the run directory? This affects every plotting script and they will not work without changes. I don’t know why this was changed or if this change is permanent?
Next, ok apparently, VariablePlots isn’t even a file in the loop? We really need to use GitHub better.
I moved my local version of the program into the loop and it ran fine manually.
I am forsaking the Generation_Data directory, so I changed the rainbowPlotter and dataConverter scripts to read in a location to match all the other plotting scripts. They both also ran fine manually.
Note: The rainbowPlotter script should be able to create a gradient from the minimum fitness score to the maximum fitness score. (currently, the range is hardcoded and this can cause basically a mono-color graph.)
FScorePlot should be good
The bash script was inputting 5 variables into colorplots when the pueo version only takes in 4 (this is also something that was already fixed before the merge.) It also requires the vEffective.csv files. After copying over the fitnessScore.csv files to vEffective.csv files, it ran fine. (The vEffective.csv files will be made automatically by rootAnalysis.py in the future) |