I've started a run of the PUEO loop to get some data for evolving the antenna width and height. We are running with 24 individuals, each with 400k neutrinos at an energy exponent of 19. Here is the output directory: /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/2023_05_01_Test_5

It's very important to note that the cross-pol data being used still comes from the data already stored in pueosim. This is because our extremely low cross polarizations were causing us to have no effective volumes. There's likely some issue in what values from XF we are choosing as our crosspols.

Slides contatining my notes on matching circuits.

https://docs.google.com/presentation/d/1x25nhiqaW7LvPZ1pNZ5O4ZzsWZbtgqxBQ5haB9uWgQY/edit?usp=sharing

Attached is a spreadsheet with the information on the parts we need for the N=14 matching circuit board.

https://docs.google.com/spreadsheets/d/1x8dX3tNE-WSHjH_slj_EH4XsHcAnCVC05XiRBFPtIUc/edit#gid=0

We presented at a workshop put on by the MODE collaboration. MODE is a collaboration dedicated to applying Automatic Differentiation to detector design. Here's the website: https://mode-collaboration.github.io/#:~:text=MODE%20(for%20Machine%2Dlearning%20Optimized,in%20space%2C%20and%20in%20nuclear

• Run Type
  • Main Arasim Loop
• Run Date
  • 02/04/2022
• Run Name
  • 2022_02_04_Rank
• Why are we doing this run?
  • To test rank selection in main loop
• What is different about this run from the last?
  • Rank Slection is being used.
  • Parents.csv introduced.
  • Elite is being turned off.
• Symmetric, asymmetric, linear, nonlinear (what order):
  • Non-linnear asymetric
• Number of individuals (NPOP):
  • 50 individuals
• Number of neutrinos thrown in AraSim (NNT):
  • 300,000
• Operatiors used (% of each):
  • 06% Reproduction
  • 72% Crossver
  • 22% Immigration
  • 1% M_rate (unused)
  • 5% sigma (unused)
• Selection methods used (% of each):
  • 0% Elite
  • 0% Reproduction
  • 10% Tournament
  • 90% Rank
• Are we using the database?
  • No.

Directory: /fs/ess/PAS1960/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/2022_02_04_Rank

Here's the video I made a few months ago demonstrating how to run the loop. There might be some changes that have been made since then, but they'll generally be minor so it should still be representative of what running the loop looks like. It might need to be shared with you through the Microsoft drive service OSU gives us in order to view it, so either message me on Slack or email me at machtay.1@osu.edu and I'll share the link through there.

https://drive.google.com/file/d/1yL_eH_w2hXNt7J7YOwS5xl3ZHwGyl5JR/view

As mentioned in the manual, using ssh to login to Nutau causes delays; the XFdtd GUI cannot be surpress and must be forwarded via X11 forwarding, and this is extremely slow. We are currently looking into using a newly suggested option, XRDP. Information will be added as this process continues. 

Previously, we have tried to straighten the sides of the best-evolved curved antenna in elog 229. However, there were potential issues with how closely this line resembled the curve of the antenna.

So, I attempted to create another straightened sides antenna using a linear regression model to find the best fitting line for the curve to create an antenna.

I made a Python notebook to separate the equations for each curve into 1000 discrete points. Then I ran a linear regression model to fit a curve of the points 1 - n, and a second curve of the n - 1000 points, looping from n = 2 to n = 999.

These results are from the combined output with the least squared error compared to the original.

Pictured is a plot showing the two sides of the curved bicone in red and blue, with the best fitting lines for each in black as well as a model in XF.

Results:

The antenna has a fitness score of 3.80627 with an error of 0.0759725.

This is much lower than the 5.71 of the curved antenna and 5.11 of the other attempt at straightening.

For the next attempt, we could consider constraining the endpoints to be the same as the original antenna to conserve the radius, and/or adding an extra line to fit the curve.

I've also attached a picture of what my notebook found for fitting 3 lines to the curve (not modeled or tested).

Professor Chen recommended using 3 sides and constraining the outer radii of the cones to match the original curved design.

Path to linear regression notebook: /users/PAS1960/dylanwells1629/developing/notebook.ipynb

Path to XF project: /users/PAS1960/dylanwells1629/straightened.xf

OSC managed to figure out our issue with running XF from an interactive job on Slurm. Previously, we were losing our connection to x11 forwarding. The solution is to use sinteractive instead of srun to obtain an interactive job. Here's the syntax:

sinteractive -A PAS0654 -t <time> -N 1 -n 8 -p serial

The -p serial flag denotes the type of partition to request. It's important to specify this, as otherwise it will default to the debug node, which has a limit of 1 hour.

We're going to begin a new run. The title is Machtay_2021_1_15_NPOP50_Asym . It is using the latest version of the GA Ryan has been working on (Latest_GA_Asym.cpp). We're using 6% reproduciton, 72% crossover, and 22% mutation. We are using 80% roulette selection. This correlates to the three numbers passed into the GA being 3 36 8 (since we're using 50 individuals).

Running IceMC:

Go into the directory ../anitaBuildTool/build/components/icemc/

And run the command

• ./icemc -i {inputFile} -o {outputDirectory} -r {runNumber} -n {numberOfNeutrinos} -t {triggerThreshold} -e {energyExponent}

*May need to chmod -R 775 ../anitaBuildTool/comonents/icemc/ if you get a permissions error

inputFile:

Must be the full path to the file

Config files are found in ../anitaBuildTool/components/icemc

Ex: /users/PAS1960/dylanwells1629/anitaBuildTool/components/icemc/inputs.anita4.conf

Config files are found in ../anitaBuildTool/components/icemc

outputDirectory:

Will be made in ../anitaBuildTool/build/components/icemc/ by default, specify full path otherwise.

runNumber: 

The run number.

numberOfNeutrinos:

The number of neutrinos generated in the simulation. 

Can be found in inputs.conf

Default is 2,000,000.

#How many neutrinos to generate

triggerThreshold:

Threshold for each band for the trigger. 

Default is 2.3

#thresholds for each band- this is only for the frequency domain voltage trigger.  If using a different trigger scheme then keep these at the default values of 2.3 because the max among them is used for the chance in hell cuts

energyExponent:

The exponent of the energy for the neutrinos

Can be found in input.conf 

Default is 1020

# Select energy (just enter the exponent) or (30) for baseline ES&S (1) for E^-1 (2) for E^-2 (3) for E^-3 (4) for E^-4 (5) for ES&S flux with cosmological constant (6) for neutrino GZK flux from Iron nuclei (16-22)not using spectrum but just for a single energy (101-114)use all kinds of theoretical flux models

Installing IceMC:

I: Getting anitaBuildTool

Clone the anitaBuildTool repository (https://github.com/anitaNeutrino/anitaBuildTool) to your user.

• git clone https://github.com/anitaNeutrino/anitaBuildTool

II: Get the Anita.sh file onto your user.

 Either copy the file from /users/PAS1960/dylanwells1629/Anita.sh

• cp /users/PAS1960/dylanwells1629/Anita.sh ~

or create a new file Anita.sh with the following code

# .bashrc

# Source global definitions

if [ -f /etc/bashrc ]; then

. /etc/bashrc

fi

#this modules were originally loading in both the env.sh, and bashrc_anita.sh files. This was redundant so it was added here, and removed from the others.                                          

 module load gnu/7.3.0

module load gnu

module load mvapich2

module load fftw3

#module load python/3.6-conda5.2

module load cmake

PATH=$PATH:$HOME/.local/bin:$home/bin

export PATH

export CC=`which gcc`

export CXX=`which g++`

export FFTWDIR=/fs/project/PAS0654/shared_software/fftw3/gnu/6.3/mvapich2/2.2/3.3.5

export ANITA_SOURCE_DIR=~/anitaBuildTool/

export ANITA_UTIL_INSTALL_DIR=~/anitaBuildTool/

export ICEMC_SRC_DIR=~/anitaBuildTool/components/icemc/

export ICEMC_BUILD_DIR=~/anitaBuildTool/build/components/icemc/

export DYLD_LIBRARY_PATH=${ICEMC_SRC_DIR}:${ICEMC_BUILD_DIR}:${DYLD_LIBRARY_PATH}

export ROOTSYS=/fs/project/PAS0654/shared_software/anita/owens_pitzer/build/root

# User specific aliases and functions

#This env.sh is for running the BiconeEvolution GENETIS software. This should only be un-commented if you are running GENETIS software. When you do this, comment out env.sh.                      

#source ~/new_root_setup.sh

source /fs/project/PAS0654/shared_software/anita/owens_pitzer/build/root/bin/thisroot.sh

#source /cvmfs/ara.opensciencegrid.org/trunk/centos7/setup.sh

#module load python/3.6-conda5.2

#BiconeGENETIS directory shortcut SHARED                                                                                                                                                           

alias GE='cd ../../../fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/'

#emacs Alias                                                                                                                                                                                       

alias emacs='emacs -nw'

#root alias                                                                                                                                                                                        

alias root='root -l'

#Alias                                                                                                                                                                                             

alias l="ls"

alias python='/cvmfs/ara.opensciencegrid.org/trunk/centos7/misc_build/bin/python3.9'

Then source the file

• source Anita.sh

Note: You will need access to PAS0654 for this step or you will get a permissions error.

III: Running the build tool.

Go into the anitaBuildTool directory

• cd anitaBuildTool

And run the building script

• ./buildAnita.sh

Note: There will be an error if you source files for running Ara in your .bashrc

Comment these out and restart your terminal before running the build. (remember to source Anita.sh before running the build tool. You could also source Anita.sh in your .bashrc)

Error if you source files for running Ara:

CMake Error at components/libRootFftwWrapper/cmake_install.cmake:238 (file):

  file INSTALL cannot copy file

  "/users/PAS1960/dylanwells1629/anitaBuildTool/components/libRootFftwWrapper/include/AnalyticSignal.h"

  to

  "/cvmfs/ara.opensciencegrid.org/v2.0.0/centos7/ara_build/include/AnalyticSignal.h":

  Read-only file system.

Call Stack (most recent call first):

Will sent me some info on how to install PUEOsim. I attached a markdown file he sent for how to install PUEOsim. He also sent the path to pre-compiled installation of PUEOsim. Here's the path to the pre-compiled installation: /users/PAS0654/wluszczak/PUEO_shared

Here is his full message, which details that you must also run set_env.sh before running PUEOsim:

a pre-compiled version is located at /users/PAS0654/wluszczak/PUEO_shared . Inside that directory, there's a set_env.sh script that you will need to source before running anything (hopefully this works, though there's a high probability you might run into permissions issues with the dependencies).

To run PUEOsim, use the following (see here for instructions: https://github.com/PUEOCollaboration/pueoSim):

./simulatePueo -i {inputFile} -o {outputDirectory} -r {runNumber} -n {numberOfNeutrinos} -e {energyExponent}

The outputs will be root files, but it should be possible (either modifying our own version directly or with a script) to print out the effective volume to a .txt to read.

This is a duplicate post of a previous post from the end of 2020 where I listed the important runs with some of their details in a table (as below). I am extending this table to include the important runs that have been conducted since this post. This includes the run used for the paper as well as the curved run done earlier this year.

In order to access the data for these runs, you can find them by going to this directory: /fs/ess/PAS1960/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs

Some of these runs can also be accessed in the old project space directory, though they should all be contained in the above directory. Here's the path if interested: 

The runs are contained in directories available in the above path. Use caution when listing files in some of these directories--some contain many files (primarily the .uan files -- more recent runs are better organized), which means it may take a long time to list files/directories.

Today I removed some of the run directories which had very little or no data or weren't worth keeping around. There are still a few that I think can be removed, but I'm keeping them until we can get a consensus that they can definitely be removed. Below I listed the names and descriptions of the runs that I think we should definitely preserve going forward. In general, the more data contained in the run directory, the more important it is to keep around. 

We are trying to do a short run of the asymmetric bicone so that we can see how it tries to evolve the antenna when the minimum length is lowered from 37.5 cm to 10 cm. Currently, there is a problem with the loop in the asymmetric version. Attached is the run detail file.

Here is our current assumption of the antenna angles needed for the icemc inputs. 

Here is a PDF with a summary/log of my work trying to integrate an ice shell surrounding the bicone antennas in the GENETIS loop. I made gain patterns of the ARA bicone with the ice shells and without them, showing the impact of the ice on the antenna's gain. Having an ice shell changes these patterns, which are also dependent on the radius of the shell. I showed that convergence on the gain patterns and expected features start ocurring for shells with radii larger than 10 meters. I also keep track of the computation time which increases with bigger shells.

Note: I also add PowerPoint slides that contain all the graphs that I made. The PDF does not contain some of those since I didn't think they were coherent with my write-up, but I may be making an omission. 

 

In this post, I'm going to give step by step instructions on everything you need to do to run the loop. See Julie's thesis for the most recent iteration of this manual. Also refer to entry 123 for a list of some errors we have encountered historically in the loop and how to resolve them, though some fixes may be outdated.

Running the loop

Getting started

Assuming you have followed the instructions from the onboarding tutorials, you should be ready to access OSC through the interactive website. Instead of using ssh to access OSC, the loop requires that you go to this website: https://ondemand.osc.edu/ . Login with your usual OSC credentials. Once there, you will need to request a Lightweight Desktop by clicking on "Interactive Apps" at the top of the page. See the screenshots attached for how to do this. Old documentation may refer to the Lightweight Desktop as a VDI.

You can request up to 24 hours for a Lightweight Desktop, and it should begin almost immediately. After you request it, the website will take you directly to the waiting page. You can navigate to this page yourself by clicking "My Interactive Sessions" at the top of the ondemand page. When the job begins, you will see a blue buttong that says "Launch Lightweight Desktop". Click it and you should have a new tab open that brings you to a remote desktop environment on OSC that you interact with through your browser. 

New runs

If you're starting a new run for the first time (that is, beginning from generation 0), you will need to edit the script that runs the loop. Here is the path:  /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Scripts/Asym_XF_Loop.sh . At the top of the script is a list of variables that can be changed. See the first screenshot attached here.
For a new run, you must change the variable <RunName>, which is the first variable in the list. It should be in quotes and the convention is to start the name off with the date of the run, using YYYY/MM/DD format--this will always keep everything in chronological order when listing directories with <ls>. Other variables are explained in the comments next to them in Asym_XF_Loop.sh and you should change them depending on the type of run you are conducting. 

To begin the loop, type in the command ./Loop_Scripts/Asym_XF_Loop.sh from the Evolutionary_Loop directory.

When the loop begins, you will be asked to press any key. After you hit a key, the loop will run Part A, which executes the GA using the <start> mode. It will then automatically move on to Part B, which creates the antennas in XFdtd. XF will ask you if you want to specify a location for the XF file to be saved. Click <No>. The loop will automatically save the XF directory into the run directory when it finishes with the first part of part B. In principle (that is, barring errors that stall the loop or interruptions due to the VDI job ending), you will never need to manually enter anything to the loop for it to run beyond this point. 

However, if it is your first time using XF, you will need to set the max GHz setting upon startup to 2 instead of the default 10. 

Continuing Runs

After starting a new run, the loop will run automatically. When your Lightweight Desktop job ends, you'll need to get another one repeat the steps above. You will not be prompted for any input. 

Stepping Back

At times, you may want to continue the loop from an earlier point or interrupt the loop and step back. Usually you'll only want to do this within a generation, but it's possible that you will encounter an error or bug at the end of a generation and wish to return back to that point to remedy it before continuing forward in the next generation. This is a very tricky and delicate process that requires a lot of attention to detail. In the future, we'd like to make an option for the loop to do this automatically when given input at the beginning, but implementing that will take a long time and will be very sophisticated.

The state of the loop is recorded in the savestate file located here: /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/saveStates . The file will be named <Run_Name>.savestate.txt . In it, you will see three numbers. The first one represents the current generation of the loop. The second one represents the state of the loop, that is, which part in the generation it is currently on. The third one represents the individual it is processing. In practice, the third number is rarely relevant, and is almost always 1, though it can be used for Part B, where the loop is processing individuals sequentially, so if it was interrupted on individual eight (for example) you can change the number to be 8 so that it will pick up where it left off without needing to redo the first seven individuals.

If you want to step back to an earlier state in a generation, you'll need to interrupt the loop and revert the state by changing the second number in the savestate file. This is NOT all you will have to do, however. Because of the way files are edited and moved around during states, you will need to manually revert files to be in the format or location needed by the earlier state. 

For example, you may need to revert from state 2 (when XF is creating the antenna geometries) to state 1 (when the GA generates new individuals). In this case, you'll need to edit the savestate file to change the second number from a 2 to a 1. Then, you'll need to change the generationDNA.csv file in Generation_Data to use the values from the previous generation, since the GA will have overwritten that file and uses it to determine what individuals are to be selected from. To do this, you'll need to locate the <gen>_generationDNA.csv file from the previous generation in the <RunName> directory in Run_Outputs. This is should be found here:  /fs/ess/PAS1960/HornEvolutionOSC/GENETIS_PUEO/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/<RunName>/Generation_Data . 

The above is a relatively simple example of how to step back in the loop. Stepping back from a late state to an early state will be more complicated, though if you're willing to sacrifice run time you can pretty easily follow the above to just restart a generation. Stepping back to a previous generation will be explained in a future update to this post.