Today's Tasks

Julie: Write bash practice assignment, assist others when they have questions, work on Git lecture. (Will finish bash assignments early next week)

Evelyn, Mitchell, Eliot: Make progress on the "Learn the Command Line" course on Code Academy. (Should be done by the end of the week?)

Leo: Start gathering information for the "physics of antennas" lecture series! Note: Remember that Leo wants to be an educator, and would like us to find opportunities to work on "teaching" items. Most of the students don't have any background on how antennas work. He started learning about them and is going to do a lecture series (likely the first 20 minutes of every meeting) starting in a few weeks! (Should start series by hopefully 2/21)

Alex M: Clean up unused code. Fix hard-coded directories (likely finished & tested by end of day 2/11)

Alex P: Fix hard-coded variables (likely finished & tested by end of day 2/11)

I've tacked on our to-do list in the meantime so we can keep carrying the items still pending along with us. As we complete them, I will drop them off the list! 

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim error fixed (Alex, Julie) 

   1. It seems that if AraSim jobs are left to run while the user is away, it doesn't properly write one or more of the flags, and makes the loop get stuck at AraSim. We must then re-run from that state by manually editing the Save State text file.

   2. We are going to try to get XF to run at the command line so that we can submit it as a job. This would alleviate us using an interactive job. We *hope* this would fix the error since the user wouldn't be "gone" while utilizing a GPU in an interactive job. 

      1. Alex emailed XF to ask a few questions! Stay tuned!

2. Make Hardcoded directories variables.

   1. Xmacros (use sed command) 

   2. All over bash script functions.

   3. AraSim job files (sed command?)

3. Make the following variables in the bash script:

   
   In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

   Grid spacing=0.1
   Frequency= factor of n higher 
   This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.
   
   
   

   In roulette_algirithm.cpp

   Length = 50cm/Multiplier_factor; 
   STD= 15cm/Multiplier_factor 
   Radius = 1.5cm/Multiplier_factor; 
   STD= 0.75cm/Multiplier_factor 
   
   Also the "Multiplier_factor" line 89.
   
   This "Multiplier_factor"has to be the same as the GeoFactor in fitnessFunction_ARA_amy.cpp AND the same as the factor higher in frequency in the xmacro.
   ​NOTE:THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   
   

   In fitnessFunction_ARA_amy.cpp:

   GeoFactor=2 
   This GeoFactor has to be the same as the Multiplier_factor in the roulette algorithm AND the same as the factor higher in frequency in the xmacro
   NOTE: THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   

4. Get rid of all code we are not using anymore.  

   1. Currently using: XF_loop_AraSeed.sh

      1. Look at what items are being called in here, and remove old versions such as XF_loop_Prototype.sh, XF_loop_Functionized.sh, and the related scripts. 

      2. These can temporarily moved to on "old code" directory and we can delete once we finalize the loop. 

5. Turn tournament on in the GA. 

6. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being seeded.

       

As of 01/30/2020, the proposal was submitted! The project description and summary are attached below. 

Bicone Loop Status Update: 

The loop currently runs; however, it does seem that if AraSim jobs are left to run while the user is away, it doesn't properly write one or more of the flags, and makes the loop get stuck at AraSim. We must then re-run from that state by manually editing the Save State text file. 

The loop currently has the following additions implemented.

• Functionalized loop (XF_Loop_AraSeed.sh)
  • Cleaner bash script which runs smaller bash scripts, instead of one massive script. 
  • Submits multiple AraSim jobs in parallel for each individual to increase speed. 
    • Ie if 100k neutrinos are thrown per individual ($NNT=100000, $Seed=10) we can, say, break it up into a "seed" of 10 -- which then submits 10 jobs with NNT=10k for each individual. 
  • Still needs this to be fixed: The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being seeded. It is just submitting one job the following way.
    • If NNT=100000 and Seed=10 (ie 10 jobs with 10k neutrinos), it will only run AraActualBicone as 1 run with 10k neutrino number (ie NNT/seed=NNT_New).
• Save State 
  • After every "part" -- ie part a, part b, etc -- of the loop, we save state by writing a series of numbers to a text file. 

    • ${gen}, ${state}, {indiv}

  • If the loop is killed for any reason during the process, the text file is read in, and the loop can continue forward from the last completed item/section. 

  • This saves the state after every individual runs through XF. 

• Antenna sizing

  • The larger the antenna, the longer XF takes to run. A true-to-size ARA antenna takes anywhere from 1.5 to 2 hours per individual. 

  • What we've done: Allowed us to scale the antenna by a factor of "x".

    • If we make the gaussian centered around something, say, 5 times smaller than the ARA antenna size, we then make the frequencies 5 times larger. This should have the same effect. 

    • This also means we've had to scale these sizes back up when plotting. 

    • Still needs this to be fixed: all of this has been hardcoded in! We need to make these variables in the MANY places we need to adjust for this scaling. See the to-do list below for details. 

Moving forward, I will be doing the following:

1. Getting our less coding-savvy teammates familiar with bash, python, and c++. This will be crucial for us when we start adding more parameters to our GA. 
2. Mastering XF (along with Alex, and Alex). 
3. Getting the team to clean up some messy coding done during the proposal hustle. 
4. Fixing some errors. 
5. Teach the ENTIRE team git. 

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim error fixed (Alex, Julie) 

   1. It seems that if AraSim jobs are left to run while the user is away, it doesn't properly write one or more of the flags, and makes the loop get stuck at AraSim. We must then re-run from that state by manually editing the Save State text file.

   2. We are going to try to get XF to run at the command line so that we can submit it as a job. This would alleviate us using an interactive job. We *hope* this would fix the error since the user wouldn't be "gone" while utilizing a GPU in an interactive job. 

      1. Alex emailed XF to ask a few questions! Stay tuned!

2. Make Hardcoded directories variables.

   1. Xmacros (use sed command) 

   2. All over bash script functions.

   3. AraSim job files (sed command?)

3. Make the following variables in the bash script:

   
   In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

   Grid spacing=0.1
   Frequency= factor of n higher 
   This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.
   
   
   

   In roulette_algirithm.cpp

   Length = 50cm/Multiplier_factor; 
   STD= 15cm/Multiplier_factor 
   Radius = 1.5cm/Multiplier_factor; 
   STD= 0.75cm/Multiplier_factor 
   
   Also the "Multiplier_factor" line 89.
   
   This "Multiplier_factor"has to be the same as the GeoFactor in fitnessFunction_ARA_amy.cpp AND the same as the factor higher in frequency in the xmacro.
   ​NOTE:THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   
   

   In fitnessFunction_ARA_amy.cpp:

   GeoFactor=2 
   This GeoFactor has to be the same as the Multiplier_factor in the roulette algorithm AND the same as the factor higher in frequency in the xmacro
   NOTE: THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   

4. Get rid of all code we are not using anymore.  

   1. Currently using: XF_loop_AraSeed.sh

      1. Look at what items are being called in here, and remove old versions such as XF_loop_Prototype.sh, XF_loop_Functionized.sh, and the related scripts. 

      2. These can temporarily moved to on "old code" directory and we can delete once we finalize the loop. 

5. Turn tournament on in the GA. 

6. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being seeded.

The following items from our to-do list have been successfully completed. Below this list, you can find the remaining urgent items for our proposal due date of 1/28 (items must be completed with plenty of time to spare). 

Completed items: 

*Note* please check the highlighted item. This one still must be verified. 

1. Made software plot Theta VS Gen
2. Fixed current L and R plots
   1. These now have correct axis labels, a legend, and connect data for each individual between generations.
3. Produced Veff for ARA's actual bicone parameters. 
   1. Made XF_Loop.sh submit a job for the ARA gain AraSim input file in BiconeEvolutionOSC/AraSim/Ara_Bicone6in_outputs.txt
   2. STILL MUST: Make sure that the "wait function" has been properly edited to consider the submission of this extra job in the first iteration of the loop. (Not sure if $NPOP+1 instead of $NPOP fixed it correctly) ----> CADE SHOULD LOOK AT THIS :)
4. Added plotting software for V_eff VS Generation (includes ARA's actual bicone V_eff as generated in #3).
5. Fixed the ARA hole constraint in fitnessFunction_ARA.cpp -- ie fixed the calculation to consider the correct radius "r". See http://radiorm.physics.ohio-state.edu/elog/GENETIS/28 for details. 
6. Successfully made energy a variable in setup.txt. 
   1. used the same "sed" format as before in the bash script (XF_Loop.sh).
7. Pushed to Github. 

Items to still do:

1. Save CAD models in XF: Due 1/15                                                   ------>(Cade-- second most urgent one)
2. Stop plots from popping up during run: Due 1/15                            ------>(Alex P.)
3. Add "save state" idea to loop: Due 1/15                                         ------> (Cade and maybe Alex P can help -- This is the most urgent one)
4. Figure out why fitnessFunction_ARA.cpp isn't plotting: Due 1/15     ------>  (Julie, and Alex M)
5. Run the loop and evolve for results: Due 1/24                               ------>(Alex M, Mitchell, Cade)
6. Write proposal: Final Due 1/28                                                   ------> (Amy, Kai, Julie)

Note that the people running the loop (Alex M, Mitchell, Cade) should be producing plots and saving them. Please save all plots in this dropbox folder: https://www.dropbox.com/home/GP_Antennas/NSF%20Proposal%20Plots%20(1_28_2020)

When submitting plots, please make a new folder in there and label it "Date_of_run, Energy, Generations, Individuals". For example, my folder within this dropbox link would be: "1_10, 10^19, 10, 5". I have made sample folders in this dropbox link if needed.

You can export the plots from OSC to your machine (using CP command) or can type "display NameOfPlot.png" then screenshot and crop your plot to upload to dropbox.

Updated To-Do List:

1. Add ARA constants in plotting code: ie add Veff, theta, R for ARA. This will sit as a constant line for us to compare each evolving parameter to. For example, in the plot Veff vs Generation, we will see a constant line for ARA's Veff to see how it is evolving in time. 

TASK 1: MAKE Veff PLOT. Note: One of our variables in this run is Energy. Unfortunately, we care about how Veff changes with Energy for ARA_actualBicone. This means that before plotting "Veff versus Generation" we need to generate Veff for ARA_actualBicone at the energy we are assigning the run. 

STEP 1: Make plotting code to plot "Veff vs Generation". 

• You can follow the way we make LRPlot.py.
  • However, in this case, we will need to take the files /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Ouputs/$RunName/AraOut${gen}_${i}.txt -- ie the AraSim outputs for each individual in ALL generations. ie Gen 0 - ${gen}  -- INCLUDING the one we created in Gen 1 for ARA_actualBicone -- and plot Veff from those. 
  • We also want to plot error bars with Veff. The error bars for each are labeled "and Veff (water eq)" at the bottom of these AraSim output files we are already reading in for Veff. 
    • Note, to know how to read in Veff, you can look at how we are reading that *exact* line from the AraSim output files in on the fitnessFunction_ARA.cpp code in the Antenna_Performance_Metric directory, since we are doing it there. 
    • Note that since we are plotting "Veff vs Generation", we will either have to read in all the files AraOut${gen}_${i}.txt (up to how may ${gen} we are in) every time -- including the ARA_actualBicone one-- or we can append them all to one .txt file. It's your choice. 
  • Make sure you label the units for each axis every time, and that you have a key! If not, Amy will hate the plot!

STEP2: Add V_eff for ARA_actualBicone as a constat in the newly made "Veff vs Gen" plotting code. Remember, as I mentioned above, we must first put the ARA_actualBicone through AraSim to see Veff at the same energy as the rest of the individuals. Here's what to do:

•  Edit the Bash script to submit a job that runs the gain patterns of ARA_actualBicone through AraSim in the initial gen (gen 0)! 
  • There's a current file that exits for ARA_actualBicone that has the gain patterns of the current antenna, in the correct format for AraSim to run. In the bash script (XF_Loop.sh) we need to create ONE more AraSim job submission for this antenna (as a comparison to what we see with our genetically evolved ones) just in gen 0 (the first generation).
    • This current AraSim input file for ARA_actualBicone exists in BiconeEvolutionOSC/AraSim/Ara_Bicone6in_output.txt
      • This needs to be the input for our .job script. 
• Add lines to your newly made "Veff vs Gen" plotting code that reads in the Veff output for this ARA_actualBicone AraSim output file (whatever you've chosen to name it when you wrote it into the job script. 

 STEP 3: Move the ARA_actualBicone output file into the $Run_Name directory so it is not overwritten ever. 

• We will want to make sure this is not overwritten after the run ends, and we start a new one. 
• We will also be using it for plotting "Veff vs Generation". 

STEP 4: Correct the "wait" command for the AraSim job submissions. 

• The "wait" command, on the first iteration, will have to wait for $NPOP+1 now-- not $NPOP.

TASK 2: MAKE "Theta vs Gen" Plot.  

STEP 1: Make plotting code to plot "Theta vs Generation". 

• You can follow the way we make LRPlot.py; however, MAKE SURE YOU LABEL AXIS WITH UNITS, AND HAVE A KEY.
• This will be exactly the same as this other code, except you will need to:
  1. Read in theta from your generation DNA files
  2. Add the ARA_actualBicone constant to the code as a reference point

TASK 3: Fix "L vs Gen" and "R vs Gen" plot.  

STEP 1: Correct both plots to have labeled the units for each axis every time, and a key. This is plotted in the "LRPlot.py" code. 

STEP 2: Fix our 'R' values. I do believe we are plotting the incorrect "R". Here is what I think is happening:

• We need to double-check and see what ARA defines "R" as on their current antennas. 
  • Check with Amy?
  • What does Brian Clark's thesis say?
  • Can you find a paper that shows a picture of how they define it?
• If we are wrong, this means we need to change the "R vs Gen" part in the "LRPlot.py" code so that we:
  1. Read in R, L, and Theta for each generation.
     • See how this is already done for reading in R, and repeat it for L and theta. 
  2. Do trig to calculate the same "R" as ARA_actualBicone defines. See picture below!
  3. Store that newly calculated "R" as an array, and plot THAT.
  4. Don't forget to add the ARA_actualBicone Radius as a constant line. 

TASK 4: See PDF for other items!

In my last post, I mentioned (1) our to-do list for adding the ARA hole constraints, and (2) that we've made a mistake upon implementing it. I've attached detailed steps of that to-do list that I've written, as well as an explanation for the correction that is currently being worked on. Please see the pdf attached for an explanation of corrections. 

Attendance: Julie, Alex M., Alex P., Cade, Scott.

What we have done today:
(1) Fixed Concerns regarding copying the output .uan files properly so they are not overwritten. (see #8 in this elog entry: http://radiorm.physics.ohio-state.edu/elog/GENETIS/22 for details on the error). This was corrected in part E as:

for i in `seq 1 $NPOP`

do

    for freq in `seq 1 60`

    do

    #Remove if plotting software doesnt need                                                                                                                                                             

    #cp data/$i.uan ${i}uan.csv                                                                                                                                                                          

        cp Antenna_Performance_Metric/${i}_${freq}.uan "$WorkingDir"/Run_Outputs/$RunName/0_${i}_${freq).uan

    done

done

And in the "looping part" -- ie part G we have corrected it to say the following. Note that every time we run an iteration/generation in the loop, it writes the name of the output files the exact same (individual_frequencystep.uan). It will overwrite these files in the next generation unless we move them and rename them like so. We don't want this, because we will likely want to plot the gain patterns of the individuals we like (gain pattern data is in the .uan files). 

 for i in `seq 1 $NPOP`

        do

            for freq in `seq 1 60`

            do

    # Gens data used to create a .csv file for the uan file for gain plotting                                                                                                                            

    # cp Antenna_Performance_Metric/$i.uan ${i}uan.csv                                                                                                                                                   

                cp Antenna_Performance_Metric/${i}_${freq}.uan "$WorkingDir"/Run_Outputs/$RunName/${gen}_${i}_${freq}.uan

            done

        done

(2) Added scale factor "A" (actually called "Scale Factor" in our bash script and fitnessFunction_ARA.cpp), and the generationDNA.csv as an input file (this file is where the radii of all individuals in the current generation live).

• We have added this as a variable in the fitnessFunction_ARA.cpp. It now runs as a flag pass in when it runs in the bash script. In section E:

./fitnessFunction.exe $NPOP $ScaleFactor $AntennaRadii/generationDNA.csv $InputFiles #Here's where we add the flags for the generation

and in the looping part (part G):

     ./fitnessFunction.exe $NPOP $ScaleFactor $AntennaRadii/generationDNA.csv $InputFiles #Here's where we add the flags for the generation

and at the top where variables are declared:

RunName='Julie_12_2_2'           ## Replace when needed                                                                                                                                                  

TotalGens=6                     ## number of generations (after initial) to run through                                                                                                                  

NPOP=4                          ## number of individuals per generation; please keep this value below 99                                                                                                 

FREQ=60                         ## frequencies being iterated over in XF (Currectly only affects the output.xmacro loop)                                                                                 

NNT=10000                          ##Number of Neutrinos Thrown in AraSim                                                                                                                                

ScaleFactor=1.0                   ##ScaleFactor used when punishing fitness scores of antennae larger than holes used in fitnessFunctoin_ARA.cpp 

Note that every time the roulette algorithm runs, it's written into generationDNA.csv. We copy those right before rerunning the roulette alg in the next generation -- ie cp generationDNA.csv Run_Outputs/$RunName/${gen}_generationDNA.csv -- so that each generations parameters don't get overwritten. However, the current generation's parameters live in generationDNA.csv until the roulette alg runs in the next generation, so we can read in that file to the fitnessFunction_ARA.cpp file to determine the constraint (ie R_antenna) 

(4) Implemented Alex P.'s version of his optimized AraSim. This is now fully functional in the loop. 

NOTE: WE HAVE MADE A MISTAKE! WE MADE THE CONSTRAINT THE R-VALUE WE READ IN FROM THE GENERATIONDNA.CSV FILE. THIS IS NOT THE RADIUS WE THINK IT IS! WE NEED TO DO SOME TRIG ON IT TO FIX IT!!

Attendance: Julie, Alex P., Cade, Alex M., Mitchell, Evelyn (Go team!)

I. Adding Constraints on the in-ice hole size.

Today we worked on adding constraints on the drilled hole size. Our plan is to do so by penalizing the fitness scores of indiviuals with a radius larger than that of the hole in the ice. Ie in fitnessFunction_ara.cpp we would have:

if R_antenna>=R _hole

fitnessScore =fitnessScore x (e^{-[A(R_antenna-R_hole)/R_hole)]^2}) 

else

fitnessScore= veff (or whatever way we decide to calculate the fitness score in the future, if this ever changed)

Where "A" is a constant we can adjust to tweak how heavy the penalty is. Other things to do in order to make this work:

(1) Make constant "A" a variable in the bash script. 

• Do this the same way $NPOP is used in the roulette algorithm cpp program! :)

(2)Read in the roulette algorithm output file for your radius of the antenna. 

• Note that this isn't a super obvious thing. Right now our roulette alg outputs are saved as $(generation)_filename.csv. So, for the first gen -- ie gen 0-- it will be 0_filename.csv. For the second gen -- ie gen 1-- it will be 1_filename.csv...etc. This means we need to make sure the fitness score calculator has a way of knowing what generation we are on, in order to be sure it is grabbing the correct radius for each individual (not that of an old generation). 
   

We will be working on this more in our meeting tomorrow. As for now, the game plan has been to make comments on what we should be adding in the correct sections of fitnessFunction_ara.cpp (and will be doing the same in the bash script shortly, too --ie XF_Loop.sh). 

II. Adding Ara Values to Plot. 

We haven't started this yet, but it is something we need to do. When we plot Veff vs Generation, Theta vs Generation, and R vs Generation we need to add in the value of these for the current ARA bicone as a baseline for our evolutions.

Note: All of the contents over the last month need to be added to the manual! DO NOT FORGET!

Requesting an interactive job:

qsub -I -X -l walltime=1:00:00 -l nodes=1:ppn=40

(ppn=24 makes it quicker FYI)

Note that walltime is the length of time you'll need it. You will likely need it for 4+ hours if you are running the loop... Probably longer. Note that before you enter this command into your command line, you must be logged into OSC.

For submitting a job with GPUs (which we will be doing to speed up XF now as of 1/24/20), use this command:

qsub -I -X -l walltime=1:00:00 -l nodes=1:ppn=40:gpus=1:default      OUTDATED

srun -A PAS0654 -t 1:00:00 -N 1 -n 40      CURRENT

GENETIS Loop Work: Fixing Consol Errors By Julie Rolla

1. Python programs are not running: I think either (1) the user isn't module loading python, or (2) they are loading the wrong version of Python (remember there's a discrepancy for Python2, and Python3). 

Solution: I added a line "module load python/3.6-conda5.2" in our araenv.sh script to address the issue. All of the users are sourcing the araenv.sh script in their .bashrc when they login. It's in the directory: /fs/project/PAS0654/BiconeEvolutionOSC. I also added a line in the XF_Loop.sh script (at the top in the "Initialization of Variables" section) that sources araenv.sh just in case the user hasn't put that in their .bashrc. The line reads: source /fs/project/PAS0654/BiconeEvolutionOSC/araenv.sh

Error message fixed:

 0 
Data successfully read. Data successfully written.
Fitness scores successfully written.
Fitness function concluded.
rm: cannot remove 'runData.csv': No such file or directory
Unable to parse the pattern
^CTraceback (most recent call last):
  File "LRPlot.py", line 99, in <module>
    plotLR(generations, lengths, radii, g.numGens, g.destination)
  File "LRPlot.py", line 45, in plotLR
    plt.show()
  File "/usr/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 143, in show
    _show(*args, **kw)
  File "/usr/lib64/python2.7/site-packages/matplotlib/backend_bases.py", line 108, in __call__
    self.mainloop()
  File "/usr/lib64/python2.7/site-packages/matplotlib/backends/backend_gtk.py", line 83, in mainloop
    gtk.main()
KeyboardInterrupt

2. rm runData.csv error: The following error was printed to consol

rm: cannot remove 'runData.csv': No such file or directory

I found these line in section (E) of the loop.

cd "$WorkingDir"
rm runData.csv
python gensData.py 0
cd Antenna_Performance_Metric
python LRPlot.py "$WorkingDir" "$WorkingDir"/Run_Outputs/$RunName 1 $NPOP
cd ..
# Note: gensData.py floats around in the main dir until it is moved to         

 I think this error is fine. This is how it works: (Step 1) removes runData.csv, if it exists. (Step 2) Runs gensData.py, which creates a new runData.csv. We wanted to delete it before the .py program was run at the start of the run, because gensData.py appends data to runData.csv, if it already exists. (alternatively if runData.csv doesn't exists it creates it-- which is what we want in this case). That's also why we don't delete it later in the "looping part" of XF_Scripts.sh (part G). We want it to append at this point, so we only delete it in the very beginning to make sure it doesn't append to an old run.

3. cp: cannot stat 'Antenna_Performance_Metric/1.uan': No such file or directory (similarly for 2.uan, 3.uan...etc): This error exists just as the fitness score is being calculated -- ie part E. Here is the print:

cp: cannot stat 'Antenna_Performance_Metric/1.uan': No such file or directory
cp: cannot stat 'Antenna_Performance_Metric/2.uan': No such file or directory
Congrats on getting a fitness score!
  File "FScorePlot.py", line 1
    -*- coding: utf-8 -*-
    ^
IndentationError: unexpected indent

Note that we have a few errors here. Right now I am directly addressing the error with the "cp" command. It looks like these files are actually being saved as gen_individual#.uan. For example 0_1.uan. The current line in part E says:

for i in `seq 1 $NPOP`

do

    #Remove if plotting software doesnt need                                                                                                                                                             

    #cp data/$i.uan ${i}uan.csv                                                                                                                                                                          

    cp Antenna_Performance_Metric/$i.uan "$WorkingDir"/Run_Outputs/$RunName/${i}.uan

done

However, this is not how these files are being saved. I corrected the first iteration of the loop (ie Part E for generation 0) by editing the line to say:

for i in `seq 1 $NPOP`

do

    #Remove if plotting software doesnt need                                                                                                                                                             

    #cp data/$i.uan ${i}uan.csv                                                                                                                                                                          

    cp Antenna_Performance_Metric/0_$i.uan "$WorkingDir"/Run_Outputs/$RunName/0_${i}.uan

done

And for the looping part of XF_Loop.sh (ie part G) it has been edited to say:

for i in `seq 1 $NPOP`

        do

    # Gens data used to create a .csv file for the uan file for gain plotting                                                                                                                            

    # cp Antenna_Performance_Metric/$i.uan ${i}uan.csv                                                                                                                                                   

                cp Antenna_Performance_Metric/${gen}_${i}.uan "$WorkingDir"/Run_Outputs/$RunName/${gen}_${i}.uan

        done

4. Made Number of Neutrinos thrown (NNU) in AraSim's setup.txt file a variable: I've added the following line in part D -- as well as the looping section in part G. 

These lines replace the number of neutrinos thrown in our setup.txt AraSim file with what ever number you assigned NNT at the top of XF_Loop.sh in the "variables" section. "setup_dummy.txt" is a copy of setup.txt that has  NNU=num_nnu instead of a number. NNU is the number of neutrinos thrown (as seen in setup.txt). This "sed" line finds every instance of num_nnu in setup_dummy.txt and replaces it with $NNT (the number you assigned NNT in the variable section at the top of XF_Loop.sh). It then pipes this information into setup.txt (and overwrites the last setup.txt file allowing the number of neutrinos thrown to be as variable changed at the top of this script instead of manually changing it in setup.txt each time. Command works the following way: sed "s/oldword/newwordReplacingOldword/" path/to/filewiththisword.txt > path/to/fileWeAreOverwriting.txt. Both setup.txt and setup_dummy.txt can be found in the directory: /fs/project/PAS0654/BiconeEvolutionOSC/AraSim/

cd "$AraSimExec"

for i in `seq 1 $NPOP`

do

#This next line replaces the number of neutrinos thrown in our setup.txt AraSim file with what ever number you assigned NNT at the top of this program. setup_dummy.txt is a copy of setup.txt that has \

NNU=num_nnu (NNU is the number of neutrinos thrown. This line finds every instance of num_nnu in setup_dummy.txt and replaces it with $NNT (the number you assigned NNT above). It then pipes this infor\

mation into setup.txt (and overwrites the last setup.txt file allowing the number of neutrinos thrown to be as variable changed at the top of this script instead of manually changing it in setup.txt e\

ach time. Command works the following way: sed "s/oldword/newwordReplacingOldword/" path/to/filewiththisword.txt > path/to/fileWeAreOverwriting.txt                                                      

        sed "s/num_nnu/$NNT/" /fs/project/PAS0654/BiconeEvolutionOSC/AraSim/setup_dummy.txt > /fs/project/PAS0654/BiconeEvolutionOSC/AraSim/setup.txt

        #We will want to call a job here to do what this AraSim call is doing so it can run in parallel                                                                                                  

        cd $WorkingDir

        qsub -v num=$i AraSimCall.sh

        rm outputs/*.root

done

5. rm cannot remove 'simulation_PEC.xmacro': No such file or directory error appearing in generations after the first (ie not seen in gen 0, but seen in gen 1+).

Statistics initialized.
Tournament parents selected.
Tournament complete.
rm: cannot remove 'simulation_PEC.xmacro': No such file or directory

I found that it tries to remove it twice. See here:

########  Loop (G)  ######################################################################################                                                                                                                                                                                                                                                                                        

#     1. Does steps A-F for each generation                                                                                                                                                                                                    

###########################################################################################################                                                                                              

for gen in `seq 1 $TotalGens`

do

# used for fixed number of generations                                                                                                                                                                   

#gen=0; while [ `cat highfive.txt` -eq 0 ]; do (( gen++ ))                                                                                                                                               

# use for runs until convergence                                                                                                                                                                         

#should the line below be double indented?                                                                                                                                                               

        read -p "Starting generation ${gen}. Press any key to continue... " -n1 -s

#Part A                                                                                                                                                                                                  

        cd $XmacrosDir

        rm simulation_PEC.xmacro                                               

        cd "$WorkingDir"

        ./roulette_algorithm.exe cont $NPOP

        cp generationDNA.csv Run_Outputs/$RunName/${gen}_generationDNA.csv

    ##chmod -R 777 $WorkingDir                                                                                                                                                                           

#we can make a function that does this with the flag $gen                                                                                                                                                

#doing this in part A would mean setting $gen=0                                                                                                                                                          

#Part B                                                                                                                                                                                                  

                # First, remove the old .xmacro files                                                                                                                                                    

        #when do that, we end up making the files only readable; we should just overwrite them                                                                                                           

        #alternatively, we can just set them as rwe when the script makes them                                                                                                                           

        cd $XmacrosDir

        #I'm commenting out the next two lines. They are written to below (around lines 154-170)                                                                                                         

        #If we keep them this way then the files have restricted permissions                                                                                                                             

        rm output.xmacro

        rm simulation_PEC.xmacro

I have commented the first removal of simulation_PEC.xmacro. If there are any issues with it running, we can go ahead and comment it back in; however, I suspect this is not a necessary line to have added in. A run has not been performed since these edits. So, once it has been tested, we can delete the comments out:

#Part A                                                                                                                                                                                                  

        #I think these next two lines can be deleted. Its repeated again just below...11/30/19 comment by Julie  #cd $XmacrosDir #rm simulation_PEC.xmacro                                               

        cd "$WorkingDir"

        ./roulette_algorithm.exe cont $NPOP

        cp generationDNA.csv Run_Outputs/$RunName/${gen}_generationDNA.csv

6. Python Error from FScorePlot.py: The error can be seen in the line below.        

Congrats on getting a fitness score!

  File "FScorePlot.py", line 1

    -*- coding: utf-8 -*-

    ^

IndentationError: unexpected indent

Congrats on getting some nice plots!

I've commented out this strange line at the start of this python program:   -*- coding: utf-8 -*-. This fixed the immediate error, but threw another one when I tried to run it outside of the loop with the command 

python FScorePlot.py /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/Julie_11_30/ /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/Julie_11_30/ 2

The next error it threw can be seen below. 

Traceback (most recent call last):

  File "FScorePlot.py", line 79, in <module>

    fScores.append(fScorei[2:])

AttributeError: 'numpy.ndarray' object has no attribute 'append'

            This is Alex P. I edited the append() and changed it to np.append(fScores, fScorei[2:]) on line 79 in FScorePlot.py

7. rm: cannot remove 'outputs/*.root': No such file or directory: This error comes up right after the AraSim job is submitted in part D. 

1011007.pitzer-batch.ten.osc.edu
rm: cannot remove 'outputs/*.root': No such file or directory
Waiting for AraSim jobs to finish...

It's definitely coming from the lines:

for i in `seq 1 $NPOP`
do        #We will want to call a job here to do what this AraSim call is doing so it can run in parallel
        cd $WorkingDir
        qsub -v num=$i AraSimCall.sh        rm outputs/*.rootdone

We are currently in $WorkingDir at this point-- which is: WorkingDir= /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop. There is not directory called "outputs" in this location. This is why we are getting an error.Can someone explain what .root files we are trying to remove, and why? I cannot proceed unless someone makes this clear to me, as I do not know what files we are looking to remove (and where they exist/when they are created).

Immediately after that, I get the following error:

Waiting for AraSim jobs to finish...
rm: remove write-protected regular file 'AraSimFlags/1.txt'? y
rm: remove write-protected regular file 'AraSimFlags/2.txt'? y
mv: cannot stat '*.root': No such file or directory

It looks like it is coming from the following line:

cd "$WorkingDir"/Antenna_Performance_Metric
mv *.root "$WorkingDir/Run_Outputs/$RunName/RootFilesGen0/"

However-- again-- it looks like it's trying to move .root files from the directory: /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Antenna_Performance_Metric and put them into: /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/$RunName/RootFilesGen0/

It can't move these, because they do not exist where it is trying to take them from. Similarly, the question in bold above would help me with this a lot. From what I can see, we are only utilizing an .txt file output from Arasim. Does Arasim output this to .txt, or are we somehow converting from a .root to a .txt? 

 This error has not yet been corrected!!!!       

Cade's comment here:

The .root files live in BiconeEvolutionOSC/AraSim/outputs/ folder. Thus, we were looking at the wrong directory. They were being removed because it is a giant file (GB size) and are created everytime AraSim runs. It just overwrites itself everytime though so it doesnt add up too much. We can have them deleted later if needed. It's our choice on whether we take it out completely, or redirect to delete the .root files in the correct directory.

8. New errors found after running with these corrections: These need to be fixed!

cp: cannot stat ‘Antenna_Performance_Metric/0_1.uan’: No such file or directory

cp: cannot stat ‘Antenna_Performance_Metric/0_2.uan’: No such file or directory

Congrats on getting a fitness score!

Traceback (most recent call last):

  File "FScorePlot.py", line 86, in <module>

    Gen = np.zeros(fScores.shape[0]*fScores.shape[1])

I found the cp error. It looks like I was incorrect in #3 about this cp command for the .uan files. I moved all of the currently existing .uan files into a new directory in

 /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Antenna_Performance_Metric/UAN

With all of them out of the Antenna_Performance_Metric directory, I did a run from scratch to see what it creates. Here's the outcome: So on generation 1, for individual one its going

1_1.uan
1_2.uan
1_3.uan

ie-- > individual_frequencyStep.uan

So once it hits all 60 frequencies for individual 1, it does all 60 for individual 2 by saying:
2_1.uan
2_2.uan
2_3.uan

...etc.This isn't great because it never actually cares about generation number. In section E (so generation 0) we currently have it as:

for i in `seq 1 $NPOP`
do
    #Remove if plotting software doesnt need                                                                                                                                                             
    #cp data/$i.uan ${i}uan.csv                                                                                                                                                                          
    cp Antenna_Performance_Metric/0_$i.uan "$WorkingDir"/Run_Outputs/$RunName/0_${i}.uan
done

and in section G of the looping part (gens 1+) we have:

    for i in `seq 1 $NPOP`
        do
    # Gens data used to create a .csv file for the uan file for gain plotting                                                                                                                            
    # cp Antenna_Performance_Metric/$i.uan ${i}uan.csv                                                                                                                                                   
                cp Antenna_Performance_Metric/${gen}_${i}.uan "$WorkingDir"/Run_Outputs/$RunName/${gen}_${i}.uan
      done

This doesn't seem like it's being output from XF the way we thought it is. I think the easiest way to do this is to edit the xmacros to have it output like:

generation_individual_frequency.uan

and thus

cp Antenna_Performance_Metric/${gen}_${i}_${frequency}.uan

 If so, we'd have to make a new variable in the bash script for $frequency to go from 1-60, and have it loop moving all of those. I will be looking into this with Cade tomorrow 12/3/19 (since he is the xmacro script expert). Stay tuned!

Phew....thanks for sticking with me on this long update.

​Today I spent some time looking at task 1 from the update here: http://radiorm.physics.ohio-state.edu/elog/GENETIS/20. Below are my updates. 

Task-- Fix error at fitness function: 

Currently, the loop stops at the fitness function calculation, and aborts (program fitnessFunction_ARA.cpp). The following are the things I have tried.

1.Run this outside of the loop: Julie and Alex did this 11/19/19. It gave a segmentation fault. UPDATE: HERE'S WHAT I DID TO CHECK THIS TODAY:

(a) Checked using cout: I used some cout commands to print sections looking for errors. It seems we were not properly giving it arguments when running it outside of the loop. Since it's running outside of the loop, these variables are not initialized. Thus, to run you need to type: ./fitnessFunction.exe $NPOP $AraOut_i.txt

• For example: ./fitnessFunction.exe 1 AraOut_1.txt

Status: The output file is verified! Running this way does properly generate our output file named fitnessScore.csv. I've double-checked this file, and it is properly being filled. At the moment it is giving a Veff of 0, thus no valid fitness score; however, this should be fixed automatically once we get real outputs from AraSim without a Veff of 0. 

 2. Check file location: See if fitnessScore.csv is being located in the correct place when the genetic algorithm initializes the next generation. 

(a) Searched bash script for the final location of fitnessScores.cvs: 

In lines 298, 299 of the bash script, it is doing the following 

cp fitnessScores.csv "$WorkingDir"/Run_Outputs/$RunName/0_fitnessScores.csv

mv fitnessScores.csv "$WorkingDir"

This is taking the .csv output generated from fitnessFunction_ARA.cpp, and is moving it to the run name directory and renaming it based on the generation -- ie first gen is 0_fitnessScores.csv-- so that data isn't overwritten. Then, for some reason, it moves the original copy -- ie fitnessScores.csv-- to the location /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop

In line 374 we then run our genetic algorithm again. 

./roulette_algorithm.exe cont $NPOP

This does not seem to take in the input file as an argument.

(b) Checked roulette_algorithm.cpp for the location it accesses fitnessScores.csv:

I checked if it was hard-coded into the roulette_algorithm.cpp script. I found the following lines.

 // Now we need to read the fitness scores:                                                                                                                                                    

        ifstream fitnessScores;

        fitnessScores.open("fitnessScores.csv");

        string fitnessRead[NPOP+2];

        for(int i=0;i<(NPOP+2);i++)

    {

                getline(fitnessScores,fitnessRead[i]);

                if(i>=2)

                {

                        fitness[i-2] = atof(fitnessRead[i].c_str());

                        if (fitness[i-2]<0)

                        {

                                fitness[i-2] = 0; // If the fitness score is less than 0, we set it to 0 to not throw things off.                                                                     

                        }

                }

    }

        fitnessScores.close();

}

This means that our script roulette_algorithm.cpp is grabbing the fitnessScores.csv from its current directory-- which is accurate due to the line (mv fitnessScores.csv "$WorkingDir") referenced above in the bash script. Something we could verify here is to make sure this function is properly working by having it cout elements from the fitnessScores.csv within this function and to run the roulette algorithm separately. We could do this now, however, we don't have real outputs in fitnessScores.csv anyway, since we are getting a veff=0 for AraSim temporarily. 

In the meantime, I've added the following line after section E (the section that runs fitnessFunction_ARA.cpp, and generates/moves/copies fitnessScore.csv):

echo 'Congrats on getting a fitness score!'

This should check to be sure that it runs until after this point. I've also added the following line after section E (section that plots the fitness score versus gen):

echo 'Congrats on getting some nice plots!'

This should check to be sure that it runs until after this point. Note that I have added these lines to both our original bash script, and the one Alex M. is editing to turn every section into a function. I have added these lines to the .sh scripts containing the functions to be sure the function scripts for these tasks ran completely.  Once we have real AraSim outputs, we should be able to create some cout lines in the instructions highlighted in yellow above. 

3. Look at how it's calculating the fitness score.

I have not checked this yet. However, I do not think it is likely this is the issue. I will check later though. 

Attendance for today's session: Cade, Julie, Alex M, Mitchell, Scott. 

Tasks/to-do list:

1. ​Get loop to run up to the fitness function.
   • Right now we have the bash script setup to run AraSim by submitting them as parallel jobs. Today Cade added the "wait" function within the job in order to make sure the loop doesn't continue to the next step before the Arasim data is ready. Cade will give us an update on Thursday letting us know whether or not this completed or gave an error. 
2. Fix error at fitness function. 
   • Currently, the loop stops at the fitness function calculation, and aborts (program fitnessFunction_ARA.cpp). We are unsure of why at the moment. Here are the things we are looking into. 
     • Run this outside of the loop-- ie compile the fitnessFunction_ARA.cpp (g++ -std=c++11 fitnessFunction_ARA.cpp -o fitnessFunction.exe), and run the executable(./fitnessFunction.exe). What happens?
       • Julie and Alex did this. It gave a segmentation fault. We need to look into this further. 
       • We double-checked that this .exe was properly created from compiling, and it was.
     • Double-check our AraSim output by checking:
       • Format
       • File type
     • Is there an error past the fitness function, and it's killing the loop after this runs?
       • Is his output not properly going into the gen alg?
       • Are the output files properly:
         • Filled
         • Formatted
         • Put in the right place
     • Look at how it's calculating the fitness score.

Since it seems like Alex and I may have found a problem at our first "test", this could be the issue. We will investigate it further. If (1) Cade's "wait" function works, and (2) we can fix this issue with the loop stopping at the fitness score calculation, ideally this means that the loop is then running. 

Today we also found that it is better to run on the terminal through an interactive job than a virtual desktop. This is MUCH faster -- ie we don't have to wait nearly as long as we do to get a node via a virtual desktop. This means that everyone with a PC needs to either (1) work out their issues with X11 forwarding crashing (Mitchell succeeded at this today by making sure Xquartz was set up to work via ssh to Pitzer in the settings), or (2) dual boot Linux. It seems that Mac users are not having issues, but that some of the Windows users are having their OS completely crash when XF GUI tries to forward. Users can test whether or not their X11 forwarding is working by doing the following:

1.  Logging onto Pitzer
2. Typing "module load xfdtd"
3. Then typing "xfdtd"

This should successfully open the GUI for XF. If it crashes, users may have to do one of the things I mentioned above. Once we do succeed at having this setup, you can start an interactive job via the command: 

qsub -I -X -l walltime=1:00:00 -l nodes=1:ppn=10

You then have to wait until the node is free. This has taken about 1/3 of the time to get than when requesting a virtual desktop. Note that, however, we may need a higher ppn (maybe something like 40). This is not totally clear yet. 

Attendance for today's session: Cade, Julie, Alex M, Mitchell, Scott. 

Tasks worked on:

• Continue to fix continuity of loop: Cade and Julie worked on fixing the following errors:
  1. In section D of the loop, it currently
     • Moves the output from the python conversion to $AraSimExec directory. 
     • Renames it as a .txt file (note the $i is renaming each in line with $NPOP). 

for i in `seq 1 $NPOP`

do

      mv evol_antenna_model_$i.dat $AraSimExec

done

for i in `seq 1 $NPOP`

do

      cp evol_antenna_model_$i.dat a_$i.txt

done

However, this wasn't working and it was only properly renaming individual 1. To check this,

• We confirmed it is properly iterating the first loop by using the "echo" command. The second loop was not properly iterating, however.
• We were able to fix it by combining these steps. This was confirmed to have worked. 

for i in `seq 1 $NPOP`

do

      mv evol_antenna_model_$i.dat $AraSimExec/a_$i.txt

done

              2. We got this iteration $i to work in the job script -- ie as an input file variable. The loop now successfully submits AraSim as a job with the proper input files. 

• Clean bash script
  • Mitchell and Alex M worked again today on turning every function possible into functions (and putting them in their own .sh scripts that are being called by XF_Loop.sh). 
    • The loop runs the first generation "manually" -- ie each step was spelled out, and nothing is put into a loop. The second+ generations then run as a loop. This is because we need to initialize the first XF run -- which then has no problems in the later iterations.
      • Alex M. and Mitchell are now turning each section A-F into individual functions and put into their own .sh scripts. Thus section A would just be calling another .sh script. In gen 2+ we then loop over these SectionA.sh through SectionF.sh functions. All of these just got completed. He just needs to add the change Julie and Cade made today, and clean some things up. 
        • For example, he has an "if else" statement that tells it whether or not it's the first (XF initializes) generation, or in generation 2+. Cade has a few changes he would like Alex to make to clean up this part of the run. 
    • Idea: The loop used to run the first generation "manually" -- ie each step was spelled out, and nothing was put into a loop. The second+ generations then ran as a loop. Since we were able to get XF to import the xmacro scripts automatically -- as well as save the project with the correct name (matching the expected name initialize in the bash script)-- they've gotten rid of that first "manual" iteration, and are now fully running all generations as a loop. We haven't tested whether or not we can make this change yet, but it is worth looking into now that some changes have been successfully made with XF. 

Quick note from Julie:

I tried to run from the terminal once Cade left. I get the following output/error that we should make sure isn't happening for other users (since most of this is a permissions issue that I thought we fixed. I can look into the other errors and see if anyone else gets them). For whatever reason, Cade is not having the same output errors; he is able to run the same copy just fine. 

Roulette algorithm initialized.

./XF_Loop.sh: line 136: output.xmacro: Permission denied

./XF_Loop.sh: line 137: output.xmacro: Permission denied

./XF_Loop.sh: line 138: output.xmacro: Permission denied

./XF_Loop.sh: line 139: output.xmacro: Permission denied

./XF_Loop.sh: line 140: output.xmacro: Permission denied

./XF_Loop.sh: line 146: simulation_PEC.xmacro: Permission denied

./XF_Loop.sh: line 147: simulation_PEC.xmacro: Permission denied

./XF_Loop.sh: line 149: simulation_PEC.xmacro: Permission denied

./XF_Loop.sh: line 158: simulation_PEC.xmacro: Permission denied

 

Opening XF user interface...

*** Please remember to save the project with the same name as RunName! ***

 

1. Import and run simulation_PEC.xmacro

2. Import and run output.xmacro

3. Close XF

Lmod has detected the following error:  The following module(s) are unknown: "xfdtd"

 

Please check the spelling or version number. Also try "module spider ..."

It is also possible your cache file is out-of-date; it may help to try:

  $ module --ignore-cache load "xfdtd"

Also make sure that all modulefiles written in TCL start with the string #%Module

./XF_Loop.sh: line 172: xfdtd: command not found

./XF_Loop.sh: line 177: cd: /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/Julie/Julie.xf/Simulations/000001/Run0001/: No such file or directory

./XF_Loop.sh: line 178: xfsolver: command not found

./XF_Loop.sh: line 177: cd: /fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Run_Outputs/Julie/Julie.xf/Simulations/000002/Run0001/: No such file or directory

./XF_Loop.sh: line 178: xfsolver: command not found

./XF_Loop.sh: line 183: xfdtd: command not found

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/AraSimTestJob.sh’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Antenna_Performance_Metric/AraOut_%num%.txt’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Functions/XF_Loop3.sh’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Functions/#Part_C.sh#’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Functions/#Part_A.sh#’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957233’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957230’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957234’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957224’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957220’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957220’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957234’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957227’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957233’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957218’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957224’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957231’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957226’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957221’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957223’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957217’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957227’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957218’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957223’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957225’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957230’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957225’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957222’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957231’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957217’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957222’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957226’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957221’: Operation not permitted

 

Resuming...

 

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/AraSimTestJob.sh’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Antenna_Performance_Metric/AraOut_%num%.txt’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Functions/XF_Loop3.sh’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Functions/#Part_C.sh#’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Functions/#Part_A.sh#’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957233’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957230’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957234’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957224’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957220’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957220’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957234’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957227’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957233’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957218’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957224’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957231’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957226’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957221’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957223’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957217’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957227’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957218’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957223’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957225’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957230’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957225’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957222’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957231’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957217’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957222’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957226’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957221’: Operation not permitted

Resuming...

NOTE:  You may want to change the following in your job for Owens

 

    nodes=1:ppn=1 defaulting to mem=4571MB

 

Please review the Owens documentation:

    https://www.osc.edu/supercomputing/computing/owens

 

Feel free to contact oschelp@osc.edu if you have any questions.

 

8485416.owens-batch.ten.osc.edu

rm: cannot remove ‘outputs/*.root’: No such file or directory

NOTE:  You may want to change the following in your job for Owens

 

    nodes=1:ppn=1 defaulting to mem=4571MB

 

Please review the Owens documentation:

    https://www.osc.edu/supercomputing/computing/owens

 

Feel free to contact oschelp@osc.edu if you have any questions.

 

8485417.owens-batch.ten.osc.edu

rm: cannot remove ‘outputs/*.root’: No such file or directory

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/AraSimTestJob.sh’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Antenna_Performance_Metric/AraOut_%num%.txt’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Functions/XF_Loop3.sh’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Functions/#Part_C.sh#’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/Loop_Functions/#Part_A.sh#’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957233’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957230’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957234’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957224’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957220’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957220’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957234’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957227’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957233’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957218’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957224’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957231’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957226’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957221’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957223’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957217’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957227’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957218’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957223’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957225’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957230’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957225’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957222’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957231’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957217’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957222’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.o957226’: Operation not permitted

chmod: changing permissions of ‘/fs/project/PAS0654/BiconeEvolutionOSC/BiconeEvolution/current_antenna_evo_build/XF_Loop/Evolutionary_Loop/scriptEOFiles/AraSimCall.sh.e957221’: Operation not permitted

mv: cannot stat ‘*.root’: No such file or directory

Fitness function initialized.

0 

Data successfully read. Data successfully written.

Attendance for today's session: Cade, Julie, Alex M, Alex P, Mitchell, Scott. 

Monday was a holiday, and we will be replaceing that meeting later this week. Today we worked on the following tasks:

Tasks:

• Check continuity of loop
  • Alex M will try another run tonight, and has already requested a virtual desktop. 
• Install Arasim to /fs/projects/PAS0654/BiconeEvolutionOCS to be edited and messed with
  • Cade is testing whether or not this is running within the loop. He had gotten a few errors, and it is not completing. It's submitting the job with the new bash script, but it is not actually running.
    • Found an error. The bash script was not grabbing the correct AraSim directory. More updates to come. 
• Clean bash script
  • Mitchell and Alex M are putting all current functions hard-coded in into their own .sh scrips that are now being called in the main bash script.
• Update Manual
  • Cade and I are updating the manual with new changes. We have officially retired the old version of the manual that was used for runs with Nutau. We have saved that version and uploaded it to the elog here: http://radiorm.physics.ohio-state.edu/elog/GENETIS/17

Right now Elliot and Leo have decided to focus on working with Keith. They will not be contributing to this project for the remainder of the semester. Students have been posting individual updates here: https://www.dropbox.com/home/GP_Antennas/Updates

I've also attached the meeting minutes from the call last week, in order to keep track of updates. 

We are off again with a new team. The new team consists of Alex Machtay, Alex Patton, Cade Sbrocco, Eliot Ferstl, Evelyn Shank, Mitchell Halley, Scott Janse. A spreadsheet of tasks and deadlines can be found here: https://docs.google.com/spreadsheets/d/1CkcdbediFbripdlwPw6UmhAHdAl5IdkL2efaseJkTTM/edit#gid=0

Updates:

We are now running on OSC. We have acquired a version of XF on OSC and have now transferred the loop to OSC. All of the xmacros for XF should be functioning, and we have managed to make the bash script auto import the xmacro scripts to XF when it boots. The loop exists on /fs/projects/PAS0654/BiconeEvolutionOCS. This is a shared copy so that everyone can run off of the same installation. This is important so that we can make sure everyone is pushing edits regularly, and so that everyone is always running with the most up-to-date edits. 

The loop had not yet been tested for it's current status. Alex M. is testing it and we are waiting to hear back. 

Tasks:

• Check continuity of loop
  • Update: Alex M is testing this
• Install Arasim to /fs/projects/PAS0654/BiconeEvolutionOCS to be edited and messed with
  • Update: this has been completed as of 11/5/19
• Test Alex M's code for editing Arasim's Setup.txt variables
• Clean bash script
  • Update: Mitchell is working on this.
• Make version of bash script which writes .job scripts for Arasim
  • This must write a different .job for each individual so it parallelizes Arasim running for each individual
  • The number of .jobs written must depend on $NPOP
    • ie this bash script should be writing .jobs from scratch each time as (1) the number of them we need depends on $NPOP, (2) the names of the input and output files depends on $NPOP. 
      • It should also delete .jobs at the end -- ie when all generations have completed so that we don't have a random number of .jobs floating around. For example, say we run with 10 individuals one time, and it writes 10 .jobs ($NPOP=10), and then the next run we only have $NPOP=5, we will have 5 extra .jobs floating around not being used unless we clean them all at the end line of the run in the bash script. 
  • Update: Work on this has begun by Alex M, Cade, and Evelyn (Alex P has also been messing with Arasim).
• Make XF be submitted as a job
  • Cade has ideas for this.

Students have been posting individual updates here: https://www.dropbox.com/home/GP_Antennas/Updates

The following are links to the 2018 and 2018 APS slides.

2018 APS talk:

https://www.dropbox.com/home/GP_Antennas/APS%20Talk?preview=April+APS+Talk_final.pdf

2019 APS talk:

https://absuploads.aps.org/presentation.cfm?pid=15183

We ran valgrind (a profiler) on AraSim to find out where it was spending most of its time, we ran it with callgrind to give an output file of the time and opened that file with qcashegrind to display it in a legible format. This let us know that the function Param_RE_Tterm_approx was taking up about 75% of the time because it was being called hundreds of millions of times. Now more specifically the standard math functions pow() and exp() were being called in this function every time and took up about 44% and 20% respectively. They were called about the same number of times as Param_RE_Tterm_approx but they are only called in the else block in an if-else statement.

JULY 31st Update:

I used the C++ library future and set up iterations of a loop to run asynchronously as they do not rely on each other's calculations. This launches a seperate thread whenever I call one. The struggle with this is that launching a thread and getting its results takes time as well, so there had to be enough calulations in a thread in order for it to make up any time. The purpose of the function GetVm_FarField_Tarray is to create arrays of doubles called Earray and Tarray and those are the only values that get altered in this function. Tarray is created with simple calculations so I left that out of threads but making Earray calls the other function that takes up so much time.

I edited the files signal.cc and signal.hh to work with 4 threads and ran 10,000 neutrinos with 4 cores. The cpu time was 1:48:41 but the real walltime it used only took 00:54:40. The time these tests for 10,000 neutrinos with base AraSim normally take anywhere from 1:35:00 to 1:45:00 or in that range. It should still work with less threads available it would just run slower and probably at the cpu time, which might be slightly slower than base AraSim but as soon as more threads are available it runs much faster as we see here. I still need to check for the accuracy of these edits, the number passed is in a reasonable range, but I will need to investigate further by giving the exact same neutrinos to both programs and making sure their outputs are identical.

I also have some further ideas to speed it up even more. Currently every iteration of the loop calls a thread but what would be faster would be to either

A: Set it up so it can call more threads at a time and thus go faster, which would be an easy edit but only work if more cores are given

B: Set it up so the current threads run more than one iteration at a time, I've started experimenting with this but having trouble making sure that everything involving pointers stays safe, but the benefit of this is that the threads would get called and joined significantly less and would cause a great speed increase without adding more threads.

All these developments are new after me spending a while trying many different ways to speed it up or run parts in parallel like this, so even if this isn't accurate or valid I believe I would be able to make it so with more time and work.

Attached I have a zip that contains the two files I edited as well as an output file from my first test. I will continue to test and improve this but am unable to work on this again until August 19th so I wanted to share my current status before I am unavailable.

Right now the following people are working on this project:

Julie Rolla, Cade Sbrocco, Scott Janse, Alex Patton, Eliot Ferstl, Abassi Brown, Evelyn Shank, Sophie Dunlap

Today we met for our first working meeting. Scott, Alex, and Eliot were asked to speed up and optimize AraSim. Sophie and Evelyn are working on editing the .csv file with the beam patterns coming out of XF. Currently it's written in a different format than AraSim takes in. Abassi is helping Evelyn and Sophie by giving them a sample program helping reorganize the cells in a .csv file. Since there's confusion on how to optimize AraSim, we've started dissecting the software so we can understand how it works. The following are questions on the setup.txt document. 

Setup.txt

What is "timestep"? Could we increase the timestep? Is this timestep big or small compared to 2 GHz?

Answer: Timestep is the unit of discrete time the computer operates at. The timestep by default is set to 2 GHz, or 5 * 10^-10 seconds. We might be able to make up some cheap time by lowering that but would have to check if it makes the data less accurate.

What is "TRIG_WINDOW"? Is this a "coincidence"? Meaning if this signal is picked up within a certain time between stations it's considered the same event?

What is "V_MIMIC_MODE"? Comments in Setup.CC says "// default : 0 - write out all chs where global triggered bin is center of the window"

Answer: Not sure exactly what it does but we can't change it from default as the other values are for the test bed mode only.

Random Question from Setup.txt, but relevant for other things. What is "testbed mode"? Answer: The "testbed" is the original test station only 40m under. This mode is for if we do detector=3 in the first part of the setup.txt file; thus, we won't use this. 

What's the difference between "DETECTOR" and "DETECTOR_SIMULATION"? We found "DETECTOR_STATION" in Settings.h. The comments say "// for DETECTOR=4, indicates the single station to be simulated // 0 = testbed, 1 = A1, 2 = A2, 3 = A3"

In this file, there's an ANTENNA_MODE variable. You can look at what it does in Settings.h. In this file, it says that you can decide whether to use a bicone antenna response or "a different response for the top vpol antennas". How does this function work? What does it mean by "top vpol antennas"? Someone should look up the ARA antennas, and look into this function. 

Our goal is to meet and work as a group twice a week. Collaboration in work should help us progress faster. 

We have 2 physical XF flash drive keys in existence. These are non-replaceable and cannot be lost! Until further notice, the keys are held by:

(1) Cade Sbrocco (sbrocco.6@buckeyemail.osu.edu )

and

(2) Abasi Brown (brown.7146@buckeyemail.osu.edu)

                                                        General update

The following were the fixes we made to make AraSim run:

(1) As of March, the loop breaks at AraSim. We have found that our program XFintoARA.py does not properly format the output in the form that AraSim is looking for (as an input). Jorge gave us a sample AraSim input file to help us edit XFintoARA.py so that it formats the file correctly. Note that we found white space is also must match in order for the format to be correct. 

Now that we have fixed this, AraSim now runs. Attached is the following sample Jorge gave us of the proper AraSim inputs. If needed, it's available to refer back to. 

(2) After correcting the AraSim input file formats (outputs of XFintoARA.py), we ran it. It failed to finish running after hours of run time. We found that we were using default input parameters for AraSim -- which wasn't triggering in a timely fashion. Once we edited the parameters, it ran successfully. Note that this was not run within the loop. Confirmation of the full loop running has not been completed. 

The parameters of interest need to be edited in setup.txt. To see which file in AraSim I am referring to, please see the .txt file attached. Note that we will be editing the bash script to allow the user to change these input parameters there. The following red lines are what need to be set:

EXPONENT=21
NNU=100 // number of neutrino events
NNU_PASSED=10 // number of neutrino events that are allowed to pass the trigger
ONLY_PASSED_EVENTS=1 // 0 (default): AraSim throws NNU events whether or not they pass; 1: AraSim throws events until the number of events that pass the trigger is equal to NNU_PASSED (WARNING: may cause long run times if reasonable values are not chosen)
NOISE_WAVEFORM_GENERATE_MODE=0 // generate new noise waveforms for each events
NOISE_EVENTS=16 // number of pure noise waveforms
TRIG_ANALYSIS_MODE=0 // 0 = signal + noise, 1 = signal only, 2 = noise only
DETECTOR=1 // ARA stations 1 to 7
NOFZ=1
core_x=10000
core_y=10000

EXPONENT gives the energy of the neutrinos. Note that higher energies will have more pass, as the energies don't dissipate as rapidly. For now, we are running at this energy. Eventally, we will want to use the full Kotera spectrum. Otherwsie, we would be finding the performance of an antenna at only one energy. Just to check the loop, we are going to use EXPONENT=21 to get events to trigger faster. 

ONLY_PASSED_EVENTS decides which of the two options to use to determine when AraSim is finished running: (1) Throws 'NNU' number of events at antennas and stops regardless of how many pass (in this case ONLY_PASSED_EVENTS=0) , or (2) throws as many events at the antenna as needed until 'NNU_PASSED' number triggers (in this case ONLY_PASSED_EVENTS=1). Ie if this number is 10, we throw as many neutrinos at the antenna as necessary until 10 events trigger. Note that this can result in a longer run time. Additionally, a higher EXPONENT will make them trigger faster. 

TRIG_ANALYSIS_MODE is a setting that determines what is being thrown. Ie- 0=signal + noise (which is what we want, and is the 'real' case), 1=signal only (would make it run faster), 2=noise only (not of interest for us). 

                                                                To-Do List

(1) Commit version of XFintoARA.py since recent formatting change. 

(2) Figure out how to add in Kotera spectrum (note that we wont want to do this until the WHOLE loop is functional)

(3) Edit the bash script to recreat the setup.txt making the variables in red above an editable option in the top of the bash script. Ie our variables in red above will be at the top of the bash script. Later in the bash script we will need to have it create this .txt file with the user inputs for those variables. We can do this by either (1) creating a skeleton, or (2) creating another .sh script that is run in our XFLoop.sh -- which writes the first handfull of lines based on the user inputs for our variables in red above, and then pastes in the rest of what is unchanged in setup.txt. 

(4) Run the whole loop: update- we tried this today and keep getting a Veff-0 for our first antenna along with the error 

Warning in <TFile::Init>: no StreamerInfo found in /datapool/software/anita/lib/G__RootFftwWrapper_rdict.pcm therefore preventing schema evolution when reading this file.
Error in <TStreamerInfo::Build>: AraEventCalibrator, discarding: unsigned short* fAtriPeds, no [dimension]
others run as expected

(5) Work on the paper: Amy added a skeleton of sections to add for the GA. Date of completion should be in-line with APS. 

(6) APS talk slides. 

(7) Edit the manual and add: 

• The changes that will occur when task (3) is done.
• A sample of how AraSim inputs -- or XFintoARA.py outputs -- should look. 
• This stuff: http://radiorm.physics.ohio-state.edu/elog/GENETIS/8
• Max's AraSim plotting software (when it's done)
• Evelyn and Sophie's plotting software info (when it's done). 

(8) Finish Max's AraSim plotting software.

(9) Finish Evenlyn and Sophie's plotting softare.