ELOG GENETIS

Monday Updates

Alex M	Kicked the loop back up. Helped Ethan and Parker with their projects during the working meeting.
Ryan	Changed the tournament/roulette ratio, reproduction_no, and crossover_no to be read in variables in the GA to increase the quality of life when running the algorithm and to prep the code for some testing. Format for how to call the code is written at the top of the cpp program file.
Ethan F	Worked with Alex M on further fixing AREA. The first generation works now, we are making minor fixes to get subsequent generations up and running.

Mon Nov 30 16:59:39 2020

Daily Update 10/30

Name	Progress	Plans
Alex M
Alex P
Ryan	Attempted to make a new mutation function for the algorithm to try and address the concerns about hitting local maximums from Wednesday. Unfortunately, the idea was unsuccessful when I put it through testing. I would like some input from some of the experts before trying something else. Otherwise, the version I had earlier this week has still been very consistent about optimizing the runs outside of sometimes hitting a local max at about 90/100.
Ben
Ethan	Moved AREA onto my own directory with Alex's help. Began fixing issues with it (small win, no more permissions issues!).	Continue fixing issues with AraSim on my user.
Parker
Elliot
Leo
Evelyn

120

Mon Nov 23 18:02:40 2020

Ryan Debolt

Monday Updates

Alex M	Kept working with Amy, Alex P, Julie, and Ben on the AraSim fix. We fixed our issue from last week but have a new one in stage 2. It looks like the issue has to do with resetting the values for V_forfft right before stage 2 (around line 963). Check here for the current version of Report.cc: /users/PAS0654/pattonalexo/EFieldProject/11_23_20
Ryan	Fixed the issue with the segmentation faults in the GA (simple fix one line changed) and worked with Kai to start creating a spreadsheet of results. Results seem to suggest that 2 tournament to 8 roulette seems to be the ideal selection ratio and the initial test seem to suggest a small amount of reproduction is ideal but more testing is needed to confirm this.

119

Fri Nov 20 17:50:05 2020

Working Meeting 11/20/20

Alex M	We worked with Amy to track down the error we were getting with the AraSim fix. Alex P and I had gotten AraSim to compile, but running gave the error (at the bottom). We did a binary search for the source of the error and determined it comes from the if statement around line 640 in Report.cc: if ( event->Nu_Interaction[0].LQ > 0 && (fabs(viewangle-signal->CHANGLE_ICE)<=settings1->OFFCONE_LIMIT*RADDEG) ) {
Ryan	Kept testing different rations of tournament and roulette. A trend seems to appear that while full roulette was the worst run overall, it rises until a ratio of about 8R:2T with a max score of 99.22% match and then it slowly falls from that point. Need to do more testing to verify averages and to fix strange segmentation faults in crossover when running 6T and 8T ratio.

AraSim EField project Error:

*** Break *** segmentation violation

===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0 0x00002b845aeb445c in waitpid () from /lib64/libc.so.6
#1 0x00002b845ae31f52 in do_system () from /lib64/libc.so.6
#2 0x00002b84562bebf4 in TUnixSystem::StackTrace() () from /cvmfs/ara.opensciencegrid.org/trunk/centos7/root_build/lib/libCore.so
#3 0x00002b84562c13ea in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/ara.opensciencegrid.org/trunk/centos7/root_build/lib/libCore.so
#4 <signal handler called>
#5 0x000000000044fd70 in void std::vector<double, std::allocator<double> >::emplace_back<double>(double&&) ()
#6 0x00000000004c3c0e in Report::Connect_Interaction_Detector(Event*, Detector*, RaySolver*, Signal*, IceModel*, Settings*, Trigger*, int) ()
#7 0x0000000000436eb3 in main ()
===========================================================

The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5 0x000000000044fd70 in void std::vector<double, std::allocator<double> >::emplace_back<double>(double&&) ()
#6 0x00000000004c3c0e in Report::Connect_Interaction_Detector(Event*, Detector*, RaySolver*, Signal*, IceModel*, Settings*, Trigger*, int) ()
#7 0x0000000000436eb3 in main ()
===========================================================

118

Thu Nov 19 16:42:06 2020

Daily Update 11/19/20

I hopped on with Alex M to catch him up with what has been done and changed within AraSim so far. I looked into using pointers to access arrays outside of their declared scope and it seemed reletively easy to set up. I set up a pointer corresponding to the arrays V_forfft, T_forfft, and vmmhz_filter and set them at the end of stage one in order to be used in stage two. Also made sure to delete the pointers at the end of stage 2 (Very Important). After this and finding the correct scope to put the deletes in, we complied and didn't get an error in Report.cc. Once it compiled I signed off to study for exam and Alex M will now take over and start to run tests to make sure that with the same seed this gives the same results as a base copy of AraSim. After this the next step would be to start implementing a way to save all the variables from stage one and reading them into stage 2.

117

Fri Oct 30 17:30:16 2020

Daily Update 10/30

Name	Progress	Plans
Alex M
Alex P
Ryan	Attempted to make a new mutation function for the algorithm to try and address the concerns about hitting local maximums from Wednesday. Unfortunately, the idea was unsuccessful when I put it through testing. I would like some input from some of the experts before trying something else. Otherwise, the version I had earlier this week has still been very consistent about optimizing the runs outside of sometimes hitting a local max at about 90/100.
Ben
Ethan	Moved AREA onto my own directory with Alex's help. Began fixing issues with it (small win, no more permissions issues!).	Continue fixing issues with AraSim on my user.
Parker
Elliot
Leo
Evelyn

116

Mon Oct 26 17:59:10 2020

Daily Update 10/26

Name	Progress	Plans
Alex M
Alex P
Ryan	I have the testing loop finished with plotted results now. The program was able to reach optimal results very quickly. It used 100% tournament selection with a cutoff on the outer radius. The algorithm was using an asymmetric algorithm and the ideal bicone it was being compared to was an arbitrarily picked one from an actual symmetric run individual we knew to stay within the outer radius. All individuals for each generation are plotted on this graph. And the fact that these bicones started as asymmetric shows that we can very easily find symmetric answers if they are indeed ideal.
Ben
Ethan	Tried to fix permission issues with AREA, as well as learned about running the loop and listening to Jorge's thoughts on our project.	Possibly move the AREA project onto the project space as it may solve permissions issues. Alex is looking into it.
Parker
Elliot
Leo
Evelyn

Attachment 1: fitness.png

115

Mon Oct 19 17:02:12 2020

Daily Update 10/19

Name	Progress	Plans
Alex M
Alex P
Ryan	Alex P. and I put the outer radius constraint into the Asymmetric our version of the algorithm. I have also created the pseudo-fitness function to be able to do some optimization testing that bypasses the time-consuming parts of the run. All I need to do to finish the pseudo tests is to create a loop to run through the generations and plotting procedures.
Ben
Ethan	Tried to fix permission issues, as well as work on running the loop (invalid id)	Continue fixing AREA permission issues with Ben's help.
Parker
Elliot
Leo
Evelyn

Fri Oct 16 17:32:49 2020

Daily Update 10/16/20

	Update	Plans for Monday
Alex M
Alex P and Ryan	Made new version of our modified GA that is asymetric. We didn't include the options to start with symetetry but just wanted to get it to a compilable state so that we can test how it would run in the loop to make sure it functions properly there. We also have a real run going with our previous symetrical GA edit.	Watch this run over the weekend and keep it going and then eventually test how the asymetric version works
Eliot
Ben
Leo
Evelyn
Ethan	Worked with Ben and Alex M on getting the AREA project working on slurm. It works now, but there are some permissions issues that we still need to fix. Simply put, Ben can run it perfectly right now.	Find and address permissions issues so anyone can run a job.

113

Mon Oct 12 17:34:29 2020

Daily Update 10/12/20

Name	Update	Plans for Week
Alex M
Alex P, Ryan, and Ben	Finished up writing the updated GA and got it compilable and then got some run time segmentation faults but managed to get all of those fixed. But once we got it running we encountered an error where a lot of the generationDNA ended up being 0,0,0. We looked through our functions and confirmed this was only happening with indiviudals developed through crossover and not mutation or reproduction	Get this problem fixed in crossover and continue to test and make sure this works as intended but so far it is able to generate genDNA files and just has that one bug. The bug is also every other individual so we think we might have messed up how crossover generates two individuals from parents so it should hopefully be an easy fix.
Eliot
Leo
Evelyn
Ethan	.

112

Fri Oct 9 17:37:57 2020

Daily Update 10/9/20

Name	Update	Plans for Monday
Alex M
Alex P, Ryan, and Ben	Got all functions finished and worked on adding them to main. We set up their calls and commented out the unfunctionalized code with a comment explaining what it was and what date it was commented out. Had to define new variables for limits on our mutation function now that our mutation function is uniformly distributed within a range rather than using standard deviations. Also created some more global variables for maximum outer radius and minimum length in order to have them more accessible.	Finish up implementing and defining variables so that it can run the functions properly. Hoping to have it ready to test on monday.
Eliot
Leo
Evelyn
Ethan	.

111

Mon Oct 5 21:06:29 2020

Everyone

Data Runs

Machtay_20200831_Asym_Length_and_Angle 10 individuals
Machtay_20200911_Symmetric 10 individuals, fewer neutrinos
Machtay_20200914_Asymmetric_50_Individuals 50 individuals, fewer neutrinos
Machtay_20200929_Asymmetric_test_2 50 individuals, fewer neutrinos, broaden parameter range

Mon Oct 5 17:34:32 2020

Leo Deer

Update Monday 10/5/2020

Name	Update	Plans for Monday
Alex M
Alex P		Finish up mutation function and then make sure it functions properly, currently our added functions all compile but we haven't tested if they are functioning as intended yet. Also have some questions about handling the standard deviations in mustations but first we want to get it functioning before applying more tweaks
Eliot	Worked on learning about 3D plotting in python. Made a simple surface plot that can be expanded upon for our purposes.
Leo	Fixed the averge Fscore Plot so that it connects the mean values with a line instead of making them discrete points. Then I made Kai's plot suggestion. For that I copied the current FScore plotting software and made some edits so that it takes in 2 runs and plots the fitness scores of both in either red or green along with the average fitness score of both. I'll attach images of what these 2 plots look like.	For Friday I'm going to look over the other plotting suggestions but most likely begin on fixing the random seperation in the RLTS plots.
Evelyn
Ryan
Ben
Ethan

Attachment 1: FixedFScore.png

Attachment 2: RedPlot.png

109

Fri Oct 2 17:11:25 2020

Update Friday 10/2/2020

Name	Update	Plans for Monday
Alex M
Alex P
Eliot
Leo	Today I started working on improving the plots. I started with plotting the average fitness score for each generation on top of all the individuals data points. I made the average fitness score a star with higher contrast so it will stand out. Here is the plot of the first 26 generations of the "Machtay_20200914_Asymmetric_50_Individuals" run with the new averages.	On monday I'm going to try and create the plot of 2 runs overlaid on top of eachother as Kai suggested.
Evelyn
Ryan	Worked with Ben and Alex P. on making the Bicone function modular on our copy. We finished writing a new roulette selection function, tournament selection function, and reproduction function.	Finish writing the mutation and crossover functions. After these are complete apply these functions in place of the existing function that is wrong and test to make sure the results are not producing errors.
Ben
Ethan	Updated the AREA code with Alex M's help to work using SLURM. Began a trial run to see if our updates work.	Begin making changes that allow the job to run faster as well as investigate what is causing some individuals to have fitness scores beyond what should be physically possible.

Attachment 1: Screen_Shot_2020-10-02_at_5.39.33_PM.png

108

Mon Sep 14 17:23:26 2020

GENETIS Update 9/14/20

I just wanted to give an update on what I've been doing today for GENETIS. I went ahead and recorded myself running the loop. It should be useful for new people we're onboarding but I also went through a whole generation, which I think is interesting for everyone to understand the timescale of each part of the loop. I'm making a small edit (I have no clue how to edit videos haha) so once I'm done I'll try to send it via email and post it on slack. I also started a long run Amy requested using antennas which are asymmetric in length and opening angle and 50 individuals per generation. Since it has so many individuals it will take a long time to go through each generation. I also had to decrease the number of AraSim jobs (thus increasing the number of neutrinos per job) because otherwise we'd be submitting too many jobs and have a lot of them blocked, holding us back from running. If anyone has problems with me using up too many jobs let me know and I can try to decrease the numer of jobs per individual futher (right now I have 10 jobs per individual with 3000 neutrinos per job, which gives us 500 AraSim jobs per generation).

107

Wed Sep 9 17:28:54 2020

Update 9/9/20

Name	Update	Plans for tomorrow
Alex M	I've been working on readying the loop for the transition to slurm. OSC is switching from its current job managing software (torque/moab) to slurm, which uses some different commands but is functionally similar. Since we have places where jobs are called in the script, I've been changing them (with a backup copy of those files that we know works on torque). Right now I'm testing all of the changes I've made on the space OSC has set aside for playing with slurm. I have changes in all of the places I think they belong, but I need to make sure there aren't typos/different commands from what I've used.	I'll keep testing the changes I've made for slurm. I'll also try to organize some of the dfferences I've found in using slurm vs torque (for example, requesting an interactive job) so that everyone can see them and be able to find out what to use when OSC makes the switch (Pitzer changes on the 22nd, Owens is in November I think).
Alex P
Eliot
Leo
Evelyn
Ryan
Ben
Ethan

106

Mon Aug 31 18:32:21 2020

Daily Update 8/31/20

Name	Update	Plans for tomorrow
Alex M	I started working on the slides for our meeting with Wolfgang this Friday. I'll share it via one drive/email/slack tomorrow for comments from everyone. Over the weekend I finished the asymmetric length run. I still have a couple of fixes to make on plotting scripts, so I'll update those plots tomorrow after making the fixes. I also started a new run today using both an asymmetric length and an asymmetric opening angle. The plots aren't automatically uploading to dropbox anymore because we hit our limit, so I'll have to add them manually. You can see the asymmetric run on dropbox under Machtay_2020_08_27_Asymmetric_Length_Real_Run(SECOND REAL RUN).	I'll finish up and share rthe power point with everyone and then fix up the plotting scripts and continue the current run. I also need to try getting onto the UW super computer in advance of the ARA bootcamp on Wednesday/Thursday.
Alex P
Eliot
Leo
Evelyn
Ryan
Ben
Ethan	Futher worked on documentation of the genetic algorithm. Ben is currently doing a big run to test the program.	Once the big run is finished, we will look at the data and attempt to fix any issues that may arrise.

105

Fri Aug 28 16:31:56 2020

First Results Slides

Here's a power point I showed today at the GENETIS meeting for the two most recent runs we did. The first run was symmetric and we ended it at 15 generations. The second one is ongoing and is shown up through generation 9 on the power point.

Attachment 1: GENETIS_Meeting_Loop_Results_2020_08_24.odp

Fri Aug 21 15:19:10 2020

Ryan Debolt

Friday Updates

Name	Update	Plans for next week
Alex M
Alex P
Eliot
Leo
Evelyn
Ryan	This week I completed work on a set of graphs that shows many instances of of a run type on a graph to show the spread that runs give. I am still working on finishing up the plots that show averages of roulette to tournament for each sigma but the should be finished soon.	Finish the aforementioned plot.
Ben
Ethan	Ben and I spent the week working on two seperate python scripts, one to plot the fitness scores of each child in each generation, and another to replace dummy Veff scores with Veff scores found in another file. We also updated the documentation on OverLeaf.	On monday we will continue to keep updating the documentation as well as continueing to imporve AREA.

Tue Aug 11 19:56:45 2020