For each new student joining the group, we ask them to onboard by completing each of the following items in the order given:

(1) Go to osc.edu and request an account

• https://www.osc.edu/supercomputing/portals/client_portal/self_signup_for_accounts
• Fill out the info and for the Project Code put PAS1960 (and PAS0654 if you're able to do both).

(2) Complete the following tutorials and reading. Note that the order of these does not matter, however, I would suggest "how to use Git" comes last, as it is the lease important for you to start. Finally, I advise you work on a tutorial concurrently with reading the pages assigned below from my Dissertation. 

• OSC tutorials
  • Because this project uses a lot of computation power, we run on the Ohio Super Computing Center's clusters. Please utilize these tutorials to learn more. 
• Read Julie's Dissertation pages 1-9, 12-20 (starting at section 1.5.2), 28-37 (starting at section 2.3)
  • Feel free to read chapter 3, as it has the history of this project and goes into details of our most up-to-date endeavors. This is optional and not mandatory. 
• Complete the introduction to Bash project
• Complete the introduction to Python Project
• How to use Git (optional)
  • Git is a resource used for version control when writing software. See more for an introduction here: https://www.simplilearn.com/tutorials/git-tutorial/what-is-git
  • BEFORE reviewing this power point, I highly advise you take the code academy class https://www.codecademy.com/learn/learn-git. We use Git and GitHub to keep records and versions of the changes we make, so reverting back to other versions is doable. 
• Make sure to get an XFdtd license on OSC
  • See instructions here 

(3) Finally, review the full manual and complete the bullets below. The full manual is Appendix A of Julie's Dissertation.

• Appendix A of my dissertation is our manual and acts as the holy grail of our research. Anything you need can be found here (from where our info exists on the web, to how to run the code). 
• Once you have read this through entirely, we advise you log on to OSC and explore the code for PAEA in PAS1960. Once you have done so, complete this homework by making a copy of this google doc and filling it out on your own. 

Once these are completed, the students are ready to start helping. This is all of the coding information and basic background information on evolutionary algorithms students must know to be proficient in assisting. 

Other useful materials we have may include:

1. Our old GENETIS manual 
   1. This is somewhat out of date; however, it does have information on the general working on the loop. 
2. Julie's candidacy paper 
3. Complete the writing a genetic algorithm in C++ project
   1. This is still a work in progress! I'll complete it shortly!
4. Read chapter 3 of this book (more if you have time) by Eiben and Smith, "Introduction to Evolutionary Computation" for a quick introduction. The book should be available electronically from a university library.  Let us know if you can't find it there.
5. Get to know XFdtd
   1. XFdtd manual is also attached; however, the link above (in 5) is more useful for coding a .xmacro.
       

Important notes:

MAC USERS: Using a GUI when connecting to OSC via SSH from your terminal.

You will need to set up X11 forwarding. Otherwise, when you go to plot via SSH, they will not display (they don't forward from OSC to your machine). If you are unable to run GUI's on OSC through your computer's terminal, follow these instructions:

1. In your own system's terminal, type: sudo vi /etc/ssh/sshd_config
2. Search for the following line: X11Forwarding no
3. Change it to: X11 Forwarding yes. Save the file and exit.
4. In your own system's terminal, type: sudo systemctl restart sshd
5. For a Mac, download and install XQuartz.
6. Connect to OSC using ssh -XY username@hostname.
7. You can test that this works by typing 'xeye' or 'xclock' in the terminal when logged into OSC. 

Now you should be able to use OSC's GUIs.

PC USERS: 

As a PC user, there are a few things you need to do:

(1) Enable the Windows Subsystem for Linux on your device (https://developerinsider.co/stepwise-guide-to-enable-windows-10-subsystem-for-linux/#:~:text=To%20enable%20the%20Windows%20Subsystem,list%20here%20and%20click%20OK.)

(2) Get Bash on Ubuntu installed on your machine. You are going to need to have Bash on Ubuntu installed prior to starting these trainings; otherwise Bash commands will not be recognized.

•  https://ubuntu.com/tutorials/install-ubuntu-on-wsl2-on-windows-10#1-overview

(3) Get X11 forwarding working. Otherwise, when you go to plot via SSH, they will not display (they don't forward from OSC to your machine).

• Follow these steps here.  
  • http://courses.cms.caltech.edu/cs11/misc/xwindows.html
• Change your config file. To do so, in terminal (with Bash on Ubuntu open) 
  • type: sudo vi /etc/ssh/sshd_config
  • Then change '#X11Forwarding no' to '#X11Forwarding yes' and save it.
    • https://www.businessnewsdaily.com/11035-how-to-use-x11-forwarding.html
• Make sure Putty and Xming are running every time before you ssh in. Use Xlaunch application to run Xming and press yes to everyrthing.
  • Xming should now be running as a background task.
  • Open PuTTY and open the pre-saved like you saved in PuTTY. (This will open a Bash terminal that asks for your OSC login password.)

Today's Tasks

Julie: Write bash practice assignment, assist others when they have questions, work on Git lecture. (Will finish bash assignments early next week)

Evelyn, Mitchell, Eliot: Make progress on the "Learn the Command Line" course on Code Academy. (Should be done by the end of the week?)

Leo: Start gathering information for the "physics of antennas" lecture series! Note: Remember that Leo wants to be an educator, and would like us to find opportunities to work on "teaching" items. Most of the students don't have any background on how antennas work. He started learning about them and is going to do a lecture series (likely the first 20 minutes of every meeting) starting in a few weeks! (Should start series by hopefully 2/21)

Alex M: Clean up unused code. Fix hard-coded directories (likely finished & tested by end of day 2/11)

Alex P: Fix hard-coded variables (likely finished & tested by end of day 2/11)

I've tacked on our to-do list in the meantime so we can keep carrying the items still pending along with us. As we complete them, I will drop them off the list! 

TO-DO List

The following is our coding to-do list:

1. Get our Ara sim error fixed (Alex, Julie) 

   1. It seems that if AraSim jobs are left to run while the user is away, it doesn't properly write one or more of the flags, and makes the loop get stuck at AraSim. We must then re-run from that state by manually editing the Save State text file.

   2. We are going to try to get XF to run at the command line so that we can submit it as a job. This would alleviate us using an interactive job. We *hope* this would fix the error since the user wouldn't be "gone" while utilizing a GPU in an interactive job. 

      1. Alex emailed XF to ask a few questions! Stay tuned!

2. Make Hardcoded directories variables.

   1. Xmacros (use sed command) 

   2. All over bash script functions.

   3. AraSim job files (sed command?)

3. Make the following variables in the bash script:

   
   In xmacros/simulationPECmacroskeleton_prototype.txt (use sed command):

   Grid spacing=0.1
   Frequency= factor of n higher 
   This factor of "n" higher in frequency has to be the same as the Multiplier_factor in the roulette algorithm AND the same as GeoFactor in fitnessFunction_ARA_amy.cpp.
   
   
   

   In roulette_algirithm.cpp

   Length = 50cm/Multiplier_factor; 
   STD= 15cm/Multiplier_factor 
   Radius = 1.5cm/Multiplier_factor; 
   STD= 0.75cm/Multiplier_factor 
   
   Also the "Multiplier_factor" line 89.
   
   This "Multiplier_factor"has to be the same as the GeoFactor in fitnessFunction_ARA_amy.cpp AND the same as the factor higher in frequency in the xmacro.
   ​NOTE:THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   
   

   In fitnessFunction_ARA_amy.cpp:

   GeoFactor=2 
   This GeoFactor has to be the same as the Multiplier_factor in the roulette algorithm AND the same as the factor higher in frequency in the xmacro
   NOTE: THIS MUST BE RECOMPILED AFTER CHANGING.
   
   
   

4. Get rid of all code we are not using anymore.  

   1. Currently using: XF_loop_AraSeed.sh

      1. Look at what items are being called in here, and remove old versions such as XF_loop_Prototype.sh, XF_loop_Functionized.sh, and the related scripts. 

      2. These can temporarily moved to on "old code" directory and we can delete once we finalize the loop. 

5. Turn tournament on in the GA. 

6. Parallelize the AraActualBicone job.

   1. The AraSim run in Gen 0 that gets Veff for the ARA actual bicone input file is not currently being seeded.

       

Dropbox link: https://www.dropbox.com/home/GP_Antennas/Updates/DailyFitnessScoreImages/Machtay_20200803_Master_Symmetric_Database_Real

Attendance for today's session: Cade, Julie, Alex M, Alex P, Mitchell, Scott. 

Monday was a holiday, and we will be replaceing that meeting later this week. Today we worked on the following tasks:

Tasks:

• Check continuity of loop
  • Alex M will try another run tonight, and has already requested a virtual desktop. 
• Install Arasim to /fs/projects/PAS0654/BiconeEvolutionOCS to be edited and messed with
  • Cade is testing whether or not this is running within the loop. He had gotten a few errors, and it is not completing. It's submitting the job with the new bash script, but it is not actually running.
    • Found an error. The bash script was not grabbing the correct AraSim directory. More updates to come. 
• Clean bash script
  • Mitchell and Alex M are putting all current functions hard-coded in into their own .sh scrips that are now being called in the main bash script.
• Update Manual
  • Cade and I are updating the manual with new changes. We have officially retired the old version of the manual that was used for runs with Nutau. We have saved that version and uploaded it to the elog here: http://radiorm.physics.ohio-state.edu/elog/GENETIS/17

Right now Elliot and Leo have decided to focus on working with Keith. They will not be contributing to this project for the remainder of the semester. Students have been posting individual updates here: https://www.dropbox.com/home/GP_Antennas/Updates

I've also attached the meeting minutes from the call last week, in order to keep track of updates. 

Copy this slide deck to use when presenting on run updates!

https://docs.google.com/presentation/d/1Tk-B1QbFTP_5pQZovfn0_wbTswZTmOrJURpi374xJWo/edit?usp=sharing

William Luszczak  1:30 PM

This is the directory with the current PUEO antenna data files: https://github.com/PUEOCollaboration/pueoSim/tree/main/data/antennas. Each type of antenna will need several files:

• vv_0 and hh_0 describe the on axis v and h pol gain as a function of frequency. Gain is listed as a multiplicative factor (not in dB, if I'm remembering correctly)
• hv_0 and vh_0 describe the on axis cross polarization gain
• vv_az and hh_az describe the off-axis gain via multiplicative factors of the boresight gain. These are currently listed as a function of frequency for reference angles of 0,10,20,30,40,50,60,70,80, and 90 degrees, though we can always adjust the reference angles if needed.
• vv_el and hh_el are similar to above, but for gain as a function of elevation angle instead of azimuthal angle.

So for example, the total vpol gain in a particular frequency bin for a signal incident at azimuthal angle az and elevation angle el would be vv_0(f)*vv_az(f, az)*vv_el(f, el)

New

1:31

We can also potentially adjust this if it would be more convenient for the GENETIS people to output antenna information in a different format (or at different reference angles or whatever). The important thing is that the boresight h, v, and cross pol gains are all defined, as well as the off-axis response (as a function of azimuth and elevation)

Slides shown at experts building meeting Aug. 2nd, 2023.

https://drive.google.com/drive/folders/1iDamk46R2_oOLHtvsOg4jNy05mCiB7Sn?usp=sharing

We are beginning a new run with several improvements to the ARA VPol loop to try to evolve antennas that are optimally matched just by their geometry. To do this, we are evolving with the RealizedGain instead of just Gain in the Xmacros (thus taking into account impedance mismatch). We also have the speed up that parallelizes the AraSim jobs on each run. Attached is the run_details.txt file, but the GA parameters are subject to change. Here is what they are to begin:

We have a population size of 50 individuals per generation.

Selection operators (NUMBER selected by each):

• Roulette: 10
• Tournamente: 10
• Rank: 30

Genetic Operators (NUMBER generated by each):

• Reproduction: 6
• Crossover: 36
• Immigration: 8
• Mutation: 2
  • Sigma: 5%

The name of this run is Data_Base_Test_7_10.

The name of the previous run we are comparing against is Length_Cutoff_Test_3.

Mostly just wrote more on the paper in the Genetic Algorithm section. I added some citation that we used in ICRC but there are still more places that should have citations.

I might check tonight when OSC is back up to try to push in more updates to the loop because I wanna get Evelyn and Ryan started on running the loop. Putting in those fixes is a big priority because we want to be able to correct the potential issue with XF simulation folders being overwritten and thus uan data not corresponding to the write individuals. The two big things for me in the loop are getting the simulation data to save correctly (and also putting that in the database) and testing that we can replicate results using the specific seed. I'll probably only focus on loop stuff tomorrow.

Got up before OSC was down to check progress of overnight run, it seems to have worked but I noticed a problem with the database that it wasn't writing to it probably due to a permissions issue but I would have to run another time to see. Shouldn't have affected data but just the use of the database. Run got up to 8 generations with non-zero fitness score which is positive and seems to have fixed the error we originally encountered. Talked to Eliot about pointers and possible errors but was unable to look at the specific error because it is on OSC.

Read about pointers and vectors in C++. Talked to Alex P a bit, and have some ideas of things to change to get the GA running. Began reading about antennas. Mostly a down day due to OSC being down.

Will implement changes to GA and continue familiarizing myself with how XF reads these values. 

I've been building in XF and have ran into some issues but currently have the following completed:

- Struts are placed, though they are not evenly spaced and the same. I have contacted XF to try and figure out how to get rotation/patterns to do what we need

- Circuit components and circuit board are in the xf file, though the components are not on the circuit board (I'm not totally sure how to put them on, that's the next step). The circuit components should be of spec, I looked up the part numbers from the previous ELOG post and grabbed the values from the manufacturer.

Here are the things that have not been completed/need answered:

- How to evenly space the struts connecting the shape (as stated previously, I've contacted XF to figure out how to do this easily. Im just bad at modeling in XF)

- Add circuit components to circuit board

- Add voltage and ground connections to the circuit board

- On the circuit diagram, figure out J1 and J2 connections. Meaning, which one connects to the signal from the cable and which connects to the other cone that is used as ground?

To see current antenna build go to: " /users/PAS1977/jacobweiler/GENETIS/XFzone/straightened_antenna_for_Jack.xf "

Attached are pictures in case there are issues pulling up model in XF. :)

Attendance: Julie Rolla, Alex M. 

Today Alex and I met to discuss how to move forward given the COVID-19 restrictions. We have resent out a message polling people's schedules to create online working sessions. Here is what we are planning:

Once we collect times for each group, we will have separate meetings for all of these entities. This way it's easier for us to meet without all having different items to discuss/work on. All of these will be zoom meetings moving forward. Meeting info will be announced shortly!

Tasks for "the Loop Group":

1. Make wall time a variable for XF sim.
2. Test current software edits.
   1. Such as item #1 above
   2. Make sure the bottleneck function is working.
      1. Right now it seems to be working; however, we keep hitting a wall time error. If we can make the wall time a variable then we can be sure the bottleneck function is working, and the wall time is truly the error. 
3. Get Ruby working on XF
   1. when I type:
      /usr/local/xfdtd/7.8.1.4/remcom/bin/xfdtd        I get:

      Info: Using latest available version (7.8.1.4)
      Info: Running /usr/local/xfdtd/7.8.1.4/remcom/XFdtd_7.8.1.4/bin/xfdtd 
      Error: Unable to find executable to run.  This command must exist

4. Get Alex's account of Ruby working. 
   1. He received an email and is still unable to log in. He is touching base with them again. 

Tasks for "the Training Group":

1. Julie and Alex need to finish the C++ project! 

One of our foundational questions tied to the optimization of the GA has been "How many individuals should we simulate". Up to now, our minds were made up for us by the speed of arasim being great enough that the time cost of simulating individuals was great enough that the improvements made from having more were not enough to justify the slowdown. However, with the upgrade to the faster, more recent version of arasim, I decided to re-examine this. This was also spurred on by the fact that the last time I ran this test we were testing GA performance by final generation metrics rather than by how many generations it took to reach a benchmark. So in one of my optimization tests, I tracked this data. 

To start, using the same run proportions, using a .5 chi-squared benchmark, the average time across all 89 run types used in this run was 25.4 generations for 50 individuals as compared to 8.3 generations running the same test for 100. Furthermore, the minimum number of generations for 50 individuals was 4.8 while using 100 individuals yielded 2.4. So on average running 100 individuals was about 3 times fast at reaching this benchmark than with 50. And when comparing the best result regardless of run type, 100 individuals was still 2 times quicker than the min for 50 individuals. Finally, the run that yielded an average of 2.4 generations for 100 individuals took an average of 29.2 generations with 50 individuals or roughly 12 times the generations. 

For the test we will discuss, we ran 89 different run types that all used 60% rank, 20% roulette, and 20% tournament selection respectively. These test had the following ranges:

6-18% of individuals through reproduction (steps of 3%)

64-88% of individuals through crossover (steps of 12%)

0-10% mutation rate (steps of 5%)

1-5% sigma on mutation (steps of 1%)

These tests also used our fitness scores with simulated error of .1 to imitate arasim's behavior and as such we used the chi-squared value to evaluate these scores as there is no error on those values. 

Comparing this same test with a tighter chi-squared benchmark of .25, we see similar results. On average 50 individuals took 37.1 generations to reach this point while 100 individuals took 16.0 generations. Similarly, the minimums amount of gens for 50 individuals was 15.4 while 100 individuals was 5. Finally, the corresponding run for the 5 generation min with 100 individuals took 41.8 generations with 50 individuals. These correspond to speed up's of 2.3, 3.08, and 8.36 respectively. 

This data implies that on average, independent of run type, we should expect to have to use 2-3 times fewer generations while running 100 individuals than we would running 50 individuals but we could see up to 8-12 times fewer generations to reach benchmarks. Another data set using a different set of selection methods was also tested for this and again yielded similar results, though overall the runs from the first batch were better across both 50 and 100 individuals and so those results are likely to be more indicative of the parameters we use in a true run. 

The data being examined in these results can be found here: https://docs.google.com/spreadsheets/d/1GlfnjQSO6VI8MuUGYTUcLkjwDZU98nyFFysgTTfVFOE/edit?usp=sharing  

Mostly just wrote more on the paper in the Genetic Algorithm section. I added some citation that we used in ICRC but there are still more places that should have citations.

I might check tonight when OSC is back up to try to push in more updates to the loop because I wanna get Evelyn and Ryan started on running the loop. Putting in those fixes is a big priority because we want to be able to correct the potential issue with XF simulation folders being overwritten and thus uan data not corresponding to the write individuals. The two big things for me in the loop are getting the simulation data to save correctly (and also putting that in the database) and testing that we can replicate results using the specific seed. I'll probably only focus on loop stuff tomorrow.

Got up before OSC was down to check progress of overnight run, it seems to have worked but I noticed a problem with the database that it wasn't writing to it probably due to a permissions issue but I would have to run another time to see. Shouldn't have affected data but just the use of the database. Run got up to 8 generations with non-zero fitness score which is positive and seems to have fixed the error we originally encountered. Talked to Eliot about pointers and possible errors but was unable to look at the specific error because it is on OSC.

Read about pointers and vectors in C++. Talked to Alex P a bit, and have some ideas of things to change to get the GA running. Began reading about antennas. Mostly a down day due to OSC being down.

Will implement changes to GA and continue familiarizing myself with how XF reads these values.