Updates and Results Talks and Posters Advice Ideas Important Figures Write-Ups Outreach How-To Funding Opportunities GENETIS
  Important Plots, Tables, and Measurements  ELOG logo
Entry  Sun Apr 23 14:54:50 2017, Hannah Hasan, Other, Simulation, Plotting ShelfMC Parameter Space, Other Veff_A_I.pdfVeff_R_S.pdf
    Reply  Fri Oct 6 15:15:53 2017, Hannah Hasan, Other, Simulation, Plotting ShelfMC Parameter Space, Other 6x
Message ID: 27     Entry time: Fri Oct 6 15:15:53 2017     In reply to: 16
Author: Hannah Hasan 
Type: Other 
Category: Simulation 
Subject: Plotting ShelfMC Parameter Space 
Project: Other 

Attached are instructions and scripts for carrying out a parameter space scan with ShelfMC, the simulation package for the ARIANNA detector.

Because some of the plotted outputs looked like colored stripes and did not offer any insight into how effective volume changed with some variables, I made some changes to the simulation and plotting scripts so that different maximum, minimum, and increment values can be chosen for each variable. Now rather than having fixed, hard-coded values for all variables, the parameter space scan and plotting is more flexible for use with variable inputs.

Quote:

I am trying to write a script that will plot a 2d histogram of effective volume versus two of ShelfMC's parameters.

The script prompts the user for which two parameters (out of five that we vary in our parameter space scan) to plot along the x- and y-axes, as well as what values to hold the other 3 parameters constant at. It then collects the necessary root files from simulation results, generates a plotting script, and runs the plotting script to produce a plot in pdf form.

After many struggles I have the script written to the point where it functions, but the plots don't look right. Some plots look like they could be actual data (like Veff_A_I), and others just look flat-out wrong (like Veff_R_S).

I have yet to pin down the cause of this, but hopefully will be able to sometime in the near future.

 

Attachment 1: ParameterSpaceScan_instructions.txt  3 kB  | Show | Hide all | Show all
Attachment 2: setup.sh  8 kB  | Show | Hide all | Show all
Attachment 3: scheduler.sh  2 kB  | Hide | Hide all | Show all
#!/bin/bash
#Jude Rajasekera 3/20/17

tmpShelfmc=$HOME/ShelfMC/git_shelfmc #location of Shelfmc
runName=e18_ParamSpaceScan #name of run




cd $tmpShelfmc #move to home directory




AttenMin=500                      # MINIMUM attenuation length for the simulation
AttenMax=1000                     # MAXIMUM attenuation length
AttenInc=100                      # INCREMENT for attenuation length

RadiusMin=3                       # MINIMUM radius
RadiusMax=31                      # MAXIMUM radius
RadiusInc=7                       # INCREMENT

IceMin=500                        # etc...
IceMax=2900
IceInc=400

FirnMin=60
FirnMax=140
FirnInc=20

StDepthMin=0
StDepthMax=200
StDepthInc=50


if [ ! -f ./jobList.txt ]; then #see if there is an existing job file
    echo "Creating new job List"
    for ((L=$AttenMin;L<=$AttenMax;L+=$AttenInc)) #Attenuation length
    do
	for ((AR=$RadiusMin;AR<=$RadiusMax;AR+=$RadiusInc)) #Station radius (measured from center of radius to antenna
	do
            for ((T=$IceMin;T<=$IceMax;T+=$IceInc)) #Ice thickness
            do
		for ((FT=$FirnMin;FT<=$FirnMax;FT+=$FirnInc)) #Firn thickness
		do
                    for ((SD=$StDepthMin;SD<=$StDepthMax;SD+=$StDepthInc)) #Station depth
                    do
		    echo "cd $runName/Atten_Up$L/AntennaRadius$AR/IceThick$T/FirnThick$FT/StationDepth$SD" >> jobList.txt
                    done
		done
            done
	done
    done
else 
    echo "Picking up from last job"
fi


numbLeft=$(wc -l < ./jobList.txt)
while [ $numbLeft -gt 0 ];
do
    jobs=$(showq | grep "cond0092") #change username here
    echo '__________Current Running Jobs__________'
    echo "$jobs"
    echo ''
    runningJobs=$(showq | grep "cond0092" | wc -l) #change username here
    echo Number of Running Jobs = $runningJobs 
    echo Number of jobs left = $numbLeft
    if [ $runningJobs -le 20 ];then
	line=$(head -n 1 jobList.txt)
	$line
	echo Submit Job && pwd
	qsub run_shelfmc_multithread.sh
	cd $tmpShelfmc
	sed -i 1d jobList.txt
    else
	echo "Full Capacity"
    fi
    sleep 1
    numbLeft=$(wc -l < ./jobList.txt)
done
Attachment 4: colorplot_instructions.txt  8 kB  | Hide | Hide all | Show all
------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------
***Summary/Instructions for colorplot.sh (followed by same for colorplot_loop.sh)
------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------

colorplot
------------------------------------------------------------------------------------------------------------
  ______         __                        __         __
 /  ___/  _____ |  | _____  __ ___ __ ___ |  | _____ |  |__
|  /     /  _  \|  |/  _  \|  `__/|  `_  \|  |/  _  \|   _/
|  \____|  (_)  |  |  (_)  |  |   |  |_)  |  |  (_)  |  |__
 \______|\_____/|__|\_____/|__|   |  .___/|__|\_____/ \____\
                                  |__|
------------------------------------------------------------------------------------------------------------

Interactive script - to be run after parameter space scan has been performed. Produces a single plot.


Will plot, for a user-selected ShelfMC scan, the effective volume versus any two variables of the user's
choice out of the 5 parameters we vary:
   attenuation length, antenna radius, ice thickness, firn depth, and station depth


Also will allow user to choose what values to hold the other 3 variables constant at

Will add together necessary files, create root script for plotting, and execute root script


------------------------------------------------------------------------------------------------------------


***To use colorplot.sh:

- On line 6 of colorplot.sh, set $shelfmc to your ShelfMC directory. This directory should hold the
  parameter space scan directory, and a directory called "outputs". The "outputs" directory may be empty.

- In the variable lines, on line23, line27, and line30, change 20 to the number of processors your ShelfMC runs were
  split across ($ppn in setup.sh)

- Ensure that the parameter values on lines 8-26 match those for the run you wish to plot



Make sure colorplot.sh has execute permission - if not, type 'chmod u+x colorplot.sh'

Run the script by typing './colorplot.sh'

The following instructions will also be provided in the execution of the script.
- Enter the name of the run as prompted
- Make your selections for the two variables to plot along the X- and Y- axes. Your options are:
  A - Attenuation Length
  R - Antenna Radius
  I - Ice Thickness
  F - Firn Thickness
  S - Station Depth
*****Please only use capital letters in your selection
- Select values at which to hold the remaining parameters constant. Please select only the numbers allowed
  as prompted.
- Your plot will be in the $shelfmc/outputs/[name of scan]_[x variable]_[y variable] directory


If you wish to keep the root file and/or the plotting script, comment out line 488 and/or line 489
accordingly in colorplot.sh



------------------------------------------------------------------------------------------------------------


colorplot_loop
------------------------------------------------------------------------------------------------------------

  ______         __                        __         __             __
 /  ___/  _____ |  | _____  __ ___ __ ___ |  | _____ |  |__         |  | _____   _____  __ ___
|  /     /  _  \|  |/  _  \|  `__/|  `_  \|  |/  _  \|   _/         |  |/  _  \ /  _  \|  `_  \
|  \____|  (_)  |  |  (_)  |  |   |  |_)  |  |  (_)  |  |__         |  |  (_)  |  (_)  |  |_)  |
 \______|\_____/|__|\_____/|__|   |  .___/|__|\_____/ \____\ ______ |__|\_____/ \_____/|  .___/
                                  |__|                                                 |__|
------------------------------------------------------------------------------------------------------------

A more flexible extension of colorplot!
Interactive script to be run after the parameter space scan has been performed. Produces multiple plots.


Like colorplot.sh, colorplot_loop.sh will prompt the user for two variables against which to plot the
effective volume, from the options:
   attenuation length, antenna radius, ice thickness, firn depth, and station depth


Unlike colorplot.sh, colorplot_loop.sh does not prompt for constant values of the remaining three variables,
but instead loops over all combinations of constants to produce many plots with the selected X- and Y- axes


Before using this script, please ensure that you have space for hundreds of thousands of files (some
clusters may have a file limit for each user - for instance, Ruby has a 1 million file limit per user).
The script will delete all files except the .pdf plots, but prior to this deletion there could be a few
hundred thousand new files in your system.


------------------------------------------------------------------------------------------------------------


***To use colorplot_loop.sh:

- On line 6 of colorplot.sh, set $shelfmc to your ShelfMC directory. This directory should hold the
  parameter space scan directory, and a directory called "outputs". The "outputs" directory may be empty.

- Ensure that lines 8-26 match lines 15-33 of setup.sh and scheduler.sh for the desired parameter space scan

- On lines 50, 54, and 57, change 20 to the number of processors your ShelfMC runs were
  split across ($ppn in setup.sh)



Make sure colorplot_loop.sh has execute permission - if not, type 'chmod u+x colorplot_loop.sh'

OPTIONAL: Since colorplot_loop can take several minutes to run, it may be desirable to do $screen prior to
executing the script, in case the connection to the host is interrupted.

Run the script by typing ./colorplot_loop.sh

The following instructions will also be provided in the execution of the script.
- Enter the name of the run as prompted
- Make your selections for the two variables to plot along the X- and Y- axes. Your options are:
  A - Attenuation Length
  R - Antenna Radius
  I - Ice Thickness
  F - Firn Thickness
  S - Station Depth
*****Please only use capital letters in your selection

- Your plots will be in the $shelfmc/outputs/[name of scan]_[x variable]_[y variable] directory


If you wish to keep the collected root files and/or plotting scripts, comment out line 1034
and/or line 1035 accordingly in colorplot_loop.sh




------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------

                                  __ -- __
                              _-`         --_
                            /             ___\
                          /         -```-``   \
                         /        /   /..\     -_ 
                         |        |   \``/       --__      
                         |         \                  - ____
                         |          |_-``--___       o |    --_
                          |           ``--____``--__   |       \
                           \              -`` ``--__`-- \       |
                             \             ``-_     \     \     |   
                               \                `_    --___ \  /
                                |                  \.        |/
                          -_    |                    \
                             -_/                      \
                                                       \
                                                        )
                                                        )
                                                        )
                                                        |
                                                       /
                                                      /
                                                    /
                                                  /
                                               -` 
                                                                
Attachment 5: colorplot.sh  14 kB  | Show | Hide all | Show all
Attachment 6: colorplot_loop.sh  32 kB  | Show | Hide all | Show all
ELOG V3.1.5-fc6679b