- Created by Alice Barthel on May 07, 2024
-
Attachments
You are viewing an old version of this page. View the current version.
Compare with Current View Page History
Version 1 Next »
Scripts are available at /global/cfs/cdirs/e3sm/www/Tutorials/2024/scripts/mpas-analysis-practicum
. Copy the available scripts to your local directory (from which you will run the scripts) on perlmutter.
Exercise 1
Goal: Run a short MPAS-Analysis over existing data. Files have been prepared to give you a good starting point.
copy the
template.mpas_analysis_short.cfg
into a new filemy_exercise1.cfg
. Open it with your editor of choice.
The config file is split into different sections defined by section title denoted by []
(such as [run]
or [execute]
).
specify your own output directory:
update [output]/baseDirectory
to your own directory /pscratch/sd/<first-letter-of-moniker>/<moniker>/e3sm2024tutorial/mpas-analysis/exercise1
and htmlSubdirectory
to /global/cfs/cdirs/ntrain6/www/<moniker>/e3sm2024tutorial/exercise1
read the
[generate]
line. This line specifies the analysis tasks that will be performed.read the date ranges (defined by
startYear
andendYear
). Note that these appear twice, once in a section under[climatology]
and once under[timeSeries]
.make sure you save your changes.
Q1.a. Describe in your own words the tasks and plots you expect this script to produce.
open
job_script_template.slurm
read and check the
#SBATCH
options at the top of the file. Key options for today are--account=ntrain6
,--reservation=e3sm_day2
,--time=00:10:00
.modify the mpas-analysis command to point to your
my_exercise1.cfg
(replacingtemplate.mpas_analysis_short.cfg
)in the directory, run the following command: Note the job ID generated
sbatch job_script_template.slurm
You can check the job status by running:
squeue -u $USER
The analysis script should take approx. 2min to run once it starts. It may spend longer in the queue before starting. (If queue wait times are long, we can switch to the debug
queue)
Q1.b. How would you know that the job is done? how would you know if it was done successfully?
open the html directory in your favorite browser:
https://portal.nersc.gov/cfs/ntrain6/www/<moniker>/e3sm2024tutorial/exercise1
Option 1: if the page does not exist, or is empty, double-check that the html path is consistent with your config file, then proceed to the Troubleshooting section.
Option 2: if the page exists and the plots are present, congratulations! Navigate the available plots and answer the following questions.
Q2.a. Navigate the available plots (you can click on plots). Note which plots are available and compare to your answer to Q1.a. Is there anything that you had not anticipated?
Q2.b. Summarize the key features that you notice (either positive or negative) in the ocean and sea ice analysis plots produced. Make sure to take note of the plots axis labels, title and legend.
Q2.c. Pick one concerning feature you noted in Q2.b. How would you go about determining whether this signal is significant? How would you go about determining whether this feature is new?
Exercise 2
Goal: Rerun MPAS-Analysis with a modified config file. We suggest a few changes that still allow the analysis to run quickly.
copy the
template.mpas_analysis_short.cfg
into a new config filemy_exercise2.cfg
modify the output directories (both the
baseDirectory
andhtmlDirectory
)make one or two modifications in the tasks. We suggest:
changing the year range in either climatology, timeseries, or both. (note that the available range is 2000-2014, go beyond that at your own peril!)
adding tasks, such as
indexNino34
,climatologyMapSchmidtko
, orstreamfunctionMOC
to theexecute
line.
You can find the list of tasks by running mpas_analysis --list
in the terminal window (provided the e3sm-unified environment has been loaded). Some tasks are more time-consuming (e.g. regional profiles, regional time series) and will not be able to run within this 1hr tutorial. If attempting more time-consuming tasks, make sure to update the wall time and/or nodes in your job script (e.g. --time
in job_script_template.cfg
).
modify the mpas-analysis command in
job_script_template.slurm
to point to yourmy_exercise2.cfg
. Submit by runningsbatch job_script_template.slurm
.Once the job is done, open the html directory in your browser.
Option 1: if the page does not exist, or is empty, double-check that the html path is consistent with your config file, then proceed to the Troubleshooting section.
Option 2: if the page exists and the plots are present, congratulations! Navigate the available plots and answer the following questions.
Q3.a. What additional plots are present? Note any additional features or differences compared to Q2.b.
Beyond the plots produced, the post-processed output is available for your own analysis. In the terminal window, navigate to the analysis directory (the path is specified in your
[output]/baseDirectory
) and locate some example output files (.nc
files):
Q3.b. Find the post-processed seasonal climatology of the ocean mixed layer depth in January-February-March. What is the full path to the relevant netcdf file?
Q3.c. If you included the MOC
task, find the longest post-processed timeseries of the meridional overturning circulation (“moc”). What is the full path to the relevant netcdf file?
If you have not encountered any technical issues, take time to go through the Troubleshooting section to familiarize yourself with logs.
Exercise 3 (offline)
Choose between the 2 options below, and open the relevant link.
Option 1: standard MPAS-Analysis output
Option2: enhanced output with polar focus
Q4.a First, identify the simulation and period(s) analyzed. What additional plots are present (compared to Exercises 1-2 above)? Are there any that are missing?
Q4.b Find the .cfg
file. (hint: check the html landing page). What are the main differences between this config file and the ones you used in the previous exercises? (you can use vimdiff if you have local copies of both scripts).
Q4.c Which lines would you modify to reproduce this analysis for the run provided in this tutorial? Do you expect to modify other files? (hint: consider why we are not running this analysis now)
Exercise 4 (offline)
Open this example of MPAS-Analysis output
Q4.a First, identify the simulation(s) and period(s) analyzed. What are the main differences between these plots and the ones shown previously?
Q4.b Find the .cfg
file. (hint: check the html landing page). What are the main differences between this config file and the ones you used in the previous exercises? (you can use vimdiff if you have local copies of both scripts).
Q4.c Find the complete .cfg
file. (hint: check the html landing page). In the plots provided, the primary simulation is plotted in black, and the control simulation is plotted in red. Which lines would you modify to switch the color of the control to black, and the primary simulation result to red?
Troubleshooting
- Is the html path correct?
- Was the job submitted?
If an error occurred at submission, double-check the options and commands in the job_script
. Note the job ID if it got successfully submitted.
- Did the job finish successfully?
You can check the job history and status by running the command
sacct --starttime 2024-05-08 --format=User,JobID,Jobname,partition,state,time,start,end,elapsed,nnodes
Check the mpas_analysis.o<jobID>
file for any runtime information. If it exists, check the mpas_analysis.e<jobID>
file for any error information.
- Check the logs in
baseDirectory/logs
Each task has produced a log. The taskProgress.log
shows the progress through the tasks.
In the log directory, you can search quickly for an error by running the command
grep -i error *.log
- You can rerun the job with options (e.g.
mpas-analysis --purge --verbose my_config.cfg
) to provide additional log information.
Extra notes on alternative workflows
Alternatively, you can choose to run mpas-analysis on an interactive job. To try this out, use the following commands:
salloc -A ntrain6 --reservation=e3sm_day2 -C cpu -N 1 -c 32 -t 30 -q shared
Once the interactive job is active, run the following commands:
export OMP_NUM_THREADS=1 source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh export HDF5_USE_FILE_LOCKING=FALSE vim template.mpas_analysis.cfg mpas_analysis template.mpas_analysis.cfg
You will see updates on progress in the terminal window, e.g.:
Detected E3SM supported machine: pm-cpu Using the following config files: ... Running tasks: 100% |################################################################################################################| Time: 0:01:44 Log files for executed tasks can be found in <path>/logs Total setup time: 0:00:32.58 Total run time: 0:02:22.48 Generating webpage for viewing results... Web page: https://portal.nersc.gov/cfs/e3sm/<html-path>/interactive/
You can run MPAS-Analysis within zppy (will be covered on later days).
This is done by adjusting two main section in the zppy configuration file. Default
include the input and output paths, mpas_analysis
contains details for the analysis.
[default] input = /pscratch/sd/t/terai/E3SM_simulation/extendedOutput.v3.LR.historical_0101 output = /pscratch/sd/a/abarthel/data/E3SMv3/20240429.tutorial.zppy case = v3.LR.historical www = /global/cfs/cdirs/e3sm/www/abarthel/tutorial/20240429.tutorial.zppy partition = pm-cpu environment_commands = "source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh"
[mpas_analysis] active = True walltime = "01:00:00" parallelTaskCount = 6 anomalyRefYear = 2000 ts_years = "2000-2004", enso_years = "2000-2004", climo_years = "2000-2004", mesh = "IcoswISC30E3r5" shortTermArchive = True purge = True
To run the analysis, use the following commands in the terminal window:
source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh zppy -c my_zppy.cfg
Troubleshooting:
- navigate to the
<outputdirectory>/post/scripts
- check the status file(s) (you can use
grep -i error *.status
if you ran many tasks) - check the
.o
and.e
files for additional information - Log files for executed tasks can be found in
<outputdirectory>/analysis/mpas_analysis/<task>/logs
- No labels