Day 2- MPAS-Analysis Practicum

Scripts are available at /global/cfs/cdirs/e3sm/www/Tutorials/2024/scripts/mpas-analysis-practicum. Copy the available scripts to your local directory (from which you will run the scripts) on perlmutter. Open a blank document either on perlmutter or your computer to answer the questions.

Exercise 1

Goal: Run a short MPAS-Analysis over existing data. Learn to check job status, find and interpret analysis plots. Troubleshoot if necessary.

Files have been prepared to give you a good starting point.

  • copy the template.mpas_analysis_short.cfg into a new file my_exercise1.cfg. Open it with your editor of choice.

The config file is split into different sections defined by section title denoted by [] (such as [run] or [execute]).

  • specify your own output directory:

update [output]/baseDirectory to your own directory /pscratch/sd/<first-letter-of-moniker>/<moniker>/e3sm2024tutorial/mpas-analysis/exercise1

and htmlSubdirectory to /global/cfs/cdirs/ntrain6/www/<moniker>/e3sm2024tutorial/exercise1

  • read the [generate] line. This line specifies the analysis tasks that will be performed.

  • read the date ranges (defined by startYear and endYear). Note that these appear twice, once in a section under [climatology]and once under [timeSeries].

  • make sure you save your changes.

Q1.a. Describe in your own words the tasks and plots you expect this script to produce.

  • open job_script_template.slurm

  • read and check the #SBATCH options at the top of the file. Key options for today are --account=ntrain6, --reservation=e3sm_day2, --time=00:10:00.

  • modify the mpas-analysis command to point to your my_exercise1.cfg (replacing template.mpas_analysis_short.cfg)

  • in the directory, run the following command: Note the job ID generated

sbatch job_script_template.slurm

You can check the job status by running:

squeue -u $USER

The analysis script should take approx. 2min to run once it starts. It may spend longer in the queue before starting. (If queue wait times are long, we can switch to the debug queue).

Q1.b. How would you know that the job is done? how would you know if it was done successfully?

 

  • once the run has succesfully finished, update the permissions to the html directory by running the following commands:

cd $CFS/ntrain6/www/ chmod -R o+rx $USER
  • open the html directory in your favorite browser: https://portal.nersc.gov/cfs/ntrain6/<moniker>/e3sm2024tutorial/exercise1

Option 1: if the page does not exist, or is empty, double-check that the html path is consistent with your config file, then proceed to the Troubleshooting section.

Option 2: if the page exists and the plots are present, congratulations! Navigate the available plots and answer the following questions.

Q2.a. Navigate the available plots (you can click on plots). Note which plots are available and compare to your answer to Q1.a. Is there anything that you had not anticipated?

Q2.b. Summarize the key features that you notice (either positive or negative) in the ocean and sea ice analysis plots produced. Make sure to take note of the plots axis labels, title and legend.

Q2.c. Pick one concerning feature you noted in Q2.b. How would you go about determining whether this signal is significant? How would you go about determining whether this feature is new?

Exercise 2

Goal: Rerun MPAS-Analysis with a modified config file. Learn to add analysis tasks, find the post-processed output files. Learn to troubleshoot (find and logs).

We suggest a few changes that still allow the analysis to run quickly.

  • copy the template.mpas_analysis_short.cfg into a new config file my_exercise2.cfg

  • modify the output directories (both the baseDirectory and htmlDirectory)

  • make one or two modifications in the tasks. We suggest:

    • changing the year range in either climatology, timeseries, or both. (note that the available range is 2000-2014, go beyond that at your own peril!)

    • adding tasks, such as indexNino34, climatologyMapSchmidtko, or streamfunctionMOC to the generate line.

You can find the list of tasks by running mpas_analysis --list in the terminal window (provided the e3sm-unified environment has been loaded). Some tasks are more time-consuming (e.g. regional profiles, regional time series) and will not be able to run within this 1hr tutorial. If attempting more time-consuming tasks, make sure to update the wall time and/or nodes in your job script (e.g. --time in job_script_template.cfg).

  • modify the mpas-analysis command in job_script_template.slurm to point to your my_exercise2.cfg. Submit by running sbatch job_script_template.slurm.

  • Once the job is done, open the html directory in your browser. (if you get a permission error, repeat the Ex1 step to change permissions using the chmod command).

Option 1: if the page does not exist, or is empty, double-check that the html path is consistent with your config file, then proceed to the Troubleshooting section.

Option 2: if the page exists and the plots are present, congratulations! Navigate the available plots and answer the following questions.

Q3.a. What additional plots are present? Note any additional features or differences compared to Q2.b.

  • Beyond the plots produced, the post-processed output is available for your own analysis. In the terminal window, navigate to the analysis directory (the path is specified in your [output]/baseDirectory) and locate some example output files (.nc files):

Q3.b. Find the post-processed seasonal climatology of the ocean mixed layer depth in January-February-March. What is the full path to the relevant netcdf file?

Q3.c. If you included the MOC task, find the longest post-processed timeseries of the meridional overturning circulation (“moc”). What is the full path to the relevant netcdf file?

  • If you have not encountered any technical issues, take time to go through the Troubleshooting section to familiarize yourself with logs.

Exercise 3 (optional)

Goal: Examine a more comprehensive analysis (standard or polar options). Learn to find and compare config files.

This exercise does not require compute time; it can be done while waiting for Ex1 and Ex2 simulations, or at your leisure any time.

  • Choose between the 2 options below, and open the relevant link.

Option 1: standard MPAS-Analysis output

Option2: enhanced output with polar focus

Q4.a First, identify the simulation and period(s) analyzed. What additional plots are present (compared to Exercises 1-2 above)? Are there any that are missing?

Q4.b Find the .cfg file. (hint: check the html landing page). What are the main differences between this config file and the ones you used in the previous exercises? (you can use vimdiff if you have local copies of both scripts).

Q4.c Which lines would you modify to reproduce this analysis for the run provided in this tutorial? Do you expect to modify other files? (hint: consider why we are not running this analysis now)

Exercise 4 (optional)

Goal: Examine an existing model-to-model comparison. Learn to find the complete config files and extra options.

This exercise does not require compute time, so it can be done while waiting for Ex1 and Ex2 simulations, or at your leisure any time.

Q4.a First, identify the simulation(s) and period(s) analyzed. What are the main differences between these plots and the ones shown previously?

Q4.b Find the .cfg file. (hint: check the html landing page). What are the main differences between this config file and the ones you used in the previous exercises? (you can use vimdiff if you have local copies of both scripts).

Q4.c Find the complete .cfg file. (hint: check the html landing page). In the plots provided, the primary simulation is plotted in black, and the control simulation is plotted in red. Which lines would you modify to switch the color of the control to black, and the primary simulation result to red?

Troubleshooting: when things go wrong…

Is the html path correct?
Was the job submitted?

If an error occurred at submission, double-check the options and commands in the job_script. Note the job ID if it got successfully submitted.

Did the job finish successfully?

You can check the job history and status by running the command

Check the mpas_analysis.o<jobID> file for any runtime information. If it exists, check the mpas_analysis.e<jobID> file for any error information.

Check the logs in baseDirectory/logs

Each task has produced a log. The taskProgress.log shows the progress through the tasks.

In the log directory, you can search quickly for an error by running the command

You can rerun the job with options (e.g. mpas-analysis --purge --verbose my_config.cfg) to provide additional log information.

Extra notes on alternative workflows