...

cd <simulations_dir>/<case_name>/case_scripts
# Make any changes necessary to avoid the crash
./case.submit

If you need to change a XML value, the following commands in the case_scripts directory are useful:

> ./xmlquery <variable>                   # Get value of a variable
> ./xmlchange -id <variable> -val <value> # Set value of a variable

Before re-submitting:

• Check that the rpointer files all point to the last restart. On very rare occasions, there might be some inconsistency if the model crashed at the end.

  • Run head -n 1 rpointer.* to see the restart date.

• gzip all the *.log files from the faulty segment so that they get moved during the next short-term archiving. To gzip log files from failed jobs, run gzip *.log.<job ID>.* (where <job ID> has no periods/dots in it).

• Delete core or error files if there are any. MPAS components will sometimes produce a large number of them. The following commands are useful for checking for these files:

  • ls | grep -in core

  • ls | grep -in err

• If you are re-submitting the initial job, you will need to run ./xmlchange -id CONTINUE_RUN -val TRUE

Post-Processing with zppy

...