Bit comparison

The standard job xcpsa now b/cs across pe configs and restarts for a 3 day run with daily dumps. This has been tested on HPCx, quest and Hector with both PGI and pathscale compilers.

Fixes

For bit-comparison the following changes are required:

Choose the following scientific sections in the UMUI:
Dynamical Adjustment: 1A
Primary Field Advection: 1C
Diffusion and Filtering: 1A
QSAT: 2A
Include this updated modset:
ctile_09_new_param.xcpsa

For the quest cluster:

A new version of gcom is required ~moxsw/gcom2.9b5 and a new set of safer compile options should be used. To include these, re-compile with the following compiler overrides:
~rgxao/um/overrides/moxsw_gcom2.9b5
~rgxao/um/overrides/O2_fastsse_ieee

The old version of ctile_09_new_param contained a bug which mean polar values for heat flux through seaice, 3201, were overwritten with inconsistant values. 3201 is also a coupling field. The overwriting was caused by calls in BL_CTL to POLAR_UV for disgnostics which weren’t included in the job. The solution was to put some if statements around these calls.

Tests

Standard robustness tests involve varying the following:

restart frequency
number of processors

Restart Test

Set the restart frequency to 1 day and the run length to 3 days. Then run two tests:

A continuous 3 day run
A run in 1 day chunks

Compare dumps using cumf.

Processor Test

Set up a 1 day run with a 1 day restart dump. Then run the job on several arrangements of processors. It’s best to vary the EW number, NS number and overall number of processors, eg:

1×8
2×2
4×4
2×4

Compare dumps using cumf.

Debugging

If fields don’t compare, set the model to write extra dumps at various points in the code to narrow down where the output first diverges. Then work out array points which differ and step through the code, writing out values.
Further testing

Some calculations are not done every timestep and so it can sometimes be useful to repeat these tests over longer periodds of time.