Notes from discussion on creating ancillary files and start dumps for FAMOUS on Monday, 23rd April 2007 with further notes added Friday, 27th April
Present: Robin, Lois, Jeff, Jonathan
To create ancillary files for arbitrary areas and resolutions from standard datasets, the UM ancillary file generation program can be used, hosted by BADC. This is not useful if you want to use different datasets or a different geography. To change the geography of the world, start dumps are needed. pptoanc can create arbitrary start dumps given PP input and namelist control, but it can be difficult to get it to work. Also, you have to know which fields should be included in the dump. This logic is coded in the UM, which will fail if the start dump omits required fields or includes unwanted ones.
The reconfiguration program can be used to change resolution by interpolation in the atmosphere, but not the ocean. However, providing exactly the data you want is preferable. The reconfiguration program can also be used to replace ancillary fields with others for the same grid and mask, but not prognostic (non-ancillary) fields.
Jeff’s program xancil (with underlying Fortran) creates ancillary fields given netCDF input. Dumps are similar in format to ancillaries. We agreed that a good method for making arbitrary start dumps would be to adapt xancil to do it. The user would provide xancil with an input dump, to indicate which fields should be included, and one or more netCDF files, containing data with the same grid and mask to replace fields. xancil can check that all the fields have the same grid, and the same vertical levels for 3D fields. Fields with no replacement data would be copied from the input dump to the output dump. Data which is not needed in the output dump would be ignored. If the output dump has to contain new prognostics, not included in the input dump, by definition they must be user prognostics, and in that case they can be configured into the dump by the reconfiguration program from ancillary files. In order to enable this, single-time ancillary files for arbitrary stash codes would need to be created. It would be useful to be able to create user ancillaries. An alternative for dumps would be to read the additional fields from a separate netCDF file. For additional prognostics and user ancillaries, a stashmaster file (standard or user) will probably have to be supplied, to specify the grid of the quantity.
xancil will have to work out somehow the correspondence between the fields in the netCDF files and stash quantities. To specify these to xancil would be quite laborious. A better method would be to use attributes in the netCDF file.
The modified xancil would create a start dump that could be used directly, without reconfiguration. The time can be specified by the user to xancil, or taken from the netCDF input files.
For the atmosphere, there is a complexity about land-only fields. These are compressed in the dump, but not in ancillaries. A land mask field would be supplied to xancil; it will be either the one in the input dump, or a replacement in the netCDF files. xancil should insist that the mask of land-only fields to be written to the output dump match the supplied mask. Fields coming from the netCDF files should indicate their mask by missing/non-missing data. Fields coming from the input dump must have been compressed using the mask from that dump. Requiring the mask to be correct is a stronger test than the UM applies, which is only that the number of land points is correct.
The ocean does not have a mask field. xancil could be provided with a mask, or could assume that 101 (ocean potential temperature) defines the 3D TS mask. From the TS mask the 3D UV mask can be derived. From the 3D masks, the fields of number of levels on TS and UV can be calculated. These fields are written into the header of the output dump. xancil should insist that all ocean fields, both 3D and surface, should have a matching mask, indicated by missing/non-missing data. xancil will also need namelist input specifying the island boxes. Usually, the ocean vertical levels will not be changed, and so can be copied from the input start dump. If they are to be changed, xancil will need this information; it could be obtained from coordinate information in CF-netCDF fields, but xancil does not handle that yet.
A further complexity for the ocean is the wrap-around columns which need to be added in dumps.
This enhancement to xancil to create start dumps deals only with the technical aspects. Jeff thinks that he could carry it out during the next two months.
Generating the data is a separate scientific task to be done by a library of facilities, which must produce the fields in netCDF files as required by xancil. This design means that all ad-hoc alteration of data will be done before it is supplied to xancil, not in the step of creating the dump. Programs in the library will be tested on one platform and will be accompanied by example input and output and a README. Some of Paul’s programs may be suitable. Facilities required include:
The third and fourth will still be useful if UM ancillary format is superseded.