FAMOUS

delete

How to Run FAMOUS on the QUEST Cluster

The current “base” FAMOUS job on quest is xcpsb under username annette on the puma UMUI. This is scientifically equivalent to xbyva, see FAMOUS Standard Versions. By default, the job produces restarts every 6 months and means over annual and decadal periods.

We assume users are accessing the UMUI through puma, see NCAS-CMS.

The FAMOUS files on quest are installed under the directory /exports/gpfsbig/um/PUM64/famous. The queues at the moment have a limit of 2 days, and after that you will have to resubmit manually, as there is no automatic resubmission.

Submission needs to be done in several stages.

• Submit compile job (runs automatically)
• Submit run job:
  • copy files over to quest using umsubmit
  • run directly on quest using qsub-um

1) Set up your environment on quest

• Copy the setvars file to your home directory

cd $HOME
cp /exports/gpfsbig/um/PUM64/setvars $HOME
ln -s setvars setvars_4.5

• Append to your .profile (or equivalent):

module add default-infiniband-pgi-251
. $HOME/setvars

Note: If you have already been running on quest, you may have references to the old module default-infiniband-pgi in your .profile and/or setvars, so just check that they are consistent.

2) Create a job

• Copy the job to one of your own folders. Change the user details and DATA_DIR environment variable.

3) Compile the job

The compilation and the run must be done in separate stages.

• In “Compile Options” in the UMUI, select “compile and build then stop”.
• Under “Job resources and re-submission”, select “at”.

• Submit using umsubmit in the usual way:

~jeff/bin/umsubmit -h quest-hpc.bris.ac.uk -u USERID -r scp RUNID

I’ve also found that the SUBMIT button works here.

Compilation output will be sent to $HOME/umui_out

4) Run the job

Copy the files over from puma:

• Under “Compile Options”, select “Run from existing exec”.
• Under “Job resources and re-submission”, select “qsub”.
• Submit using the SUBMIT button or umsubmit.

Submit the job directly on quest:

• cd to the most recent directory in $HOME/umui_runs

cd $HOME/umui_runs
ls -lrt
cd ...

• submit the job with:

~um/bin/qsub-um qsubmit.quest

Job output will be sent to this directory.

Use qstat -an to get a summary of the running jobs.