For basic procedures to run FAMOUS version 4.5.3 on the QUEST cluster, look at RunningQuest. Here I put some notes from my own experience of running a copy of xcpsb [tcmia] (although these contain some repetition of the Running Quest page to provide the contexts). Similar notes for running FAMOUS with reconfiguration are also available.
1. Copy the setvars file to your home directory
cd $HOME
cp /exports/gpfsbig/um/PUM 64?/setvars $HOME
ln -s setvars setvars_4.5
2. Append to your .profile (or equivalent):
module add default-infiniband-pgi-251
. $HOME/setvars
3. also see trouble shooting [5] 4-8
1. in my first attempts UMUI froze during processing. it was found (by Annette, Thanks!) that the following lines in my puma:~/.kshrc were somehow disturbing the processing;
set prompt=“/% “
set history=250
set savehist=250
also
--- Make ^H into erase
if /usr/bin/tty -s ;then
stty erase ^H
fi
so I commented out these lines.
2. If processing still doesn’t go well or you don’t want to comment out the lines causing a problem in processing, you may have to turn off all post processing scripts. To do this, go to UMUI: [local post-processing scripts] and set “N” to all lines there. (You could find out which one is disturbing the processing by turning on and off from one script to another and try processing.) So now post processing needs to be done manually every time after processing.
/home/famous/bin/he_namelist_new_phase5 [jobid]
/home/famous/bin/vfdrift_pp.sh [jobid]
/home/famous/bin/quest_queue.sh [jobid]
The compilation and the run must be done in separate stages.
1. In “Compile Options” in the UMUI, select “compile and build then stop”
2. Under “Job resources and re-submission”, select “at”
3. Save, process, and post-process if necessary (see [2]−2)
4. Submit;
~jeff/bin/umsubmit -h quest-hpc.bris.ac.uk -u [userid] -r scp [jobid]
OR press the [SUBMIT] button on UMUI. Yes, this works.
- compilation takes several minutes.
- output file will be created as ***.leave in quest:~/umui_out. This file will typically be of 150~250 KB when compilation finishes successfully.
5. also see trouble shooting [5] 3-5
1. Under [Compile Options], select “Run from existing exec”
2. Under [Job resources and re-submission], select “qsub” (the top one)
3. Save, process, and post-process if necessary (see [2]−2)
4. Submit using the [SUBMIT] button or “umsubmit” (see [3]−4). I get messages like below but it turned out submission is actually working in spite of the messages.
qsub: invalid option — x
qsub: invalid option — s
usage: qsub [-a date_time] [-A account_string] [-b secs]
[-c c[=<INTERVAL›] ] [-C directive_prefix] [-d path] [-D path]
[-e path] [-h] [-I] [-j oe] [-k {oe}] [-l resource_list] [-m {abe}]
[-M user_list] [-N jobname] [-o path] [-p priority] [-q queue] [-r y|n]
[-S path] [-u user_list] [-X] [-W otherattributes=value…] [-v variable_list]
[-V ] [-z] [script]
* output file doesn’t seem to be created.
* this only copies the files from puma to quest and DOES NOT START THE RUN. GO TO THE NEXT STEP!!!
6. Submit to run
* on quest cd to the newest directory in ~/umui_runs
cd $HOME/umui_runs
ls -lrt
cd [jobid]−012345678 (go to the latest directory)
* submit the job with
~um/bin/qsub-um qsubmit.quest# (# = 1 or 2; look in the directory)
* output will be created as ~/umui_runs/[jobid]−012345678/[jobid]000.o1234 when the run completes in any way (finished or stopped due to error). This should contain information that would previously be contained in .leave files.
7. check the status of the run by typing;
qstat -u [userid]
8. also see trouble shooting [5] 4-8
cd ~/umui_runs(go to the latest directory)
ls -lrt
cd [jobid]−012345678
./qsubmit.quest# > qsubmit.out 2>&1(# = 1 or 2; look in the directory)
output qsubmit.out will be created. it contains what .leave file would contain.
~/umui_runs/[jobid]−012345678/[jobid]000.o1234;/var/spool/PBS/mom_priv/jobs/8357.master.SC: /exports/gpfsbig/home/[userid]/umui_runs/ [jobid]−012345678/SCRIPT[42]: .: /vn4.5/scripts/setglobalvars: not found
In “SCRIPT” in the same directory, there are lines;
VN=4.5 # Model version number . $UMDIR/vn$VN/scripts/setglobalvars # set UI system variables
So the problem seems that UMDIR is not given a value.
UMDIR is set in quest:~/setvars (which should be linked to /exports/gpfsbig/um/PUM 64?/setvars) as “UMDIR=~um/PUM 64?/um” and should be read by the shell through “. $HOME/setvars” in quest:~/.profile. However it wasn’t read for some reason.
So a line “. $HOME/.profile” is added to quest:~/.bashrc so bash reads .profile. If other shell is used this line needs to be added to .kshrc, .tcshrc etc.
Whether this is working Ok can be checked by typing “echo $UMDIR” on UNIX command line.