! ###################################################################### ! ### ! ### TM5 ! ### ! ### Template settings for standard run. ! ### ! ### VARIABLE EXPANSION IN RC FILE ! ### ------------------------------ ! ### The variables '${..}' in this file are expanded on reading. ! ### ! ### Special variables are (read 'S' as '$'): ! ### S{pid} : evalutates to the process id of the calling script ! ### S{script} : " " " name " " " " ! ### ! ### Convention : variables 'my.*' are NOT read by the scripts or ! ### the program, but only used to fill settings in this file or ! ### the expert settings. ! ### ! ### In post/preprocessing settings, <...> keys are expanded by the ! ### underlying python script. Eg: ! ### ! ### Finally, there is the %{...} keys, similar to '${..}' but ! ### expanded only at the end of the run in the "rs" file. See ! ### post-processing settings, where this is used. ! ### ! ### ! ### QUICKSTART ! ### ----------------------------- ! ### Take the following steps for a first experience. ! ### ! ### 1. Ensure that the setup script is linked: ! ### ! ### TM5> ln -s base/trunk/bin/pycasso_setup_tm5 setup_tm5 ! ### ! ### 2. Obtain a copy of this file to edit: ! ### ! ### TM5> cp base/trunk/rc/pycasso-tm5.rc pycasso-tm5.rc ! ### ! ### 3. Select an approriate file with machine specific settings to be included; ! ### if your system is not supported yet, use: ! ### ! ### my.machine.rc : pycasso-machine-template.rc ! ### ! ### 4. The scripts will try to install, compile, and run ia TM5 code in ! ### the following directory structure (adopted from Carbon-Tracker): ! ### ! ### //build/ ! ### /run/ ! ### /output/ ! ### ... ! ### Edit the rcfile and fill a proper base directory name: ! ### ! ### my.basedir : /scratch/yourname/run-TM5 ! ### ! ### 5. Try to install and compile a code, use '-v' for verbose mode: ! ### ! ### TM5> ./setup_tm5 pycasso-tm5.rc -v ! ### ! ### The first runs will probably fail due to incorrect settings. ! ### Some hints on where to change them: ! ### ! ### o library paths and flags : pycasso-machine-*.rc ! ### ! ### o compiler flags : pycasso-machine-*.rc ! ### (or the 'pycasso-compiler-*.rc' included from there) ! ### ! ### If the code is compiled correctly, also also try to submit it: ! ### ! ### TM5> ./setup_tm5 pycasso-tm5.rc -v -s ! ### ! ### This will start the run in foreground. ! ### ! ### The first runs will probably fail due to missing input files. ! ### See the settings in this file, and eventually the definition ! ### of some common input paths in the machine.rc : ! ### my.data.dir : /data/TM5/input ! ### my.meteo.dir : /data/TM5/METEO ! ### ! ### Use the '-h' flag for info on how to submit to a queue. ! ### Ensure that correct queue settings are specfified in the machine.rc. ! ### ! ###################################################################### ! ! ------------------------------------------------ ! run description ! ------------------------------------------------ ! ! Project Name : a convenient variable, you can use for whatever need ! you may have in defining your keys. Here it is only used to define ! the project top directory, under which build/run/output/archive/ ! restart/save... directories are set): my.project : meteo-nc ! Id : short description for this run. It is used to name default ! standard output, but can be used anywhere else you want. ! ~ to convert glb100x100 meteo: runid : glb100x100 ! ~ to convert coarsened meteo: !runid : coarsened ! benchmark settings ? run.benchmark : T ! ---------------------------------------- ! build and run directories ! ---------------------------------------- ! The code is build and compiled in : ! ! /build/ ! ! It is suggested (but you do not have to) to start my.run.dir with ! ${my.project.dir}, so you end up with the following tree : ! ! /build ! / ! ! Output/restart/save/archive directories -see below- can also be ! set below ${my.project.dir} : ! ! / build ! / ! / ! / ! / ! ! Following these suggestions, the ${my.project.dir} fully becomes a ! project top directory, and not just the parent of the build dir. ! ! Note that ! (1) the 'my.scratch' variable (defined in machine.rc) is available ! (2) you can always define and use intermediate key. For example, ! my.basedir: ! base directory for build/run/etc : choose an appropriate run ! directory on a scratch disk, or use ${my.scratch} my.basedir : ${SCRATCH}/run-TM5 ! project directory: my.project.dir : ${my.basedir}/${my.project} ! run directory: my.run.dir : ${my.project.dir}/run ! ---------------------------------------- ! build (See expert rcfile for more settings) ! ---------------------------------------- ! basename for executable, runtime rcfile, logfiles, etc: my.basename : tm5-${my.project}-${runid} ! keys to select compiler flags; see compiler.*.rc for all values: !my.build.configure.flags : optim-none check-all my.build.configure.flags : optim-fast ! ---------------------------------------- ! Start run automatically (True|False) ? ! ---------------------------------------- ! Forced to 'True' by the '--submit' option to 'setup_tm5'. The job is ! started in the background, foreground or managed by a job scheduler, ! according to ${submit.to}, defined in pycasso-tm5-expert.rc. But ! this can also be overwritten at the command line. submit.auto : False ! ---------------------------------------- ! User specified queue options. ! Only used for submission of a job to a queue system. ! ---------------------------------------- ! Account is to specify scheduler (eg loadleveller) account. ! Leave empty for default account associated to your user name. my.queue.account : nlchekli ! Run with high memory load ? ! Value of this queue is used in machine.rc to specfiy destination ! queue or other settings. Supported values: ! normal large huge absurd ! For ecmwf/c1a, 'large' means twice the default amount of consumable ! memory, 'huge' three times, and 'absurd' four times. ! Note that this is per MPI task. ! A pure OpenMP run (meteo100x100 production) therefore needs an ! 'absurd' memory load. ! Full chemistry run with 2-4 MPI tasks needs 'large' . ! Leave empty to not have any memory specific options. my.queue.memoryload : ! ---------------------------------------- ! machine specific stuff (compiler name & flags, lib paths, ... ) ! ---------------------------------------- !my.machine.rc : pycasso-machine-template.rc my.machine.rc : pycasso-machine-ecmwf-ecgate.rc !my.machine.rc : pycasso-machine-ecmwf-c1a.rc !my.machine.rc : pycasso-machine-jrc-clust7.rc ! include settings: #include base/${my.branch}/rc/${my.machine.rc} ! ---------------------------------------- ! expert settings ! ---------------------------------------- ! include expert settings to build source code #include base/${my.branch}/rc/pycasso-tm5-expert.rc !================================================================ ! GRIDS & LEVELS ! ---------------------------------------- ! vertical layers ! ---------------------------------------- ! Met fields levels depend on the meteo class (od/ei) and year. ! But number of model levels is set here. ! Choose between: all (default), tropo25, tropo34, ml40 #if "${runid}" == "glb100x100" my.levs : all #elif "${runid}" == "coarsened" my.levs : tropo25 #else #error could not set my.levs for runid ${runid} #endif ! source codes with level definition: #if "${my.levs}" == "all" my.levs.sources : proj/levels/${my.eclevs}/${my.branch} #else my.levs.sources : proj/levels/${my.eclevs}/${my.branch} \ proj/levels/${my.eclevs}/${my.levs}/${my.branch} #endif ! ---------------------------------------- ! horizontal grid (leave region name empty to not use them) ! ---------------------------------------- !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #if "${runid}" == "glb100x100" !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! regions name my.region1 : glb100x100 my.region2 : my.region3 : my.region4 : my.region5 : ! grid names for meteo input (allows on the flight regridding): my.region1m : ${my.region1} my.region2m : my.region3m : my.region4m : my.region5m : !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #elif "${runid}" == "coarsened" !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! regions name my.region1 : glb600x400 my.region2 : eur300x200 my.region3 : eux100x100 my.region4 : nam300x200 my.region5 : nam100x100 ! grid names for meteo input (allows on the flight regridding): my.region1m : ${my.region1} my.region2m : ${my.region2} my.region3m : ${my.region3} my.region4m : ${my.region4} my.region5m : ${my.region5} ! parent regions for zoom region.${my.region2}.parent : ${my.region1} region.${my.region3}.parent : ${my.region2} region.${my.region4}.parent : ${my.region1} region.${my.region5}.parent : ${my.region4} !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #else #error could not set my.levs for runid ${runid} #endif !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ !=====================================================================! ! SOURCE CODE ! ! The source code is collected from a number of different directories. ! A base code for the model could be found in the directory: ! ! base/${my.branch}/src ! ! where the 'my.branch' variable is set to 'trunk', or 'release/X.X' ! for a standard release, or 'branch/' for development ! versions. ! The files in this directory are coppied to the build directory. ! ! Then, modifications to the base source could be copied from project ! directories. These could replace files from the base code. ! Common used projects: ! ! o level selection of 34 layers out of 91 for tropospheric studies; ! base is 25 out of 60 : ! ! proj/levels/${my.eclevs}/ ! proj/levels/${my.eclevs}/tropo34/ ! ! o do not only run, but write something out too: ! ! proj/user_output/trunk ! ! The base and project directories to be used should be specified ! as a space seperated list to 'my.source.dirs' . ! ! NOTES: ! ! *** USE THE '-n' OPTION IF YOU CHANGED THE PROJECT LIST ! *** ! The files from the new project directories are probably older than ! the current object files, and there 'make' will not recompile. ! With the '-n' option a complete new build will be created. ! ! Which branch to use ? !---------------------- ! This key defines the branch of the TM5 "base" model. It sets the ! path to sources (here) but also to several other rc files (machine, ! expert, compiler, region,...). ! It can also be used to define other src directories (see ! pycasso-tm5_chembase.rc for example). ! Possible values : trunk, "a branch", or a release [must be higher ! than 3.0] !my.branch : release/4.0 my.branch : trunk ! which code directories ? my.source.dirs : base/${my.branch} \ ${my.levs.sources} \ proj/chem/one/${my.branch} !=====================================================================! ! FLAGS ! These are C-preprocessors that you can set here. ! ! You must define 3 sets of flags in the following keys: ! ! my.df.define --> for scientific data format lib (for output & restart) ! my.tmm.define --> for meteo reading (format) & processing ! my.tm5.define --> everythnig else for tm5 model (processes, output, emiss,...) ! !=====================================================================! ! ---------------------------------------- ! ~~ Data Format (DF) ! ---------------------------------------- ! To specify the scientific data libraries to use (via MDF module) ! ! with_hdf4 : HDF ! with_hdf5 : HDF5 ! with_hdf5_par : HDF5 with parallel features ! with_netcdf : NetCDF ! with_netcdf4 : NetCDF with NetCDF-4 features, requires 'with_hdf5' ! with_netcdf4_par : NetCDF with NetCDF-4 and parallel features, requires 'with_hdf5_par' ! !! ~ only hdf4: !my.df.define : with_hdf4 ! ! ~ hdf4 and netcdf4 (used for restart files and cf output): #if "${par.mpi}" in ["T","True"] : my.df.define : with_hdf4 with_hdf5_par with_netcdf4_par #else my.df.define : with_hdf4 with_hdf5 with_netcdf4 #endif ! ---------------------------------------- ! ~~ meteo (tmm module) ! ---------------------------------------- ! ! ~~ meteo input ! ! with_tmm_tmpp : support TMPP HDF files ! with_tmm_tm5 : support TM5 HDF or NetCDF files ! with_tmm_ecmwf : support ECMWF grib files for TMPP or TM5 ! with_tmm_ncep : support NCEP cdc and gfs files ! with_tmm_msc : support data from Meteo Service Canada ! ! with_udunits : UniData Units library, used for TM5 meteo in NetCDF ! ! ~~ meteo production ! ! with_tmm_convec_raw : compute convec from p, T, Q, etc ! with_tmm_convec_ec : compute convec from ecmwf convective fields ! with_tmm_convec_ec_gg : compute convec from ecmwf convective fields ! on original gaussian grid ! ! macro's for meteo input: #if "${my.meteo.format}" == "tm5-nc" my.tmm.define : with_tmm_tm5 with_udunits #else my.tmm.define : with_tmm_tm5 #endif ! ---------------------------------------- ! ~~ Other TM5 flags ! ---------------------------------------- ! ! ~~ advection ! ! slopes : use slopes advection ! secmom : use second moments advection; requires 'slopes' ! ! ~~ convection levels: ! ! without_lmax_conv : convective fields not on limited number of layers ! ! ~~ to skip some processes: ! ! without_advection ! without_boundary ! without_convection ! without_diffusion ! without_dry_deposition ! without_wet_deposition ! without_chemistry ! without_photolysis ! without_emission ! without_sedimentation (sedimentation requires emis and dry depos) ! ! ~~ coupling ! ! with_prism ! oasis3 ! oasis4 ! with_tendencies ! with_feedback ! ! ~~ user output ! ! with_cf_output : to use CF compliant netCDF output in ! user_output routines. ! with_retro_output : ! ! ~~ miscellaneous ! ! with_budgets : to produce budget output files ! ! ~~ special libraries: ! ! with_lapack : to use LAPACK lib for linar algebra, ! and speed up convection ! ! ~~ Chemistry flags ! ! with_cariolle : to use Cariolle parameterization of ! stratospheric O3 ! with_gems_emis : to use GEMS emissions instead of default EDGAR ! with_GFED_8day : to use GFED2-8days BB emissions when ! using GEMS (requires with_gems_emis) ! without_nmhc_gfed : to not use VOC from GFED2 ! with_m7 : to use m7 aerosols model ! with_optics : m7 flag - ! with_pm : m7 flag - ! with_online_bvoc : to use MEGAN for biogenic emissions ! with_online_nox : ! with_ch4_emis : to switch on CH4 emissions ! without_o3_nudging : to switch off O3 nudging ! !=====================================================================! ! YOU SHOULD ***NOT*** USE ANY OF THOSE : ! !=====================================================================! ! with_barrier : ??? ! ! with_budget : should be with_budgets ! ! with_go : use GO module instead ! ! with_mpi : use MPI instead: #ifdef MPI ! ! with_okdebug : use okdebug flag below, available thru ! ! the dims module: ! ! ! ! use dims, only : okdebug ! ! ... ! ! if (okdebug) ... ! ! ! ! with_openmp : use _OPENMP instead in your code ! ! without_wetdeposition : should be without_wet_deposition ! ! with_restart : deprecated ! !=====================================================================! my.tm5.define : slopes ! ---------------------------------------- ! parallel ! ---------------------------------------- ! compile with MPI library (T|F) ? par.mpi : F ! number of mpi tasks: ! (used by scripts to launch job; source code gets npes from mpi routines) par.ntask : 4 ! enable openmp ? par.openmp : F ! threads per task: par.nthread : 1 ! ---------------------------------------- ! times ! ---------------------------------------- ! A run from start to end is break into a series of chunk jobs. The ! length of each chunk is set by 'jobstep.length'. ! ! start and end time in "yyyy-mm-dd hh:mn:ss" format: timerange.start : 2008-01-01 00:00:00 timerange.end : 2008-01-04 00:00:00 ! ! break run after: 'n' days, or 'inf' (never), or a 'month' !jobstep.length : 1 jobstep.length : inf ! ! Forecast mode (F|T) ? ! If true, then specify the 'zero' forecast day yyyy-mm-dd ! (otherwise, comment or leave empty) ! NOTE: in forecast mode, start and final time should ! not differ more than 10 days from fcday0 time.fc : F time.fc.day0 : ! --------------------------------------------------------- ! job chain ! --------------------------------------------------------- ! number of job step in chain; ! this is used to setup the next job in the chain, ! thus initialize to zero: jobstep : 0 ! time range of this step; ! ==> filled automatically by submit scripts! jobstep.timerange.start : ${timerange.start} jobstep.timerange.end : ${timerange.end} ! output directory for restart ! ==> filled automatically by submit scripts! prev.output.dir : ! --------------------------------------------------------- ! model timestep ! --------------------------------------------------------- ! time steps are set such that at least every 'ntimestep' seconds ! is the end of a dynamic time step: ! 10800 = every 3 hour ! 5400 = every 1.5 hour ! 3600 = every 1 hour ! 1800 = every 0.5 hour time.ntimestep : 1800 ! maximum dynamic timestep (sec): time.ndyn_max : 1800 ! ---------------------------------------- ! processes ! ---------------------------------------- ! to switch certain processes off, use the 'without_' defined above ! ! advection: #if "${runid}" == "glb100x100" : proces.advection.reduced : T #elif "${runid}" == "coarsened" : proces.advection.reduced : F #else #error reduced flag not set for runid ${runid} #endif ! to become OBSOLETE -- uncomment if you are still using the grid project !!#if len("${regions}") == 0 : !! file with reduced grid definition: !proces.advection.RedGrid : ${my.data.dir}/RedGrid !#endif ! wet removal (applied in convec and sources_sinks) proces.wet_removal.cp_scale : 0.5 ! ---------------------------------------- ! compute global budgets ? ! ---------------------------------------- apply.budget.global : T ! ---------------------------------------- ! initial fields ! ---------------------------------------- ! ! Start options: ! 1 coldstart with initial fields set to 0 ! 2 coldstart with initial fields computed ! in sr trace1 in sources_sinks... ! 30 save file specified here with start.30. ! 31 initial rm read from save file, no slopes, convert grids ! 33 restart file ! 4 coldstart with initial ! fields read from model output stored ! in mixing ratio (no slopes). ! 5 coldstart with initial ! fields read from an mmix output file ! in mixing ratio (no slopes, missing field set to zero) ! 9 user defined in module 'user_input' ! istart : 2 !! For istart=30, specify save files with nonstandard names: !start.30.glb600x400 : /data/old-save-files/save.hdfglb6x4 !start.30.eur300x200 : /data/old-save-files/save.hdfeur3x2 !start.30.eur100x100 : /data/old-save-files/save.hdfeur1x1 ! For initial istart=31, specify where the save files should be found; ! optionally, specify a TM4 save file to overwrite fields if possible: !start.31.glb300x200 : ${my.datadir}/save/tm5save-aero-tropo25-glb6x4.hdf !start.31.glb300x200.TM4 : ${my.datadir}/save/TM4-ml31-save_20051101_glb3x2.hdf !! For initial istart=4, specify where the save files should be found: !start.4.glb600x400 : ${my.datadir}/save/savetm5m_glb6x4.hdf !start.4.eur300x200 : ${my.datadir}/save/savetm5m_eur3x2.hdf !start.4.eur100x100 : ${my.datadir}/save/savetm5m_eur1x1.hdf ! ---------------------------------------- ! save files ! ---------------------------------------- ! New save files are writen to the output directory. ! THIS IS NOT DONE ANYMORE: ! After the model run is finished, the job script will link ! the save files to this special directory: ! SO THE FOLLOWING HAS BEEN MADE OBSOLETE IN PYCASSO: !savedir : ${my.project.dir}/tm5_save !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! OBSOLETE -- TO BE DELETED ! ensure creation of directory ! add 'save' to the 'install.tasks' list : !save.install.dir : ${savedir} ! ! save files should be copied from output directory ! to the save file directory; ! add 'save' to the 'store.tasks' list : !save.store.from : !save.store.files : save_*.hdf !save.store.arch : ${savedir} ! ! NOTE : save.cmd has been retired too.... !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! ---------------------------------------- ! restart files ! ---------------------------------------- ! ! Write and read restart files (T|F) ? If empty, read as F. Does not ! apply to initial conditions (see "initial fields" above). ! Recommendation: set to T so subsequent jobs use restart files. ! Set to F for subsequent jobs to use "save files" instead. ! ! Ensure to link with the NetCDF4 library by defining the macro's: ! with_netcdf4 with_hdf5 # serial run ! with_netcdf4_par with_hdf5_par # MPI parallel run ! restart.write : T ! write extra restart files (T|F) ? ! write at specified hour (00,12,..) plus every dhour (24,06,...) restart.write.extra : F restart.write.extra.hour : 00 restart.write.extra.dhour : 24 ! where to write restart files: restart.write.dir : ${my.project.dir}/restart ! ensure creation of directory; ! the key 'restart' should remain included in the 'install.tasks' list: restart.install.dir : ${restart.write.dir} ! * ! where to search for restart files for istart=33 : restart.read.dir : ${restart.write.dir} ! ---------------------------------------- ! meteo settings ! ---------------------------------------- ! Time ranges of different meteo archives: ! ! * od L60 1999-10-12 00:00:00 2006-02-01 00:00:00 ! * od L91 2005-10-01 00:00:00 now ! * ei 1989-01-01 00:00:00 now ! ! Operational Data (od) or ERA Interim (ei) ? my.meteo.class : ei ! o TMPP produced meteo: !include base/${my.branch}/rc/pycasso-meteo.tmpp.rc ! o TM5 produced meteo: ! ! what kind of meteo archive ? !my.meteo.resol : glb100x100 ! original archive with all ECMWF layers and 1.00x1.00 grid my.meteo.resol : coarsened ! data already regridded to ${my.levs} and regions resolution ! ! file format: "tm5-nc" for netcdf, "tm5-hdf" for good old hdf: !my.meteo.format : tm5-hdf my.meteo.format : tm5-nc ! ~ load meteo settings: !#include base/${my.branch}/rc/pycasso-meteo-tm5.rc ! ~ load meteo settings to convert from hdf to netcdf; also: ! - add "tmm.output" to "install.tasks" to create output directory; ! - add call to "tm5-tmm-store" script to "output.user.scripts" ! to have the new files stored in the archive; ! - use a timestep of 1800 sec to avoid errors about problematic intervals; ! - select single dummy tracer "proj/chem/one/" to speedup #include base/${my.branch}/rc/pycasso-meteo-tm5-convert-hdf-to-nc.rc ! switch: call meteo setup script (T|F) ? !my.tmm.setup.apply : F my.tmm.setup.apply : T ! ---------------------------------------- ! coupling ! ---------------------------------------- !include base/trunk/rc/prism.settings.rc ! ---------------------------------------- ! input ! ---------------------------------------- ! location of standard input files: inputdir : ${my.data.dir}/input/glb1x1 ! OBSOLETE -- add input.sc to input.user.scripts below ! ! run the 'input.sc' script (T|F) ? ! input.link : F ! locations of input files: !include base/${my.branch}/rc/input.base.rc ! for TIPP input: !include base/${my.branch}/rc/tipp.settings.rc ! for chemistry versions: !include base/${my.branch}/rc/chem.input.rc ! ---------------------------------------- ! standard output ! ---------------------------------------- ! ! The 'outputmode' determines what is done with standard output. ! SCREEN : not redirected, but written to screen; ! in case of background or submitted jobs, ! it is therefore added to 'jb..out' in launch dir. ! DEFAULT : redirected to default output file '${runid}.out' in run dir ! : your favorite file name ! !outputmode : benchmark.out ! print debug info (T|F) ? okdebug : F ! ! message formatting ! ! print messages from all processors (T|F) ? ! default is root only go.print.all : F ! print processor id prompt (T|F) ? ! default is only if npes>1 go.print.prompt.pe : T ! print label at begin and end of every routine ? go.print.trace : F ! print messages to file (T|F) ? ! provide first part of file name, full names will be: ! /.0 (proces 0) ! .1 (proces 1) ! : go.print.file : F go.print.file.base : ctm.tm5 !=====================================================================! ! output !=====================================================================! ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! Default Output Directory ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ output.dir : ${output.dir.base} ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! Alternative Output Directory ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! The tm5 setup_rcfile can overwrite the default output.dir with a ! runtime-dependent value, defined as a series of subdirectories in a ! top dir. For an example, to get: ! ! top-output-dir/sub1/sub2 ! ! you would set those keys: ! ! output.dir.base : top-output-dir ! output.dir.extensions : sub1 sub2 ! ! The dependence on runtime is done by using <> keys (they are ! recognized by the python script), or any rc keys from this file, ! in the extensions. Are available: ! ! : to use the jobstep number ! : for the timerange as yyyymmdd_hhmnss__yyyymmdd_hhmnss ! ! This alternative output.dir is ignored if "extensions" is empty output.dir.base : ${my.project.dir}/output output.dir.extensions : ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! Output Sub-Directory (who uses that?) ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! A set of subdir can be created below "output.dir". Use "types" to ! list their keynames, and for each of these keynames, give the subdir ! name, and a flag to switch on/off creation. Example, for two subdir: ! ! output.types : sub1 sub2 ! sub1.output : T ! sub1.output.subdir : name1 ! sub2.output : T ! sub2.output.subdir : name2 output.types : ! OBSOLETE -- all instances of outputdir should be ! replaced with output.dir ! for backwards compatibility ... ! outputdir : ${output.dir} ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! write grid definition(s) to netcdf files ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ settings.output : T ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! station output ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ output.station : F !!inputdir.station : ${datadir}/stationfiles/ !inputdir.station : ${my.data.dir}/output/ground/ ! !input.station.filename : station_test.list !input.station.filename2 : ! !input.station.fileformat : 1 ! !station.speclist : TEST1 TEST2 TEST3 TEST4 TEST5 ! !output.station.filename : station.hdf ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! flight tracks ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ output.flight : F ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! mixing ratio's ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ output.mix : F output.mix.dhour : 6 ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! mean mixing ratio's ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ output.mmix : F ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! timing ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ! write timing profiles ? timing.output : F ! optionally define a subdir of output.dir : timing.output.subdir : ! --------------------------------------------------------- ! Pre-processing I - Installation of input files ("install" keys) ! --------------------------------------------------------- ! ! Specify space seperated list with install tasks: ! ! install.tasks : obs diffusion ! ! For each task, an number of specific settings should be defined in ! one rc file. ! ! To have a number of files copied from an archive into a local directory, ! specify a space seperated list with file names: ! ! .install.files : krep.dat zooi.tar morezooi.tar ! .install.dir : /local/data/ ! .install.arch : ecfs:/tmx/TM/data ! ! If a file ends with '.tar' it is unpacked automatically. ! ! The archive description should be understandible by the 'gss' scripts: ! ! /archive/data/ ! ecfs:/tmx/TM/data # ECMWF tape archive when running on ECMWF computer ! ec:ecfs[tmx]:TM/data # idem from a remote computer using EcAccess tools ! mos:/fa/ks/TM/data # KNMI tape archive ! ! If an alternative file is available in case of a primary file is ! missing, use the 'optional' list instead of next to the 'files' list: ! ! .install.optional : eventually.tar ! ! On some systems only a temporary scratch disk is present at run time. ! The user should ensure that all input data is present on this local disk. ! A useful UNIX command for this is 'rsync', which syncronizes the content ! of a local directory with another directory. Use the following specification ! for a list of files or directories that need to be present in the 'dir' ! and should be synchronized with the version present in 'arch' : ! ! .install.rsync : TMtree ! ! The installtion performs a time loop over the 'jobstep.timerange' ; ! the 'dir', 'arch', 'files', 'optional', and 'rsync' values could then include ! special keys '', '', or '
' ! that are replaced by the year, month, and day values respectively: ! ! .install.files : zooi_.tar ! ! Default install tasks: ! o machine specific installation only ! o create destination of restart files !install.tasks : ${my.install.tasks} restart install.tasks : ${my.install.tasks} restart tmm.output ! --------------------------------------------------------- ! Pre-processing II - User scripts to setup run ("install" keys) ! --------------------------------------------------------- ! ! Provide a ';' seperated list with script commands to be exectuted: ! ! input.user.scripts : mypath/myscript ; important.sc ! ! The path should be either absolute or relative to the run directory. ! For scripts in the 'bin' subdirectories of the source, use : ! ! input.user.scripts : /input.sc ! ! Arguments/options can use , which is replaced by ! the name of the runtime rcfile. For example, to install meteo files ! for all days of the run: ! ! input.user.scripts : /tm5-tmm-setup -v ! input.user.scripts : /tm5-tmm-setup !! for the old, input.sc : !input.user.scripts : /input.sc ! --------------------------------------------------------- ! Post-processing I - Archiving output files ("store" keys) ! --------------------------------------------------------- ! ! Settings to store output files in archive. ! ! Specify space seperated list with store tasks: ! ! store.tasks : task1 task2 ! ! For each task, an number of task specific settings is defined in the ! rest of this file; some values are initialized below using a dummy ! task 'default' : ! ! ! from where ? leave empty for run directory, use '' for output directory: ! .store.from : ! ! ! copy some extra files to output subdirectory for storage: ! .store.copy : station-list.txt ! .store.copy.to : STATION/ ! ! ! name(s) of target file(s) for storage; ! ! if only one file is specified with extension '.tar' ! ! then an archive file is created. Default is a *.tar defined by ! ! sources.store.files in pycasso-tm5-expert.rc file. ! .store.files : all.tar ! ! ! if a tarfile has to be created, specify which files to archive ! ! through a filter: ! .store.tarfilter : STATION/*.hdf STATION/*.txt ! ! ! zip files before archiving ? empty, or a zipper command (gzip, bzip2, compress, ...) : ! default.zipper : gzip ! ! ! archive into one or more destinations (space seperated list): ! default.store.arch : ecfs:/xxx/MAIN_ARCHIVE/ ! .store.arch : ecfs:/xxx/SPECIAL_OUTPUT/ /extra/backup/archive/ ! ! ! If the 'arch' does not contain any ':' characters it is assumed to be ! ! a local directory. In this case, a 'cp' command is used to archive ! ! the files rather than the 'gss' script, and the files might therefore ! ! include filters: ! .store.files : save*.hdf ! .store.arch : ${SCRATCH}/savefiles/ ! ! ! store in archive even if already present ? ! ! by default probably True, but might not be necessary for some precomputed data: ! default.store.renew : True ! correlations.store.renew : False ! ! ! extra shell command to be called at the end of the task; ! ! for example to cleanup or do other important stuff: ! .store.extra.command : rm -f STATION/*.hdf STATION/*.txt ! ! Storage might be subject to a special condition, e.g. end of iteration in a 4D-var run. ! Specifiy a condition line that could be evaluated by python. ! If not specified, the default condition is True . ! Keys '%{..}' in the condition line are expanded with the values in the restart file ! if present; this file is written by the main program and has the same name as ! the rcfile but with extension '.rs' instead of '.rc' . ! Note the '%' instead of '$', otherwise the line is evaluated too early! ! ! store.condition : %{m1qn3.finished} == 1 ! ! only store for some condition ? store.condition : True ! space seperated list with store tasks: store.tasks : sources ! zipper: empty, or a zipper command (gzip, bzip2, compress, ...) : ! HERE: hdf files already have internal compression, so do not waste time on this ... default.store.zipper : ! default archive location(s): default.store.arch : ${my.project.dir}/ARCHIVE ! renew files ? default.store.renew : True ! --------------------------------------------------------- ! Post-processing II - Touch files ! --------------------------------------------------------- ! To avoid that output files are removed from scratch ! during long, long runs. ! Enable this flag to touch recursively the access time ! of all files in 'output.dir' . ! --------------------------------------------------------- ! touch files to prevent removal (True|False) ? output.touch : False ! --------------------------------------------------------- ! Post-processing III - User scripts ! --------------------------------------------------------- ! For more elaborate post-processing, you can write your own ! script. Then add the full command (including options and arguments) ! to the output.user.script key, so it will automatically be executed: ! ! output.user.scripts : /myscript ! ! More than one command can be specified: use ';' to separate them. ! The path should be either absolute or relative to the run directory. ! For scripts in any of the 'bin' subdirectories of the source, use ! . Arguments/options can use , which is replaced by ! the name of the runtime rcfile. ! ! Conditional call is possible. Just specifiy a condition line ! that can be evaluated with python eval() function. ! Set with the `output.user.scripts.condition` key, ! and default to True if not set or empty. ! The same condition apply to all listed scripts. ! --------------------------------------------------------- ! condition line: output.user.scripts.condition : True ! list with scripts to be called: output.user.scripts : /tm5-tmm-store ! -- Will output.sc become obsolete, and superseded by the "store" ! keys above?? -- ! Anyway, if one of the user script is the existing output.sc, you ! must specify the correct arguments: ! ! output.sc ${my.project.dir}/build/src ! ! and: one or more destination directories (gss descriptions), ! if all files must be first compressed using gzip (T|F), and ! if outpudir is cleaned after storing is completed (T|F). !output.store.destdirs : ecfs:/nl5/TM5/output !output.store.compress : T !output.store.clean : T ! --------------------------------------------------------- ! DIADEM postprocessing ! --------------------------------------------------------- ! ! To run the postprocessor, uncomment the include line and start: ! ! ./tools/diadem/py/diadem ! ! include settings: !#include tools/diadem/rc/diadem.rc