Coral/README.md

Coral - simulation manager for NEMO

Purpose

Coral is a collaborative job submission script manager for the NEMO (Nucleus for European Modelling of the Ocean) modeling framework Documentation. With Coral, you can use and re-use job submission scripts designed and tested by your colleagues. You can ammend them in an easy way, and generate a submission script tailored to your needs. Then, you can share your script with your colleages.

Overview

• Run 'coral -h' to get a list of Coral commands, and 'coral <command> -h' to get help for a specific <command>
• Please see the Documentation for common usages and change log.
• Having trouble? Get help with Troubleshooting

Features

Once Nemo and Coral are properly installed, you can create a working submission script in three commands:

• coral init: initiate a set of configuration files (with working default values on the cluster) for the experiment features, data location, restart options, etc.
• coral build: create a submission script from the configuration files
• coral submit: submit the job, with options to run tests locally, ignore automatic job resubmission, restart from scratch, etc.
  

At each stage, you can act manually to adapt the workflow to your liking.

Once the job is submitted, you can simply run 'coral status' in the run directory to get an overview of what is happening. No need to look for job ids, or output files.

Requirements

NEMO & XIOS source code: see How to get NEMO from scratch

Install

Clone the Coral repository (for instance in the EXTERNAL directory of your NEMO install)

cd $HOME/modeles/nemo/3.6/EXTERNAL
git clone ssh://git@www.climate.be:3022/TECLIM/Coral.git coral

and update your shell startup script to source the 'setup-coral.sh' file to add 'coral' to your PATH variable.

configure_nemo()
{
  [...]
  source $HOME/modeles/nemo/3.6/EXTERNAL/coral/setup-coral.sh
}

Getting started

NEMO & XIOS must be compiled: see How to compile NEMO

Create a NEMO config experiment

cd $HOME/modeles/nemo/3.6/CONFIG
# Create new experiment
./makenemo -n ORCA1_LIM3_REF -d "OPA_SRC LIM_SRC_3" -m myARCHfile -j0
# Check your CPP keys
vi ORCA1_LIM3_REF/cpp_ORCA1_LIM3_REF.fcm
# Compile
./makenemo -n ORCA1_LIM3_REF -m myARCHfile -j4

Create a Coral instance

cd ORCA1_LIM3_REF
mkdir CORAL00
cd CORAL00
coral -h
coral init -h
coral list
coral init lemaitre3

Benchmarks

Lemaitre3

Blocking Switch = 2 x 576 cores & 1 x 768 cores

• Compiler: intel-2016a
• Flags: -O2 -i4 -r8 -fp-model precise -xHost

XIOS	NEMO	WTIME per YEAR	#CORES	MEM/CPU
2	22	133min17	24	3072
2	46	77min14	48	3072
2	140	44min20	142	3072
4	260	43min15	264	3584
4	520	36min09	524	3584
8	760	40min31	768	3584
8	1048	50min51	1056	3584
8	1912	53min49	1920	3584

Zenobe

• Compiler: intel-2015b
• Flags: -O2 -i4 -r8 -fp-model precise -xHost

XIOS	NEMO	WTIME per YEAR	#CORES	MEM/CPU
2	22	229min28	24	2625
2	46	146min	48	2625
2	140	58min29	142	2625
4	260	48min30	264	2625
4	520	34min54	524	2625
4	760	38min46	768	2625
8	1048	43min	1056	2625
8	1936	36min26	1944	2625

Acknowledgments

• Thanks @dfr for initializing this project.

Contributors

• @dfr, @pbarriat.

License

This project is under the Creative Commons CC0 1.0 Universal License. See the LICENSE file for the full license text.