November 2016
Updated  November 2017

François Massonnet
francois.massonnet@bsc.es/uclouvain.be

Everything available to set up the ensemble Kalman filter for NEMO and EC-Earth

|-----------------------------------------------------------------------|
|ALL COMPILATIONS AND CREATION OF BINARY FILES SHOULD BE DONE ON THE HPC|
|-----------------------------------------------------------------------|

1/ Go to the HPC

2/ Clone the git repository (some HPCs don't allow communication with the web, in this case clone locally and rsync the git project to HPC)

>> git clone https://earth.bsc.es/gitlab/fmassonnet/EnKF-assimilation.git

3/ Check-out the appropriate branch
>> git checkout develop-${HPC}
where ${HPC} is for instance mn4

The file enkf_modules_to_load.txt is a set of modules to be load at compilation and execution to make sure that all libraries are found


A/ Compilation of the EnKF code
-------------------------------
The folder EnKF-MPI-TOPAZ is a modified copy of the source code found at NERSC (http://enkf.nersc.no/Code/NERSC_implementation/V2/)

It has been modified as to work with NEMO and LIM, by F. Massonnet and C. König Beatty.

___THE CODE SHOULD NOT BE MODIFIED___


>> cd EnKF-MPI-TOPAZ
>> make clean
>> source ../enkf_modules_to_load.txt
>> make 

An executable EnKF should be produced. Try to run it to see if you get the following information message:

   bsc32526@login2:/gpfs/projects/bsc32/bsc32526/enkf/EnKF-assimilation/EnKF-MPI-TOPAZ> ./EnKF
   Inne i start_mpi: qmpi_proc_num =           0  master = T
   MPI started with 1 processors
   Usage: EnKF <parameter file>
          EnKF -h
   Options:
     -h -- describe parameter fie format

B/ Compilation of the program to convert NetCDF files to binaries
-----------------------------------------------------------------
>> cd ../conversion_uf/
>> make clean
>> source ../enkf_modules_to_load.txt
>> make

Two executables named prep_obs_ORCA1 and prep_obs_ORCA25 should be created. Try them:
>> ./prep_obs_ORCA1
           0
 
  (prep_obs) takes a real obs, extracts the desired variable and outputs
  it in a format that the EnKF can read & treat ('observations.uf').
 
  A file named mask.nc containing the variables tmaskutil, nav_lon and nav_lat
  is expected to be in the current directory (ORCA-file)
 
  Three command line arguments are expected:
  1. Path to the nc file of which the data is to be extracted.
  2. Variable name that can be found in there, 'h_i_htc1' or
     'at_i'. or dxdy_ice
  3. A tag with the date, e.g. 19790520
 
  Hope to see you again soon.
 
(prep_obs): Stopped.

C/ Compilation of the program to post-process the restarts after assimilation ("sanity_check")
----------------------------------------------------------------------------------------------
The folder sanity_check is a set of routines that make sure that the restarts provided by the EnKF can be read by NEMO. These are home-made (François Massonnet, see also description in http://www.sciencedirect.com/science/article/pii/S1463500315000050


>> cd ../sanity_check/
>> make clean
>> source ../enkf_modules_to_load.txt
>> make

An executable named sanity_checked is created, try it:
>> ./sanity_check
 
  sanity_check_LIM3 needs arguments: 
  -analysis_file_ice 
  -forecast_file_ice 
  -analysis_file_oce 
  -forecast_file_oce 
  Checks NEMO-LIM3 ice and ocean analyses restarst (netcdf) file for sanity and 
 fixesthem if necessary.
 
  Sanity means for now:
  Strongly follow limupdate.F90
  Files mask.nc and mesh_hgr.nc need to be in the current directory
 
  Hope to see you again soon.
 
  Chris König Beatty 
  Francois Massonnet -- francois.massonnet@uclouvain.be
  Last update: 2013
  Last update: 2016 (to work with NEMO3.6)
(sanity_check): Stopped.



D/ Convert NetCDF observations (already interpolated on the ORCA grids) to UF format:
-------------------------------------------------------------------------------------
>> cd ../conversion_uf/

open convert.bash, check it, and run it (always on HPC)