In some instances one might need to run different simulations in parallel. For instance, I'm currently running a long control run with EC-Earth 3.3.0 (experiment E006). While this run proceeds in the background, I would like to test several features like outputting new data from LIM (see https://dev.ec-earth.org/issues/632). To do this, I need to run another simulation while E006 is running.
However, simulations need common files: forcings, restarts, output specifications. While forcings and restarts are not expected to change, this is not the case of the XIOS xml files, in my case. Currently, the ece-ifs+nemo.sh.tmpl template links those XML files to the run directory. I would suggest to rather copy them to make sure they do not change while the simulation goes on. This has been done in commit a9c8f92 of the branch develop-fmasson. Note that by doing so, the XML files will not be changing over the simulation since they are only copied once (at the beginning of the run). The only caveat is that if a second run is submitted while the first one is still in queue, the first one will import the second one's XML files.
Note also that if a simulation is submitted as usual by:
qsub ece-ifs+nemo.sh
and then some changes are done in ece-ifs+nemo.sh to prepare another simulation, there is a risk that the first one imports the new changes since the script automatically resubmits itself once the legs are done.
To do so, I would follow @ddocquie strategy and make one submission script per experiment: E000.sh, E001.sh, etc.
Hi @pbarriat et al.,
In some instances one might need to run different simulations in parallel. For instance, I'm currently running a long control run with EC-Earth 3.3.0 (experiment E006). While this run proceeds in the background, I would like to test several features like outputting new data from LIM (see https://dev.ec-earth.org/issues/632). To do this, I need to run another simulation while E006 is running.
However, simulations need common files: forcings, restarts, output specifications. While forcings and restarts are not expected to change, this is not the case of the XIOS xml files, in my case. Currently, the `ece-ifs+nemo.sh.tmpl` template *links* those XML files to the run directory. I would suggest to rather *copy* them to make sure they do not change while the simulation goes on. This has been done in commit `a9c8f92` of the branch `develop-fmasson`. Note that by doing so, the XML files will not be changing over the simulation since they are only copied once (at the beginning of the run). The only caveat is that if a second run is submitted while the first one is still in queue, the first one will import the second one's XML files.
Note also that if a simulation is submitted as usual by:
```
qsub ece-ifs+nemo.sh
```
and then some changes are done in `ece-ifs+nemo.sh` to prepare another simulation, there is a risk that the first one imports the new changes since the script automatically resubmits itself once the legs are done.
To do so, I would follow @ddocquie strategy and make one submission script per experiment: `E000.sh`, `E001.sh`, etc.
Hi @pbarriat et al.,
In some instances one might need to run different simulations in parallel. For instance, I'm currently running a long control run with EC-Earth 3.3.0 (experiment E006). While this run proceeds in the background, I would like to test several features like outputting new data from LIM (see https://dev.ec-earth.org/issues/632). To do this, I need to run another simulation while E006 is running.
However, simulations need common files: forcings, restarts, output specifications. While forcings and restarts are not expected to change, this is not the case of the XIOS xml files, in my case. Currently, the
ece-ifs+nemo.sh.tmpltemplate links those XML files to the run directory. I would suggest to rather copy them to make sure they do not change while the simulation goes on. This has been done in commita9c8f92of the branchdevelop-fmasson. Note that by doing so, the XML files will not be changing over the simulation since they are only copied once (at the beginning of the run). The only caveat is that if a second run is submitted while the first one is still in queue, the first one will import the second one's XML files.Note also that if a simulation is submitted as usual by:
and then some changes are done in
ece-ifs+nemo.shto prepare another simulation, there is a risk that the first one imports the new changes since the script automatically resubmits itself once the legs are done.To do so, I would follow @ddocquie strategy and make one submission script per experiment:
E000.sh,E001.sh, etc.