Croco_pytools install on datarmor

Hello,

Several users have encounter issues with env installation of croco_pytools 2.0.0 on datarmor (HPC center at Ifremer, Brest France).

Here is some help to avoid some tricky things :slight_smile:
Note that this is really Datarmor specific, no need to do that in other host.

To create env on datarmor follow the instructions

  1. Launch an interactive job on ftp queue (to have internet access):
qsub -I -q ftp -l mem=10g -l walltime=02:00:00
  1. Go to your croco_pytools directory:
cd my_croco_pytools_path
  1. Activate conda:
source /appli/anaconda/latest/etc/profile.d/conda.csh
conda activate
  1. Create the env using mamba (faster than conda) and a modified env.yml where you add h5py in the conda part (not the pip one) to the original env.yml of the repository, (adding h5py in conda part will use a compiled package from conda instead of trying to compile it when installing copernicusmarine from pip and having compilers versions problems) :
mamba env create -f env_with_h5py.yml
  1. Compile fortran part:
conda activate croco_pyenv
cd Modules/toolsf
make clean
make

Then you will encounter at runtime the error:

./make_grid.py
Traceback (most recent call last):
  File "/scale/user/slegac/DEVS/test_croco_pytools/croco_pytools/prepro/./make_grid.py", line 25, in <module>
    from Modules.croco_class import CROCO
  File "/scale/user/slegac/DEVS/test_croco_pytools/croco_pytools/prepro/Modules/croco_class.py", line 3, in <module>
    from Modules.grid_tools import create_mask_and_topo, smooth_after_mask_edit, create_grid
  File "/scale/user/slegac/DEVS/test_croco_pytools/croco_pytools/prepro/Modules/grid_tools.py", line 6, in <module>
    import Modules.toolsf as toolsf
ImportError: dlopen: cannot load any more object with static TLS

To avoid this, you will have to import toolsf at the begining of the script you want to use

Exemple of where to add it in make_grid.py:

...
Smoothing uses Fortran routines for performance.

===========================================================================
"""

# Dependencies
import Modules.toolsf
import argparse
from Modules.croco_class import CROCO


def main():
...

We are still investigating on how to avoid these modifications… if you have found a clean solution, please share it :slight_smile:

After all this, you will finally be able to use croco_pytools on datarmor.

Note that this is really Datarmor specific, no need to do that in other host.

I have a less code invasive but still unsatisfactory solution:

export LD_PRELOAD=/home2/datahome/jcollin/conda-env/crd
oco_pyenv/lib/libgomp.so.1

Nice thanks !

in csh it will be

setenv LD_PRELOAD $CONDA_PREFIX/croco_pyenv/lib/libgomp.so.1

Yes it is definitly better than modify .py