pisces segmentation runtime error

biogeochemistry
1

Hi Everyone,
Big thanks for dev team and the update of the croco code and tools. I try testing a PISCES run with v1.3. After successfully compiling, I ran into this issue during my run:

           *** Total number of passive tracer jptra =           25
[spirit1:304773:0:304773] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x7ffcc449bf58)
==== backtrace (tid: 304773) ====
 0  /net/nfs/tools/u20/22.3/PrgEnv/intel/linux-ubuntu20.04-zen2/ucx/1.11.2-intel-2021.4.0-vqv63n3gh6g4gnchf25ezmkxui4mvyix/lib/libucs.so.0(ucs_handle_error+0x365) [0x7f8a9d9606d5]
 1  /net/nfs/tools/u20/22.3/PrgEnv/intel/linux-ubuntu20.04-zen2/ucx/1.11.2-intel-2021.4.0-vqv63n3gh6g4gnchf25ezmkxui4mvyix/lib/libucs.so.0(+0x2f2eb) [0x7f8a9d9612eb]
 2  /net/nfs/tools/u20/22.3/PrgEnv/intel/linux-ubuntu20.04-zen2/ucx/1.11.2-intel-2021.4.0-vqv63n3gh6g4gnchf25ezmkxui4mvyix/lib/libucs.so.0(+0x2eddb) [0x7f8a9d960ddb]
 3  /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f8a9eee4420]
 4  ./croco() [0x40a9fd]
 5  ./croco() [0x4b7d98]
 6  ./croco() [0x4b9123]
 7  ./croco() [0x52fd27]
 8  ./croco() [0x402692]
 9  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f8a9ed02083]
10  ./croco() [0x40259e]
=================================
Segmentation fault (core dumped)

I not sure what the issue is, but i suspect it has to do with the namelist. Is there anyone who had encountered this issue before? Or any ideas to solve this issue.

Thanks

2

Hello,
I just ran CROCO PISCES v1.3 on the spirit cluster (it seems you are running on this machine) using the Benguela_LR configuration for a one month run (with MPI). Everything went well.

Can you confirm hat you are running on the spirit cluster? In that case, i can send you my environment and check your run directory as well.

3

Hi Renaud, Yes, I running on spirit cluster. I just login again to try again but still the issue persist. If you can send me your environment to cross-check mine, that’ll perfect.
Thanks

4

The module loaded:

  1. intel/2021.4.0 2) openmpi/4.0.7 3) hdf5/1.10.7-mpi 4) netcdf-fortran/4.5.3-mpi 5) netcdf-c/4.7.4-mpi

Compilation option in the jobcomp:
FC=ifort

Check that the key_ligand is activated in the cppdefs.h :
define key_ligand

You can also send me the path to your run directory by mail

5

Hello. I encountered the same “Segmentation fault”. In my case, I was running an interannual simulation coupled with PISCES. Could you please give me a hint?

MPI (ifort) with ligand cppkey also activated

6

Thanks for your comment. I try to avoid using MPI but I did after @Renaud suggestion. My MPI flag uses ifort and the ligand cpp key for the ligand is activated.

7

Here is my cppdefs.h and jobcomp file.cppdefs.h(/upfiles/16702744387487833.h)

8

Here, the problem comes from the initialization of bgc values to Nan at some points of the domain close to the bottom (deepest points in fact) when creating croco_ini with crocotools. This is due to creation of HR initialisation file with LR data.

9

This can be fixed by setting hmax (crocotools_parama.m) to the nearest lower value of the depth of the NaN filled points or by modifying the initialization file by changing the Nan points to the nearest values.