error in linking ww3_gspl when compiling ww3 in coupled mode

Gump · July 9, 2020, 12:49pm

I am compiling ww3 in an ocean-wave coupled mode on a Mac system. An error occurs as follows:

Processing ww3_gspl

ad3 : processing ww3_gspl
Linking ww3_gspl
link with oasis
*** error in linking ***

Undefined symbols for architecture x86_64:
“mpi_abort”, referenced from:
___mod_oasis_sys_MOD_oasis_abort in libpsmile.MPI1.a(mod_oasis_sys.o)
__m_dropdead_MOD_diex in libmpeu.a(m_dropdead.o)
__m_dropdead_MOD_die in libmpeu.a(m_dropdead.o)
“mpi_allgather”, referenced from:
__m_globalmap_MOD_initd in libmct.a(m_GlobalMap.o)
“mpi_allreduce”, referenced from:
___w3ogcmmd_MOD_rcv_fields_from_ocean in w3ogcmmd.o
___mod_oasis_mpi_MOD_oasis_mpi_maxr1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_maxr0 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_maxi1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_maxi0 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_minr1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_minr0 in libpsmile.MPI1.a(mod_oasis_mpi.o)
…
“mpi_alltoallv”, referenced from:
__m_rearranger_MOD_rearrange in libmct.a(m_Rearranger.o)
“mpi_barrier”, referenced from:
__m_globalsegmap_MOD_initd in libmct.a(m_GlobalSegMap.o)
___mod_oasis_mpi_MOD_oasis_mpi_barrier in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_timer_MOD_oasis_timer_print in libpsmile.MPI1.a(mod_oasis_timer.o)
___mod_oasis_timer_MOD_oasis_timer_start in libpsmile.MPI1.a(mod_oasis_timer.o)
“mpi_bcast”, referenced from:
__m_mctworld_MOD_initr in libmct.a(m_MCTWorld.o)
__m_attrvectcomms_MOD_bcast in libmct.a(m_AttrVectComms.o)
__m_attrvectcomms_MOD_gm_scatter in libmct.a(m_AttrVectComms.o)
__m_globalsegmap_MOD_initr in libmct.a(m_GlobalSegMap.o)
___mod_oasis_mpi_MOD_oasis_mpi_bcastr3 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_bcasti2 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_bcastr2 in libpsmile.MPI1.a(mod_oasis_mpi.o)
…
“mpi_comm_dup”, referenced from:
__m_mctworld_MOD_initr in libmct.a(m_MCTWorld.o)
“mpi_comm_free”, referenced from:
__m_mctworld_MOD_clean in libmct.a(m_MCTWorld.o)
“mpi_comm_rank”, referenced from:
___mod_oasis_auxiliary_routines_MOD_oasis_set_couplcomm in libpsmile.MPI1.a(mod_oasis_auxiliary_routines.o)
___mod_oasis_method_MOD_oasis_enddef in libpsmile.MPI1.a(mod_oasis_method.o)
___mod_oasis_method_MOD_oasis_init_comp in libpsmile.MPI1.a(mod_oasis_method.o)
___mod_oasis_part_MOD_oasis_part_create in libpsmile.MPI1.a(mod_oasis_part.o)
___mod_oasis_part_MOD_oasis_part_setup in libpsmile.MPI1.a(mod_oasis_part.o)
__m_mctworld_MOD_initr in libmct.a(m_MCTWorld.o)
__m_mctworld_MOD_initm in libmct.a(m_MCTWorld.o)
…
“mpi_comm_size”, referenced from:
___mod_oasis_auxiliary_routines_MOD_oasis_set_couplcomm in libpsmile.MPI1.a(mod_oasis_auxiliary_routines.o)
___mod_oasis_method_MOD_oasis_enddef in libpsmile.MPI1.a(mod_oasis_method.o)
___mod_oasis_method_MOD_oasis_init_comp in libpsmile.MPI1.a(mod_oasis_method.o)
___mod_oasis_part_MOD_oasis_part_create in libpsmile.MPI1.a(mod_oasis_part.o)
___mod_oasis_part_MOD_oasis_part_setup in libpsmile.MPI1.a(mod_oasis_part.o)
__m_mctworld_MOD_initr in libmct.a(m_MCTWorld.o)
__m_mctworld_MOD_initm in libmct.a(m_MCTWorld.o)
…
“mpi_comm_split”, referenced from:
___mod_oasis_auxiliary_routines_MOD_oasis_create_couplcomm in libpsmile.MPI1.a(mod_oasis_auxiliary_routines.o)
___mod_oasis_method_MOD_oasis_enddef in libpsmile.MPI1.a(mod_oasis_method.o)
___mod_oasis_method_MOD_oasis_init_comp in libpsmile.MPI1.a(mod_oasis_method.o)
___mod_oasis_part_MOD_oasis_part_setup in libpsmile.MPI1.a(mod_oasis_part.o)
“mpi_error_string”, referenced from:
___mod_oasis_mpi_MOD_oasis_mpi_chkerr in libpsmile.MPI1.a(mod_oasis_mpi.o)
__m_mpif90_MOD_perr in libmpeu.a(m_mpif90.o)
“mpi_finalize”, referenced from:
___mod_oasis_method_MOD_oasis_terminate in libpsmile.MPI1.a(mod_oasis_method.o)
___mod_oasis_mpi_MOD_oasis_mpi_finalize in libpsmile.MPI1.a(mod_oasis_mpi.o)
“mpi_gather”, referenced from:
___mod_oasis_method_MOD_oasis_init_comp in libpsmile.MPI1.a(mod_oasis_method.o)
__m_mctworld_MOD_initm in libmct.a(m_MCTWorld.o)
___mod_oasis_mpi_MOD_oasis_mpi_gathscatvinitr1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_timer_MOD_oasis_timer_print in libpsmile.MPI1.a(mod_oasis_timer.o)
___m_fccomms_MOD_fc_gather_fp in libmpeu.a(m_FcComms.o)
___m_fccomms_MOD_fc_gather_int in libmpeu.a(m_FcComms.o)
__m_zeit_MOD_allflush in libmpeu.a(m_zeit.o)
…
“mpi_gatherv”, referenced from:
___mod_oasis_mpi_MOD_oasis_mpi_gathervr1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___m_fccomms_MOD_fc_gatherv_fp in libmpeu.a(m_FcComms.o)
___m_fccomms_MOD_fc_gatherv_int in libmpeu.a(m_FcComms.o)
“mpi_get_processor_name”, referenced from:
___mod_oasis_method_MOD_oasis_init_comp in libpsmile.MPI1.a(mod_oasis_method.o)
“mpi_init”, referenced from:
___mod_oasis_method_MOD_oasis_init_comp in libpsmile.MPI1.a(mod_oasis_method.o)
___mod_oasis_mpi_MOD_oasis_mpi_init in libpsmile.MPI1.a(mod_oasis_mpi.o)
“mpi_initialized”, referenced from:
___mod_oasis_method_MOD_oasis_init_comp in libpsmile.MPI1.a(mod_oasis_method.o)
___mod_oasis_mpi_MOD_oasis_mpi_initialized in libpsmile.MPI1.a(mod_oasis_mpi.o)
__m_dropdead_MOD_diex in libmpeu.a(m_dropdead.o)
__m_dropdead_MOD_die in libmpeu.a(m_dropdead.o)
“mpi_intercomm_create”, referenced from:
___mod_oasis_auxiliary_routines_MOD_oasis_get_intercomm in libpsmile.MPI1.a(mod_oasis_auxiliary_routines.o)
“mpi_intercomm_merge”, referenced from:
___mod_oasis_auxiliary_routines_MOD_oasis_get_intracomm in libpsmile.MPI1.a(mod_oasis_auxiliary_routines.o)
“mpi_irecv”, referenced from:
__m_mctworld_MOD_initm in libmct.a(m_MCTWorld.o)
___m_fccomms_MOD_fc_gatherv_fp in libmpeu.a(m_FcComms.o)
___m_fccomms_MOD_fc_gatherv_int in libmpeu.a(m_FcComms.o)
___m_fccomms_MOD_fc_gather_fp in libmpeu.a(m_FcComms.o)
___m_fccomms_MOD_fc_gather_int in libmpeu.a(m_FcComms.o)
__m_rearranger_MOD_rearrange in libmct.a(m_Rearranger.o)
__m_transfer_MOD_irecv.part.1 in libmct.a(m_Transfer.o)
…
“mpi_isend”, referenced from:
__m_rearranger_MOD_rearrange in libmct.a(m_Rearranger.o)
__m_transfer_MOD_isend.part.3 in libmct.a(m_Transfer.o)
__m_exchangemaps_MOD_exgsmapgsmap in libmct.a(m_ExchangeMaps.o)
___m_spmdutils_MOD_m_swapm_fp in libmct.a(m_SPMDutils.o)
___m_spmdutils_MOD_m_swapm_int in libmct.a(m_SPMDutils.o)
“mpi_op_create”, referenced from:
___mod_oasis_reprosum_MOD_oasis_reprosum_ddpdd in libpsmile.MPI1.a(mod_oasis_reprosum.o)
“mpi_recv”, referenced from:
__m_attrvectcomms_MOD_recv in libmct.a(m_AttrVectComms.o)
___mod_oasis_mpi_MOD_oasis_mpi_reducelists in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_recvr3 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_recvr1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_recvr0 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_recvi1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_recvi0 in libpsmile.MPI1.a(mod_oasis_mpi.o)
…
“mpi_reduce”, referenced from:
___mod_oasis_mpi_MOD_oasis_mpi_maxr1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_maxr0 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_maxi1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_maxi0 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_minr1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_minr0 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_mini1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
…
“mpi_scatter”, referenced from:
___mod_oasis_method_MOD_oasis_init_comp in libpsmile.MPI1.a(mod_oasis_method.o)
“mpi_scatterv”, referenced from:
__m_attrvectcomms_MOD_gm_scatter in libmct.a(m_AttrVectComms.o)
___mod_oasis_mpi_MOD_oasis_mpi_scattervr1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
“mpi_send”, referenced from:
__m_mctworld_MOD_initm in libmct.a(m_MCTWorld.o)
__m_attrvectcomms_MOD_send in libmct.a(m_AttrVectComms.o)
___mod_oasis_mpi_MOD_oasis_mpi_reducelists in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_sendr3 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_sendr1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_sendr0 in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_mpi_MOD_oasis_mpi_sendi1 in libpsmile.MPI1.a(mod_oasis_mpi.o)
…
“mpi_wait”, referenced from:
___m_fccomms_MOD_fc_gatherv_fp in libmpeu.a(m_FcComms.o)
___m_fccomms_MOD_fc_gatherv_int in libmpeu.a(m_FcComms.o)
___m_fccomms_MOD_fc_gather_fp in libmpeu.a(m_FcComms.o)
___m_fccomms_MOD_fc_gather_int in libmpeu.a(m_FcComms.o)
__m_rearranger_MOD_rearrange in libmct.a(m_Rearranger.o)
__m_exchangemaps_MOD_exgsmapgsmap in libmct.a(m_ExchangeMaps.o)
___m_spmdutils_MOD_m_swapm_fp in libmct.a(m_SPMDutils.o)
…
“mpi_waitall”, referenced from:
__m_mctworld_MOD_initm in libmct.a(m_MCTWorld.o)
__m_rearranger_MOD_rearrange in libmct.a(m_Rearranger.o)
__m_transfer_MOD_waitsend.part.2 in libmct.a(m_Transfer.o)
__m_transfer_MOD_waitrecv in libmct.a(m_Transfer.o)
___remap_bi_interp_MOD_remap_bi in libscrip.a(remap_bi_interp.o)
___remap_bicubic_reduced_MOD_remap_bicub_reduced in libscrip.a(remap_bicubic_reduced.o)
___remap_conservative_MOD_remap_conserv in libscrip.a(remap_conserv.o)
…
“mpi_waitany”, referenced from:
__m_rearranger_MOD_rearrange in libmct.a(m_Rearranger.o)
“mpi_wtime”, referenced from:
___mod_oasis_method_MOD_oasis_init_comp in libpsmile.MPI1.a(mod_oasis_method.o)
___mod_oasis_advance_MOD_oasis_advance_run in libpsmile.MPI1.a(mod_oasis_advance.o)
___mod_oasis_mpi_MOD_oasis_mpi_wtime in libpsmile.MPI1.a(mod_oasis_mpi.o)
___mod_oasis_timer_MOD_oasis_timer_stop in libpsmile.MPI1.a(mod_oasis_timer.o)
___mod_oasis_timer_MOD_oasis_timer_start in libpsmile.MPI1.a(mod_oasis_timer.o)
__m_zeit_MOD_allflush in libmpeu.a(m_zeit.o)
__m_zeit_MOD_flush in libmpeu.a(m_zeit.o)
…
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status

I am not sure if this means my oasis is not compiled correctly or there is some issue with the open-mpi I installed. I appreciate any help or comments. Thank you all!

sjullien · July 9, 2020, 1:30pm

Hi,

Can you check that you have you correctly set your OASISDIR environment variable? (it is the one used by WW3 compilation scripts to find OASIS libraries). It should point towards the directory where your OASIS libraries are compiled.
Can you provide details on your compilation settings (compilers, compilation options) for both OASIS and WW3?

Gump · July 9, 2020, 2:05pm

Hi Swen,

Thanks for your answer! 1)for OASIS: MAKE= /usr/bin/make; F90= $(MPIBIN)/mpif90 -I$(MPI_INCLUDE); CC= /usr/local/gfortran/bin/gcc -I$(MPI_INCLUDE); LD = $(MPIBIN)/mpif90 $(MPILIB); 2)for WW3 (the comp file): comp=mpif90; opt=“-c -g -O3 -static -fconvert=big-endian -J$path_m”;

Gump · July 9, 2020, 2:08pm

(to be continued) I also checked that OASISDIR is defined when compiling WW3 and it points towards the directory of OASIS libs.

sjullien · July 10, 2020, 8:20am

Hi, in WW3 you have several comp examples. If you look in comp.Gnu the compilation options are: opt=“-c -O3 -fno-second-underscore -ffree-line-length-none -fconvert=big-endian -J$path_m”. It seems that your error may be related to the double underscore problem, so I suggest that you try this compilation option line.

Gump · July 10, 2020, 12:58pm

part 1) Thanks for your suggestion! I have tried the comp option you suggested but the same error still occurs. Do you think it is because my OASIS is not compiled correctly? When I compiled OASIS, the directories are generated but there are some warnings in COMP.err as follows:

Gump · July 10, 2020, 12:59pm

part2 ) /opt/local/bin/ranlib: file: libmpeu.a(m_SortingTools.o) has no symbols
/opt/local/bin/ranlib: file: libmpeu.a(m_realkinds.o) has no symbols
/opt/local/bin/ranlib: file: libmpeu.a(m_stdio.o) has no symbols
ranlib: file: libmpeu.a(m_SortingTools.o) has no symbols

sjullien · July 10, 2020, 1:07pm

I have not seen this kind of warnings yet, so I don’t know if your compilation is ok or not. First check that you actually have in your OASIS compiled directory: libmpeu.a, libmct.a, libscrip.a, libpsmile.MPI1

Gump · July 10, 2020, 1:16pm

I just checked again and I have libmpeu.a, libmct.a, libscrip.a, libpsmile.MPI1.a in the OASIS compiled directory. Directories “build” “lib” are both created and 4 sub-directories in “build/lib” are also there.

sjullien · July 10, 2020, 1:32pm

2 suggestions:

check what is in your $MPILIB
try to re-compile oasis with CC=mpicc (add the full path)
provide your full compilation line, as used during compilation to check every lib and include