/var/spool/slurmd/job7807171/slurm_script: line 28: limit: command not found /var/spool/slurmd/job7807171/slurm_script: line 29: limit: command not found Loading module 'gcc/9.2.0' Loading module 'szip/2.1.1' Loading module 'hdf5/1.12.0' Loading module 'netcdf/4.7.4' Loading module 'openmpi/4.1.1rc1' Loading netcdf-fortran/4.5.3 Loading requirement: openmpi/4.1.1rc1 ERROR: Unable to locate a modulefile for 'openmpi/4.0.4' [cn117:3402175:0:3402175] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402176:0:3402176] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402181:0:3402181] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402188:0:3402188] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402191:0:3402191] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402196:0:3402196] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402199:0:3402199] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402200:0:3402200] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402201:0:3402201] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402202:0:3402202] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402203:0:3402203] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402204:0:3402204] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402206:0:3402206] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402207:0:3402207] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402209:0:3402209] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402210:0:3402210] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402212:0:3402212] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402213:0:3402213] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402214:0:3402214] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402216:0:3402216] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402217:0:3402217] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402218:0:3402218] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402220:0:3402220] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402221:0:3402221] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402223:0:3402223] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402224:0:3402224] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402225:0:3402225] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402226:0:3402226] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402227:0:3402227] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402228:0:3402228] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402229:0:3402229] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402230:0:3402230] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402231:0:3402231] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402232:0:3402232] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402233:0:3402233] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402234:0:3402234] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402235:0:3402235] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402236:0:3402236] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402237:0:3402237] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402238:0:3402238] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402239:0:3402239] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402240:0:3402240] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402241:0:3402241] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402242:0:3402242] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402243:0:3402243] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402244:0:3402244] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402245:0:3402245] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402246:0:3402246] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402247:0:3402247] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402248:0:3402248] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402249:0:3402249] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402250:0:3402250] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402251:0:3402251] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402252:0:3402252] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402253:0:3402253] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402254:0:3402254] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402255:0:3402255] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402256:0:3402256] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402257:0:3402257] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402258:0:3402258] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402259:0:3402259] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402260:0:3402260] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402261:0:3402261] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402262:0:3402262] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402263:0:3402263] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402264:0:3402264] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402265:0:3402265] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402266:0:3402266] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402267:0:3402267] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402268:0:3402268] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402269:0:3402269] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402270:0:3402270] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402271:0:3402271] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402272:0:3402272] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402273:0:3402273] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402274:0:3402274] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402275:0:3402275] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402276:0:3402276] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402277:0:3402277] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402278:0:3402278] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402279:0:3402279] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402280:0:3402280] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402281:0:3402281] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402282:0:3402282] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402283:0:3402283] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402284:0:3402284] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402285:0:3402285] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402286:0:3402286] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402287:0:3402287] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402288:0:3402288] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402289:0:3402289] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402290:0:3402290] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402291:0:3402291] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402292:0:3402292] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402293:0:3402293] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402294:0:3402294] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402295:0:3402295] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402296:0:3402296] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402297:0:3402297] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402298:0:3402298] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402299:0:3402299] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402300:0:3402300] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402301:0:3402301] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402174:0:3402174] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402177:0:3402177] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402178:0:3402178] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402179:0:3402179] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402180:0:3402180] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402182:0:3402182] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402183:0:3402183] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402184:0:3402184] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402185:0:3402185] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402186:0:3402186] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402187:0:3402187] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402189:0:3402189] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402190:0:3402190] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402192:0:3402192] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402193:0:3402193] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402194:0:3402194] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402195:0:3402195] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402197:0:3402197] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402198:0:3402198] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402205:0:3402205] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402208:0:3402208] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402211:0:3402211] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402215:0:3402215] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402219:0:3402219] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn117:3402222:0:3402222] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590296:0:3590296] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584912:0:584912] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110344:0:4110344] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590305:0:3590305] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590307:0:3590307] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590319:0:3590319] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590329:0:3590329] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590337:0:3590337] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590340:0:3590340] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590341:0:3590341] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590342:0:3590342] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590343:0:3590343] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590344:0:3590344] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590345:0:3590345] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590346:0:3590346] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590347:0:3590347] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590348:0:3590348] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590349:0:3590349] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590350:0:3590350] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590351:0:3590351] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590352:0:3590352] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590353:0:3590353] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590354:0:3590354] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590355:0:3590355] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590356:0:3590356] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590357:0:3590357] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590358:0:3590358] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590359:0:3590359] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590360:0:3590360] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590361:0:3590361] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590362:0:3590362] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590363:0:3590363] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590364:0:3590364] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590365:0:3590365] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590366:0:3590366] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590367:0:3590367] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590368:0:3590368] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590369:0:3590369] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590370:0:3590370] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590371:0:3590371] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590372:0:3590372] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590373:0:3590373] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590374:0:3590374] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590375:0:3590375] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590376:0:3590376] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590377:0:3590377] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590378:0:3590378] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590379:0:3590379] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590380:0:3590380] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590381:0:3590381] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590382:0:3590382] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590383:0:3590383] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590385:0:3590385] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590386:0:3590386] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590387:0:3590387] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590388:0:3590388] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590389:0:3590389] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590390:0:3590390] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590391:0:3590391] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590392:0:3590392] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590393:0:3590393] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590394:0:3590394] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590395:0:3590395] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590396:0:3590396] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590397:0:3590397] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584937:0:584937] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590398:0:3590398] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584940:0:584940] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590399:0:3590399] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584944:0:584944] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590400:0:3590400] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584952:0:584952] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590401:0:3590401] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584957:0:584957] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590402:0:3590402] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584963:0:584963] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590403:0:3590403] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584967:0:584967] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590404:0:3590404] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584968:0:584968] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590405:0:3590405] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584969:0:584969] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590406:0:3590406] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584970:0:584970] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590407:0:3590407] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584972:0:584972] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590408:0:3590408] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584973:0:584973] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590409:0:3590409] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584974:0:584974] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590410:0:3590410] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584975:0:584975] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590411:0:3590411] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584976:0:584976] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590412:0:3590412] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584977:0:584977] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590414:0:3590414] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584978:0:584978] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590415:0:3590415] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584979:0:584979] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590416:0:3590416] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584980:0:584980] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590417:0:3590417] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584981:0:584981] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590418:0:3590418] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584982:0:584982] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590419:0:3590419] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584983:0:584983] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590420:0:3590420] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584984:0:584984] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590421:0:3590421] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584985:0:584985] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590422:0:3590422] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584986:0:584986] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590295:0:3590295] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584987:0:584987] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590297:0:3590297] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584988:0:584988] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590298:0:3590298] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584989:0:584989] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590299:0:3590299] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584990:0:584990] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590300:0:3590300] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584991:0:584991] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590301:0:3590301] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584992:0:584992] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590302:0:3590302] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584993:0:584993] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590303:0:3590303] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584994:0:584994] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590304:0:3590304] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584995:0:584995] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590306:0:3590306] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584996:0:584996] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590308:0:3590308] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584997:0:584997] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590309:0:3590309] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584998:0:584998] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590310:0:3590310] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584999:0:584999] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590311:0:3590311] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585000:0:585000] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590312:0:3590312] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585001:0:585001] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590313:0:3590313] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585002:0:585002] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590314:0:3590314] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585003:0:585003] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590315:0:3590315] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585004:0:585004] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590316:0:3590316] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585005:0:585005] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590317:0:3590317] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585006:0:585006] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590318:0:3590318] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585007:0:585007] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590320:0:3590320] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585008:0:585008] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590321:0:3590321] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585009:0:585009] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590322:0:3590322] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585010:0:585010] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590323:0:3590323] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585011:0:585011] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590324:0:3590324] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585012:0:585012] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590325:0:3590325] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585013:0:585013] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590326:0:3590326] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585014:0:585014] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590327:0:3590327] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585015:0:585015] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590328:0:3590328] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585016:0:585016] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590330:0:3590330] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110350:0:4110350] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585017:0:585017] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590331:0:3590331] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110352:0:4110352] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585018:0:585018] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590332:0:3590332] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110355:0:4110355] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585019:0:585019] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590333:0:3590333] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110356:0:4110356] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585020:0:585020] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590334:0:3590334] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110370:0:4110370] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585021:0:585021] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590335:0:3590335] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110382:0:4110382] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585022:0:585022] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590336:0:3590336] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110392:0:4110392] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585024:0:585024] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590338:0:3590338] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110395:0:4110395] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585025:0:585025] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590339:0:3590339] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110396:0:4110396] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585026:0:585026] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590384:0:3590384] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110397:0:4110397] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585027:0:585027] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn142:3590413:0:3590413] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110398:0:4110398] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585028:0:585028] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110399:0:4110399] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585029:0:585029] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110400:0:4110400] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585030:0:585030] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110401:0:4110401] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585031:0:585031] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110402:0:4110402] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585032:0:585032] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110403:0:4110403] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585033:0:585033] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110404:0:4110404] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585034:0:585034] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110405:0:4110405] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585035:0:585035] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110406:0:4110406] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585036:0:585036] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110407:0:4110407] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584909:0:584909] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110408:0:4110408] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584910:0:584910] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110409:0:4110409] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584911:0:584911] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110410:0:4110410] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584913:0:584913] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110411:0:4110411] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584914:0:584914] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110412:0:4110412] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584915:0:584915] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110413:0:4110413] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584916:0:584916] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110414:0:4110414] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584917:0:584917] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110415:0:4110415] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584918:0:584918] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110416:0:4110416] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584919:0:584919] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110417:0:4110417] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584920:0:584920] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110418:0:4110418] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584921:0:584921] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110419:0:4110419] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584922:0:584922] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110420:0:4110420] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584923:0:584923] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110421:0:4110421] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584924:0:584924] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110422:0:4110422] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584925:0:584925] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110423:0:4110423] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584926:0:584926] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110424:0:4110424] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584927:0:584927] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110425:0:4110425] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584928:0:584928] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110426:0:4110426] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584929:0:584929] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110427:0:4110427] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584930:0:584930] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110428:0:4110428] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584931:0:584931] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110429:0:4110429] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584932:0:584932] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110430:0:4110430] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584933:0:584933] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110431:0:4110431] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584934:0:584934] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110432:0:4110432] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584935:0:584935] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110433:0:4110433] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584936:0:584936] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110434:0:4110434] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584938:0:584938] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110436:0:4110436] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584939:0:584939] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110437:0:4110437] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584941:0:584941] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110438:0:4110438] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584942:0:584942] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110439:0:4110439] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584943:0:584943] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110440:0:4110440] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584945:0:584945] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110441:0:4110441] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584946:0:584946] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110442:0:4110442] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584947:0:584947] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110443:0:4110443] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584948:0:584948] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110444:0:4110444] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584949:0:584949] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110445:0:4110445] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584950:0:584950] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110446:0:4110446] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584951:0:584951] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110447:0:4110447] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584953:0:584953] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110448:0:4110448] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584954:0:584954] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110449:0:4110449] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584955:0:584955] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110450:0:4110450] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584956:0:584956] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110451:0:4110451] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584958:0:584958] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110452:0:4110452] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584959:0:584959] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110453:0:4110453] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584960:0:584960] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110455:0:4110455] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584961:0:584961] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110456:0:4110456] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584962:0:584962] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110457:0:4110457] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584964:0:584964] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110458:0:4110458] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584965:0:584965] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110459:0:4110459] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584966:0:584966] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110460:0:4110460] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:584971:0:584971] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110461:0:4110461] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn237:585023:0:585023] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110462:0:4110462] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110463:0:4110463] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110464:0:4110464] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110465:0:4110465] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110466:0:4110466] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110467:0:4110467] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110468:0:4110468] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110469:0:4110469] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110470:0:4110470] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110343:0:4110343] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110345:0:4110345] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110346:0:4110346] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110347:0:4110347] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110348:0:4110348] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110349:0:4110349] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110351:0:4110351] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110353:0:4110353] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110354:0:4110354] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110357:0:4110357] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110358:0:4110358] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110359:0:4110359] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110360:0:4110360] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110361:0:4110361] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110362:0:4110362] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110363:0:4110363] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110364:0:4110364] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110365:0:4110365] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110366:0:4110366] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110367:0:4110367] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110368:0:4110368] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110369:0:4110369] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110371:0:4110371] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110372:0:4110372] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110373:0:4110373] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110374:0:4110374] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110375:0:4110375] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110376:0:4110376] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110377:0:4110377] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110378:0:4110378] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110379:0:4110379] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110380:0:4110380] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110381:0:4110381] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110383:0:4110383] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110384:0:4110384] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110385:0:4110385] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110386:0:4110386] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110387:0:4110387] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110388:0:4110388] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110389:0:4110389] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110390:0:4110390] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110393:0:4110393] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110394:0:4110394] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110435:0:4110435] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110454:0:4110454] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) [cn164:4110391:0:4110391] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x30) ==== backtrace (tid: 585016) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110390) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402236) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402289) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402275) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402245) ==== ==== backtrace (tid:4110414) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590324) ==== ==== backtrace (tid:4110420) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 ==== backtrace (tid:3402251) ==== 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 12 0x00000000004a1fa1 def_avg_() ???:0 ==== backtrace (tid:4110351) ==== 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 13 0x00000000004afc24 wrt_avg_() ???:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 14 0x000000000048da4e output_() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402298) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 15 0x00000000004051fc step_() ???:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 16 0x0000000000404a5e MAIN__() main_.f:0 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 19 0x000000000040361e _start() ???:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 ================================= 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. 9 0x00000000000212d8 nc_open_par() ???:0 Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110384) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402279) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid: 584969) ==== Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402185) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584960) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110411) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110368) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590409) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 ==== backtrace (tid:3402235) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 5 0x00000000000387d7 closemagic() dinfermodel.c:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110363) ==== 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 ==== backtrace (tid: 584962) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 12 0x00000000004a1fa1 def_avg_() ???:0 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 ==== backtrace (tid:3402249) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 6 0x00000000000383cd check_file_type() dinfermodel.c:0 ==== backtrace (tid:3402178) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 7 0x0000000000037f96 NC_infermodel() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402281) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 ==== backtrace (tid:3590372) ==== 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 ==== backtrace (tid:4110450) ==== 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 ==== backtrace (tid:3590370) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402290) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 ==== backtrace (tid:3590402) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 ==== backtrace (tid:3590414) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590364) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402186) ==== ==== backtrace (tid:3402197) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402294) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 ==== backtrace (tid: 584963) ==== 5 0x00000000000387d7 closemagic() dinfermodel.c:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 6 0x00000000000383cd check_file_type() dinfermodel.c:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 8 0x0000000000023b35 NC_open() ???:0 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 9 0x00000000000212d8 nc_open_par() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584978) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 ==== backtrace (tid: 584999) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402190) ==== 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 19 0x000000000040361e _start() ???:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 ================================= 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584967) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590323) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402243) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 ==== backtrace (tid:3402193) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 ==== backtrace (tid:3402301) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 ==== backtrace (tid: 585036) ==== ==== backtrace (tid:3402184) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid: 584952) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 14 0x000000000048da4e output_() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 15 0x00000000004051fc step_() ???:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 16 0x0000000000404a5e MAIN__() main_.f:0 ==== backtrace (tid:4110410) ==== ==== backtrace (tid:3402300) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584936) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 17 0x00000000004035e7 main() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 ==== backtrace (tid: 585012) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 18 0x00000000000236a3 __libc_start_main() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 ==== backtrace (tid:3402265) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 19 0x000000000040361e _start() ???:0 ================================= 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. ==== backtrace (tid:3590358) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584954) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3590333) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110349) ==== 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 ==== backtrace (tid: 584996) ==== 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 5 0x00000000000387d7 closemagic() dinfermodel.c:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 7 0x0000000000037f96 NC_infermodel() ???:0 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 ==== backtrace (tid:4110353) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 ==== backtrace (tid:4110452) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 6 0x00000000000383cd check_file_type() dinfermodel.c:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 8 0x0000000000023b35 NC_open() ???:0 ==== backtrace (tid: 585028) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:4110387) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 ==== backtrace (tid:3590309) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3590339) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 7 0x0000000000037f96 NC_infermodel() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 ==== backtrace (tid: 584992) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 12 0x00000000004a1fa1 def_avg_() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 13 0x00000000004afc24 wrt_avg_() ???:0 ==== backtrace (tid: 585023) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 16 0x0000000000404a5e MAIN__() main_.f:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 17 0x00000000004035e7 main() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ==== backtrace (tid: 585032) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ================================= ==== backtrace (tid:4110459) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3402208) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402179) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 ==== backtrace (tid:3590412) ==== 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 16 0x0000000000404a5e MAIN__() main_.f:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 17 0x00000000004035e7 main() ???:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 18 0x00000000000236a3 __libc_start_main() ???:0 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 19 0x000000000040361e _start() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 ================================= 5 0x00000000000387d7 closemagic() dinfermodel.c:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 ==== backtrace (tid:3402220) ==== ==== backtrace (tid:3402277) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 9 0x00000000000212d8 nc_open_par() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 10 0x0000000000021331 nc_open_par_fortran() ???:0 Backtrace for this error: ==== backtrace (tid:3402177) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402250) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 13 0x00000000004afc24 wrt_avg_() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402273) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402192) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 14 0x000000000048da4e output_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 15 0x00000000004051fc step_() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 16 0x0000000000404a5e MAIN__() main_.f:0 ==== backtrace (tid:3402241) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 17 0x00000000004035e7 main() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 18 0x00000000000236a3 __libc_start_main() ???:0 ==== backtrace (tid:3402242) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3590336) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402174) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 19 0x000000000040361e _start() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 ================================= 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Backtrace for this error: ==== backtrace (tid:3402217) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 ==== backtrace (tid: 585002) ==== 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 ==== backtrace (tid:4110391) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 ==== backtrace (tid: 584997) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 7 0x0000000000037f96 NC_infermodel() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590354) ==== ==== backtrace (tid:4110457) ==== 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 ==== backtrace (tid:3402291) ==== ==== backtrace (tid: 585017) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3402215) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 ==== backtrace (tid:3402228) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590398) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 ==== backtrace (tid:3402255) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402219) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 6 0x00000000000383cd check_file_type() dinfermodel.c:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3590413) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 7 0x0000000000037f96 NC_infermodel() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402247) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 8 0x0000000000023b35 NC_open() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590304) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 585031) ==== Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 12 0x00000000004a1fa1 def_avg_() ???:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 ==== backtrace (tid:3402194) ==== Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 13 0x00000000004afc24 wrt_avg_() ???:0 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 ==== backtrace (tid:3402269) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 14 0x000000000048da4e output_() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 15 0x00000000004051fc step_() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 16 0x0000000000404a5e MAIN__() main_.f:0 ==== backtrace (tid: 584975) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 17 0x00000000004035e7 main() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 18 0x00000000000236a3 __libc_start_main() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 19 0x000000000040361e _start() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402272) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 ================================= 6 0x00000000000383cd check_file_type() dinfermodel.c:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. 7 0x0000000000037f96 NC_infermodel() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Backtrace for this error: 8 0x0000000000023b35 NC_open() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 9 0x00000000000212d8 nc_open_par() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 ==== backtrace (tid:3590392) ==== ==== backtrace (tid:4110413) ==== ==== backtrace (tid:3402233) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 ==== backtrace (tid: 584924) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 ==== backtrace (tid: 584939) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 8 0x0000000000023b35 NC_open() ???:0 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590418) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584946) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 ==== backtrace (tid:3590355) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 ==== backtrace (tid: 584970) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 6 0x00000000000383cd check_file_type() dinfermodel.c:0 ==== backtrace (tid: 585007) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 12 0x00000000004a1fa1 def_avg_() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 ==== backtrace (tid:4110393) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ==== backtrace (tid: 584947) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 13 0x00000000004afc24 wrt_avg_() ???:0 7 0x0000000000037f96 NC_infermodel() ???:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 8 0x0000000000023b35 NC_open() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584995) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590316) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 ==== backtrace (tid:3590322) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 ==== backtrace (tid:4110430) ==== ==== backtrace (tid:3402257) ==== 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402222) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110470) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590381) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 9 0x00000000000212d8 nc_open_par() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 ==== backtrace (tid:3590384) ==== 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 10 0x0000000000021331 nc_open_par_fortran() ???:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 12 0x00000000004a1fa1 def_avg_() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 ==== backtrace (tid: 584920) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 #0 0x1555530badcf in ??? 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors ==== backtrace (tid: 585013) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3402278) ==== ==== backtrace (tid:3590327) ==== at ../opal/class/opal_object.h:481 ==== backtrace (tid:3402175) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #2 0x15555454d4a1 in ompi_file_close 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 at file/file.c:153 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #3 0x155554570d15 in PMPI_File_close Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590406) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #4 0x155552a0d7d6 in ??? 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 #5 0x155552a0d3cc in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 585018) ==== 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 ==== backtrace (tid: 584971) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 5 0x00000000000387d7 closemagic() dinfermodel.c:0 ==== backtrace (tid: 584968) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 #0 0x1555530badcf in ??? 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? 6 0x00000000000383cd check_file_type() dinfermodel.c:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 7 0x0000000000037f96 NC_infermodel() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 8 0x0000000000023b35 NC_open() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 9 0x00000000000212d8 nc_open_par() ???:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 ==== backtrace (tid:3590313) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 12 0x00000000004a1fa1 def_avg_() ???:0 ==== backtrace (tid: 584965) ==== 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 9 0x00000000000212d8 nc_open_par() ???:0 14 0x000000000048da4e output_() ???:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 15 0x00000000004051fc step_() ???:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 16 0x0000000000404a5e MAIN__() main_.f:0 12 0x00000000004a1fa1 def_avg_() ???:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 19 0x000000000040361e _start() ???:0 14 0x000000000048da4e output_() ???:0 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 ================================= 15 0x00000000004051fc step_() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3590298) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590342) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 ==== backtrace (tid:3402209) ==== ==== backtrace (tid:3590360) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid: 585019) ==== Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 ==== backtrace (tid: 584927) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 ==== backtrace (tid:3402182) ==== #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 ==== backtrace (tid: 584987) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 ==== backtrace (tid: 584944) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 7 0x0000000000037f96 NC_infermodel() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 ==== backtrace (tid: 584941) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584942) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 ==== backtrace (tid:4110377) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 585025) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110397) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110440) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 12 0x00000000004a1fa1 def_avg_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110466) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 14 0x000000000048da4e output_() ???:0 ==== backtrace (tid:4110392) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 15 0x00000000004051fc step_() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 16 0x0000000000404a5e MAIN__() main_.f:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590348) ==== ==== backtrace (tid: 585021) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 ==== backtrace (tid: 584926) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 6 0x00000000000383cd check_file_type() dinfermodel.c:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 7 0x0000000000037f96 NC_infermodel() ???:0 5 0x00000000000387d7 closemagic() dinfermodel.c:0 8 0x0000000000023b35 NC_open() ???:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 9 0x00000000000212d8 nc_open_par() ???:0 7 0x0000000000037f96 NC_infermodel() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 8 0x0000000000023b35 NC_open() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 9 0x00000000000212d8 nc_open_par() ???:0 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 14 0x000000000048da4e output_() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584945) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 15 0x00000000004051fc step_() ???:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 16 0x0000000000404a5e MAIN__() main_.f:0 #0 0x1555530badcf in ??? 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #1 0x15555454d4a1 in opal_obj_run_destructors Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close ==== backtrace (tid:3590419) ==== at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590301) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:4110441) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590400) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 #0 0x1555530badcf in ??? 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #0 0x1555530badcf in ??? 12 0x00000000004a1fa1 def_avg_() ???:0 ==== backtrace (tid:4110374) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? 13 0x00000000004afc24 wrt_avg_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 #0 0x1555530badcf in ??? 14 0x000000000048da4e output_() ???:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 15 0x00000000004051fc step_() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3590314) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 16 0x0000000000404a5e MAIN__() main_.f:0 #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110425) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590312) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #3 0x155554570d15 in PMPI_File_close 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #0 0x1555530badcf in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #11 0x4a1fa0 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #12 0x4afc23 in ??? ==== backtrace (tid:4110407) ==== #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? ==== backtrace (tid:3590337) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590388) ==== 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590378) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:4110371) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #0 0x1555530badcf in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid:3590299) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:4110348) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402229) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590349) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110468) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? at ../opal/class/opal_object.h:481 #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590334) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590417) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590308) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590320) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590340) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590373) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590382) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590422) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590367) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590386) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110366) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110433) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #0 0x1555530badcf in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590347) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590351) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590352) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590369) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590362) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590375) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590397) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590410) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110380) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 ==== backtrace (tid: 584957) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 584931) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 584974) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 #0 0x1555530badcf in ??? 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110429) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? ==== backtrace (tid:4110373) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #18 0x40361d in ??? #19 0xffffffffffffffff in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110442) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110469) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590295) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590356) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590408) ==== #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110428) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590343) ==== ==== backtrace (tid:3590344) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 ==== backtrace (tid:3590332) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #0 0x1555530badcf in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid: 584918) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110372) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590329) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3590407) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? ==== backtrace (tid:3590319) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close ==== backtrace (tid:3590365) ==== at file/file.c:153 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402205) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402196) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402232) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402183) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590318) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590366) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? ==== backtrace (tid:3590377) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590371) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590401) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #0 0x1555530badcf in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590325) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? ==== backtrace (tid:3590415) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid:3590310) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #0 0x1555530badcf in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid: 585005) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #19 0xffffffffffffffff in ??? ================================= #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #0 0x1555530badcf in ??? #19 0xffffffffffffffff in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590368) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3402231) ==== 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 ==== backtrace (tid:3590420) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? 5 0x00000000000387d7 closemagic() dinfermodel.c:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 6 0x00000000000383cd check_file_type() dinfermodel.c:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #0 0x1555530badcf in ??? 7 0x0000000000037f96 NC_infermodel() ???:0 #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 8 0x0000000000023b35 NC_open() ???:0 #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590405) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590300) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3590302) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584984) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? ==== backtrace (tid:3590307) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584909) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3590395) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584959) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 ==== backtrace (tid:3590330) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 585000) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402212) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590385) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3402195) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590331) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? ================================= #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3402266) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #1 0x15555454d4a1 in opal_obj_run_destructors #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? at ../opal/class/opal_object.h:481 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402187) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? ==== backtrace (tid:3402271) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402198) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid: 585014) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584923) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402224) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584938) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402227) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584940) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid: 584949) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584955) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584991) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584994) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3402210) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 585022) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid: 585034) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 585035) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid: 584981) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? ==== backtrace (tid: 584935) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? ==== backtrace (tid:4110346) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? ==== backtrace (tid:4110356) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584961) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 ==== backtrace (tid:4110443) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid: 584989) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3402256) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:4110369) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584928) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110379) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584956) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close #0 0x1555530badcf in ??? ==== backtrace (tid: 584922) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402199) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:4110354) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 #0 0x1555530badcf in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:4110409) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110422) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110431) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402180) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402218) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402262) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402280) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402286) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402223) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402248) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402276) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402299) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402221) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402288) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402267) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402254) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402284) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402285) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402201) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 ==== backtrace (tid:3402189) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402297) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402259) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402263) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590363) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid:3590297) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590380) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3590305) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3402207) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 ==== backtrace (tid:3402213) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402216) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? ==== backtrace (tid:3402200) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? ==== backtrace (tid:3402214) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402253) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590404) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3402239) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 ==== backtrace (tid:3590411) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590306) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590346) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584985) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584910) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590353) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid: 584953) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590326) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? ==== backtrace (tid:3590379) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 ==== backtrace (tid: 585008) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590359) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402211) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590399) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 ==== backtrace (tid: 584990) ==== 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 ==== backtrace (tid:3590303) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 ==== backtrace (tid:3590296) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584976) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584915) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590403) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590421) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 5 0x00000000000387d7 closemagic() dinfermodel.c:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 6 0x00000000000383cd check_file_type() dinfermodel.c:0 ==== backtrace (tid:3402226) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590393) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 7 0x0000000000037f96 NC_infermodel() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 8 0x0000000000023b35 NC_open() ???:0 ==== backtrace (tid:3590321) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 12 0x00000000004a1fa1 def_avg_() ???:0 ==== backtrace (tid:3590335) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 14 0x000000000048da4e output_() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid:3590317) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 15 0x00000000004051fc step_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3590387) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 17 0x00000000004035e7 main() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 18 0x00000000000236a3 __libc_start_main() ???:0 ==== backtrace (tid:3402204) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590390) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584948) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402287) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590311) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid: 585001) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3590376) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584943) ==== ==== backtrace (tid:3590345) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 ==== backtrace (tid:3590357) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid:3590361) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 5 0x00000000000387d7 closemagic() dinfermodel.c:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590389) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584972) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590396) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 585010) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 584921) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid: 584930) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 584986) ==== #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590350) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? ==== backtrace (tid:3590416) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? ==== backtrace (tid: 584980) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590391) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 584993) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors #0 0x1555530badcf in ??? at ../opal/class/opal_object.h:481 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #2 0x15555454d4a1 in ompi_file_close #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close ==== backtrace (tid:3402293) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? ==== backtrace (tid: 584950) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #19 0xffffffffffffffff in ??? ==== backtrace (tid: 584964) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid: 584929) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid: 584998) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? ==== backtrace (tid: 584934) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #0 0x1555530badcf in ??? #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 584977) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: at ../opal/class/opal_object.h:481 ==== backtrace (tid: 585004) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #2 0x15555454d4a1 in ompi_file_close 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid: 584982) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid:3402283) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid:3402258) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110451) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? ==== backtrace (tid:4110435) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590374) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? ==== backtrace (tid:3590338) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3590328) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590341) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 585003) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 585027) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584916) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 585015) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 ==== backtrace (tid: 584925) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584951) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #4 0x155552a0d7d6 in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 585009) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #0 0x1555530badcf in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #0 0x1555530badcf in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? ==== backtrace (tid: 585029) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 584973) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #0 0x1555530badcf in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #0 0x1555530badcf in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid: 584917) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 584937) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 584979) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid: 584911) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid: 585024) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402203) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402240) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close ==== backtrace (tid:3402244) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402260) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402181) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402292) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402225) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402234) ==== ==== backtrace (tid:3402191) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402296) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402264) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402295) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402246) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402268) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402270) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402252) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402202) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402238) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402282) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402176) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3402188) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402206) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid:3402274) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:3402237) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:3402261) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3402230) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110375) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:3590383) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584912) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590315) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 584913) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:3590394) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 ==== backtrace (tid: 585006) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 ==== backtrace (tid: 584988) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 5 0x00000000000387d7 closemagic() dinfermodel.c:0 Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? 6 0x00000000000383cd check_file_type() dinfermodel.c:0 ==== backtrace (tid: 585033) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 7 0x0000000000037f96 NC_infermodel() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 8 0x0000000000023b35 NC_open() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 ==== backtrace (tid: 584914) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid: 584932) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 ==== backtrace (tid: 585030) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? 12 0x00000000004a1fa1 def_avg_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 13 0x00000000004afc24 wrt_avg_() ???:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? ==== backtrace (tid: 584933) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 14 0x000000000048da4e output_() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? 15 0x00000000004051fc step_() ???:0 #0 0x1555530badcf in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #1 0x15555454d4a1 in opal_obj_run_destructors #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? at ../opal/class/opal_object.h:481 #0 0x1555530badcf in ??? 16 0x0000000000404a5e MAIN__() main_.f:0 #2 0x15555454d4a1 in ompi_file_close #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 17 0x00000000004035e7 main() ???:0 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? 18 0x00000000000236a3 __libc_start_main() ???:0 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid: 584958) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? 19 0x000000000040361e _start() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? ================================= #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. ==== backtrace (tid: 585020) ==== #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? ==== backtrace (tid: 584966) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #18 0x40361d in ??? #19 0xffffffffffffffff in ??? Backtrace for this error: 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid: 584919) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid: 584983) ==== #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors ==== backtrace (tid: 585011) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 at ../opal/class/opal_object.h:481 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #2 0x15555454d4a1 in ompi_file_close ==== backtrace (tid: 585026) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 at file/file.c:153 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #3 0x155554570d15 in PMPI_File_close Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #4 0x155552a0d7d6 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #0 0x1555530badcf in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #0 0x1555530badcf in ??? #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #0 0x1555530badcf in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #19 0xffffffffffffffff in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? ==== backtrace (tid:4110343) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110360) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110434) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 #0 0x1555530badcf in ??? 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 ==== backtrace (tid:4110465) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 ==== backtrace (tid:4110406) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. ==== backtrace (tid:4110396) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110344) ==== ==== backtrace (tid:4110408) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:4110449) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110467) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:4110458) ==== #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 #0 0x1555530badcf in ??? 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 ==== backtrace (tid:4110436) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 5 0x00000000000387d7 closemagic() dinfermodel.c:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 6 0x00000000000383cd check_file_type() dinfermodel.c:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 7 0x0000000000037f96 NC_infermodel() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 8 0x0000000000023b35 NC_open() ???:0 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 9 0x00000000000212d8 nc_open_par() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 #0 0x1555530badcf in ??? 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 15 0x00000000004051fc step_() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 #0 0x1555530badcf in ??? 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 ==== backtrace (tid:4110399) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110350) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid:4110445) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #0 0x1555530badcf in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #0 0x1555530badcf in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110386) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110365) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110418) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110419) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110432) ==== ==== backtrace (tid:4110438) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? ==== backtrace (tid:4110376) ==== #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110370) ==== ==== backtrace (tid:4110463) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= ==== backtrace (tid:4110345) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110427) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110401) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110415) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110385) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110364) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110367) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110400) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110460) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110347) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110454) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110453) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110405) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? ==== backtrace (tid:4110395) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110357) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110446) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110417) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110352) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110358) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110423) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110362) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110389) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110412) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110398) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110437) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110355) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 #0 0x1555530badcf in ??? 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 5 0x00000000000387d7 closemagic() dinfermodel.c:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 ==== backtrace (tid:4110402) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 ==== backtrace (tid:4110426) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= #0 0x1555530badcf in ??? 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 13 0x00000000004afc24 wrt_avg_() ???:0 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110359) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110361) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110421) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110381) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110403) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110378) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110404) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110448) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110464) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110388) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110444) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110394) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110424) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110456) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110416) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110461) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110382) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110383) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: ==== backtrace (tid:4110462) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? ==== backtrace (tid:4110447) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110439) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? ==== backtrace (tid:4110455) ==== 0 0x00000000000577e3 ucs_debug_print_backtrace() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ucx-96422ce/src/ucs/debug/debug.c:656 1 0x0000000000012dd0 .annobin_sigaction.c() sigaction.c:0 2 0x00000000000464a1 opal_obj_run_destructors() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/../opal/class/opal_object.h:481 3 0x00000000000464a1 ompi_file_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/file/file.c:153 4 0x0000000000069d16 PMPI_File_close() /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 5 0x00000000000387d7 closemagic() dinfermodel.c:0 6 0x00000000000383cd check_file_type() dinfermodel.c:0 7 0x0000000000037f96 NC_infermodel() ???:0 8 0x0000000000023b35 NC_open() ???:0 9 0x00000000000212d8 nc_open_par() ???:0 10 0x0000000000021331 nc_open_par_fortran() ???:0 11 0x000000000001b31e nf_open_par_() /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 12 0x00000000004a1fa1 def_avg_() ???:0 13 0x00000000004afc24 wrt_avg_() ???:0 14 0x000000000048da4e output_() ???:0 15 0x00000000004051fc step_() ???:0 16 0x0000000000404a5e MAIN__() main_.f:0 17 0x00000000004035e7 main() ???:0 18 0x00000000000236a3 __libc_start_main() ???:0 19 0x000000000040361e _start() ???:0 ================================= Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #0 0x1555530badcf in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #0 0x1555530badcf in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #0 0x1555530badcf in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #19 0xffffffffffffffff in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? #1 0x15555454d4a1 in opal_obj_run_destructors at ../opal/class/opal_object.h:481 #2 0x15555454d4a1 in ompi_file_close at file/file.c:153 #3 0x155554570d15 in PMPI_File_close at /build-result/src/hpcx-v2.8.1-gcc-MLNX_OFED_LINUX-5.1-2.5.8.0-redhat8.2-x86_64/ompi-ae74eaa/ompi/mpi/c/profile/pfile_close.c:60 #4 0x155552a0d7d6 in ??? #5 0x155552a0d3cc in ??? #6 0x155552a0cf95 in ??? #7 0x1555529f8b34 in ??? #8 0x1555529f62d7 in ??? #9 0x1555529f6330 in ??? #10 0x1555550cb31d in nf_open_par_ at /tmp/apps/spack-stage/spack-stage-netcdf-fortran-4.5.3-3nnqwld3gtakuphoqe4q4pqnnuglpqm6/spack-src/fortran/nf_nc.f90:59 #11 0x4a1fa0 in ??? #12 0x4afc23 in ??? #13 0x48da4d in ??? #14 0x4051fb in ??? #15 0x404a5d in ??? #16 0x4035e6 in ??? #17 0x155552d096a2 in ??? #18 0x40361d in ??? #19 0xffffffffffffffff in ??? slurmstepd: error: mpi/pmix_v3: _errhandler: cn117 [0]: pmixp_client_v2.c:211: Error handler invoked: status = -25, source = [slurm.pmix.7807171.0:124] slurmstepd: error: *** STEP 7807171.0 ON cn117 CANCELLED AT 2024-06-03T11:25:11 *** slurmstepd: error: mpi/pmix_v3: _errhandler: cn237 [3]: pmixp_client_v2.c:211: Error handler invoked: status = -25, source = [slurm.pmix.7807171.0:477] srun: Job step aborted: Waiting up to 17 seconds for job step to finish. slurmstepd: error: mpi/pmix_v3: _errhandler: cn142 [1]: pmixp_client_v2.c:211: Error handler invoked: status = -25, source = [slurm.pmix.7807171.0:239] slurmstepd: error: mpi/pmix_v3: _errhandler: cn164 [2]: pmixp_client_v2.c:211: Error handler invoked: status = -25, source = [slurm.pmix.7807171.0:290] srun: error: cn117: tasks 0-127: Segmentation fault (core dumped) srun: launch/slurm: _step_signal: Terminating StepId=7807171.0 srun: error: cn142: tasks 128-255: Segmentation fault (core dumped) srun: error: cn237: tasks 384-511: Segmentation fault (core dumped) srun: error: cn164: tasks 256-383: Segmentation fault (core dumped) real 25m54.371s user 0m0.200s sys 0m1.291s