Tractoflow/nextflow not finding memory resource

Following up on some previous posts, I’m trying to test run Tractoflow on an interactive node and am running into a memory allocation issue. I am on a node with 128G mem/16 cores but when I launch Tractoflow, I get this error (full output at the bottom of post):

Local avail `memory` attribute cannot zero. Expression: (availMemory > 0). Values: availMemory = 0

 -- Check script 'tractoflow/main.nf' at line: 352 or see '.nextflow.log' file for more details

I’m checking with my HPC admins to see if there’s some permissions error in accessing memory, for whatever reason. Not sure if this could be caused by the fact that I’m using a locally installed Nextflow? In the meantime, if you have any ideas about this, please let me know! Thanks.

Here is the full output and also the lines referred to in main.nf.

Full output:

nsharif@c29:/nafs/narr/nooralsh>/nafs/narr/nooralsh/nextflow run /nafs/narr/nooralsh/tractoflow/main.nf --input /nafs/narr/nooralsh/input/ --fodf_shells "5 1500 3000" -with-singularity /nafs/narr/nooralsh/tractoflow_2.2.1.sif -w /nafs/narr/nooralsh/tf_results/ -resume
NOTE: Nextflow is not tested with Java 1.8.0_282 -- It's recommended the use of version 11 up to 18

N E X T F L O W  ~  version 22.04.5
Launching `/nafs/narr/nooralsh/tractoflow/main.nf` [festering_fermi] DSL1 - revision: 3fb5c2dda4
TractoFlow pipeline
===================

Start time: 2022-09-28T10:01:16.835-07:00

Max DTI shell extracted: 1200
DTI shells extracted: 5 1500 3000
Input: /nafs/narr/nooralsh/input/
Local avail `memory` attribute cannot zero. Expression: (availMemory > 0). Values: availMemory = 0

 -- Check script 'tractoflow/main.nf' at line: 352 or see '.nextflow.log' file for more details

main.nf lines:

 352 process README {
 353     cpus 1
 354     publishDir = params.Readme_Publish_Dir
 355     tag = "README"
 356
 357     output:
 358     file "readme.txt"
 359
 360     script:
 361     String list_options = new String();
 362     for (String item : params) {
 363         list_options += item + "\n"
 364     }
 365     """
 366     echo "TractoFlow pipeline\n" >> readme.txt
 367     echo "Start time: $workflow.start\n" >> readme.txt
 368     echo "[Command-line]\n$workflow.commandLine\n" >> readme.txt
 369     echo "[Git Info]\n" >> readme.txt
 370     echo "$workflow.repository - $workflow.revision [$workflow.commitId]\n" >> readme.txt
 371     echo "[Options]\n" >> readme.txt
 372     echo "$list_options" >> readme.txt
 373     """
 374 }

Hi @nooralsh ,

First thing I would like you to try is to use nextflow 21.* version instead of 22.x and make sure there is no warning about java version.
We’ve seen a couple of issues with nextflow 22.* also never related to memory.

Arnaud

Hi Arnaud,

I reran as you suggested, now using nextflow 21.20.6 and I used the “nextflow-21.10.6-all” version to avoid other dependency errors (is that the correct thing to do?) but I still got the same error. It doesn’t show the java warning anymore though. This occurs both in an interactive node and when submitted via sbatch.

Our IT admin found this message (nextflow-io/nextflow - Gitter) from the old NF forums but they resolved their issue by resetting java or loading a different version. I don’t seem to be having an issue with java versions anymore using Nextflow 21.x and using unset _JAVA_OPTIONS didn’t do anything.

nsharif@c19:/nafs/narr/nooralsh>/nafs/narr/nooralsh/nextflow-21.10.6-all run /nafs/narr/nooralsh/tractoflow/main.nf --input /nafs/narr/nooralsh/input/ --fodf_shells "5 1500 3000" -with-singularity /nafs/narr/nooralsh/tractoflow_2.2.1.sif -w /nafs/narr/nooralsh/tf_results/ -resume

N E X T F L O W  ~  version 21.10.6
Launching `/nafs/narr/nooralsh/tractoflow/main.nf` [stupefied_almeida] - revision: 3fb5c2dda4
TractoFlow pipeline
===================

Start time: 2022-09-30T08:25:33.880-07:00

Max DTI shell extracted: 1200
DTI shells extracted: 5 1500 3000
Input: /nafs/narr/nooralsh/input/
Local avail `memory` attribute cannot zero. Expression: (availMemory > 0). Values: availMemory = 0

 -- Check script 'tractoflow/main.nf' at line: 352 or see '.nextflow.log' file for more details

I have a small update to this even though the issue is not resolved: I launched a run on the login node instead of the interactive node or the queue and everything worked as expected (or at least, the typical process begins; I cancelled the process before it completed). So there must be some disconnect between Nextflow settings on the login node and on the cluster nodes? Any idea what that might be or how to fix it?