I’m hoping this question isn’t too dependent on the site that I’m working at, but I’m a bit unclear about how I should be using the
mem_mb argument when submitting jobs to an HPC. I am running fmriprep through singularity. I am submitting the job using a command
bsub, which requires specification of both the number of processors to use and the maximum amount of memory each processor may need. If I’ve specified the number of processors and RAM through
bsub, should I also specify
mem_mb in the call to fmriprep? Or, will fmriprep only attempt to use the resources specified by the
It depends on how your HPC is configured. If it uses cgroups or some other sandboxing technique to limit which resources are visible/available to each job you do not need to pass any parameters to fmriprep (it will figure out the optimal resource allocation by itself).
However, very few HPC actually use proper sandboxing. In such case passing the resource parameters to fmriprep will ensure that it does not try to use other resources allocated to other jobs running on the same node.
In short setting
mem_mb to match your job submission parameters does not hurt and depending on the HPC configuration it can prevent overallocation of resources.