I’m hoping this question isn’t too dependent on the site that I’m working at, but I’m a bit unclear about how I should be using the n_threads and mem_mb argument when submitting jobs to an HPC. I am running fmriprep through singularity. I am submitting the job using a command bsub, which requires specification of both the number of processors to use and the maximum amount of memory each processor may need. If I’ve specified the number of processors and RAM through bsub, should I also specify n_threads and mem_mb in the call to fmriprep? Or, will fmriprep only attempt to use the resources specified by the bsub command?
It depends on how your HPC is configured. If it uses cgroups or some other sandboxing technique to limit which resources are visible/available to each job you do not need to pass any parameters to fmriprep (it will figure out the optimal resource allocation by itself).
However, very few HPC actually use proper sandboxing. In such case passing the resource parameters to fmriprep will ensure that it does not try to use other resources allocated to other jobs running on the same node.
In short setting n_threads and mem_mb to match your job submission parameters does not hurt and depending on the HPC configuration it can prevent overallocation of resources.