Updated: Fmriprep: Workaround for running subjects in parallel

1

Currently, trying to run multiple fmriprep subjects in parallel leads to errors. The solution is to make sure the first job runs a bit before all the other jobs. One potential way to do this is simply to add sleep 1m && before every job except for the first:

[submit job: ] fmriprep data/bids_root/ out/ participant --participant_label 01 -w work/
[submit job: ] sleep 1m && fmriprep data/bids_root/ out/ participant --participant_label 02 -w work/
[submit job: ] sleep 1m && fmriprep data/bids_root/ out/ participant --participant_label 03 -w work/
[submit job: ] sleep 1m && fmriprep data/bids_root/ out/ participant --participant_label 04 -w work/

Note that [submit job: ] should be replaced with whatever command you use to submit a job. This solution assumes you already know more generally how to run tasks in parallel on your system.

2 Likes
2

Reading your response jbwexler, I am wondering if have any recommendations for a similar thing in hpc with fMRIprep via singularity and a parallel GNU, e.g. https://github.uconn.edu/HPC/parallel-slurm.git

I am trying to set the code to read-in consecutive subjectIDs from a .txt file and submit the same job using:
cat subjID.txt | parallel myscript.sh

but I keep getting errors, and I am not sure if this is because the variable for subjID was defined as s=$1 instead of s=+$1, or loop with something like

for line in subjID.txt:
submit myscript.sh line

Thank you in advance!