Problem adding fmriprep container to subdataset with datalad containers-add

#1

Hi DataLad Team,

I want to add an mriqc and fmriprep container to the pipeline subdataset of my superdataset using datalad containers-add on our lab server running Ubuntu 20.04.3 (multiple users within the lab, no HPC) and DataLad 0.15.3.

For mriqc, this works without problems pulling the image from singularity hub:

datalad containers-add mriqc --url shub://ReproNim/containers:bids-mriqc--0.16.1 --call-fmt 'singularity run --cleanenv -B "$PWD,/data/singularity/:/data/singularity/" {img} {cmd}'

After this, datalad containers-run works perfectly

datalad containers-run -m "run mriqc on sub-KUL001" -i "BIDS/sub-KUL001/*" -o "mriqc/*" -n pipeline/mriqc BIDS mriqc participant --participant-label KUL001 --verbose --verbose-reports --fd_thres 0.9

However, since I cannot find the latest or a recent fmriprep container on singularity hub, I tried to add the image from Docker locally

datalad containers-add fmriprep --url docker://nipreps/fmriprep:20.2.6 --call-fmt 'singularity run --cleanenv -B "$PWD,/data/singularity/:/data/singularity/" {img} {cmd} --fs-license-file "$PWD/{img_dspath}/license.txt"'

I first ran into errors with permissions, but after solving those the container was succesfully added and saved to my subdataset, albeit with the following kind of warnings during installation

2021/11/25 11:22:48 warn rootless{dev/agpgart} creating empty file in
place of device 10:175

However, when trying to run fmriprep from the container

datalad containers-run -m "run fmriprep on sub-KUL001" -i "BIDS/sub-KUL001/*" -o "derivatives/fmriprep/*" -n pipeline/fmriprep BIDS derivatives participant --participant-label KUL001 --fs-no-reconall --fd-spike-threshold 0.9 --dvars-spike-threshold 2 --verbose --output-spaces MNI152NLin2009cAsym:res-2 --ignore fieldmaps --use-syn-sdc

I got the following output (hence no fmriprep run)

subdataset(ok): BIDS (dataset)
subdataset(ok): code (dataset)
subdataset(ok): derivatives (dataset)
subdataset(ok): mriqc (dataset)
subdataset(ok): pipeline (dataset)
subdataset(ok): sourcedata (dataset)
[INFO ] Making sure inputs are available (this may take some time)
[INFO ] == Command start (output follows) =====
WARNING: passwd file doesn’t exist in container, not updating
WARNING: group file doesn’t exist in container, not updating
INFO: No runscript found in container, executing /bin/sh
[INFO ] == Command exit (modification check follows) =====
action summary:
get (notneeded: 17)
save (notneeded: 7)

Do you have any suggestion on how to solve this? I guess datalad containers-remove and then datalad containers-add again from another location? Do you know of a singularity image for fmriprep 20.2.6 I can use for this purpose? Any other issues?

Thanks a lot in advance!

Best wishes,

Lukas

#2

quick one: didn’t look in detail, but fmriprep docker->singularity conversion can be tricky due to its sheer size. FWIW, I have managed to make it work and that image is available as a part of the ///repronim/containers . Try it out (just install as a subdataset), see e.g. https://github.com/ReproNim/containers#a-typical-workflow for more info etc.

#3

FWIW and FYI, singularity hub is now in a read only mode so no new containers are built, and it is also available (and served from) as a DataLad dataset at DataLad Repository

#5

Thanks Yarik!

I was looking for a solution to add one of the containers to my already existing pipeline subdataset, FYI this worked fine with

datalad containers-add fmriprep --url http://datasets.datalad.org/repronim/containers/images/bids/bids-fmriprep--20.2.6.sing --call-fmt 'singularity run --cleanenv -B "$PWD,/data/singularity/:/data/singularity/" {img} {cmd} --fs-license-file "$PWD/{img_dspath}/license.txt"'

MANAGED BY INCF