Question about running abcd-hcp-pipeline in singularity

1

I am interested in comparing fmriprep versus abcd-hcp-pipeline on my data. I am using singularity-ce version 3.8.0 on CentOS Linux 7

fmriprep
I have successfully built an fmriprep singularity container and executed it on my validated BIDS formatted dataset. There were no issues with permissions for input/output directories or finding my fsl license file. The container runs correctly and produces the desired outputs.

abcd-hcp-pipeline
I am having trouble getting the abcd-hcp-pipeline to run correctly. I built the container as follows

singularity build abcd-hcp-pipeline.img docker://dcanlabs/abcd-hcp-pipeline

This build was successful and I can run the container to get the version number as follows:

singularity run /home/kabush/singularity_images/abcd-hcp-pipeline.img --version

which returns:

abcd-hcp-pipeline 0.0.1

But when I try to give it a bids dataset, output path, and license path:

singularity run /home/kabush/singularity_images/abcd-hcp-pipeline.img 
/home/kabush/workspace/bids/DIAL /home/kabush/workspace/data/EmoReg/derivatives 
--freesurfer-license /home/kabush/licenses/fsl_license.txt

it returns:

Traceback (most recent call last):
 File "/app/run.py", line 364, in <module>
_cli()
File "/app/run.py", line 67, in _cli
return interface(**kwargs)
File "/app/run.py", line 238, in interface
validate_license(freesurfer_license)
File "/app/helpers.py", line 315, in validate_license
'freesurfer license.txt not located. You can provide a license ' \
AssertionError: freesurfer license.txt not located. You can provide a license file using the -- 
freesurfer-license <LICENSE> argument.
/home/kabush/workspace/data/EmoReg/derivatives: Permission denied.

I have verified all of the paths. Given that the fmriprep container works (finds the fsl license file and writes to the same derivatives output path), this would suggest it is not a singularity configuration/permissions problem. Does anyone have thoughts?

2

Hi @kabush

I think you need to rename your file fsl_license.txt to license.txt. And try binding your “project directory” /home/kabush to make it visible inside the container.

I.e., singularity run -B /home/kabush /home/kabush/singularity_images/abcd-hcp-pipeline.img etc etc

3

I moved everything into a csh script to simplify and ran the code according to the suggestion.

#! /bin/csh
set abcdhcppath = /home/kabush/singularity_images/abcd-hcp-pipeline.img
set projpath = /home/kabush
set inpath = /home/kabush/workspace/bids/DIAL
set outpath = /home/kabush/workspace/data/EmoReg/derivatives
set licensepath = /home/kabush/fsl_license/license.txt

singularity run \
-B ${projpath} \
${abcdhcppath} \
${inpath} \
${outpath} \
–freesurfer-license ${licensepath}

which returned the same errors:

Traceback (most recent call last):
File “/app/run.py”, line 364, in
_cli()
File “/app/run.py”, line 67, in _cli
return interface(**kwargs)
File “/app/run.py”, line 238, in interface
validate_license(freesurfer_license)
File “/app/helpers.py”, line 315, in validate_license
'freesurfer license.txt not located. You can provide a license ’
AssertionError: freesurfer license.txt not located. You can provide a license file using the --freesurfer-license argument.
/home/kabush/workspace/data/EmoReg/derivatives: Permission denied.

I also tried explicitly binding all of the individual paths. Same result.

4

In my experience, most issues with running singularity images has to do with environmental variables messing up stuff and how the binding is performed. Binding to locations overlapping with your $HOME directory is especially tricky (at least on our cluster).

Safest is to bind to folders you know exist within the container. From the Dockerfile https://github.com/DCAN-Labs/abcd-hcp-pipeline/blob/master/Dockerfile you see that the developers have made some directories available for input/output etc.

image
image720×98 10.2 KB
.

As Freesurfer is included in the image, you can put the licence.txt directly into the $FREESURFER_HOME director which should be `/opt/freesurfer (as seen in https://github.com/DCAN-Labs/abcd-hcp-pipeline/blob/master/app/SetupEnv.sh).

The command would then be something like:

singularity run --contain --cleanenv \
-B /home/kabush/workspace/bids/DIAL:/bids_input \
-B /home/kabush/workspace/data/EmoReg/derivatives:/output \
-B /home/kabush/fsl_license/license.txt:/opt/freesurfer/license.txt \
/home/kabush/singularity_images/abcd-hcp-pipeline.img \
/bids_input \
/output

As you can see, the image is now run looking for a bids-structure in /bids_input within the container (and you just put it there via binding!). And output will be places in /output within the container (which corresponds to your derivatives folder on your computer). Don’t think there’s any need for the --freesurfer-license argument when the license already is present within $FREESURFER_HOME (but including --freesurfer-license /opt/freesurfer/license.txt wouldn’t hurt)

5

That worked and your description was very helpful! Thank you. I’m now receiving an error related to
“PixelBandwidth”

Traceback (most recent call last):
File “/app/run.py”, line 364, in
_cli()
File “/app/run.py”, line 67, in _cli
return interface(**kwargs)
File “/app/run.py”, line 263, in interface
session_spec = ParameterSettings(session, out_dir)
File “/app/pipelines.py”, line 106, in init
self.bids_data[‘t1w_metadata’])
File “/app/helpers.py”, line 231, in get_realdwelltime
pBW = metadata[‘PixelBandwidth’]
KeyError: ‘PixelBandwidth’

This seems to be a known problem with this pipeline:

As this data does comes from a Philips scanner which aligns with the troubleshooting help, this is a problem with the formatting of the BIDS rather than singularity. I think this ticket is solved. Thank you very much.