MRIQC error on Singularity

,
1

Hi,

I am trying to run mriqc (v 0.15.2) on our HPC. I first built the image with the following command

singularity pull --name mriqc_0.15.2.sif docker://poldracklab/mriqc:0.15.2

and then I run it with

singularity run --bind /scratch/punim1233:/scratch/punim1233 /scratch/punim1233/mriqc_0.15.2.sif /scratch/punim1233/BIDS /scratch/punim1233/ELM_mriqc participant 
-w /scratch/punim1233/work/ -vvv

It think it starts, but it crashes shortly after

Running MRIQC version 0.15.2:
      * BIDS dataset path: /scratch/punim1233/BIDS.
      * Output folder: /scratch/punim1233/ELM_mriqc.
      * Analysis levels: ['participant'].

200415-20:33:19,45 nipype.utils INFO:
	 No new version available.
200415-20:33:20,278 nipype.utils INFO:
	 No new version available.
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/mriqc/cli/workflow.py", line 21, in build_workflow
    retval["workflow"] = init_mriqc_wf()
  File "/usr/local/miniconda/lib/python3.7/site-packages/mriqc/workflows/core.py", line 17, in init_mriqc_wf
    workflow.add_nodes([fmri_qc_workflow()])
  File "/usr/local/miniconda/lib/python3.7/site-packages/mriqc/workflows/functional.py", line 95, in fmri_qc_workflow
    ema = epi_mni_align()
  File "/usr/local/miniconda/lib/python3.7/site-packages/mriqc/workflows/functional.py", line 716, in epi_mni_align
    template_resolution=2, ),
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/mixins/reporting.py", line 34, in __init__
    super(ReportCapableInterface, self).__init__(**kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/interfaces/mni.py", line 98, in __init__
    super(RobustMNINormalization, self).__init__(**inputs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 183, in __init__
    self.inputs = self.input_spec(**inputs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/specs.py", line 66, in __init__
    super(BaseTraitedSpec, self).__init__(**kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 329, in validate
    value = super(File, self).validate(objekt, name, value, return_pathlike=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 134, in validate
    self.error(objekt, name, str(value))
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 172, in error
    value )
traits.trait_errors.TraitError: The 'reference_image' trait of a _RobustMNINormalizationInputSpecRPT instance must be a pathlike object or string representing an 
existing file, but a value of '[]' <class 'str'> was specified.

I am not sure what I am doing wrong. I have run mriqc previously via docker and that was fine, but I would like to run it on the HPC now.

Any help would be really appreciated, thanks in advance!

2

Hi,

Just a follow up. I am now trying to run mriqc on Docker and I still have some troubles with version 15.0.2. From the terminal, it looks like it is stuck on looking for updates (and found none) and when I kill the process it displays:

KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/concurrent/futures/process.py", line 232, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 70, in run_node
    result["result"] = node.result
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 217, in result
    op.join(self.output_dir(), "result_%s.pklz" % self.name)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/utils.py", line 291, in load_resultfile
    raise FileNotFoundError(results_file)
FileNotFoundError: /Users/elena/Desktop/work/mriqc_wf/funcMRIQC/fMRI_HMC_afni/_in_file_..data..sub-E127..func..sub-E127_task-renewal_bold.nii/motion_correct/result_motion_correct.pklz
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/concurrent/futures/_base.py", line 324, in _invoke_callbacks
    callback(self)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
  File "/usr/local/miniconda/lib/python3.7/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/local/miniconda/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
FileNotFoundError: /Users/elena/Desktop/work/mriqc_wf/funcMRIQC/fMRI_HMC_afni/_in_file_..data..sub-E127..func..sub-E127_task-renewal_bold.nii/motion_correct/result_motion_correct.pklz

I am now trying v 15.0 and it seems to be running. The command I was using is the following:

docker run -it --rm -v /Users/elena/Desktop/BIDS:/data:ro -v /Users/elena/Desktop/ELM_mriqc:/out poldracklab/mriqc:0.15.2 /data /out participant group -w /Users/elena/Desktop/work

Thanks in advance, and sorry for the multiple messages.

3

Hi @ElenaP,

I don’t have anything to about your errors right now, but I did notice something odd from your command.

As far I know, you cannot specify participant and group in the same invocation of mriqc. First you only specify participant, then when you have all the individual results you replace participant with group.

Hope that helps with something,
James

4

Thank you @jdkent! Unfortunately I have the same issues even if I only specify participant.

5

hmm,

I also just noticed this in your docker command: -w /Users/elena/Desktop/work, which is a path that probably doesn’t exist in the docker container unlike /data and /out which you explicitly say to exist, I’m unsure of what the behavior would be if the docker image did not have access to a working directory (my guess would be that mriqc would complain loudly and not hang), but you can try this command:

docker run -it --rm \
-v /Users/elena/Desktop/BIDS:/data:ro \
-v /Users/elena/Desktop/ELM_mriqc:/out \
-v /Users/elena/Desktop/work:/work \
poldracklab/mriqc:0.15.2 /data /out participant -w /work

and for testing the command out, I highly recommend using --participant-label to select a particular participant label (i.e., 01), especially if you have a large dataset as this will also help narrow down what could be going wrong.

Hope these suggestions will help point you towards a solution.
James

1 Like
6

Thank you very much for the suggestions! I tried with only one subject and the changes you suggested, and it hung for ages on nipype.utils INFO: No new version available. and did not print any of the usual information in the terminal about the process but it worked.

With Singularity, the command

singularity run --cleanenv --bind /scratch/punim1233/ mriqc_0.15.2.sif /scratch/punim1233/BIDS/ /scratch/punim1233/ELM_mriqc participant --participant-label E005

did not work for v 0.15.2 and gave the error I posted

The 'reference_image' trait of a _RobustMNINormalizationInputSpecRPT instance must be a pathlike object or string representing an existing file, but a value of '[]' <class 'str'> was specified

but run successfully when I used v 0.15.0.

I will now try to process the whole dataset !

7

Awesome that v0.15.0 worked on a participant.

Your error with v0.15.2 is starting to look like a bug to me, pinging @oesteban since he may be able to give more insight on this potential bug.

Best,
James

8

Thanks so much @jdkent for your help. Just confirming that:

  1. with the command I posted just above singularity run --cleanenv --bind /scratch/punim1233/ mriqc_0.15.0.sif /scratch/punim1233/BIDS/ /scratch/punim1233/ELM_mriqc participant I ran successfully my whole dataset and produced group plot afterwards - with mriqc v 0.15.0 on HPC via Singularity. However, running the exactly same command with v 0.15.2 produced that reference_image error.

  2. with the command suggested above by @jdkent , I run successfully my whole dataset with v 0.15.2 on the local machine via Docker. On the terminal I cannot see any of the usual printing about the processing and it looks like it is hanging, but this could be a change implemented in the new version, whereby if you don’t add verbose in your command it won’t print what it is doing?

I would like to understand why I couldn’t get v 0.15.2 to work with Singularity, so if @oesteban has some clues that would be really appreciated!

Thank you very much.