MRIQC / ModuleNotFoundError: No module named ‘niworkflows.interfaces.utils’

WenjunSu · October 11, 2021, 3:09am

Hi there,

I’m running MRIQC on one subject with docker but running into the following problem:

ndyxs@ndyxs-thinkserver:~$ docker run -it --rm -v /media/ndyxs/WD12_3/test20211008/bidsdata/:/data:ro -v /media/ndyxs/WD12_3/test20211008/MRIQC_output/:/out poldracklab/mriqc:latest /data /out participant --participant_label sub-0100
211009-05:25:22,386 cli IMPORTANT:

Running MRIQC version 0.16.1+81.g0093e62.dirty:
  * BIDS dataset path: /data.
  * Output folder: /out.
  * Analysis levels: ['participant'].

Process Process-2:
Traceback (most recent call last):
File “/opt/conda/lib/python3.8/multiprocessing/process.py”, line 315, in _bootstrap
self.run()
File “/opt/conda/lib/python3.8/multiprocessing/process.py”, line 108, in run
self._target(*self._args, **self._kwargs)
File “/opt/conda/lib/python3.8/site-packages/mriqc/cli/workflow.py”, line 21, in build_workflow
retval[“workflow”] = init_mriqc_wf()
File “/opt/conda/lib/python3.8/site-packages/mriqc/workflows/core.py”, line 22, in init_mriqc_wf
workflow.add_nodes([fmri_qc_workflow()])
File “/opt/conda/lib/python3.8/site-packages/mriqc/workflows/functional.py”, line 43, in fmri_qc_workflow
from niworkflows.interfaces.utils import SanitizeImage
ModuleNotFoundError: No module named ‘niworkflows.interfaces.utils’

I have tried to install niworkflows but the same problem still exists.

Anyone know how to fix it I would appreciate your help

Thank you very much!

Steven · October 12, 2021, 1:28pm

Are you by chance using a working directory that has results from a different MRIQC version? Try specifying a clean working directory and see if you get the same error.

WenjunSu · October 13, 2021, 4:24pm

Hi Steven - Nope. That was my first time downloading MRIQC. However, this issue was fixed after I switching to nipreps/mriqc:21.0.0rc2.

Suzzie · December 6, 2021, 12:29pm

Hello, I have encountered the same error while using the MRIQC:latest singularity image on HPC. It is also my first time using the MRIQC.
Any idea what would be causing the error?

Steven · December 6, 2021, 2:29pm

Hi Suzzie, it would help us to see

The MRIQC command
Log/crash files
Whether your dataset is BIDS valid

Best,
Steven

Suzzie · December 7, 2021, 4:19am

Hi Steven,
Thank you for the help!

part of PBS file
==========================================================

/usr/local/bin/singularity run $HOME/mriqc.simg
bids_root_dir/sub-{subj} bids_root_dir/derivatives/mriqc/sub-{subj}
participant
–participant-label $subj
–correct-slice-timing \

==========================================================

Crash file
==========================================================

Running MRIQC version 0.16.1+81.g0093e62.dirty:

BIDS dataset path: /home/user1_/BIDS/01/sub-01.
Output folder: /home/user_1/BIDS/01/derivatives/mriqc/sub-01.
Analysis levels: [‘participant’].

Process Process-2:
Traceback (most recent call last):
File “/opt/conda/lib/python3.8/multiprocessing/process.py”, line 315, in _bootstrap
self.run()
File “/opt/conda/lib/python3.8/multiprocessing/process.py”, line 108, in run
self._target(*self._args, **self._kwargs)
File “/opt/conda/lib/python3.8/site-packages/mriqc/cli/workflow.py”, line 21, in build_workflow
retval [“workflow”] = init_mriqc_wf()
File “/opt/conda/lib/python3.8/site-packages/mriqc/workflows/core.py”, line 22, in init_mriqc_wf
workflow.add_nodes([fmri_qc_workflow()])
File “/opt/conda/lib/python3.8/site-packages/mriqc/workflows/functional.py”, line 43, in fmri_qc_workflow
from niworkflows.interfaces.utils import SanitizeImage
ModuleNotFoundError: No module named ‘niworkflows.interfaces.utils’

==========================================================

Yes, it has been validated by bids-validator.

Thank you:)

Steven · December 7, 2021, 4:34am

Hi Suzzie, like the original poster, you should try using the 21.0.0 prerelease (rc2).

Suzzie · December 7, 2021, 4:50am

Sure, I will try using that, but is it possible to get the singularity image of a 21.0.0 prerelease from the docker itself?

WenjunSu · December 7, 2021, 4:58am

Hi Suzzie,

docker pull nipreps/mriqc:21.0.0rc2 will do it:)

Suzzie · December 7, 2021, 7:29am

It works! Thank you all:)

kpunjabi · February 7, 2022, 10:44am

Hi,
I’m using the latest version of MRIQC and nipreps, but I’m still getting the same error

I used this command:
sudo docker run -it --rm -v /Volumes/DATA_TEAM/CNS_HUMAN/kpunjabi/7T/bids_epinov/:/data:ro -v /Volumes/DATA_TEAM/CNS_HUMAN/kpunjabi/7T/derivatives/mriqc/:/out poldracklab/mriqc:latest /data /out participant -m t1w -m bold --verbose-reports --no-sub --nprocs 10

And I get the following error:

===============================
Running MRIQC version 0.16.1+81.g0093e62.dirty:
* BIDS dataset path: /data.
* Output folder: /out.
* Analysis levels: [‘participant’].

220207-10:38:36,881 nipype.utils WARNING:
A newer version (1.7.0) of nipy/nipype is available. You are using 1.6.1
220207-10:38:38,627 nipype.utils WARNING:
A newer version (1.7.0) of nipy/nipype is available. You are using 1.6.1
Process Process-2:
Traceback (most recent call last):
File “/opt/conda/lib/python3.8/multiprocessing/process.py”, line 315, in _bootstrap
self.run()
File “/opt/conda/lib/python3.8/multiprocessing/process.py”, line 108, in run
self._target(*self._args, **self._kwargs)
File “/opt/conda/lib/python3.8/site-packages/mriqc/cli/workflow.py”, line 21, in build_workflow
retval[“workflow”] = init_mriqc_wf()
File “/opt/conda/lib/python3.8/site-packages/mriqc/workflows/core.py”, line 22, in init_mriqc_wf
workflow.add_nodes([fmri_qc_workflow()])
File “/opt/conda/lib/python3.8/site-packages/mriqc/workflows/functional.py”, line 43, in fmri_qc_workflow
from niworkflows.interfaces.utils import SanitizeImage
ModuleNotFoundError: No module named ‘niworkflows.interfaces.utils’

======================================
Your help is much appreciated
Thanks

WenjunSu · February 7, 2022, 11:01am

Hi,

The poldracklab version is not the latest release. You can try pull nipreps/mriqc:21.0.0.rc2.

kpunjabi · February 8, 2022, 8:19am

Thanks, it worked but generates this log and doesn’t proceed

==================
A newer version (1.7.0) of nipy/nipype is available. You are using 1.6.1
220208-02:15:26,445 nipype.utils WARNING:
A newer version (1.7.0) of nipy/nipype is available. You are using 1.6.1
exception calling callback for <Future at 0x7f367194f730 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
exception calling callback for <Future at 0x7f3671967fd0 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
exception calling callback for <Future at 0x7f36719a7d90 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

Steven · February 8, 2022, 1:52pm

This might be related to memory usage. How much memory/cpu are you devoting to the job. Are you using a fresh work directory?

kpunjabi · February 8, 2022, 2:29pm

12gb memory and 10 procs. This is the command I’m using sudo docker run -it --memory="12g" --rm -v /media/khushboo/Seagate_Mac/7T/bids:/data:ro -v /media/khushboo/Seagate_Mac/7T/derivatives/mriqc:/out nipreps/mriqc:21.0.0rc2 /data /out participant --participant_label pilote3a -m bold --verbose-reports --no-sub --nprocs 12

Steven · February 8, 2022, 6:48pm

Can you try to specify a work directory with the -w argument for mriqc?

kpunjabi · February 9, 2022, 12:47pm

I tried adding a working directory, but the same problem persists.

Steven · February 9, 2022, 7:23pm

Can you try removing all mriqc flags besides the participant_label and work directory?

kpunjabi · February 9, 2022, 10:16pm

Used the following command
sudo docker run -it --rm -v /media/khushboo/Seagate_Mac/7T:/data:ro -v /media/khushboo/Seagate_Mac/7T/derivatives/mriqc:/out nipreps/mriqc:21.0.0rc2 /data /out participant --participant_label pilote3a -m bold -w /media/khushboo/Seagate_Mac/7T/derivatives/temp

Still the same error.

==============
220209-23:10:26,253 nipype.utils WARNING:
A newer version (1.7.0) of nipy/nipype is available. You are using 1.6.1
exception calling callback for <Future at 0x7f754f743670 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
exception calling callback for <Future at 0x7f758337b550 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
exception calling callback for <Future at 0x7f758337b3a0 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 328, in _invoke_callbacks
callback(self)
File “/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py”, line 159, in _async_callback
result = args.result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 432, in result
return self.__get_result()
File “/opt/conda/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.