Nipype: TCompCor error

Hi! I’ve been working on implementing a ACompCor/TCompCor workflow (similar to fmriprep’s confound workflow), but I’m experiencing some difficulty with the TCompCor node. (I’m using Nipype version 0.13.0-dev.)

The error seems to be in the “Collecting precomputed outputs” stage of the node, as shown below:

170429-10:10:05,94 workflow INFO:
Collecting precomputed outputs
170429-10:10:05,99 workflow ERROR:
[u’Node tcompcor failed to run on host laptop.’]
170429-10:10:05,100 workflow INFO:
Saving crash info to /home/lukas/spynoza/crash-20170429-101005-lukas-tcompcor-414c6dea-19a7-4bc4-9b17-065374a55a88.pklz
170429-10:10:05,100 workflow INFO:
Traceback (most recent call last):
File “/home/lukas/anaconda2/lib/python2.7/site-packages/nipype-0.13.0_ga17af6b.dev-py2.7.egg/nipype/pipeline/plugins/linear.py”, line 43, in run
node.run(updatehash=updatehash)
File “/home/lukas/anaconda2/lib/python2.7/site-packages/nipype-0.13.0_ga17af6b.dev-py2.7.egg/nipype/pipeline/engine/nodes.py”, line 366, in run
self._run_interface()
File “/home/lukas/anaconda2/lib/python2.7/site-packages/nipype-0.13.0_ga17af6b.dev-py2.7.egg/nipype/pipeline/engine/nodes.py”, line 1281, in _run_interface
updatehash=updatehash))
File “/home/lukas/anaconda2/lib/python2.7/site-packages/nipype-0.13.0_ga17af6b.dev-py2.7.egg/nipype/pipeline/engine/nodes.py”, line 1166, in _collate_results
values.insert(i, node.result.outputs.get()[key])
AttributeError: ‘unicode’ object has no attribute ‘insert’

I specify my TCompCor node, which is a MapNode iterating over two functional files (each with it’s own mask_file), as follows:

tcompcor = pe.MapNode(TCompCor(components_file=‘tcomcor_comps.txt’),
iterfield=[‘realigned_file’, ‘mask_file’],
name=‘tcompcor’)

The execution outputs (from the report.rst files) looks fine, as pasted below:

  • components_file : /tmp/spynoza/workingdir/compcor/tcompcor/mapflow/_tcompcor0/tcomcor_comps.txt
  • header :
  • high_variance_mask : /tmp/spynoza/workingdir/compcor/tcompcor/mapflow/_tcompcor0/mask.nii
  • ignore_exception : False
  • mask_file :
  • num_components : 6
  • realigned_file :
  • regress_poly_degree : 1
  • use_regress_poly : True

Maybe I’m doing something stupid, but after hours of debugging, I gave up. Hope you guys can help me!

All the best,
Lukas

There was a recent API change - see https://github.com/nipy/nipype/issues/1982

In short use mask_files input instead of mask_file.

Out of curiosity - is there anything in FMRIPREP that you were missing that motivated you to build your own pipeline?

Ah, using mask_files instead of mask_file helped to fix the error for the ACompCor node, but unfortunately not in the TCompCor node … I still get the values.insert(i, node.result.outputs.get()[key]) error from the _collate_results method from the Node object. The outputs (high-variance mask, mask.nii, and the compcor-textfile) are correctly created, though … All the code/tests are organized in the following repo: https://github.com/spinoza-centre/spynoza/tree/develop (develop branch!), and the compcor-workflow is defined in https://github.com/spinoza-centre/spynoza/blob/develop/spynoza/denoising/compcor/workflows.py, it you want to take a look. (I can also move the conversation to the nipype/issues thread?)

We actually started our nipype-based preprocessing workflow a while ago, and when I discovered fmriprep I in fact considered using that instead. While I think fmriprep is a great tool and has much potential, I prefer some more ‘control’/flexibility over the workflow myself. For example, I’d rather use the FSL registration workflow than the ANTs one. Also, I would prefer if the workflow would ‘sink’ some more data, like the registration parameters (epi --> T1, T1 --> standard).

Ah, using mask_files instead of mask_file helped to fix the error for the ACompCor node, but unfortunately not in the TCompCor node … I still get the values.insert(i, node.result.outputs.get()[key]) error from the _collate_results method from the Node object. The outputs (high-variance mask, mask.nii, and the compcor-textfile) are correctly created, though … All the code/tests are organized in the following repo: https://github.com/spinoza-centre/spynoza/tree/develop (develop branch!), and the compcor-workflow is defined in https://github.com/spinoza-centre/spynoza/blob/develop/spynoza/denoising/compcor/workflows.py, it you want to take a look. (I can also move the conversation to the nipype/issues thread?)

Sorry to hear that. Could you open an issue at https://github.com/nipy/nipype/issues?

While I think fmriprep is a great tool and has much potential, I prefer some more ‘control’/flexibility over the workflow myself. For example, I’d rather use the FSL registration workflow than the ANTs one.

Good to know - could you say why do you prefer FSL for doing coregistration over ANTs?

Also, I would prefer if the workflow would ‘sink’ some more data, like the registration parameters (epi --> T1, T1 --> standard).

Actually we do store the T1 -> MNI transformation in *T1w_target-MNI152NLin2009cAsym_warp.h5 file (see http://fmriprep.readthedocs.io/en/latest/workflows.html#derivatives). Since our output volumetric BOLD data is in either T1w or MNI space there is no need for storing the epi --> T1 transformation (unless I am missing some use case - please correct me if I am wrong).

@lukassnoek is there a crashfile created? If so, could you paste its contents?

Sure thing! See below:

File: /media/lukas/data/Software/Spynoza/spynoza/crash-20170502-174644-lukas-tcompcor-0bdc0ae1-bda4-4092-a936-f48bfbf9e5bf.pklz
Node: compcor.tcompcor
Working directory: /tmp/spynoza/workingdir/compcor/tcompcor

Node inputs:

components_file = tcomcor_comps.txt
header =
ignore_exception = False
mask_file =
mask_files = [[’/tmp/spynoza/workingdir/compcor/erode_csf/mapflow/_erode_csf0/erodd_mask.nii.gz’], [’/tmp/spynoza/workingdir/compcor/erode_csf/mapflow/_erode_csf1/erodd_mask.nii.gz’]]
mask_index =
merge_method =
num_components = 6
percentile_threshold = 0.02
realigned_file = [’/media/lukas/data/Software/Spynoza/spynoza/spynoza/data/test_data/sub-0020/func/sub-0020_task-harriri_bold_cut_mcf.nii.gz’, ‘/media/lukas/data/Software/Spynoza/spynoza/spynoza/data/test_data/sub-0020/func/sub-0020_task-wm_bold_cut_mcf.nii.gz’]
regress_poly_degree = 1
use_regress_poly = True

Traceback:
Traceback (most recent call last):
File “/home/lukas/anaconda3/lib/python3.6/site-packages/nipype-0.13.0_gae7af2d.dev-py3.6.egg/nipype/pipeline/plugins/linear.py”, line 43, in run
node.run(updatehash=updatehash)
File “/home/lukas/anaconda3/lib/python3.6/site-packages/nipype-0.13.0_gae7af2d.dev-py3.6.egg/nipype/pipeline/engine/nodes.py”, line 372, in run
self._run_interface()
File “/home/lukas/anaconda3/lib/python3.6/site-packages/nipype-0.13.0_gae7af2d.dev-py3.6.egg/nipype/pipeline/engine/nodes.py”, line 1287, in _run_interface
updatehash=updatehash))
File “/home/lukas/anaconda3/lib/python3.6/site-packages/nipype-0.13.0_gae7af2d.dev-py3.6.egg/nipype/pipeline/engine/nodes.py”, line 1172, in _collate_results
values.insert(i, node.result.outputs.get()[key])
AttributeError: ‘str’ object has no attribute ‘insert’

The preference for FSL over ANTs is mostly due to legacy issues (old code/other packages assuming FSL registration files) to be honest. Ah, yeah, I missed the h5 file.

I’m somewhat confused about the output of T1w/MNI-space BOLD data, through. My original BOLD-images have the dimensions (80x80x37, 3mm isotropic) and my T1 has the dimensions (240x240x220), but fmriprep’s *_bold_space-T1w_preproc.nii.gz files have the dimensions (91x86x74xDyns), which seems neither in the original BOLD-space nor in T1-space. The reason I’d like the data in original BOLD-space it because we normally fit uni/multivariate models on original BOLD-space data (which saves time in the model fitting procedure) and subsequently transform the estimates to MNI for group-level analyses. But I might be missing something …

I’m somewhat confused about the output of T1w/MNI-space BOLD data, through. My original BOLD-images have the dimensions (80x80x37, 3mm isotropic) and my T1 has the dimensions (240x240x220), but fmriprep’s *_bold_space-T1w_preproc.nii.gz files have the dimensions (91x86x74xDyns), which seems neither in the original BOLD-space nor in T1-space. The reason I’d like the data in original BOLD-space it because we normally fit uni/multivariate models on original BOLD-space data (which saves time in the model fitting procedure) and subsequently transform the estimates to MNI for group-level analyses. But I might be missing something …

The outputs of FMRIPREP are in the T1w space, but are using the resolution of the original input BOLD images (to save space). This facilitates exactly the modelling workflow you described. You can do your run/session/subject level models in T1w space and then apply the .h5 warp to move the contrast maps to a common MNI space.

However, we also do the same resolution matching trick for MNI outputs. Therefore the outputs in the MNI space are as small as the ones in the T1w space which means you can do all of your models using those files without the need to apply the warp to your contrast images. Hopefully this can simplify your modelling pipeline.

ah cool, that makes sense! Thanks for the info!