BIDSDataGrabber does not iterate although iterables are defined

,
1

Background

I want to set up a pre-processing workflow that iterates over subject and sessions. Specifically, I want to have a set of sub-workflows that are e.g., preprocessing fieldmap data, nested within a meta-workflow that combines all preprocessing steps. My data-set is in BIDS and I want to use the BIDSDataGrabber to get the raw data into the workflow. I will try to give a minimal example below:

Code for my meta-workflow:

workflow_name = "preprocessing"
wf_preproc = Workflow(name=workflow_name)
wf_preproc.base_dir = "path/to/base_dir"

Code for my BIDSDataGrabber node:

bg = Node(interface=BIDSDataGrabber(iterables=['subject', 'session']), name ="bg")
bg.inputs.base_dir = "path/to/base_dir"
bg.inputs.subject = ["41002","41001"]
bg.inputs.session = ["01","02"]
bg.inputs.output_query['fmap_nii'] = dict(acq='prenorm', modality='fmap', type='epi', extensions='nii.gz', dir=('AP', 'PA'))

Code for my datasink node:

datasink = Node(DataSink(), name='datasink')
datasink.inputs.base_directory = "path/to/base_dir"
datasink.inputs.container = "preprocessing" + '_results'
datasink.inputs.parameterization = False
substitutions = [("_subject_id_", ""), ("_session_id_", "")]
datasink.inputs.substitutions = substitutions

Code for my sub-workflow:

wf_fmap = Workflow(name="fieldmaps")
wf_fmap.base_dir = experiment_dir

Code for some example nodes:

roi_fmap = MapNode(ExtractROI(), name='roi_fmap', iterfield=['in_file'])
roi_fmap.inputs.t_min = 0
roi_fmap.inputs.t_size = num_vol
merge_fmap = Node(interface=Merge(), name='merge_fmap')
merge_fmap.inputs.dimension = 't'

Connecting nodes of the sub-workflow 

wf_fmap.connect(roi_fmap, 'roi_file', merge_fmap, 'in_files')

Input and output stream for meta-workflow 

wf_preproc.connect(bg, 'fmap_nii', wf_fmap, 'roi_fmap.in_file')
wf_preproc.connect(wf_fmap, 'roi_fmap.roi_file', datasink, 'fmap.@roi_fmap')
wf_preproc.connect(wf_fmap, 'merge_fmap.merged_file', datasink, 'fmap.@merge_fmap')

The problem

Then I run the meta-workflow. It executes cleanly, and no problems occur regarding the functionality of the nodes themselves. However, the workflow does not iterate over subjects and sessions separately
as it should since I specified subject and session as iterables of the BIDSDataGrabber. I went over my code many times but could not figure out why it is not working. I really hope someone can help me out! Thanks in advance!

2

Ok, I kind of solved it myself (sorry for letting my frustration make me post the problem somewhat early here).
A minor change to the specification of the BIDSDataGrabber solved it:

# i took out the iterables from here:
bg = Node(interface=BIDSDataGrabber(), name ="bg", n_procs=3)
# now the iterables are defined like this:
bg.iterables = [('subject',subject_list),('session',session_list)]

However, I am not really sure why, so I would still very much appreciate an explanation. Thanks!

3

Hi @lennart,

Glad to hear you were able to get it working ! One idea: In the first code block, you were defining iterables within the BIDSDataGrabber interface, rather than within the (bg) Node itself. In the second code block, you’re assigning iterables to the bg node, so it correctly interprets them as belonging to the node rather than to the interface.

Hope that helps !

Elizabeth

1 Like
4

Hi @emdupre,

thanks for your answer! That makes a lot of sense! It also works now, so I am completely happy!

Cheers,
Lennart