Iteration of existing workflow over subjects

I’m trying to run an existing workflow from nipype over many subjects, for example create_skullstripped_recon_flow(). I’m not quite sure, however, how to provide iterables for workflows. If I feed it the subject list I get an error that it’s a list and not a string.

recon_flow = create_skullstripped_recon_flow()
recon_flow.inputs.inputspec.subjects_dir = outDir
recon_flow.inputs.inputspec.subject_id = subs
# recon_flow.inputs.inputspec.iterfield = ['subject_id', 'T1_files']  # <- this doesn't work as it's not a mapnode.
wf.connect([
    (skullstrip, recon_flow, [('outStripped', 'inputspec.T1_files')])
    ])

@Glad

When using iterables - I like to designate an ‘iterable’ node, usually an IdentityInterface to help keep track of workflow divergence.

you could set yours up like so:

from nipype import IdentityInterface, Node, Workflow

infosource = Node(IdentityInterface(fields=['subject_id'], name='infosource')
infosource.iterables = ('subject_id', subs)

# grab T1 with subject_id and pass file into skullstrip node

# and connect both subject id / T1 to recon workflow
wf.connect([
    (skullstrip, recon_flow, [('outStripped', 'inputspec.T1_files')]),
    (infosource, recon_flow, [('subject_id', 'inputspec.subject_id')])
    ])
1 Like

Hi, I have a similar use-case but the subject list is created at runtime as the output of a custom datagrabber-node.
I then want to use a subworkflow which works on single subjects, so I use the IdentityNode to split the output list of the datagrabber and “call” the subworkflow.
It would look like this:
infosource.iterables = [(‘subject-id’, datagrabber.outputs.subs)]
wf.connect(infosource, ‘subject-id’, subworkflow, ‘inputnode.subject-id’)

this does not work: TypeError: '_Undefined’ object is not iterable
This is to be expected since the output is generated later after nipype does a graph and all that so it cannot know how many nodes to generate.
Is there a more dynamic way to do this?
I basically need a MapWorkflow and hoped to build one by using iterable and join.
Something that takes a list and runs the subgraph on every item of the list. This cannot be done with the utility.Split interface can it?

The only other option I can think of is to replace every Node in the subworkflow with a MapNode, which would be a hassle.
Cheers, Niklas

@niklasf - you can approach this using a two node workflow:

  1. datagrabber - Node
  2. function node - MapNode

the function for the function node will be something like:

def create_and_run_subj_workflow(subject_id):
     #create and run workflow

alternatively you can use a metaflow concept. instead of using datagrabber to get list of subjects, generate a list with some python code:

subject_list = get_list_of_subjects()     
...
infosource_node.iterables = ('subject_id', subject_list)

and then assign that list to the iterables of infosource

Thanks a lot! Sadly I surrendered and just replaced every node with a Mapnode and made existing mapnodes nested before reading your reply.
Will keep your approach in mind for the future :wink: