Skip instance using IdentityInterface.iterables

Dear all,
I am using IdentityInterface.iterables to loop over subjects and sessions. Is there an easy way to skip certain sessions of certain subjects by adding an “exclude-list”?

found this:

I guess, I have to find another solution then.

This is a relatively common pattern for the work I do too, so I accomplish this through using itertools product to expand all possible combinations of subjects/sessions, and then remove the cases I do not wish to process after generating all possible combinations.

Then with the nipype node, I set synchronize=True to match up the subjects/sessions.

Example code:

from itertools import product
from nipype.interfaces import utility as niu
from nipype.pipeline import engine as pe

# set up the inputs
subjects = ['sub-01', 'sub-02', 'sub-03']
sessions = ['pre', 'post', 'followup']
exclude_list = [('sub-01', 'followup'), ('sub-02', 'pre'), ('sub-02', 'post')]

# test function to make sure this works
def _display_sub_ses(subject, session):
    print('\n\n' + '_'.join([subject, session]) + '\n\n')
    return subject, session

# workflow generation function
def my_preproc(subjects, sessions, exclude_list):
    # explicitly create all possible combinations of subjects/sessions
    all_subject_session_combos = set(product(subjects, sessions))
    # sets allow for easy removal of elements
    # however we want the output to be a list because lists have order (sets are unordered)
    filtered_subject_sessions = list(all_subject_session_combos - set(exclude_list))
    # the first element for each tuple is the subject (<'sub-01'>, 'ses-pre')
    subjects_iter = [tup[0] for tup in filtered_subject_sessions]
    # the second element for each tuple is the session ('sub-01', <'ses-pre'>)
    sessions_iter = [tup[1] for tup in filtered_subject_sessions]
    # since we explicitly state the combinations of subjects/sessions, we
    # do not want iterables to try to generate all combinations.
    # setting synchronize to true gives us the behavior we want.
    input_node = pe.Node(niu.IdentityInterface(fields=['subject', 'session']),
                         iterables=[('subject', subjects_iter), ('session', sessions_iter)],
    test_node = pe.Node(niu.Function(function=_display_sub_ses,
                                     output_names=['subject', 'session'],
                                     input_names=['subject', 'session']),
    wf = pe.Workflow(name='test_wf')
        (input_node, test_node, [('subject', 'subject'),
                                 ('session', 'session')]),
    return wf

my_wf = my_preproc(subjects, sessions, exclude_list)

which generates this output on my end:

200205-16:37:04,671 nipype.workflow INFO:
	 Workflow test_wf settings: ['check', 'execution', 'logging', 'monitoring']
200205-16:37:04,695 nipype.workflow INFO:
	 Running serially.
200205-16:37:04,697 nipype.workflow INFO:
	 [Node] Setting-up "test_wf.test_node" in "/tmp/tmp0ohxt7fh/test_wf/_session_pre_subject_sub-01/test_node".
200205-16:37:04,704 nipype.workflow INFO:
	 [Node] Running "test_node" ("nipype.interfaces.utility.wrappers.Function")


200205-16:37:04,710 nipype.workflow INFO:
	 [Node] Finished "test_wf.test_node".
200205-16:37:04,712 nipype.workflow INFO:
	 [Node] Setting-up "test_wf.test_node" in "/tmp/tmpltoigpbt/test_wf/_session_followup_subject_sub-03/test_node".
200205-16:37:04,716 nipype.workflow INFO:
	 [Node] Running "test_node" ("nipype.interfaces.utility.wrappers.Function")


200205-16:37:04,722 nipype.workflow INFO:
	 [Node] Finished "test_wf.test_node".
200205-16:37:04,723 nipype.workflow INFO:
	 [Node] Setting-up "test_wf.test_node" in "/tmp/tmp4zqnadyt/test_wf/_session_pre_subject_sub-03/test_node".
200205-16:37:04,727 nipype.workflow INFO:
	 [Node] Running "test_node" ("nipype.interfaces.utility.wrappers.Function")


200205-16:37:04,732 nipype.workflow INFO:
	 [Node] Finished "test_wf.test_node".
200205-16:37:04,733 nipype.workflow INFO:
	 [Node] Setting-up "test_wf.test_node" in "/tmp/tmpaud957j2/test_wf/_session_post_subject_sub-01/test_node".
200205-16:37:04,737 nipype.workflow INFO:
	 [Node] Running "test_node" ("nipype.interfaces.utility.wrappers.Function")


200205-16:37:04,741 nipype.workflow INFO:
	 [Node] Finished "test_wf.test_node".
200205-16:37:04,742 nipype.workflow INFO:
	 [Node] Setting-up "test_wf.test_node" in "/tmp/tmp8728129x/test_wf/_session_followup_subject_sub-02/test_node".
200205-16:37:04,746 nipype.workflow INFO:
	 [Node] Running "test_node" ("nipype.interfaces.utility.wrappers.Function")


200205-16:37:04,751 nipype.workflow INFO:
	 [Node] Finished "test_wf.test_node".
200205-16:37:04,752 nipype.workflow INFO:
	 [Node] Setting-up "test_wf.test_node" in "/tmp/tmpra4hc1vo/test_wf/_session_post_subject_sub-03/test_node".
200205-16:37:04,755 nipype.workflow INFO:
	 [Node] Running "test_node" ("nipype.interfaces.utility.wrappers.Function")


200205-16:37:04,760 nipype.workflow INFO:
	 [Node] Finished "test_wf.test_node".

Hope this helps!


thanks for sharing, jdkent!