Hello community,
Trying to write a node that will select files based on their name pattern using SelectFiles with the IdentityInterface to provide arguments and iterables. When a pair of (SelectFiles, IdentityInterface) is created for each file type - anatomical and functiona data, workflow selects data with no issues.
#------------------------------------------FUNC------------------------------------------
infosource_func = Node(IdentityInterface(fields=['subject_id', 'session_id', 'task_id']),
name="infosource_func")
infosource_func.iterables = [('task_id', task_id), ('subject_id', subject_id), ('session_id', session_id)]
func_file = opj('sub-{subject_id}', 'ses-{session_id}', 'func', 'sub-{subject_id}_ses-{session_id}_task-{task_id}_acq-EPI_run-*_bold.nii.gz')
templates_func = {'func': func_file}
selectfiles_func = Node(SelectFiles(templates_func,
base_directory=bids_path), name="selectfiles_func")
# #------------------------------------------ANAT------------------------------------------
infosource_anat = Node(IdentityInterface(fields=['subject_id', 'session_id']),
name="infosource_anat")
infosource_anat.iterables = [('subject_id', subject_id), ('session_id', session_id)]
anat_file = opj('sub-{subject_id}', 'ses-{session_id}', 'anat', 'sub-{subject_id}_ses-{session_id}_*T1w.nii.gz')
templates_anat = {'anat': anat_file}
selectfiles_anat = Node(SelectFiles(templates_anat,
base_directory=bids_path), name="selectfiles_anat")
But when it comes to unification of SelectFiles node:
templates = {'anat': anat_file,
'func': func_file}
selectfiles = Node(SelectFiles(templates,
base_directory=bids_path), name="selectfiles")
infosource_func = Node(IdentityInterface(fields=['subject_id', 'session_id', 'task_id']),
name="infosource_func")
infosource_func.iterables = [('task_id', task_id), ('subject_id', subject_id), ('session_id', session_id)]
infosource_anat = Node(IdentityInterface(fields=['subject_id', 'session_id']),
name="infosource_anat")
infosource_anat.iterables = [('subject_id', subject_id), ('session_id', session_id)]
func_file = opj('sub-{subject_id}', 'ses-{session_id}', 'func', 'sub-{subject_id}_ses-{session_id}_task-{task_id}_acq-EPI_run-*_bold.nii.gz')
anat_file = opj('sub-{subject_id}', 'ses-{session_id}', 'anat', 'sub-{subject_id}_ses-{session_id}_*T1w.nii.gz')
I use printer node to list the elements selected by the workflow:
def printer(files_lst):
string = 'FILES_SELECTED -> '
if type(files_lst) == str:
print(string, files_lst)
else:
for item, name in enumerate(files_lst):
print(item, name)
printer_node = Node(Function(input_names=['files_lst'], function=printer), name='printer')
It behaves in a strange manner. It works well for functional scans:
func_WF = Workflow(name='func_WF')
func_WF.connect([
(infosource_func, selectfiles, [('subject_id','subject_id'),
('task_id','task_id'),
('session_id','session_id')]),
(selectfiles, printer_node, [ ('func','files_lst')]),
])
func_WF.run()
OUTPUT:
230112-18:03:58,786 nipype.workflow INFO:
[Node] Executing "printer" <nipype.interfaces.utility.wrappers.Function>
0 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4L_acq-EPI_run-5_bold.nii.gz
1 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4L_acq-EPI_run-6_bold.nii.gz
2 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4L_acq-EPI_run-7_bold.nii.gz
3 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4L_acq-EPI_run-23_bold.nii.gz
4 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4L_acq-EPI_run-24_bold.nii.gz
5 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4L_acq-EPI_run-25_bold.nii.gz
6 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4L_acq-EPI_run-39_bold.nii.gz
7 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4L_acq-EPI_run-40_bold.nii.gz
8 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4L_acq-EPI_run-41_bold.nii.gz
230112-18:03:58,787 nipype.workflow INFO:
[Node] Finished "printer", elapsed time 0.000315s.
230112-18:03:58,789 nipype.workflow INFO:
[Node] Setting-up "func_WF.printer" in "/tmp/tmpufr7cnnu/func_WF/_session_id_FDWS_subject_id_18032021_task_id_4R/printer".
230112-18:03:58,975 nipype.workflow INFO:
[Node] Executing "printer" <nipype.interfaces.utility.wrappers.Function>
0 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4R_acq-EPI_run-15_bold.nii.gz
1 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4R_acq-EPI_run-16_bold.nii.gz
2 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4R_acq-EPI_run-17_bold.nii.gz
3 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4R_acq-EPI_run-31_bold.nii.gz
4 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4R_acq-EPI_run-32_bold.nii.gz
5 /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-4R_acq-EPI_run-33_bold.nii.gz
230112-18:03:58,977 nipype.workflow INFO:
[Node] Finished "printer", elapsed time 0.000389s.
but throws an error for anatomical (most likely my mistake but I can not find one):
anat_WF = Workflow(name='anat_WF')
anat_WF.connect([
(infosource_anat, selectfiles, [('subject_id','subject_id'),
('session_id','session_id')]),
(selectfiles, printer_node, [ ('anat','files_lst')]),
])
anat_WF.run()
OUTPUT:
230112-18:04:04,279 nipype.workflow INFO:
Workflow anat_WF settings: ['check', 'execution', 'logging', 'monitoring']
230112-18:04:04,373 nipype.workflow INFO:
Running serially.
230112-18:04:04,374 nipype.workflow INFO:
[Node] Setting-up "func_WF.selectfiles" in "/tmp/tmp1oqxt2qi/func_WF/_session_id_FDWS_subject_id_18032021/selectfiles".
230112-18:04:04,407 nipype.workflow INFO:
[Node] Executing "selectfiles" <nipype.interfaces.io.SelectFiles>
230112-18:04:04,434 nipype.workflow INFO:
[Node] Finished "selectfiles", elapsed time 0.024258s.
230112-18:04:04,435 nipype.workflow WARNING:
Storing result file without outputs
230112-18:04:04,436 nipype.workflow WARNING:
[Node] Error on "func_WF.selectfiles" (/tmp/tmp1oqxt2qi/func_WF/_session_id_FDWS_subject_id_18032021/selectfiles)
230112-18:04:04,437 nipype.workflow ERROR:
Node selectfiles.a0 failed to run on host 5ddfeb23f11d.
230112-18:04:04,439 nipype.workflow ERROR:
Saving crash info to /data/jup_notebooks/mr/Bru2BIDS/crash-20230112-180404-neuro-selectfiles.a0-c7616586-ce38-4f68-ae4b-87fe4ada7068.pklz
Traceback (most recent call last):
File "/src/nipype/nipype/pipeline/plugins/linear.py", line 47, in run
node.run(updatehash=updatehash)
File "/src/nipype/nipype/pipeline/engine/nodes.py", line 527, in run
result = self._run_interface(execute=True)
File "/src/nipype/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
return self._run_command(execute)
File "/src/nipype/nipype/pipeline/engine/nodes.py", line 771, in _run_command
raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node selectfiles.
Traceback:
Traceback (most recent call last):
File "/src/nipype/nipype/interfaces/base/core.py", line 401, in run
outputs = self.aggregate_outputs(runtime)
File "/src/nipype/nipype/interfaces/base/core.py", line 430, in aggregate_outputs
predicted_outputs = self._list_outputs() # Predictions from _list_outputs
File "/src/nipype/nipype/interfaces/io.py", line 1474, in _list_outputs
raise IOError(msg)
OSError: No files were found matching func template: /data/data/mr/testDS/mouse_BIDS/mouse_20210318/bids/sub-18032021/ses-FDWS/func/sub-18032021_ses-FDWS_task-<undefined>_acq-EPI_run-*_bold.nii.gz
230112-18:04:04,448 nipype.workflow INFO:
***********************************
230112-18:04:04,450 nipype.workflow ERROR:
could not run node: func_WF.selectfiles.a0
230112-18:04:04,451 nipype.workflow INFO:
crashfile: /data/jup_notebooks/mr/Bru2BIDS/crash-20230112-180404-neuro-selectfiles.a0-c7616586-ce38-4f68-ae4b-87fe4ada7068.pklz
230112-18:04:04,451 nipype.workflow INFO:
***********************************
When I remove ‘func’ from the template dictionary for the SelectFiles it starts to work fine. I’m puzzled.