Force DataSink to save in input directories

1

Dear all,

I am writing a nypipe workflow to be run in parallel. I have the following folder strutcture:

base_dir
|_ folder1
|      |_ t1w.nii.gz
|_ another_folder
|      |_ t1w.nii.gz
.....
|_ last_folder
|      |_ t1w.nii.gz

DataGrabber does an excellent job and takes all the t1w.nii.gz files as input

grabber = nipype.Node(interface=nipype.DataGrabber(infields=['arg'],outfields=['out_file']), name='grabber')      
grabber.inputs.base_directory = base_dir
grabber.inputs.sort_filelist = False
grabber.inputs.template = '*/%s.nii.gz'
grabber.inputs.arg = 't1w'

then I set DataSink

sink = nipype.Node(interface=nipype.DataSink(),name='sink')
sink.inputs.base_directory = base_dir

and write the remaining parts of the pipeline

# Neck removal by FSL robustfov
neck_remove=nipype.MapNode(interface=fsl.RobustFOV(), name='neck_remove', iterfield=['in_file'])
neck_remove.inputs.out_roi="t1w_fov.nii.gz"
workflow.connect([(grabber, neck_remove, [('out_file', 'in_file')]),
                  (neck_remove, sink, [('out_roi', '@in_file')]), ])
workflow.run('MultiProc', plugin_args={'n_procs': 5})

This works nicely and put the outputs in this structure

base_dir
|_ neck_remove0
|      |_ t1w_fov.nii.gz
|_ neck_remove1
|      |_ t1w_fov.nii.gz
....
|_ folder
|      |_ t1w.nii.gz
|_ another_folder
|      |_ t1w.nii.gz
.....

Is it possible to force DataSink to put the output files in the same directory as the respective inputs? The structure I had in mind was:

base_dir
|_ folder
|      |_ t1w.nii.gz
|      |_ t1w_fov.nii.gz
|_ another_folder
|      |_ t1w.nii.gz
|      |_ t1w_fov.nii.gz
.....

Thank you very much!

2

I don’t see any specific option of DataSink to do it and I think that would be ambiguous since you can have files from more than one directory as input fields.

However, you can edit the file paths using substitions, e.g.:

substitutions = [('neck_remove0', 'folder')]
sink.inputs.substitutions = substitutions

Not sure if this is what you hoped for, but you can experiment. If you’re using MapNode and you have multiple directories you can always write a loop to create a longer list of substitutions.

3

thank you for your answer!
the point is that folder is an list of strings and not a string, so your suggestion gave an error…

4

I don’t fully understand what you mean by “folder is an list of strings”. Can you please write more? What error did you get? substitutions is a list and you can include as many elements as you want.

Please note, that my original answer was missing ' for proper string formatting, just edited.

5

that was a nice hint! Unfortunately didn’t work…
I tried with a for loop

substitutions=[]
for i in range(len(subject_list)):
    substitutions+= [(""+str(i), subject_list[i])]
sink.inputs.substitutions =substitutions

with no better outcome so I tried

substitutions=[]
for i in range(len(subject_list)):
    substitutions+= [("_neck_remove"+str(i), subject_list[i])]
    substitutions+= [("subject_id_", "")]
sink.inputs.substitutions =substitutions

which turned out to work as I intended!

Thank you so much! :slight_smile: