Tell nipype to not re-run iteration nodes for which the results already exist

1

I am noticing, that nipype will not overwrite results files which already exist, however it does re-run the entire workflow up to (in my case) the datasink step, if the work directory has been removed. This makes sense.

I am wondering:

  • Whether at the last datasink step it compares just the file names or also the checksums (in the latter case meaning that it would only NOT overwrite absolutely identical files)
  • Whether there is any easy way to tell nipype to check in advance for the results directories and skip the iterator steps for which results are already present (obviously in this case independently of the checksum).
2

I think the checks happen when you run the the pipe. If you want to overwrite the files in a specific step you need to set overwrite=True in the node, e.g.

level1design = Node(Level1Design(),
                        overwrite=True,
                        name="level1design")
3

That wasn’t really my question. I was wondering whether I can prevent execution of an iteration based on just a name check of the last step.

4

@TheChymera Not currently. You could do it manually during workflow creation, though; if you can derive what the outputs should be called, you should be able to check for them and modify how you construct your workflow. (For instance, if sub-01 is done, set your subjects list to sub-02+.)

That could cause problems for iteration done by MapNode and not iterables, since MapNode-generated Nodes have an integer index in their names.

1 Like