Create nipype OutputSpec at runtime?

1

Hello,

I am writing nipype wrappers for my C++ tools. I have hit a problem that one particular tool of mine generates different numbers of output files (with different names) depending on the input parameters. This doesn’t seem to be something that can be easily supported in nipype?

I tried creating the OutputSpec in __init__(), but this doesn’t work because the cleaning stage of a workflow uses the OutputSpec as defined by the class, not the particular instance of the class.

I have resorted to writing specific wrappers for the common cases (see here https://github.com/spinicist/QUIT/blob/python/Python/QUIT/mt.py#L86), but it would be great to generalise this if possible?

Thanks!

PS - Is Neurostars the correct place for a question like this? The nipype lists here, gitter, the mailing list and Slack as venues. I picked the one that seemed to have highest traffic.
PPS - Why is the Brainhack Slack restricted to certain universities only? I can’t sign-up with my @kcl.ac.uk address.

2

Yes, I think you want to use a DynamicTraitedSpec as base class for your OutputSpec - https://github.com/nipy/nipype/blob/master/nipype/interfaces/base/specs.py#L329

With a DynamicTraitedSpec you can add new traits at any given time. This example shows how to add new output traits with names listed by the input fields.

What do you mean by “generalize” here?

Sure!

3

Thanks @oesteban, I think I have something working with DynamicTraitedSpec. However I’m not sure I’ve done it completely correctly - I ended up having to use DynamicTraitedSpec, adding attributes to it during __init__ (https://github.com/spinicist/QUIT/blob/python/Python/QUIT/mt.py#L75) and adding the outputs during _list_outputs() (https://github.com/spinicist/QUIT/blob/python/Python/QUIT/mt.py#L85). If any of these steps were missing, I received an error:

1 - TraitedSpec instead of DynamicTraitedSpec gave

traits.trait_errors.TraitError: Cannot set the undefined 'DS_A' attribute of a 'LorentzianOutputSpec' object.

2 - Not adding the attributes during __init__:

Exception: Some connections were not found

Module lorentz has no output called DS_A

3 - Not adding the outputs in _list_outputs() gave:

  File "/Users/Tobias/anaconda3/lib/python3.6/site-packages/nipype/pipeline/engine/utils.py", line 1402, in clean_working_directory
    output_files.extend(walk_outputs(outputdict[output]))
KeyError: 'DS_A'

I’m not surprised but 1 & 2, but having to both add the attributes and set the outputs seems counter-intuitive. I think the key issue is I can’t quite get my head round how the standard version of _list_outputs() works.

(By “generalize” I meant write a proper, flexible version with dynamic outputs instead of writing multiple fixed cases)

4

That is expected. enthought.traits does not allow you to set undefined traits (unless you use the DynamicTraitedSpec which has _disallow = False).

If you already know that the names of your outputs are at instantiation, then it is good to set them within __init__. You can do it later though, it should not be a problem (the example I sent you sets them during _run_interface).

This is also expected. The _list_outputs() idea was to avoid holding too big objects in memory, but it is counter-intuitive. That is right.

Generalization in that sense could be done using xml specifications of the command line that can generate corresponding nipype interfaces automatically. Slicer used to work that way.

Another example is MIA (https://sourceforge.net/p/mia/wiki/MIA%20and%20NiPyPe/)

5

Thanks again @oesteban, I have something working now. I needed to add a dynamic InputSpec as well which was a bit of work, but I have some lovely CEST maps now.

I have some other questions about controlling logging, but that’s for another day and another topic.

1 Like