CPAC timeseries extraction from command line

Hi everyone!
We recently got some new fMRI data that contains both structural and functional MRI, and we only want to extract the functional timeseries.

In the documentation, we see that the function CPAC.timeseries.get_roi_timeseries should be able to do that and save the .1D file as output.

However, when we try to run the example shown in the documentation, we get an error related to the setting of output_type:

>>> import CPAC.timeseries.timeseries_analysis as t
>>> wf = t.get_roi_timeseries()
>>> wf.inputs.inputspec.rest = '/path/to/our/rest.nii.gz'  
>>> wf.inputs.input_roi.roi = 'path/to/the/atlas'  
>>> wf.base_dir = './'
>>> wf.inputs.inputspec.output_type = [True, True]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
traits.trait_errors.TraitError: Cannot set the undefined 'output_type' attribute of a 'TraitedSpec' object.

We also noticed that when this function is called by the CPAC pipeline when this function is called in other parts of the code, the line related to output_type is commented, thus we tried running the same example without specifying this flag. This resulted in another error:

>>> wf.run()
241204-12:52:33,463 nipype.workflow INFO:
         Workflow roi_timeseries settings: ['check', 'execution', 'logging', 'monitoring']
241204-12:52:33,467 nipype.workflow INFO:
         Running serially.
241204-12:52:33,468 nipype.workflow INFO:
         [Node] Setting-up "roi_timeseries.3dROIstats" in "/root/roi_timeseries/3dROIstats".
241204-12:52:33,469 nipype.workflow INFO:
         [Node] Cached "roi_timeseries.3dROIstats" - collecting precomputed outputs
241204-12:52:33,469 nipype.workflow INFO:
         [Node] "roi_timeseries.3dROIstats" found cached.
241204-12:52:33,469 nipype.workflow INFO:
         [Node] Setting-up "roi_timeseries.clean_roi_csv" in "/root/roi_timeseries/clean_roi_csv".
241204-12:52:33,472 nipype.workflow INFO:
         [Node] Executing "clean_roi_csv" <nipype.interfaces.utility.wrappers.Function>
241204-12:52:33,474 nipype.workflow INFO:
         [Node] Finished "clean_roi_csv", elapsed time 0.000976s.
241204-12:52:33,474 nipype.workflow WARNING:
         Storing result file without outputs
241204-12:52:33,475 nipype.workflow WARNING:
         [Node] Error on "roi_timeseries.clean_roi_csv" (/root/roi_timeseries/clean_roi_csv)
241204-12:52:33,475 nipype.workflow ERROR:
         Node clean_roi_csv failed to run on host dde75858d182.
241204-12:52:33,475 nipype.workflow ERROR:
         Saving crash info to /root/crash-20241204-125233-c-pac_user-clean_roi_csv-0d211723-f180-447c-9f97-fffd2a58bc10.pklz
Traceback (most recent call last):
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/plugins/linear.py", line 47, in run
    node.run(updatehash=updatehash)
  File "/code/CPAC/pipeline/nipype_pipeline_engine/engine.py", line 428, in run
    return super().run(updatehash)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node clean_roi_csv.

Traceback:
        Traceback (most recent call last):
          File "/code/CPAC/pipeline/nipype_pipeline_engine/monkeypatch.py", line 40, in run
            runtime = self._run_interface(runtime)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/interfaces/utility/wrappers.py", line 142, in _run_interface
            out = function_handle(**args)
          File "<string>", line 58, in clean_roi_csv
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 912, in read_csv
            return _read(filepath_or_buffer, kwds)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 577, in _read
            parser = TextFileReader(filepath_or_buffer, **kwds)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1407, in __init__
            self._engine = self._make_engine(f, self.engine)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1679, in _make_engine
            return mapping[engine](f, **self.options)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 93, in __init__
            self._reader = parsers.TextReader(src, **kwds)
          File "pandas/_libs/parsers.pyx", line 550, in pandas._libs.parsers.TextReader.__cinit__
          File "pandas/_libs/parsers.pyx", line 653, in pandas._libs.parsers.TextReader._get_header
        pandas.errors.ParserError: Passed header=1 but only 1 lines in file


241204-12:52:33,476 nipype.workflow INFO:
         ***********************************
241204-12:52:33,476 nipype.workflow ERROR:
         could not run node: roi_timeseries.clean_roi_csv
241204-12:52:33,476 nipype.workflow INFO:
         crashfile: /root/crash-20241204-125233-c-pac_user-clean_roi_csv-0d211723-f180-447c-9f97-fffd2a58bc10.pklz
241204-12:52:33,476 nipype.workflow INFO:
         ***********************************
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/workflows.py", line 638, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/plugins/linear.py", line 82, in run
    raise error from cause
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/plugins/linear.py", line 47, in run
    node.run(updatehash=updatehash)
  File "/code/CPAC/pipeline/nipype_pipeline_engine/engine.py", line 428, in run
    return super().run(updatehash)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node clean_roi_csv.

Traceback:
        Traceback (most recent call last):
          File "/code/CPAC/pipeline/nipype_pipeline_engine/monkeypatch.py", line 40, in run
            runtime = self._run_interface(runtime)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/nipype/interfaces/utility/wrappers.py", line 142, in _run_interface
            out = function_handle(**args)
          File "<string>", line 58, in clean_roi_csv
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 912, in read_csv
            return _read(filepath_or_buffer, kwds)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 577, in _read
            parser = TextFileReader(filepath_or_buffer, **kwds)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1407, in __init__
            self._engine = self._make_engine(f, self.engine)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1679, in _make_engine
            return mapping[engine](f, **self.options)
          File "/usr/share/fsl/6.0/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 93, in __init__
            self._reader = parsers.TextReader(src, **kwds)
          File "pandas/_libs/parsers.pyx", line 550, in pandas._libs.parsers.TextReader.__cinit__
          File "pandas/_libs/parsers.pyx", line 653, in pandas._libs.parsers.TextReader._get_header
        pandas.errors.ParserError: Passed header=1 but only 1 lines in file

Version:

1.8.6

Environment (Docker, Singularity / Apptainer, custom installation):

We are running cpac in our local machine (with Ubuntu 22.04 LTS) using the command cpac enter to use it from command line and we adapted the path shown in the example to the ones we actually have in our docker image.

Can somebody help me? Thank you in advance!

Gianmarco

Hi Gianmarco,

Would you be able to attach the pipeline config you’re using for this run? I’ll be able to replicate this error better that way.

Thanks!
-Tamsin

As far as the pandas.errors.ParserError: Passed header=1 but only 1 lines in file error, can you take a look at the clean_roi_csv subdir in your working directory? If we can see what that CSV looks like we can likely solve that one - let’s make sure the .csv doesn’t have a header with no contents.