pyBids derivative validation error running fitlins on fmriprep cifti output

2 part question, if anyone can help, I am trying to run fitlins using cifti outputs from fmriprep.

First: I’m getting an error in fitlins from the pybids validator:

Traceback (most recent call last):
  File "/opt/miniconda-latest/envs/neuro/bin/fitlins", line 8, in <module>
    sys.exit(main())
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py", line 271, in main
    sys.exit(run_fitlins(sys.argv[1:]))
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py", line 218, in run_fitlins
    reset_database=reset_database)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/layout/layout.py", line 209, in __init__
    reset_database=reset_database
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/layout/layout.py", line 604, in add_derivatives
    MANDATORY_DERIVATIVES_FIELDS['PipelineDescription.Name'])
bids.exceptions.BIDSDerivativesValidationError: Every valid BIDS-derivatives dataset must have a PipelineDescription.Name field set inside 'dataset_description.json'.
Example: {'PipelineDescription': {'Name': 'Example pipeline'}}

but my derivative dataset_description has a name field that is filled out:
Do I need to add another field or change the formatting? I’ve experimented with this but was not successful yet.

{
    "Name": "fMRIPrep - fMRI PREProcessing workflow",
    "BIDSVersion": "1.4.0",
    "DatasetType": "derivative",
    "GeneratedBy": [
        {
            "Name": "fMRIPrep",
            "Version": "20.2.0rc0",
            "CodeURL": "https://github.com/poldracklab/fmriprep/archive/20.2.0rc0.tar.gz"
        }
    ],
    "HowToAcknowledge": "Please cite our paper (https://doi.org/10.1038/s41592-018-0235-4), and include the generated citation boilerplate within the \
Methods section of the text."
}
  1. Second question, does anyone know how to specify that I would like to run fitlins using the cifti data and not the standard nifti data? Is there something in particular to specify in the --space or --desc-label flags? I’ve been unable to figure this out from going through fitlins docs/github page

This is the command I was trying to run using singularity

singularity run --cleanenv -B /om2,/scratch/ fitlins-latest.simg /om2/user/jsmentch/data/datalad/ds001110 out/ participant
-v
–participant-label 19
-m /om2/user/jsmentch/neuroscout/SherlockMerlin/neuroscout-bundles/MNQZW/model.json
–desc-label dtseries
-d /om2/user/jsmentch/data/datalad/ds001110/derivatives
-w /scratch/Fri/jsmentch

(I’m reusing a model from neuroscout so may need to fix that part up )

Thanks!

We’re going to need to update PyBIDS to handle BIDS 1.4.0 derivatives.

To select CIFTIs, use --space fsLR.

1 Like

Ah, fsLR makes a lot of sense, thank you!

For now, before PyBIDS is updated, I’m trying to modify the dataset_description.json file to conform with PyBIDS as per the github issue but running into some errors. I’ve modified it to have a PipelineDescription.Name see below, and a few permutations of that but it doesn’t seem to be working… Do you know if there is another way to squeeze this by the validator or what might be going wrong?

{
    "Name": "fMRIPrep - fMRI PREProcessing workflow",
    "BIDSVersion": "1.4.0",
    "DatasetType": "derivative",
    "PipelineDescription": [
        {
            "Name": "fMRIPrep"
        }
    ],
    "GeneratedBy": [
        {
            "Name": "fMRIPrep",
            "Version": "20.2.0rc0",
            "CodeURL": "https://github.com/poldracklab/fmriprep/archive/20.2.0r\
c0.tar.gz"
        }
    ],
    "HowToAcknowledge": "Please cite our paper (https://doi.org/10.1038/s41592-\
018-0235-4), and include the generated citation boilerplate within the Methods \
section of the text."
}

Error:

AttributeError: 'list' object has no attribute 'get'
bash-4.2$ singularity run --cleanenv -B /om2/user/jsmentch,/scratch fitlins-latest.simg /om2/user/jsmentch/data/datalad/ds001110 out/ participant -v --participant-label 19 -m /om2/user/jsmentch/neuroscout/SherlockMerlin/neuroscout-bundles/MNQZW/model.json --space fsLR -d /om2/user/jsmentch/data/datalad/ds001110/derivatives/fmriprep -w /scratch/Fri/jsmentch
Traceback (most recent call last):
  File "/opt/miniconda-latest/envs/neuro/bin/fitlins", line 8, in <module>
    sys.exit(main())
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py", line 271, in main
    sys.exit(run_fitlins(sys.argv[1:]))
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py", line 218, in run_fitlins
    reset_database=reset_database)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/layout/layout.py", line 209, in __init__
    reset_database=reset_database
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/layout/layout.py", line 597, in add_derivatives
    'PipelineDescription', {}).get('Name')
AttributeError: 'list' object has no attribute 'get'

PipelineDescription was just one JSON object. It got changed to a list when renamed to GeneratedBy.

1 Like

Thanks, it seems I was formatting the .json file incorrectly with pipeline description as a list. I got past this error by replacing GeneratedBy with this if anyone else runs into this:

"PipelineDescription":      {
            "Name": "fMRIPrep"
        },

Since then, I’ve been running into more errors like this:

Traceback (most recent call last):
  File "/opt/miniconda-latest/envs/neuro/bin/fitlins", line 8, in <module>
    sys.exit(main())
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py", line 271, in main
    sys.exit(run_fitlins(sys.argv[1:]))
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py", line 263, in run_fitlins
    analysis.setup(**selectors)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py", line 98, in setup
    step.setup(input_contrasts, drop_na=drop_na, **selectors)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py", line 256, in setup
    **kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/layout/layout.py", line 960, in get_collections
    skip_empty=skip_empty, **kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/variables/io.py", line 93, in load_variables
    dataset = _load_time_variables(layout, dataset, scope=scope, **_kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/variables/io.py", line 186, in _load_time_variables
    raise ValueError("No functional images that match criteria found.")
ValueError: No functional images that match criteria found.

This is when i set --space to fsLR or fsLR_den-91k , neither of which finds the file(sub-19_task-MerlinMovie_space-fsLR_den-91k_bold.dtseries.nii) I’m wondering if maybe this is due to it having the double extension .dtseries.nii? But I see there was a fix for that already.

I don’t encounter this error if I set it to --space MNI152NLin2009cAsym

still unable to run fitlins with the cifti output, but another error I am receiving which neglected to mention is:
TypeError: __init__() got an unexpected keyword argument 'initializer'

Are there other differences for running CIFTI through besides specifying --space fsLR? Can’t find any more info on this.

The full error I’m receiving is:

bash-4.2$ singularity run --cleanenv -B /om2/user/jsmentch,/scratch fitlins-latest.simg /om2/user/jsmentch/data/datalad/ds001110 out/ participant -v --participant-label 19
-m /om2/user/jsmentch/neuroscout/SherlockMerlin/neuroscout-bundles/MNQZW/model-19.json --space fsLR -d /om2/user/jsmentch/data/datalad/ds001110/derivatives/fmriprep -w /scr
atch/Fri/jsmentch
200928-16:34:58,983 nipype.utils WARNING:
         A newer version (1.5.1) of nipy/nipype is available. You are using 1.5.0
200928-16:34:59,474 nipype.workflow INFO:
         Workflow fitlins_wf settings: ['check', 'execution', 'logging', 'monitoring']
200928-16:34:59,526 nipype.workflow INFO:
         Running in parallel.
200928-16:34:59,536 nipype.workflow INFO:
         [MultiProc] Running 0 tasks, and 1 jobs ready. Free memory (GB): 226.54/226.54, Free processors: 40/40.
200928-16:35:00,151 nipype.workflow INFO:
         [Node] Setting-up "fitlins_wf.loader" in "/scratch/Fri/jsmentch/fitlins_wf/loader".
200928-16:35:00,209 nipype.workflow INFO:
         [Node] Running "loader" ("fitlins.interfaces.bids.LoadBIDSModel")
200928-16:35:01,529 nipype.workflow WARNING:
         Storing result file without outputs
200928-16:35:01,535 nipype.workflow INFO:
         [MultiProc] Running 1 tasks, and 0 jobs ready. Free memory (GB): 226.34/226.54, Free processors: 39/40.
                     Currently running:
                       * fitlins_wf.loader
200928-16:35:01,541 nipype.workflow WARNING:
         [Node] Error on "fitlins_wf.loader" (/scratch/Fri/jsmentch/fitlins_wf/loader)
200928-16:35:03,537 nipype.workflow ERROR:
         Node loader failed to run on host node007.cm.cluster.
200928-16:35:03,538 nipype.workflow ERROR:
         Saving crash info to /scratch/Fri/jsmentch/crash-20200928-163503-jsmentch-loader-35421cea-9531-43db-b665-37d7ec945aa2.txt
Traceback (most recent call last):
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/plugins/multiproc.py", line 148, in __init__
    mp_context=mp_context,
TypeError: __init__() got an unexpected keyword argument 'initializer'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/interfaces/base/core.py", line 397, in run
    runtime = self._run_interface(runtime)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/interfaces/bids.py", line 220, in _run_interface
    analysis.setup(drop_na=False, **selectors)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py", line 98, in setup
    step.setup(input_contrasts, drop_na=drop_na, **selectors)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py", line 256, in setup
    **kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/layout/layout.py", line 960, in get_collections
    skip_empty=skip_empty, **kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/variables/io.py", line 93, in load_variables
    dataset = _load_time_variables(layout, dataset, scope=scope, **_kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/variables/io.py", line 186, in _load_time_variables
    raise ValueError("No functional images that match criteria found.")
ValueError: No functional images that match criteria found.

200928-16:35:03,545 nipype.workflow INFO:
         [MultiProc] Running 0 tasks, and 0 jobs ready. Free memory (GB): 226.54/226.54, Free processors: 40/40.
200928-16:35:05,536 nipype.workflow INFO:
         ***********************************
200928-16:35:05,537 nipype.workflow ERROR:
         could not run node: fitlins_wf.loader
200928-16:35:05,537 nipype.workflow INFO:
         crashfile: /scratch/Fri/jsmentch/crash-20200928-163503-jsmentch-loader-35421cea-9531-43db-b665-37d7ec945aa2.txt
200928-16:35:05,537 nipype.workflow INFO:
         ***********************************
Traceback (most recent call last):
  File "/opt/miniconda-latest/envs/neuro/bin/fitlins", line 8, in <module>
    sys.exit(main())
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py", line 271, in main
    sys.exit(run_fitlins(sys.argv[1:]))
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/fitlins/cli/run.py", line 263, in run_fitlins
    analysis.setup(**selectors)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py", line 98, in setup
    step.setup(input_contrasts, drop_na=drop_na, **selectors)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/analysis/analysis.py", line 256, in setup
    **kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/layout/layout.py", line 960, in get_collections
    skip_empty=skip_empty, **kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/variables/io.py", line 93, in load_variables
    dataset = _load_time_variables(layout, dataset, scope=scope, **_kwargs)
  File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/bids/variables/io.py", line 186, in _load_time_variables
    raise ValueError("No functional images that match criteria found.")
ValueError: No functional images that match criteria found.

For anyone else that runs into this, this particular error was bypassed by adding: --desc-label '' to the fitlins command. It was not finding the cifti file because by default it was looking for a file with *desc-preproc_bold.nii.gz

and the file in question has a name without that e.g.:
sub-19_task-MerlinMovie_space-fsLR_den-91k_bold.dtseries.nii