Specifying a custom surface template as fMRIPrep output space

alexenge · March 6, 2023, 6:36pm

Summary of what happened:

For a longitudinal fMRI study with children, I’ve created a study-specific template using the FreeSurfer longitudinal pipeline. I would like to use the generated FreeSurfer surfaces as a (non-standard) output space with fMRIPrep so that I can analyze the functional data in this custom study-specific surface space.

As pointed out in here, here, and here, it is possible to specify custom templates as --output-spaces if one puts them into the TemplateFlow home directory in the correct format as expected by TemplateFlow. However, this only seems to work for volumetric templates (e.g., a NIfTI-converted version of FreeSurfer’s brain.mgz), not for surface templates (e.g., GIfTI-converted versions of FreeSurfers *.lh/*.rh files).

I’ve tried adding all the NIfTI/GIfTI converted FreeSurfer outputs to a custom TemplateFlow template directory that now has a structure similar to tpl-fsaverge, but for my custom template (here called tpl-fslong):

/templateflow_home/tpl-fslong
├─ template_description.json
├─ tpl-fslong_hemi-L_den-164k_inflated.surf.gii
├─ tpl-fslong_hemi-L_den-164k_pial.surf.gii
├─ tpl-fslong_hemi-L_den-164k_smoothwm.surf.gii
├─ tpl-fslong_hemi-L_den-164k_sphere.surf.gii
├─ tpl-fslong_hemi-L_den-164k_white.surf.gii
├─ tpl-fslong_hemi-R_den-164k_inflated.surf.gii
├─ tpl-fslong_hemi-R_den-164k_pial.surf.gii
├─ tpl-fslong_hemi-R_den-164k_smoothwm.surf.gii
├─ tpl-fslong_hemi-R_den-164k_sphere.surf.gii
├─ tpl-fslong_hemi-R_den-164k_white.surf.gii
├─ tpl-fslong_res-01_desc-brain_mask.nii.gz
└─ tpl-fslong_res-01_T1w.nii.gz

However, when now specifying --output-spaces fslong, fMRIPrep automatically treats this as volume space (tpl-fslong:res-01) and aligns things to the volumetric tpl-fslong_res-01_T1w.nii.gz file (ignoring the surface files). When specifically asking for a mesh density (i.e., --output_spaces fslong:den164k), fMRIPrep complains about ValueError: 'den' is not a recognized entity.

I assume it’s currently simply not possible to normalize to custom surface templates, only to the “special” fsaverage and fsnative? If so, are there any plans to make this possible in the future, or do you have any ideas for a workaround to map the preprocessed functional data to a custom surface template?

Thanks a lot in advance for your help and for all the efforts your putting into these amazing software packages!

Command used (and if a helper script was used, a link to the helper script or the command generated):

fmriprep \
    /bids_dir \
    /bids_dir/derivatives/fmriprep \
    participant \
    --participant-label sub-SA27 \
    --output-spaces T1w MNIPediatricAsym:cohort-2 fsnative fsaverage fslong \
    --fs-license-file /bids_dir/code/license.txt \

Version:

fMRIPrep 21.0.2

Environment (Docker, Singularity, custom installation):

Singularity container from the ReproNim container collection

Relevant log outputs (more than 20 lines, I’m sorry!):

When using --output-spaces fslong, fMRIPrep runs through but only outputs stuff in volumetric template-aligned space:

add(ok): fmriprep/sub-SA27/figures/sub-SA27_ses-01_run-1_space-fslong_T1w.svg (file)
add(ok): fmriprep/sub-SA27/log/20230306-085704_a2c1a596-a393-4e3b-8a49-97285a05ef93/fmriprep.toml (file)
add(ok): fmriprep/sub-SA27/ses-01/anat/sub-SA27_ses-01_run-1_from-T1w_to-fslong_mode-image_xfm.h5 (file)
add(ok): fmriprep/sub-SA27/ses-01/anat/sub-SA27_ses-01_run-1_from-fslong_to-T1w_mode-image_xfm.h5 (file)
add(ok): fmriprep/sub-SA27/ses-01/anat/sub-SA27_ses-01_run-1_space-fslong_desc-brain_mask.json (file)
add(ok): fmriprep/sub-SA27/ses-01/anat/sub-SA27_ses-01_run-1_space-fslong_desc-brain_mask.nii.gz (file)
add(ok): fmriprep/sub-SA27/ses-01/anat/sub-SA27_ses-01_run-1_space-fslong_desc-preproc_T1w.json (file)
add(ok): fmriprep/sub-SA27/ses-01/anat/sub-SA27_ses-01_run-1_space-fslong_desc-preproc_T1w.nii.gz (file)
add(ok): fmriprep/sub-SA27/ses-01/anat/sub-SA27_ses-01_run-1_space-fslong_dseg.nii.gz (file)
add(ok): fmriprep/sub-SA27/ses-01/anat/sub-SA27_ses-01_run-1_space-fslong_label-CSF_probseg.nii.gz (file)
add(ok): fmriprep/sub-SA27/ses-01/anat/sub-SA27_ses-01_run-1_space-fslong_label-GM_probseg.nii.gz (file)
add(ok): fmriprep/sub-SA27/ses-01/anat/sub-SA27_ses-01_run-1_space-fslong_label-WM_probseg.nii.gz (file)
add(ok): fmriprep/sub-SA27/ses-01/func/sub-SA27_ses-01_task-language_run-1_space-fslong_boldref.nii.gz (file)
add(ok): fmriprep/sub-SA27/ses-01/func/sub-SA27_ses-01_task-language_run-1_space-fslong_desc-aparcaseg_dseg.nii.gz (file)
add(ok): fmriprep/sub-SA27/ses-01/func/sub-SA27_ses-01_task-language_run-1_space-fslong_desc-aseg_dseg.nii.gz (file)
add(ok): fmriprep/sub-SA27/ses-01/func/sub-SA27_ses-01_task-language_run-1_space-fslong_desc-brain_mask.json (file)
add(ok): fmriprep/sub-SA27/ses-01/func/sub-SA27_ses-01_task-language_run-1_space-fslong_desc-brain_mask.nii.gz (file)
add(ok): fmriprep/sub-SA27/ses-01/func/sub-SA27_ses-01_task-language_run-1_space-fslong_desc-preproc_bold.json (file)
add(ok): fmriprep/sub-SA27/ses-01/func/sub-SA27_ses-01_task-language_run-1_space-fslong_desc-preproc_bold.nii.gz (file)
...

When using --output-spaces fslong:den-164k, fMRIPrep returns the following error (because it’s still treating fslong:den-164 as a volumetric template):

230306-13:32:39,480 nipype.workflow WARNING:
	 [Node] Error on "fmriprep_wf.single_subject_SA27_wf.func_preproc_ses_01_task_language_run_1_wf.bold_std_trans_wf.select_tpl" (/ptmp/aenge/tmp/work_job_6551480/fmriprep_wf/single_subject_SA27_wf/func_preproc_ses_01_task_language_run_1_wf/bold_std_trans_wf/_std_target_fslong.den164k.resnative/select_tpl)
230306-13:32:39,484 nipype.workflow ERROR:
	 Node select_tpl.a1 failed to run on host co6508.
230306-13:32:39,484 nipype.workflow ERROR:
	 Saving crash info to /ptmp/aenge/tmp/ds_job_6551480/derivatives/fmriprep/sub-SA27/log/20230306-133005_1465a266-40d8-4454-bbe3-2812a11b09dc/crash-20230306-133239-aenge-select_tpl.a1-22bae3a5-2649-4a66-8328-cff6221bdb03.txt
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 428, in run
    runtime = self._run_interface(runtime)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/utility/wrappers.py", line 142, in _run_interface
    out = function_handle(**args)
  File "<string>", line 18, in _select_template
  File "/opt/conda/lib/python3.8/site-packages/niworkflows/utils/misc.py", line 79, in get_template_specs
    tpl_target_path = get_template(in_template, **template_spec)
  File "/opt/conda/lib/python3.8/site-packages/templateflow/conf/__init__.py", line 31, in wrapper
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/templateflow/api.py", line 66, in get
    Path(p) for p in TF_LAYOUT.get(template=template, return_type="file", **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/bids/layout/layout.py", line 619, in get
    raise ValueError(msg + "If you're sure you want to impose "
ValueError: 'den' is not a recognized entity. Did you mean ['density']? If you're sure you want to impose this constraint, set invalid_filters='allow'.

effigies · March 7, 2023, 4:03pm

Thanks for this highly detailed post. So this is not a problem in templateflow, and I think you’ve done this correctly. It is a limitation in niworkflows’s spatial reference code:

github.com

nipreps/niworkflows/blob/master/niworkflows/utils/spaces.py

# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: nil -*-
# vi: set ft=python sts=4 ts=4 sw=4 et:
#
# Copyright 2021 The NiPreps Developers <nipreps@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# We support and encourage derived works from this project, please read
# about our expectations at
#

This file has been truncated. show original

See the get_fs_spaces() function that only expects fsnative or fsaverage.

github.com

nipreps/niworkflows/blob/647c895e423ad2bf853cd858ecd6dc88636e4859/niworkflows/utils/spaces.py#L655-L683

      
        
            def get_fs_spaces(self):
                """
                Return FreeSurfer spaces.
            
            
    Discards nonlegacy fsaverage values (i.e., with nonstandard density value).
            
            
    Examples
                --------
                >>> SpatialReferences([
                ...     'fsnative',
                ...     'fsaverage6',
                ...     'fsaverage5',
                ...     'MNI152NLin6Asym',
                ... ]).get_fs_spaces()
                ['fsnative', 'fsaverage6', 'fsaverage5']
            
            
    >>> SpatialReferences([
                ...     'fsnative',
                ...     'fsaverage6',
                ...     Reference(space='fsaverage', spec={'den': '30k'})

This file has been truncated. show original

Unfortunately, this isn’t something I can commit to spending any time on for several months at the least, but given how far you’ve gotten already, you might be interested in hacking on it and submitting a pull request.

If you’re willing to have a go at this, you can try patching an edited version of niworkflows into your singularity container with -B /path/to/niworkflows/niworkflows:/opt/conda/lib/python3.8/site-packages/niworkflows. (The in-container path is derived from your traceback. More recent images use Python 3.9 and may have changed where we install conda.)

That said, experimenting locally with Docker is often simpler, so you could try:

fmriprep-docker $INPUT $OUTPUT participant [...] --patch niworkflows=/path/to/niworkflows

And a quick note that we will not be accepting patches to anything but 20.2.x (LTS), 22.1.x (most recent release) and 23.0.x (upcoming). Once 23.0.0 is released, support for 22.1.x will be dropped. Please see our version matrix for the relevant niworkflows/smriprep versions for any given fmriprep series.

alexenge · March 7, 2023, 4:25pm

Thanks so much for the quick response, Chris!

And thanks for pointing me to the relevant function in niworkflows. I’ll try and see if I can patch this myself and will open a PR if so.

Specifying a custom *surface* template as fMRIPrep output space