Use custom Dartel template in fmriprep

Summary of what happened:

Hello everyone! For the univariate fMRI analysis of my data I want to normalize the images to a custom Dartel template. We create this template in SPM12, using the raw T1w images of the full study sample. Besides the template itself I also created a brain mask and a json file. According to the fmriprep documentation, I should be able to save this template anywhere, as long as I define this correctly in my script. But I keep getting errors, so I think that I’m calling it incorrectly. I would really appreciate some help with this, because there is barely any documentation on this. I don’t want to share the template on templateflow, because this is a very study-specific template.

Command used:

#!/bin/bash
#SBATCH --account=owner-guest
#SBATCH --mail-user=u1350546@utah.edu
#SBATCH --partition=notchpeak-shared-guest
#SBATCH --job-name=fmriprep
#SBATCH --nodes=1
#SBATCH --ntasks=10
#SBATCH --mem=50G
#SBATCH --time=48:00:00
#SBATCH -o run_fmriprep_sif-%j.out-%N
#SBATCH -e run_fmriprep_sif-%j.err-%N

# * Environment
container="/uufs/chpc.utah.edu/common/home/u1350546/fmriprep_latest.sif"
base="/uufs/chpc.utah.edu/common/home/u1350546/Documents/fmriprep"
output_base="/uufs/chpc.utah.edu/common/home/bking-group1"
template_dir="/uufs/chpc.utah.edu/common/home/u1350546/Documents/fmriprep/TEMPLATEFLOW_HOME/tpl-Dartel"
fslicense="${base}/license.txt"
dat_dir="${base}/OfflineBrain_BIDS"
der_dir="${output_base}/derivatives"
wrk_dir="${output_base}/tmp"
subject="108"    # Test Subject ID, should not contain 'sub-'
mkdir -p "${der_dir}" "${wrk_dir}"


# * Load singularity/Apptainer
module load python
module load apptainer


# * Run fMRIprep
apptainer \
    run \
    --cleanenv \
    -B "${dat_dir}":/data:ro \
    -B "${der_dir}":/derivatives \
    -B "${wrk_dir}":/work \
    -B "${fslicense}":/license.txt:ro \
    -B "${template_dir}":/templateflow \
    "${container}" \
    /data \
    "/derivatives/fMRIPrep" \
    participant \
    --participant-label "${subject}" \
    -w /work/ \
    --fs-no-reconall \
    --output-spaces Dartel \
    --write-graph \
    --notrack \
    --fs-license-file /license.txt \
    --ignore slicetiming \
    -vv \
    --skip-bids-validation
exit

Version:

24.1.0

Environment

Apptainer, within a high performance computing cluster

Folder organization

TEMPLATEFLOW_HOME/
   tpl-Dartel/ 
      template_description.json
      tpl-Dartel_res-1_T1w.nii.gz
      tpl-Dartel_res-1_desc-brain_mask.nii.gz

Relevant error outputs:

ValueError: space identifier "Dartel" is invalid.
Valid identifiers are: Fischer344, MNI152Lin, MNI152NLin2009aAsym, MNI152NLin2009aSym, MNI152NLin2009bAsym, MNI152NLin2009bSym, MNI152NLin2009cAsym, MNI152NLin2009cSym, MNI152NLin6Asym, MNI152NLin6Sym, MNI305, MNIColin27, MNIInfant, MNIPediatricAsym, MouseIn, NKI, NMT31Sym, OASIS30ANTs, PNC, RESILIENT, UNCInfant, VALiDATe29, WHS, dhcpAsym, dhcpSym, dhcpVol, fsLR, fsaverage, onavg, T1w, T2w, anat, fsnative, func, run, sbref, session, individual, dwi, asl

JSON file:

{
  "Name": "Dartel",
  "TemplateFlowVersion": "0.7.1",
  "Resolution": [1],
  "ReferencesAndLinks": [
    " "
  ]
}

Hi @ankevr97,

Can you run export APPTAINTER_TEMPLATEFLOW_HOME=/templateflow before your fmriprep command?

Best,
Steven

Thank you for the suggestion, Steven! Unfortunately, that didn’t fix the problem. I still get the error below. Any suggestions on what else I could try?

Reoccuring error:

ValueError: space identifier "Dartel" is invalid.
Valid identifiers are: Fischer344, MNI152Lin, MNI152NLin2009aAsym, MNI152NLin2009aSym, MNI152NLin2009bAsym, MNI152NLin2009bSym, MNI152NLin2009cAsym, MNI152NLin2009cSym, MNI152NLin6Asym, MNI152NLin6Sym, MNI305, MNIColin27, MNIInfant, MNIPediatricAsym, MouseIn, NKI, NMT31Sym, OASIS30ANTs, PNC, RESILIENT, UNCInfant, VALiDATe29, WHS, dhcpAsym, dhcpSym, dhcpVol, fsLR, fsaverage, onavg, T1w, T2w, anat, fsnative, func, run, sbref, session, individual, dwi, asl

The updated script:

#!/bin/bash
#SBATCH --account=owner-guest
#SBATCH --mail-user=u1350546@utah.edu
#SBATCH --partition=notchpeak-shared-guest
#SBATCH --job-name=fmriprep
#SBATCH --nodes=1
#SBATCH --ntasks=10
#SBATCH --mem=50G
#SBATCH --time=48:00:00
#SBATCH -o run_fmriprep_sif-%j.out-%N
#SBATCH -e run_fmriprep_sif-%j.err-%N

export APPTAINTER_TEMPLATEFLOW_HOME=/templateflow

# * Environment
container="/uufs/chpc.utah.edu/common/home/u1350546/fmriprep_latest.sif"
base="/uufs/chpc.utah.edu/common/home/u1350546/Documents/fmriprep"
output_base="/uufs/chpc.utah.edu/common/home/bking-group1"
template_base="/uufs/chpc.utah.edu/common/home/u1350546/Documents/fmriprep/TEMPLATEFLOW_HOME/tpl-Dartel"
fslicense="${base}/license.txt"
dat_dir="${base}/OfflineBrain_BIDS"
der_dir="${output_base}/derivatives"
wrk_dir="${output_base}/tmp"
subject="108"    # Test Subject ID, should not contain 'sub-'
mkdir -p "${der_dir}" "${wrk_dir}"


# * Load singularity/Apptainer
module load python
module load apptainer


# * Run fMRIprep
apptainer \
    run \
    --cleanenv \
    -B "${dat_dir}":/data:ro \
    -B "${der_dir}":/derivatives \
    -B "${wrk_dir}":/work \
    -B "${fslicense}":/license.txt:ro \
    -B "${template_base}":/templateflow \
    "${container}" \
    /data \
    "/derivatives/fMRIPrep" \
    participant \
    --participant-label "${subject}" \
    -w /work/ \
    --fs-no-reconall \
    --output-spaces Dartel \
    --write-graph \
    --notrack \
    --fs-license-file /license.txt \
    --ignore slicetiming \
    -vv \
    --skip-bids-validation
exit

Templateflow home should point to the folder that contains the tpl-* folders, so one directory up. What are the contents of /uufs/chpc.utah.edu/common/home/u1350546/Documents/fmriprep/TEMPLATEFLOW_HOME/? Also it should be APPTAINERENV_TEMPLATEFLOW_HOME (note the “ENV” after “APPTAINER”), sorry for mistype!

Best,
Steven

Thanks, Steven! The only content of Templateflow_home is what outlined in my post:

TEMPLATEFLOW_HOME/
   tpl-Dartel/ 
      template_description.json
      tpl-Dartel_res-1_T1w.nii.gz
      tpl-Dartel_res-1_desc-brain_mask.nii.gz

You’re going to need to have a more-or-less fully populated templateflow directory, as fMRIPrep uses templates for various steps such as skull-stripping. I would prepare your directory as follows:

export TEMPLATEFLOW_HOME=<desired path>
python -m pip install templateflow
curl -sSL https://github.com/nipreps/fmriprep/raw/24.1.0/scripts/fetch_templates.py | python
cp -R <dartel dataset> $TEMPLATEFLOW_HOME

Thank you for the suggestion! I actually think that the code that Steven provided already did this, because when I check my templateflow directory now, it has a bunch of templates including the MNI ones.

That already solved a portion of it. Thank you! But of course I ran into another error now. Is this also related to the templates that I use or is there something wrong with my fieldmaps themselves?

2024-10-09 10:52:14,574 [   ERROR] Node fmap_recon.a0 failed to run on host notch425.
2024-10-09 10:52:14,575 [   ERROR] Saving crash info to /derivatives/fMRIPrep/sub-108/log/20241009-100831_af6ee638-c250-4529-8fda-ab3e55d405e1/crash-20241009-105214-u1350546-fmap_recon.a0-29d324b8-506b-4298-ac64-4c34f1fc024d.txt
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node fmap_recon.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 397, in run
	    runtime = self._run_interface(runtime)
	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/interfaces/resampling.py", line 169, in _run_interface
	    transforms = load_transforms(self.inputs.transforms, self.inputs.inverse)
	                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/utils/transforms.py", line 27, in load_transforms
	    xfm = load_ants_h5(path)
	          ^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/utils/transforms.py", line 85, in load_ants_h5
	    raise ValueError(f'Unexpected spacing: {spacing}')
	ValueError: Unexpected spacing: [1.5 1.5 1.5]

That’s the T1w-to-template transform, and you’re hitting this section that was an intentional speed bump to make sure we didn’t silently do the wrong thing:

I believe your error message indicates that you have 1.5mm isotropic voxels. Could you run nib-ls on your template files and share the output (you may need to pip install nibabel)?

If this is the case, this would be a good chance to relax this constraint. If I pushed a patched Docker image, would you be able to convert it to Apptainer and test it?

@ankevr97 If you want to test out docker://nipreps/fmriprep:isotropic-warps, I think that should resolve your issue.

Thank you for sharing that! The previous error was actually because of a stupid mistake. I put the wrong resolution in the JSON file of the Dartel template.

I did run into a new error though. This is the information provided in the crash-file:

Node: fmriprep_24_1_wf.sub_108_wf.anat_fit_wf.anat_reports_wf.norm_msk
Working directory: /work/fmriprep_24_1_wf/sub_108_wf/anat_fit_wf/anat_reports_wf/_in_tuple_Dartel.resnative/norm_msk

Node inputs:

after = /work/fmriprep_24_1_wf/sub_108_wf/anat_fit_wf/anat_reports_wf/_in_tuple_Dartel.resnative/t1w_std/sub-108_desc-preproc_T1w_trans.nii.gz
after_mask = /work/fmriprep_24_1_wf/sub_108_wf/anat_fit_wf/anat_reports_wf/_in_tuple_Dartel.resnative/mask_std/sub-108_desc-brain_mask_trans.nii.gz
before = /templateflow/tpl-Dartel/tpl-Dartel_res-1_T1w.nii.gz
function_str = def _rpt_masks(mask_file, before, after, after_mask=None):
    from os.path import abspath

    import nibabel as nb

    msk = nb.load(mask_file).get_fdata() > 0
    bnii = nb.load(before)
    nb.Nifti1Image(bnii.get_fdata() * msk, bnii.affine, bnii.header).to_filename('before.nii.gz')
    if after_mask is not None:
        msk = nb.load(after_mask).get_fdata() > 0

    anii = nb.load(after)
    nb.Nifti1Image(anii.get_fdata() * msk, anii.affine, anii.header).to_filename('after.nii.gz')
    return abspath('before.nii.gz'), abspath('after.nii.gz')

mask_file = /templateflow/tpl-Dartel/tpl-Dartel_res-1_desc-brain_mask.nii.gz

Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node norm_msk.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 397, in run
	    runtime = self._run_interface(runtime)
	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/interfaces/utility/wrappers.py", line 142, in _run_interface
	    out = function_handle(**args)
	          ^^^^^^^^^^^^^^^^^^^^^^^
	  File "<string>", line 8, in _rpt_masks
	ValueError: operands could not be broadcast together with shapes (121,145,121,3) (121,145,121) 

We’re starting to wonder if it’s even possible to use our Dartel template in fmriprep…

Could you share the results from nib-ls of your Dartel images?

nib-ls -s tpl-Dartel_res-1_T1w.nii.gz
nib-ls -c tpl-Dartel_res-1_desc-brain_mask.nii.gz

For the template:

[u1350546@notch308:fmriprep]$ /uufs/chpc.utah.edu/common/home/u1350546/.local/bin/nib-ls /uufs/chpc.utah.edu/common/home/u1350546/Documents/fmriprep/TEMPLATEFLOW_HOME/tpl-Dartel/tpl-Dartel_res-1_T1w.nii.gz
/uufs/chpc.utah.edu/common/home/u1350546/Documents/fmriprep/TEMPLATEFLOW_HOME/tpl-Dartel/tpl-Dartel_res-1_T1w.nii.gz float32 [121, 145, 121,   3] 1.50x1.50x1.50x0.00

For the brain mask:

[u1350546@notch308:fmriprep]$ /uufs/chpc.utah.edu/common/home/u1350546/.local/bin/nib-ls /uufs/chpc.utah.edu/common/home/u1350546/Documents/fmriprep/TEMPLATEFLOW_HOME/tpl-Dartel/tpl-Dartel_res-1_desc-brain_mask.nii.gz
/uufs/chpc.utah.edu/common/home/u1350546/Documents/fmriprep/TEMPLATEFLOW_HOME/tpl-Dartel/tpl-Dartel_res-1_desc-brain_mask.nii.gz int16 [121, 145, 121] 1.50x1.50x1.50

Okay, you’ve got a 3-volume T1w image. We won’t know what to do with that. What do the volumes look like? Are they different, or is it three copies of the same image?

Dartel does indeed include 3 volumes and they are not copies of each other. So does this mean that we cannot use our Dartel template in fmriprep?

Those images look like different tissue segmentations. The image you need (at a minimum) is the full T1-weighted image. Do you have one available?

No, this is what the Dartel creation step in SPM12 outputs. This is the “template” that it uses to normalize the images to. It goes through 6 iterations to create the template, which consists of 3 volumes. It also creates a flowfield. But that’s it. I had to run a separate step to get the brain mask.

Fmriprep uses the T1w for registration, so if you do not have one, you cannot use that template.

Best,
Steven

Well, we’ll just do our normalization and smoothing in SPM12 then. I really appreciate you guys’ help with this!!

Sounds good. I’d recommend normalizing Dartel to the BOLD MNI as opposed to other way around. Much less computationally expensive and you don’t over-interpolate your BOLD data.

Best,
Steven