fMRIPrep + tedana: solutions to overly-inclusive masks?

Hello,
I’m preprocessing multi-echo data in fMRIPrep with the goal of using tedana for denoising (on the individual echo images as provided by --me-output-echos). I am running both fMRIPrep 21.0.2 and Tedana 0.0.12 via Singularity on our HPC.

A number of the fMRIPrep functional reports show an overly-inclusive BOLD mask that incorporates areas of skull and dura (see example at bottom of post). Anatomical images and coregistration look good. The final fMRIPrep-derived optimally-combined image also looks good; skull-stripping appears accurate there.

The individual echo images (_echo-*_desc-preproc_bold.nii) aren’t skull-stripped so it doesn’t look like the problem is carried over from fMRIPrep. However, the tedana-derived optimally-combined images (and their masks *desc-adaptiveGoodSignal_mask.nii ) show the same problem.

I have many questions…

1. What might be causing fMRIPrep’s overly-inclusive BOLD masks? Is there a way to correct it (if it matters)? Would it be expected to affect anything later in the workflow, particularly the transform matrices that I use later for normalizing the Tedana denoised output?

2. Is the inclusion of some skull/dura areas in the adaptive mask from Tedana problematic for the ICA-based denoising (because you’re including areas of signal of no interest in the data for decomposition)?

3. My current workflow is Tedana → normalize denoised OC image to MNI space. If I wanted to use the subject’s own mask from fMRIPrep as an explicit mask, would it make sense to reverse that – so normalize the individual echoes first to MNI space, then specify --mask *_MNI152NLin2009cAsym_desc-brain_mask.nii.gz as the mask for tedana? (Or is there a better way to generate a native space mask that doesn’t involve me re-running fMRIPrep on 73+ subjects x 4 tasks w/1-4 runs each? :wink: )

4. Alternatively, would you recommend just masking out the extraneous non-brain areas later, during the single-subject-level analysis, using the participant’s normalized T1w mask from /anat/?

I also considered tweaking the compute_epi_mask parameters in tedana, but I’m reluctant to make them more conservative given that the Tedana adaptive mask often has more-than-ideal dropout.


fMRIPrep details:

  • fMRIPrep version: 21.0.2
  • fMRIPrep command: /opt/conda/bin/fmriprep /data/BIDS_data /data/BIDS_data/derivatives/fmriprep participant --participant-label 003 --fs-license-file /hpc/users/seeles01/minerva/license.txt --skip_bids_validation --me-output-echos --n_cpus 16 --omp-nthreads 2 --mem-mb 120000 -vv --notrack

Functional data:

  • Original orientation: LAS
  • Repetition time (TR): 0.882s
  • Phase-encoding (PE) direction: Anterior-Posterior
  • Multi-echo EPI sequence: 4 echoes.
  • Slice timing correction: Applied
  • Susceptibility distortion correction: PEB/PEPOLAR (phase-encoding based / PE-POLARity)
  • Registration: FreeSurfer bbregister (boundary-based registration, BBR) - 6 dof
  • Non-steady-state volumes: 2

tedana details:

tedana -d
$sub"_task-"$task"_dir-AP_run-"$r"_echo-1_desc-preproc_bold.nii.gz"
$sub"_task-"$task"_dir-AP_run-"$r"_echo-2_desc-preproc_bold.nii.gz"
$sub"_task-"$task"_dir-AP_run-"$r"_echo-3_desc-preproc_bold.nii.gz"
$sub"_task-"$task"_dir-AP_run-"$r"_echo-4_desc-preproc_bold.nii.gz"
-e 11.0 29.7 48.4 67.1 --tedpca “kundu-stabilize”
–out-dir $sub_outdir
–debug | tee $sub"_task-"$task"_run-"$r"_tedana.log"

Example from fMRIPrep report:

Screen Shot 2022-06-28 at 10.36.39 AM

Example: optimally-combined image from Tedana (in native space)

Well, I found an answer to question 3: normalization prior to Tedana is currently not recommended but needs more empirical testing…

https://tedana.readthedocs.io/en/stable/multi-echo.html#perform-distortion-correction-spatial-normalization-smoothing-and-any-rescaling-or-filtering-after-denoising

Any step that will alter the relationship of signal magnitudes between echoes should occur after denoising and combining of the echoes. For example, if echo is separately scaled by its mean signal over time, then resulting intensity gradients and the subsequent calculation of voxelwise T2* values will be distorted or incorrect. See the description of tedana’s approach for more details on how T2* values are calculated. An aggressive temporal filter (i.e. a 0.1Hz low pass filter) or spatial smoothing could similarly distort the relationship between the echoes at each time point.

Note
We are assuming that spatial normalization and distortion correction, particularly non-linear normalization methods with higher order interpolation functions, are likely to distort the relationship between echoes while rigid body motion correction would linearly alter each echo in a similar manner. This assumption has not yet been empirically tested and an affine normalzation with bilinear interpolation may not distort the relationship between echoes. Additionally, there are benefits to applying only one spatial transform to data rather than applying one spatial transform for motion correction and a later transform for normalization and distortion correction. Our advice against doing normalization and distortion correction is a conservative choice and we encourage additional research to better understand how these steps can be applied before denoising.

Hello Saren,

Yes, it’s definitely possible. Having a good brain mask computed ahead of time could be useful in this kind of case.

As you mentioned in your second post, normalizing before denoising isn’t recommended. That said, it’s important to remember that we haven’t done any actual testing to check if it’s a major problem.

I believe transforming the space-T1w brain mask (or space-MNI152NLin2009cAsym if you didn’t output T1w-space derivatives) into native BOLD space would be a good solution to your problem, rather than warping the echoes. You should have access to the transforms necessary to do that.

It’s probably better to tackle the masking issue prior to denoising, IMHO.

Best,
Taylor

1 Like

Aha, thank you so much! So transforming space-T1w brain mask into native BOLD space = transforming *_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz using *_from-T1w_to-scanner_mode-image_xfm.txt, correct?

Or because I didn’t output the T1w-space derivatives, do I need to use the /anat/ transforms *_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5 as a first step?
(so I’m doing MNI space mask → T1w space → native space)?

Exactly. You’re going to need to use both transforms.

1 Like

Hi Taylor,

antsApplyTransforms seems to be having the same issue reading the xfm.h5 transforms as described here…any suggestions? It looks like some people might have had success using ANTs within the fmriprep singularity container but not sure how to do that.

the job .out file shows:

Sender: LSF System <lsfadmin@lh06c27>
Subject: Job 66579332: <masks_norm> in cluster <chimera> Exited

Job <masks_norm> was submitted from host <li03c03> by user <seeles01> in cluster <chimera> at Fri Jul  1 14:42:35 2022
Job was executed on host(s) <lh06c27>, in queue <express>, as user <seeles01> in cluster <chimera> at Fri Jul  1 14:42:36 2022
</hpc/users/seeles01> was used as the home directory.
</hpc/users/seeles01/minerva> was used as the working directory.
Started at Fri Jul  1 14:42:36 2022
Terminated at Fri Jul  1 14:42:39 2022
Results reported at Fri Jul  1 14:42:39 2022

Your job looked like:

------------------------------------------------------------
# LSBATCH: User input
#!/bin/bash

#BSUB -J masks_norm   #job name
#BSUB -P acc_federa03a          #project allocation number
#BSUB -q express                #queue to run on
#BSUB -n 1                      #1 job slot:
#BSUB -R affinity[core(16)]     #with 16 cores
#BSUB -R rusage[mem=120000]     #with 120GB memory dynamically allocated across cores
#BSUB -W 02:00                  #wall time
#BSUB -o %J.out                 #output file to t.out
#BSUB -e %J.err                 #error file to t.err
module purge
module load ants
bash batch_norm_masks_MNI-to-native.sh 060
Exited with exit code 1.
Resource usage summary:
    CPU time :                                   2.00 sec.
    Max Memory :                                 10 MB
    Average Memory :                             10.00 MB
    Total Requested Memory :                     120000.00 MB
    Delta Memory :                               119990.00 MB
    Max Swap :                                   -
    Max Processes :                              4
    Max Threads :                                5
    Run time :                                   3 sec.
    Turnaround time :                            4 sec.
The output (if any) follows:
060
faceStroop
1
Using single precision for computations.
Input time-series image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/func/sub-060_task-faceStroop_dir-AP_run-1_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
Reference image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/tpl-MNI152NLin2009cAsym_res-02_T1w.nii.gz
Can't read initial transform /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
2
Using single precision for computations.
Input time-series image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/func/sub-060_task-faceStroop_dir-AP_run-2_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
Reference image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/tpl-MNI152NLin2009cAsym_res-02_T1w.nii.gz
Can't read initial transform /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
RMET
1
Using single precision for computations.
Input time-series image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/func/sub-060_task-RMET_dir-AP_run-1_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
Reference image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/tpl-MNI152NLin2009cAsym_res-02_T1w.nii.gz
Can't read initial transform /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
2
Using single precision for computations.
Input time-series image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/func/sub-060_task-RMET_dir-AP_run-2_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
Reference image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/tpl-MNI152NLin2009cAsym_res-02_T1w.nii.gz
Can't read initial transform /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
IFT
1
Using single precision for computations.
Input time-series image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/func/sub-060_task-IFT_dir-AP_run-1_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
Reference image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/tpl-MNI152NLin2009cAsym_res-02_T1w.nii.gz
Can't read initial transform /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
2
Using single precision for computations.
Input time-series image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/func/sub-060_task-IFT_dir-AP_run-2_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
Reference image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/tpl-MNI152NLin2009cAsym_res-02_T1w.nii.gz
Can't read initial transform /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
3
Using single precision for computations.
Input time-series image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/func/sub-060_task-IFT_dir-AP_run-3_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
Reference image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/tpl-MNI152NLin2009cAsym_res-02_T1w.nii.gz
Can't read initial transform /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
4
Using single precision for computations.
Input time-series image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/func/sub-060_task-IFT_dir-AP_run-4_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
Reference image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/tpl-MNI152NLin2009cAsym_res-02_T1w.nii.gz
Can't read initial transform /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
rest
1
Using single precision for computations.
Input time-series image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/func/sub-060_task-rest_dir-AP_run-1_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
Reference image: /sc/arion/projects/psychres/WTC_resilience_imaging/data/tpl-MNI152NLin2009cAsym_res-02_T1w.nii.gz
Can't read initial transform /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
PS:
Read file <66579332.err> for stderr output of this job.

66579332.err shows:

HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
  #000: /hpc/packages/minerva-common/itk/4.8.2/checkout/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: /hpc/packages/minerva-common/itk/4.8.2/checkout/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: /hpc/packages/minerva-common/itk/4.8.2/checkout/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Gtraverse.c line 905 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: /hpc/packages/minerva-common/itk/4.8.2/checkout/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Gtraverse.c line 688 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: /hpc/packages/minerva-common/itk/4.8.2/checkout/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'TranformFixedParameters' doesn't exist
    major: Symbol table
    minor: Object not found
Transform reader for /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5 caught an ITK exception:

itk::ExceptionObject (0x1ab0660)
Location: "unknown" 
File: /hpc/packages/minerva-common/itk/4.8.2/checkout/Modules/IO/TransformHDF5/include/itkHDF5TransformIO.hxx
Line: 281
Description: itk::ERROR: HDF5TransformIOTemplate(0x1914380): H5Dopen2 failed

These are the versions of ANTs on our HPC…tried all of them and same issue:

Oct 18 2012 1.9.y
Apr 6 2015 2.1.0-1b14aa88
Jul 27 2020 2.1.0-67b2c5e4
Mar 17 2016 2.1.0-modified
Jan 28 2016 latest

AHA, figured it out! Based on a previous post explaining how to run antsApplyTransforms inside the fmriprep container, I was able to run my batch shell script inside an interactive fMRIPrep container on our HPC:

$ module load singularity
$ singularity shell /path/to/images/fmriprep-21.0.2.sif 
$ Singularity fmriprep-21.0.2.sif:~/> cd /path/to/derivatives/fmriprep
$ Singularity fmriprep-21.0.2.sif:~/> bash ~/path/to/scripts/batch_norm_masks_MNI-to-native.sh 

^ generated the following (specified sub-060 and faceStroop in .sh script):

060
faceStroop
1
Using single precision for computations.
Input time-series image: /path/to/derivatives/fmriprep/sub-060/func/sub-060_task-faceStroop_dir-AP_run-1_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
Reference image: /path/to/templates/tpl-MNI152NLin2009cAsym_res-02_T1w.nii.gz
=============================================================================
The composite transform comprises the following transforms (in order): 
  1. /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5[0] (type = AffineTransform)
  2. /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/anat/sub-060_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5[1] (type = DisplacementFieldTransform)
  3. /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep/sub-060/func/sub-060_task-faceStroop_dir-AP_run-1_from-scanner_to-T1w_mode-image_xfm.txt (type = AffineTransform)
=============================================================================
Default pixel value: 0
Interpolation type: WindowedSincInterpolateImageFunction
  Applying transform(s) to time point 0 (out of 1).
Output warped image: /path/to/derivatives/fmriprep/sub-060/func/sub-060_task-faceStroop_dir-AP_run-1_space-native_desc-brain_mask.nii.gz

Now to see what these output masks actually look like… [edit] They look great! If they look weird, make sure you specified your transforms in the correct order and direction, e.g.:

--transform $homedir"/sub-"$sub"/func/sub-"$sub"_task-"$task"_dir-AP_run-"$r"_from-T1w_to-scanner_mode-image_xfm.txt" \
--transform $homedir"/sub-"$sub"/anat/sub-"$sub"_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5" \

Hmm, now my native space mask isn’t matching the dimensions of the image that tedana’s make_adaptive_mask is using (I think ??)

Tried using fmriprep’s partially-preprocessed echo-1 images as the reference image instead of a template, but the resulting mask looked very wrong…

INFO     tedana:tedana_workflow:479 Loading input data: ['sub-004_task-IFT_dir-AP_run-4_echo-1_desc-preproc_bold.nii.gz', 'sub-004_task-IFT_dir-AP_run-4_echo-2_desc-preproc_bold.nii.gz', 'sub-004_task-IFT_dir-AP_run-4_echo-3_desc-preproc_bold.nii.gz', 'sub-004_task-IFT_dir-AP_run-4_echo-4_desc-preproc_bold.nii.gz']
INFO     tedana:tedana_workflow:561 Using user-defined mask
Traceback (most recent call last):
  File "/hpc/packages/minerva-centos7/py_packages/3.7/bin/tedana", line 8, in <module>
    sys.exit(_main())
  File "/hpc/packages/minerva-centos7/py_packages/3.7/lib/python3.7/site-packages/tedana/workflows/tedana.py", line 933, in _main
    tedana_workflow(**kwargs)
  File "/hpc/packages/minerva-centos7/py_packages/3.7/lib/python3.7/site-packages/tedana/workflows/tedana.py", line 590, in tedana_workflow
    threshold=1,
  File "/hpc/packages/minerva-centos7/py_packages/3.7/lib/python3.7/site-packages/tedana/utils.py", line 103, in make_adaptive_mask
    masksum = masksum * mask
ValueError: operands could not be broadcast together with shapes (288000,) (1082035,) 

Can you share the tedana command you used? It might also be helpful to share the warping script your used (batch_norm_masks_MNI-to-native.sh, I believe?).

I wonder if the issue might be coming from the --reference-image I’m using, since for warping input images, the reference image defines the spacing, origin, size, and direction of the output warped image. Have been playing around with using different template and subject-specific images and not sure what is correct.

(Also - I checked the masks and some participants still had issues with non-brain areas included in their masks, so I just now tried to create a mask in native space from their T1w brain mask in /anat/ instead of the MNI mask)

batch_norm_masks_T1w-to-native.sh:

homedir='/sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep'
outdir='/sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep'

if [ ! -d "$outdir" ]; then mkdir "$outdir"; fi
cd "$homedir"
tasks=('faceStroop' 'RMET' 'IFT' 'rest')
subs=($(ls -d sub*))

for sub in "${subs[@]}"; do
	echo "$sub"
	sub_outdir="$outdir/$sub/func"

	cd "$homedir/$sub/func"
	for task in "${tasks[@]}"; do
	#if ppt completed that task:
		#if [[ -n $(find "$homedir/$sub/func/" -iname "$task"*) ]]; then
			echo "$task"

			#get number of runs for that ppt/task:
			runs=($(ls *"$task"_dir-AP_run-*_desc-confounds_timeseries.json))
			nRuns=${#runs[@]}

			#for each run:
			for r in $(seq 1 $nRuns); do #for r in $(seq 2 2); do
				echo "$task run $r"
				# normalize image from T1w to native space using fmriprep-generated
				# transforms
				antsApplyTransforms --default-value 0 --float 1 --dimensionality 3 --verbose \
				--input $homedir"/"$sub"/anat/"$sub"_desc-brain_mask.nii.gz" \
				--interpolation LanczosWindowedSinc \
				--output $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz" \
			  --reference-image $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz" \
        --transform $homedir"/"$sub"/func/"$sub"_task-"$task"_dir-AP_run-"$r"_from-T1w_to-scanner_mode-image_xfm.txt" \
			#	--transform $homedir"/"$sub"/anat/"$sub"_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5" \

# smooth and binarize resulting mask
                        fslmaths $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz" -s .2 -bin $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz"
                       fslmaths $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz" -bin $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz"

			done
		#fi
	done

done

batch_process_tedana.sh:

#!/bin/bash

##specify relevant project dirs and info
homedir='/sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/fmriprep'
outdir='/sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/tedana'

if [ ! -d "$outdir" ]; then mkdir "$outdir"; fi
cd "$homedir"
tasks=('faceStroop' 'RMET' 'IFT' 'rest')
#subs=('sub-002') #subs=($(ls -d sub*))
subs=$1

for sub in "${subs[@]}"; do
	echo "$sub"
	sub_outdir="$outdir/$sub"
	if [ ! -d "$sub_outdir" ]; then mkdir "$sub_outdir"; fi

	cd "$homedir/$sub/func"
	for task in "${tasks[@]}"; do
	#if ppt completed that task:
		#if [[ -n $(find "$homedir/$sub/func/" -iname "$task"*) ]]; then
			echo "$task"

			#get number of runs for that ppt/task:
			runs=($(ls *"$task"_dir-AP_run-*_desc-confounds_timeseries.json))
			nRuns=${#runs[@]}
			#sub_outdir_sep="$sub_outdir/task-$task/run_$r" # redefine to put tedana output in separate directories so that we can see the visual reports

			# for each run:
				for r in $(seq 1 $nRuns); do

				#process partially preproc (fmriprep) data using tedana ME-ICA:
				#(no_gscontrol, low kdaw, and removal of a few initial volumes should help with occasional convergence issue)
				tedana -d \
				$sub"_task-"$task"_dir-AP_run-"$r"_echo-1_desc-preproc_bold.nii.gz" \
				$sub"_task-"$task"_dir-AP_run-"$r"_echo-2_desc-preproc_bold.nii.gz" \
				$sub"_task-"$task"_dir-AP_run-"$r"_echo-3_desc-preproc_bold.nii.gz" \
				$sub"_task-"$task"_dir-AP_run-"$r"_echo-4_desc-preproc_bold.nii.gz" \
				-e 11.0 29.7 48.4 67.1 --tedpca "kundu-stabilize" \
				--mask $homedir"/"$sub"/func/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz" \
				--out-dir $sub_outdir"/task-"$task"/run_"$r/


		done
		#fi
	done


done

Running batch_process_tedana.sh (via lsf) results in the following, repeated x every task/run:

INFO     tedana:tedana_workflow:466 Using output directory: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/tedana/sub-002/task-rest/run_1
INFO     tedana:tedana_workflow:479 Loading input data: ['sub-002_task-rest_dir-AP_run-1_echo-1_desc-preproc_bold.nii.gz', 'sub-002_task-rest_dir-AP_run-1_echo-2_desc-preproc_bold.nii.gz', 'sub-002_task-rest_dir-AP_run-1_echo-3_desc-preproc_bold.nii.gz', 'sub-002_task-rest_dir-AP_run-1_echo-4_desc-preproc_bold.nii.gz']
INFO     io:__init__:106 Generating figures directory: /sc/arion/projects/psychres/WTC_resilience_imaging/data/BIDS_data/derivatives/tedana/sub-002/task-rest/run_1/figures
INFO     tedana:tedana_workflow:561 Using user-defined mask
Traceback (most recent call last):
  File "/hpc/packages/minerva-centos7/py_packages/3.7/bin/tedana", line 8, in <module>
    sys.exit(_main())
  File "/hpc/packages/minerva-centos7/py_packages/3.7/lib/python3.7/site-packages/tedana/workflows/tedana.py", line 933, in _main
    tedana_workflow(**kwargs)
  File "/hpc/packages/minerva-centos7/py_packages/3.7/lib/python3.7/site-packages/tedana/workflows/tedana.py", line 590, in tedana_workflow
    threshold=1,
  File "/hpc/packages/minerva-centos7/py_packages/3.7/lib/python3.7/site-packages/tedana/utils.py", line 103, in make_adaptive_mask
    masksum = masksum * mask
ValueError: operands could not be broadcast together with shapes (288000,) (325325,) 

Here’s the relative tedana code:

Here’s one of the resulting masks:
https://www.dropbox.com/s/0gugiwrvwvazfpt/sub-002_task-rest_dir-AP_run-1_space-native_desc-brain_mask.nii.gz?dl=1

Tedana should probably compare the mask and data image affines/shapes before it applies the mask. I might open an issue about that in the future.

I’m only passingly familiar with antsApplyTransforms, but your script makes sense to me, except for the reference image, which I do think should be a native-space file. I assume the MNI-to-T1w transform was commented out because the brain mask was in T1w space already.

Is it possible that the warped affine is correct, but the resolution is not? I.e., do you need to resample the mask to the same resolution as the preprocessed functional data?

Thanks Taylor! I’m only passingly familiar with antsApplyTransforms myself.

If I understand correctly what ValueError: operands could not be broadcast together with shapes means, I believe the issue is that the native-space mask generated by me from the fmriprep transforms has a different size and dimensions than *echo-1_preproc-bold.nii.gz. Whereas the size and dimensions of Tedana’s adaptive mask match those of the bold images. This is the case even when using *echo-1_preproc-bold.nii.gz as the reference image for antsApplyTransforms.

*echo-1_preproc-bold.nii.gz dimensions:
echo-1-dim

native space mask (from T1w anat mask) dimensions:
my-mask-dim

native space mask (from MNI space bold mask) dimensions:
Screen Shot 2022-07-06 at 8.41.03 AM

Tedana adaptive mask dimensions:
adap-mask-dim

EDIT: Solved it!! I ran tedana (with no user-defined mask) for one subject, then saved the resulting adaptive mask as a template. Used that image as the reference image for antsApplyTransforms (i.e., providing it a 3D image matching the dimensions of *echo-1_preproc-bold.nii.gz). I think it’s okay that I’m using one subject’s image for everyone because the reference image “defines the spacing, origin, size, and direction of the output warped image” which should be the same across subjects, but I’ll double-check this after I run a few more subjects.

Tedana is running now, not done yet but it passed the make_adaptive_mask step without issue so that’s at least progress.

Script:

#!/bin/bash

##specify relevant project dirs and info
homedir='/path/to/BIDS_data/derivatives/fmriprep'
outdir='/path/to/BIDS_data/derivatives/fmriprep'
templdir='path/to/templates/'

if [ ! -d "$outdir" ]; then mkdir "$outdir"; fi
cd "$homedir"
tasks=('name1' 'name2' 'name3' 'rest') # for 3 tasks + resting state
subs=($(ls -d sub*)) # all directories starting with sub* in /derivatives/fmriprep

for sub in "${subs[@]}"; do
	echo "$sub"
	sub_outdir="$outdir/$sub/func"

	cd "$homedir/$sub/func"
	for task in "${tasks[@]}"; do
	#if ppt completed that task:
		#if [[ -n $(find "$homedir/$sub/func/" -iname "$task"*) ]]; then
			echo "$task"

			#get number of runs for that ppt/task:
			runs=($(ls *"$task"_dir-AP_run-*_desc-confounds_timeseries.json))
			nRuns=${#runs[@]}

			#for each run:
			for r in $(seq 1 $nRuns); do #for r in $(seq 2 2); do
				echo "$task run $r"
				# normalize image from T1w space to native space using fmriprep-generated
				# transforms (using T1w mask because bold mask includes some dura/skull)

				antsApplyTransforms \
				--input $homedir"/"$sub"/anat/"$sub"_desc-brain_mask.nii.gz" \
				--interpolation LanczosWindowedSinc \
				--output $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz" \
			        --reference-image $templdir"/sample_desc-adaptiveGoodSignal_mask.nii.gz" \
                                --transform $homedir"/"$sub"/func/"$sub"_task-"$task"_dir-AP_run-"$r"_from-T1w_to-scanner_mode-image_xfm.txt" 

             # need to smooth the resulting mask to get good coverage
             # because the T1w mask had a higher resolution than the bold images
				
			fslmaths $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz" -s .5 $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz"
                        fslmaths $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz" -bin $sub_outdir"/"$sub"_task-"$task"_dir-AP_run-"$r"_space-native_desc-brain_mask.nii.gz" 

			done
		#fi
	done

done

I don’t think you can use the same reference image across subjects, since their individual native spaces will be different, but I’m glad that using a 3D image instead of the 4D run files solved the issue!

EDIT: You can probably take the mean image from one of the runs as the reference image instead of having to run tedana to get the adaptive mask first, but if you already have the script set up to use the adaptive mask then sticking with that might be easier for you.

1 Like

Oh yeah, that makes sense, thanks! Hm, I wonder if I could use 3dResample (on the bold image I want to use as a reference) to do the same thing more efficiently…