Tedana is only processing three out of four runs for each participant

Hi,

I am using fmriprep and tedana to process multi-band multi-echo images. For some reason, when it gets to the tedana step, the first run of the images is skipped on most participants (I have three out of ten where this was not the case).

When this happens there is no error code, it just stops and moves to the next run:

<pre>STEP 3 - TEDANA mac 1
<font color="#3465A4">INFO:   </font> Using cached SIF image
/opt/conda/envs/tedana_py36/lib/python3.6/site-packages/nilearn/__init__.py:69: FutureWarning: Python 3.6 support is deprecated and will be removed in release 0.10 of Nilearn. Consider switching to Python 3.8 or 3.9.
  _python_deprecation_warnings()
[DD] TASK mac RUN 1
[DD] echo_images =  [&apos;sub-014_task-mac_acq-MB2ME_run-1_echo-3_desc-preproc_bold.nii.gz&apos;, &apos;sub-014_task-mac_acq-MB2ME_run-1_echo-1_desc-preproc_bold.nii.gz&apos;, &apos;sub-014_task-mac_acq-MB2ME_run-1_echo-2_desc-preproc_bold.nii.gz&apos;]
[DD] image_prefix_list =  {&apos;sub-014_task-mac_acq-MB2ME_run-1&apos;}
/tmp/prep_data /tmp/tedana/sub-014/MB2ME
sub-014
(Stops and goes to next run at this point)

My feeling is that there is something wrong with my first run but I’m not really sure where to start looking for the problem. I’m hoping this might be something that someone else has come across or that someone here on Neurostars might have some ideas of things that we can try.

Any thoughts or ideas would be greatly appreciated.

@jmcc146 your log looks odd to me. The messages don’t look similar to what tedana prints out to the command line. Is it maybe a wrapper script? The tedana runs should produce a log file in the output directory. Can you share that file for the failing run?

1 Like

Hi @tsalo,

I have a log file for each of the successful runs, but not the run that failed. For some reason it just seems to skip over run-1 entirely.

In that case, what script are you using to run tedana?

Here is the script used for tedana:

#!/usr/bin/env python
import pandas as pd
from tedana import workflows
import json
import os
import re
import time
from pathlib import Path
import argparse

# Output from frmiprep in step 2
prep_data = "/tmp/prep_data"

# base bids data directory
bids_dir= "/tmp/bids_dir"


parser = argparse.ArgumentParser()
parser.add_argument('--task', type=str)
parser.add_argument('--run', type=int, default=0)

args = parser.parse_args()

task=args.task
run=args.run

print ("[DD] TASK %s RUN %i" % (task, run))

# # Obtain Echo files
#find the prefix and suffix to that echo #
echo_images=[f for root, dirs, files in os.walk(prep_data) for f in files if (("_task-%s")%(task) in f)  & (("_run-%i_")%(run) in f or run == 0) & ('_echo-' in f) & (f.endswith('_bold.nii.gz'))]
print ("[DD] echo_images = ", echo_images)

#Make a list of filenames that match the prefix
image_prefix_list=[re.search('(.*)_echo-',f).group(1) for f in echo_images]
image_prefix_list=set(image_prefix_list)
print ("[DD] image_prefix_list = ", image_prefix_list)

#Make a dataframe where C1 is Sub C2 is inputFiles and C3 is Echotimes
data=[]
for acq in image_prefix_list:
    #Use RegEx to find Sub
    sub="sub-"+re.search('sub-(.*)_task',acq).group(1)
    #Make a list of the json's w/ appropriate header info from BIDS
    ME_headerinfo=[os.path.join(root, f) for root, dirs, files in os.walk(bids_dir) for f in files if (acq in f)& (f.endswith('_bold.json'))]

    #Read Echo times out of header info and sort
    echo_times=[json.load(open(f))['EchoTime'] for f in ME_headerinfo]
    echo_times.sort()

    #Find images matching the appropriate acq prefix
    acq_image_files=[os.path.join(root, f) for root, dirs, files in os.walk(prep_data) for f in files if (acq in f) & ('echo' in f) & (f.endswith('_desc-preproc_bold.nii.gz'))]
    acq_image_files.sort()
    
    res = re.search('MB\dME', acq_image_files[0]).group(0)
    
    out_dir= os.path.join(
        os.path.abspath(
            os.path.dirname( prep_data )), f"tedana/{sub}/{res}")

    print(prep_data,out_dir)

    data.append([sub,acq_image_files,echo_times,out_dir])

InData_df=pd.DataFrame(data=data,columns=['sub','EchoFiles','EchoTimes','OutDir'])
args=zip(InData_df['sub'].tolist(),
        InData_df['EchoFiles'].tolist(),
        InData_df['EchoTimes'].tolist(),
        InData_df['OutDir'].tolist())

  #Changes can be reasonably made to
  #fittype: 'loglin' is faster but maybe less accurate than 'curvefit'
  #tedpca:'mdl'Minimum Description Length returns the least number of components (default) and recommeded
  #'kic' Kullback-Leibler Information Criterion medium aggression
  # 'aic' Akaike Information Criterion least aggressive; i.e., returns the most components.
  #gscontrol: post-processing to remove spatially diffuse noise. options implemented here are...
  #global signal regression (GSR), minimum image regression (MIR),
  #But anatomical CompCor, Go Decomposition (GODEC), and robust PCA can also be used

def RUN_Tedana(sub,EchoFiles,EchoTimes,OutDir):

    time.sleep(2)
    print(sub+'\n')

    if not os.path.isdir(OutDir):
        Path.mkdir(Path(OutDir),exist_ok=True, parents=True)
        
    else:
        workflows.tedana_workflow(
        EchoFiles,
        EchoTimes,
        out_dir=OutDir,
        prefix="%s_task-%s_run-%i_acq_space-Native"%(sub,task,run), # set filename prefix for output 
        fittype="curvefit",
        tedpca="kic",
        verbose=True,
        gscontrol=None)


if __name__ == "__main__":

    for arg in args:
        RUN_Tedana(*arg)

Given that the other runs ran through successfully and run 1 didn’t run at all, I think the issue might be in your script. Is it possible that the OutDir for your first run already exists? It looks like your script skips tedana if the folder is already there.

1 Like