Hello everyone,
I have applied fMRIPrep LTS v20.2.7 on resting state fmri dataset using singularity (two batches of the saame dataset with similar output directory declared in fmriprep script). fMRIPrep has been successfully applied for one batch and the html file was created. For the other batch, altough fmriprep was successfully completed, the html file was not created. I was wondering what the issue was and whether it is possible to create the html file without rerunnning the whole process? (I have checked this tread for that, but since temporary working directory was created for the jobs I do not have access to them).
fMRIPrep Error:
fMRIPrep finished successfully!
230418-06:32:42,343 nipype.workflow IMPORTANT:
Works derived from this fMRIPrep execution should include the boilerplate text found in <OUTPUT_PATH>/fmriprep/logs/CITATION.md.
Traceback (most recent call last):
File "/usr/local/miniconda/bin/fmriprep", line 10, in <module>
sys.exit(main())
File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 181, in main
packagename="fmriprep",
File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/reports/core.py", line 92, in generate_reports
for subject_label in subject_list
File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/reports/core.py", line 92, in <listcomp>
for subject_label in subject_list
File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/reports/core.py", line 72, in run_reports
reportlets_dir=reportlets_dir,
File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/reports/core.py", line 370, in generate_report
.findall((logs_path / "CITATION.html").read_text())[0]
IndexError: list index out of range
Finished tasks MTL0023 with exit code 1
fMRIPrep Command used:
#!/bin/bash
#SBATCH -J prep_MTL0023
#SBATCH --time=20:00:00
#SBATCH -n 1
#SBATCH --account=def-villens
#SBATCH --cpus-per-task=3
#SBATCH --mem-per-cpu=4G
#SBATCH --mail-type=None
# Load singularity module
SINGULARITY_IMG="/project/ctb-villens/quarantine/Core/fmriprep/20.2.7_LTS/fmriprep-20.2.7_LTS.simg"
module load VilleneuveLab
module load singularity/3.8
# Allocate your input and output directories, and allocate local scratch as the working directory
WORK_DIR=$(mktemp -d -p "${SLURM_TMPDIR}" work_dir-XXXX) || exit 1
echo "working directory is $work_dir"/home/javan/scratch/multi_prep/working_dir
# Define path for TSV file
STUDY="/home/javan/scratch/multi_prep/final/scripts/fmriprep/part1"
# Designate the location of your templateflow templates
export SINGULARITYENV_TEMPLATEFLOW_HOME=/project/ctb-villens/javan/templateflow
TEMPLATEFLOW_HOME="/project/ctb-villens/javan/templateflow"
export SINGULARITYENV_FS_LICENSE=/project/ctb-villens/quarantine/Core/freesurfer/7.1.0/license.txt
FS_LICENSE="/project/ctb-villens/quarantine/Core/freesurfer/7.1.0/license.txt"
#Parse the participants.tsv file and extract one subject ID from the line corresponding to this SLURM task
# The $((${SLURM_ARRAY_TASK_ID} + 1)) section is responsible for scipping the first line of the participant tsv file
#subject=$( sed -n -E "$((${SLURM_ARRAY_TASK_ID} + 1))s/sub-(\S*)\>.*/\1/gp" ${STUDY}/participant.tsv)
subject=MTL0023
# Build your command to tell singularity to run fmriprep, ensuring it binds to your template directory, your scratch folder
SINGULARITY_CMD="singularity run --cleanenv \
-B ${TEMPLATEFLOW_HOME}:${SINGULARITYENV_TEMPLATEFLOW_HOME} \
-B ${FS_LICENSE}:${SINGULARITYENV_FS_LICENSE} \
-B /project/ctb-villens/dataset/PreventAD/mri/sourcedata/bids:/source_bids:ro \
-B /home/javan/scratch/prep_all_part2/derivatives_final_LTSv20_2_7:/out \
-B /home/javan/scratch/prep_all_part2/script/final/code_final_LTSv20_2_7/sub-${subject}:/jbid \
-B /home/javan/scratch/prep_all_part2/derivatives/sub-${subject}:/fsdir \
-B "${WORK_DIR}":/work \
${SINGULARITY_IMG} \
/source_bids /out \
participant --participant-label ${subject} \
-w /work -vv --nthreads ${SLURM_CPUS_PER_TASK} \
--mem_mb $((${SLURM_MEM_PER_CPU}*${SLURM_CPUS_PER_TASK}-256)) \
--fs-license-file ${FS_LICENSE} \
--fs-subjects-dir /fsdir/ses-FU48A/anat \
--bids-filter-file /jbid/config_${subject}_suffix.json \
--notrack \
--skip-bids-validation \
--output-spaces MNI152NLin2009cAsym:res-2 MNI152NLin2009cAsym anat"
# Setup done, run the command
echo Running task ${subject}
echo Commandline: ${SINGULARITY_CMD}
eval ${SINGULARITY_CMD}
exitcode=$?
# Output results to a table
echo "sub-$subject $exitcode" \
>> ${SLURM_JOB_NAME}.${subject}.tsv
echo Finished tasks ${subject} with exit code $exitcode
exit $exitcode
Version: 20.2.7_LTS
Environment: Singularity
Config file used:
{"t1w": {
"datatype": "anat",
"session": "FU03A",
"acquisition": null,
"suffix": "T1w"
},
"bold": {
"datatype": "func",
"session": "FU03A",
"task": "rest",
"suffix": "bold"},
"fmap" : {
"datatype": "fmap",
"session": "FU03A",
"suffix": "magnitude1",
"suffix": "magnitude2",
"suffix": "phasediff"}}
Data used (the functional, structural and fmaps) are BIDS validated: output of validator is attached here:
bids-validator@1.6.1
^[[31m1: [ERR] Internal error. SOME VALIDATION STEPS MAY NOT HAVE OCCURRED (code: 0 - INTERNAL ERROR)^[[39m
.undefined
Evidence: Error: Could not load HED schema version "Latest" from remote repository - "Error: connect ECONNREFUSED 185.199.108.133:443".
at /src/node_modules/hed-validator/utils/schema.js:40:13
at tryCallOne (/src/node_modules/promise/lib/core.js:37:12)
at /src/node_modules/promise/lib/core.js:123:15
at flush (/src/node_modules/asap/raw.js:50:29)
at processTicksAndRejections (internal/process/task_queues.js:76:11)
^[[36m Please visit https://neurostars.org/search?q=INTERNAL ERROR for existing conversations about this issue.^[[39m
^[[31m2: [ERR] Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and ma>
./sub-MTL0023/ses-BL00A/asl/sub-MTL0023_ses-BL00A_asl.json
Evidence: sub-MTL0023_ses-BL00A_asl.json
./sub-MTL0023/ses-BL00A/asl/sub-MTL0023_ses-BL00A_asl.nii.gz
Evidence: sub-MTL0023_ses-BL00A_asl.nii.gz
./sub-MTL0023/ses-BL00A/func/sub-MTL0023_ses-BL00A_task-enc_events.edat2
Evidence: sub-MTL0023_ses-BL00A_task-enc_events.edat2
./sub-MTL0023/ses-BL00A/func/sub-MTL0023_ses-BL00A_task-enc_events.txt
Evidence: sub-MTL0023_ses-BL00A_task-enc_events.txt
./sub-MTL0023/ses-BL00A/func/sub-MTL0023_ses-BL00A_task-ret_events.edat2
Evidence: sub-MTL0023_ses-BL00A_task-ret_events.edat2
./sub-MTL0023/ses-BL00A/func/sub-MTL0023_ses-BL00A_task-ret_events.txt
Evidence: sub-MTL0023_ses-BL00A_task-ret_events.txt
./sub-MTL0023/ses-EN00A/func/sub-MTL0023_ses-EN00A_task-enc_events.edat2
Evidence: sub-MTL0023_ses-EN00A_task-enc_events.edat2
./sub-MTL0023/ses-EN00A/func/sub-MTL0023_ses-EN00A_task-enc_events.txt
Evidence: sub-MTL0023_ses-EN00A_task-enc_events.txt
./sub-MTL0023/ses-EN00A/func/sub-MTL0023_ses-EN00A_task-ret_events.edat2
Evidence: sub-MTL0023_ses-EN00A_task-ret_events.edat2
./sub-MTL0023/ses-EN00A/func/sub-MTL0023_ses-EN00A_task-ret_events.txt
Evidence: sub-MTL0023_ses-EN00A_task-ret_events.txt
^[[31m... and 24 more files having this issue (Use --verbose to see them all).^[[39m
^[[36m Please visit https://neurostars.org/search?q=NOT_INCLUDED for existing conversations about this issue.^[[39m
^[[31m3: [ERR] You have to define 'TaskName' for this file. (code: 50 - TASK_NAME_MUST_DEFINE)^[[39m
./sub-MTL0023/ses-BL00A/func/sub-MTL0023_ses-BL00A_task-enc_bold.nii.gz
./sub-MTL0023/ses-BL00A/func/sub-MTL0023_ses-BL00A_task-rest_run-1_bold.nii.gz
./sub-MTL0023/ses-BL00A/func/sub-MTL0023_ses-BL00A_task-rest_run-2_bold.nii.gz
./sub-MTL0023/ses-BL00A/func/sub-MTL0023_ses-BL00A_task-ret_bold.nii.gz
./sub-MTL0023/ses-EN00A/func/sub-MTL0023_ses-EN00A_task-enc_bold.nii.gz
./sub-MTL0023/ses-EN00A/func/sub-MTL0023_ses-EN00A_task-ret_bold.nii.gz
./sub-MTL0023/ses-FU03A/func/sub-MTL0023_ses-FU03A_task-enc_bold.nii.gz
./sub-MTL0023/ses-FU03A/func/sub-MTL0023_ses-FU03A_task-rest_run-1_bold.nii.gz
./sub-MTL0023/ses-FU03A/func/sub-MTL0023_ses-FU03A_task-rest_run-2_bold.nii.gz
./sub-MTL0023/ses-FU03A/func/sub-MTL0023_ses-FU03A_task-ret_bold.nii.gz
^[[31m... and 14 more files having this issue (Use --verbose to see them all).^[[39m
^[[36m Please visit https://neurostars.org/search?q=TASK_NAME_MUST_DEFINE for existing conversations about this issue.^[[39m
^[[33m1: [WARN] Task scans should have a corresponding events.tsv file. If this is a resting state scan you can ignore this warning or rename the task to include the word "rest". (code: 25 - EVENTS_TSV_MISSING)^[[3>
./sub-MTL0023/ses-BL00A/func/sub-MTL0023_ses-BL00A_task-enc_bold.nii.gz
./sub-MTL0023/ses-BL00A/func/sub-MTL0023_ses-BL00A_task-ret_bold.nii.gz
./sub-MTL0023/ses-EN00A/func/sub-MTL0023_ses-EN00A_task-enc_bold.nii.gz
./sub-MTL0023/ses-EN00A/func/sub-MTL0023_ses-EN00A_task-ret_bold.nii.gz
./sub-MTL0023/ses-FU03A/func/sub-MTL0023_ses-FU03A_task-enc_bold.nii.gz
./sub-MTL0023/ses-FU03A/func/sub-MTL0023_ses-FU03A_task-ret_bold.nii.gz
./sub-MTL0023/ses-FU12A/func/sub-MTL0023_ses-FU12A_task-enc_bold.nii.gz
./sub-MTL0023/ses-FU12A/func/sub-MTL0023_ses-FU12A_task-ret_bold.nii.gz
./sub-MTL0023/ses-FU24A/func/sub-MTL0023_ses-FU24A_task-enc_bold.nii.gz
./sub-MTL0023/ses-FU24A/func/sub-MTL0023_ses-FU24A_task-ret_bold.nii.gz
^[[33m... and 2 more files having this issue (Use --verbose to see them all).^[[39m
^[[36m Please visit https://neurostars.org/search?q=EVENTS_TSV_MISSING for existing conversations about this issue.^[[39m
^[[33m2: [WARN] Not all subjects/sessions/runs have the same scanning parameters. (code: 39 - INCONSISTENT_PARAMETERS)^[[39m
./sub-MTL0023/ses-FU36A/fmap/sub-MTL0023_ses-FU36A_run-03_magnitude1.nii.gz
./sub-MTL0023/ses-FU36A/fmap/sub-MTL0023_ses-FU36A_run-03_magnitude2.nii.gz
./sub-MTL0023/ses-FU36A/fmap/sub-MTL0023_ses-FU36A_run-03_phasediff.nii.gz
./sub-MTL0023/ses-FU36A/fmap/sub-MTL0023_ses-FU36A_run-04_magnitude1.nii.gz
./sub-MTL0023/ses-FU36A/fmap/sub-MTL0023_ses-FU36A_run-04_magnitude2.nii.gz
./sub-MTL0023/ses-FU36A/fmap/sub-MTL0023_ses-FU36A_run-04_phasediff.nii.gz
^[[36m Please visit https://neurostars.org/search?q=INCONSISTENT_PARAMETERS for existing conversations about this issue.^[[39m
^[[33m3: [WARN] The Authors field of dataset_description.json should contain an array of fields - with one author per field. This was triggered because there are no authors, which will make DOI registration from da>
^[[36m Please visit https://neurostars.org/search?q=NO_AUTHORS for existing conversations about this issue.^[[39m
^[[34m^[[4mSummary:^[[24m^[[39m ^[[34m^[[4mAvailable Tasks:^[[24m^[[39m ^[[34m^[[4mAvailable Modalities:^[[24m^[[39m
272 Files, 1.01GB T1w
1 - Subject T2star
8 - Sessions magnitude1
magnitude2
phasediff
bold
FLAIR
dwi
^[[36m If you have any questions, please post on https://neurostars.org/tags/bids.^[[39m
I highly appreciate your guidance on solving the error.
Kind regards,
Ali