IntendedFor fmriprep

heenam · January 8, 2022, 7:36pm

Hello,

I am trying to resolve the following error re IntendedFor in the .json fieldmap files when running fmriprep v21.0.0.

Error:
220108-14:14:48,50 nipype.workflow INFO:
B0 field inhomogeneity map will be estimated with the following 1 estimators: [<EstimatorType.PHASEDIFF: 3>].
220108-14:14:48,275 nipype.workflow INFO:
No single-band-reference found for sub-001_ses-pre_task-nback_run-01_bold.nii.gz.
220108-14:14:48,275 nipype.workflow CRITICAL:
None of the available B0 fieldmaps are associated to </data/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-01_bold.nii.gz>
220108-14:14:48,741 nipype.workflow INFO:
No single-band-reference found for sub-001_ses-pre_task-nback_run-02_bold.nii.gz.
220108-14:14:48,741 nipype.workflow CRITICAL:
None of the available B0 fieldmaps are associated to </data/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-02_bold.nii.gz> … the same error messages are displayed for all functional runs.

 Setting-up fieldmap "auto_00000" (EstimatorType.PHASEDIFF) with <sub-001_ses-pre_run-01_phasediff.nii.gz, sub-001_ses-pre_run-01_magnitude1.nii.gz, sub-001_ses-pre_run-01_magnitude2.nii.gz>

220108-14:14:58,414 nipype.workflow INFO:
fMRIPrep workflow graph with 1275 nodes built successfully.

Below is the intendedfor text in the json files:
“EchoTime”:0.00517,
“IntendedFor”:[
“func/sub-001_ses-pre_task-nback_run-01_bold.nii.gz”,
“func/sub-001_ses-pre_task-nback_run-02_bold.nii.gz”,
“func/sub-001_ses-pre_task-rest_run-01_bold.nii.gz”,
“func/sub-001_ses-pre_task-sdmt_run-01_bold.nii.gz”,
“func/sub-001_ses-pre_task-sdmt_run-02_bold.nii.gz”
],
“FlipAngle”:60,

I have checked permissions, file paths, and that the naming convention accords with BIDS. Running on a supercomputer so haven’t been able to run BIDS validator. Would greatly appreciate any advice!

Many thanks!
Heena

Steven · January 8, 2022, 8:02pm

Running BIDS validator might help here (you can install the Docker image to run the validator on a supercomputer via docker or singularity).

It would help us help you if you could provide:

BIDS validator output
Command you used to run fMRIPrep
Output of tree sub-001/ses-pre/ (when run from inside the BIDS root directory)

Hope we can get to the bottom of this,
Steven

heenam · January 8, 2022, 11:02pm

Hi Steven,

Thanks for your quick response! I was able to install and run BIDS validator on the supercomputer. Please see below requested info:

1. BIDS validator output:
1: [ERR] Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files
are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a “.bidsignore” file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder. (code: 1 - NOT_INCLUDED)
./ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii
Evidence: sub-001_ses-pre_T1w_label-lesion_roi.nii
./ses-pre/anat/sub-001_ses-pre_run-01_T1w.json
Evidence: sub-001_ses-pre_run-01_T1w.json
./ses-pre/anat/sub-001_ses-pre_run-01_T1w.nii.gz
Evidence: sub-001_ses-pre_run-01_T1w.nii.gz
./ses-pre/anat/sub-001_ses-pre_run-01_T2w.json
Evidence: sub-001_ses-pre_run-01_T2w.json
./ses-pre/anat/sub-001_ses-pre_run-01_T2w.nii.gz
Evidence: sub-001_ses-pre_run-01_T2w.nii.gz
./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.bval
Evidence: sub-001_ses-pre_run-01_dwi.bval
./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.bvec
Evidence: sub-001_ses-pre_run-01_dwi.bvec
./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.json
Evidence: sub-001_ses-pre_run-01_dwi.json
./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.nii.gz
Evidence: sub-001_ses-pre_run-01_dwi.nii.gz
./ses-pre/fmap/sub-001_ses-pre_run-01_magnitude1.json
Evidence: sub-001_ses-pre_run-01_magnitude1.json
… and 21 more files having this issue (Use --verbose to see them all).

    Please visit https://neurostars.org/search?q=NOT_INCLUDED for existing conversations about this issue.

    2: [ERR] There are no subject folders (labeled "sub-*") in the root of this dataset. (code: 45 - SUBJECT_FOLDERS)

    Please visit https://neurostars.org/search?q=SUBJECT_FOLDERS for existing conversations about this issue.

    3: [ERR] The compulsory file /dataset_description.json is missing. See Section 03 (Modality agnostic files) of the BIDS specification. (code: 57 - DATASET_DESCRIPTION_JSON_MISSING)

    Please visit https://neurostars.org/search?q=DATASET_DESCRIPTION_JSON_MISSING for existing conversations about this issue.

    4: [ERR] Ses label contain an Illegal Character hyphen or underscore. Please edit the filename as per BIDS spec. (code: 63 - SESSION_VALUE_CONTAINS_ILLEGAL_CHARACTER)
            ./ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii

    Please visit https://neurostars.org/search?q=SESSION_VALUE_CONTAINS_ILLEGAL_CHARACTER for existing conversations about this issue.

    5: [ERR] Subject label in the filename doesn't match with the path of the file. File seems to be saved in incorrect subject directory. (code: 64 - SUBJECT_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY)
            ./ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: File: /ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii is saved in incorrect subject directory as per sub-id in filename.
            ./ses-pre/anat/sub-001_ses-pre_run-01_T1w.json
                    Evidence: File: /ses-pre/anat/sub-001_ses-pre_run-01_T1w.json is saved in incorrect subject directory as per sub-id in filename.
            ./ses-pre/anat/sub-001_ses-pre_run-01_T1w.nii.gz
                    Evidence: File: /ses-pre/anat/sub-001_ses-pre_run-01_T1w.nii.gz is saved in incorrect subject directory as per sub-id in filename.
            ./ses-pre/anat/sub-001_ses-pre_run-01_T2w.json
                    Evidence: File: /ses-pre/anat/sub-001_ses-pre_run-01_T2w.json is saved in incorrect subject directory as per sub-id in filename.
            ./ses-pre/anat/sub-001_ses-pre_run-01_T2w.nii.gz
                    Evidence: File: /ses-pre/anat/sub-001_ses-pre_run-01_T2w.nii.gz is saved in incorrect subject directory as per sub-id in filename.
            ./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.bval
                    Evidence: File: /ses-pre/dwi/sub-001_ses-pre_run-01_dwi.bval is saved in incorrect subject directory as per sub-id in filename.
            ./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.bvec
                    Evidence: File: /ses-pre/dwi/sub-001_ses-pre_run-01_dwi.bvec is saved in incorrect subject directory as per sub-id in filename.
            ./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.json
                    Evidence: File: /ses-pre/dwi/sub-001_ses-pre_run-01_dwi.json is saved in incorrect subject directory as per sub-id in filename.
            ./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.nii.gz
                    Evidence: File: /ses-pre/dwi/sub-001_ses-pre_run-01_dwi.nii.gz is saved in incorrect subject directory as per sub-id in filename.
            ./ses-pre/fmap/sub-001_ses-pre_run-01_magnitude1.json
                    Evidence: File: /ses-pre/fmap/sub-001_ses-pre_run-01_magnitude1.json is saved in incorrect subject directory as per sub-id in filename.
            ... and 21 more files having this issue (Use --verbose to see them all).
    Please visit https://neurostars.org/search?q=SUBJECT_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY for existing conversations about this issue.
    6: [ERR] Session label in the filename doesn't match with the path of the file. File seems to be saved in incorrect session directory. (code: 65 - SESSION_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY)
            ./ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: File: /ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii is saved in incorrect session directory as per ses-id in filename.
            ./ses-pre/anat/sub-001_ses-pre_run-01_T1w.json
                    Evidence: File: /ses-pre/anat/sub-001_ses-pre_run-01_T1w.json is saved in incorrect session directory as per ses-id in filename.
            ./ses-pre/anat/sub-001_ses-pre_run-01_T1w.nii.gz
                    Evidence: File: /ses-pre/anat/sub-001_ses-pre_run-01_T1w.nii.gz is saved in incorrect session directory as per ses-id in filename.
            ./ses-pre/anat/sub-001_ses-pre_run-01_T2w.json
                    Evidence: File: /ses-pre/anat/sub-001_ses-pre_run-01_T2w.json is saved in incorrect session directory as per ses-id in filename.
            ./ses-pre/anat/sub-001_ses-pre_run-01_T2w.nii.gz
                    Evidence: File: /ses-pre/anat/sub-001_ses-pre_run-01_T2w.nii.gz is saved in incorrect session directory as per ses-id in filename.
            ./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.bval
                    Evidence: File: /ses-pre/dwi/sub-001_ses-pre_run-01_dwi.bval is saved in incorrect session directory as per ses-id in filename.
            ./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.bvec
                    Evidence: File: /ses-pre/dwi/sub-001_ses-pre_run-01_dwi.bvec is saved in incorrect session directory as per ses-id in filename.
            ./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.json
                    Evidence: File: /ses-pre/dwi/sub-001_ses-pre_run-01_dwi.json is saved in incorrect session directory as per ses-id in filename.
            ./ses-pre/dwi/sub-001_ses-pre_run-01_dwi.nii.gz
                    Evidence: File: /ses-pre/dwi/sub-001_ses-pre_run-01_dwi.nii.gz is saved in incorrect session directory as per ses-id in filename.
            ./ses-pre/fmap/sub-001_ses-pre_run-01_magnitude1.json
                    Evidence: File: /ses-pre/fmap/sub-001_ses-pre_run-01_magnitude1.json is saved in incorrect session directory as per ses-id in filename.
            ... and 21 more files having this issue (Use --verbose to see them all).
    Please visit https://neurostars.org/search?q=SESSION_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY for existing conversations about this issue.
    1: [WARN] The recommended file /README is missing. See Section 03 (Modality agnostic files) of the BIDS specification. (code: 101 - README_FILE_MISSING)
    Please visit https://neurostars.org/search?q=README_FILE_MISSING for existing conversations about this issue.
    Summary:                  Available Tasks:        Available Modalities: 
    31 Files, 941.87MB                                                      
    0 - Subjects                                                            
    1 - Session

2. Command to run fMRIPrep:
#!/bin/bash
#SBATCH --time=20:00:00
#SBATCH --nodes=1 --ntasks-per-node=2
#SBATCH --cpus-per-task=8
#SBATCH --job-name=fmriprep_trac
#SBATCH --mail-type=FAIL
#SBATCH --mail-user=manglani.2@osu.edu
#SBATCH --account=PAS0457
#SBATCH --output=/fs/scratch/PAS0457/trac/code/output_logs/%x.%j.o
#SBATCH --error=/fs/scratch/PAS0457/trac/code/output_logs/%x.%j.e
#SBATCH --array=1

#Use appropriate 0 padding:
#00: 1,3,4,6,7,8,9
#0: 10,12,20,22,23,24,25,27,29,31,33,35,36,37,38,41,42,43,44,45,46,49,50,51,55,56,60,61,62,63,65,66,67,70,71,72,75,76,78,79,84,85,86,87,90,92,93,94,95,97,98
#None: 100,102,103,104,105,106,107,113,115,116

export SINGULARITY_CACHEDIR=$TMPDIR
export SINGULARITY_TMPDIR=$TMPDIR

module load singularity
unset PYTHONPATH
singularity run --cleanenv --bind /fs/scratch/PAS0457/trac/1_bids/nifti:/data --bind /fs/scratch/PAS0457/trac/3_fmriprep/lesions:/out /fs/scratch/PAS0457/fmriprep-21.0.0.simg
–output-spaces {T1w,MNI152NLin2009cAsym,func}
–skip_bids_validation
–ignore slicetiming
–nthreads $SLURM_CPUS_PER_TASK
–omp-nthreads SLURM_CPUS_PER_TASK \ --low-mem \ -w /fs/scratch/PAS0457/trac/temp/sub-00{SLURM_ARRAY_TASK_ID}
–participant-label sub-00${SLURM_ARRAY_TASK_ID}
–fs-license-file /fs/ess/PAS0457/7_trac/licenses/license.txt
/data /out participant

3. Output of tree sub-001/ses-pre/

Is there a new/better convention for saving the lesion roi files? However, that’s probably separate from the current issue with the fieldmaps.

Thank you!
Heena

Steven · January 9, 2022, 1:41am

I think you ran BIDS validator on the subject directory. BIDS validator should be run on the BIDS root directory (that is, the one that contains all of the subject folders, /fs/scratch/PAS0457/trac/1_bids/nifti). Can you paste the outputs from that?

Steven · January 9, 2022, 1:45am

I think the main problem is you might also need the session level in the IntendedFor elements. See here.

heenam · January 9, 2022, 2:23am

Here is the (long but not verbose) output from running the BIDS validator on the root dir:
bash-4.2$ ./validator_latest.sif /fs/scratch/PAS0457/trac/1_bids/nifti/
bids-validator@1.8.9
bids-specification@disable
(node:5814) Warning: Closing directory handle on garbage collection
(Use node --trace-warnings ... to show where the warning was created)
1: [ERR] Invalid JSON file. The file is not formatted according the schema. (code: 55 - JSON_SCHEMA_VALIDATION_ERROR)
./sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-01_bold.json
Evidence: .CogAtlasID should match format “uri”
./sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-02_bold.json
Evidence: .CogAtlasID should match format “uri”
./sub-001/ses-pre/func/sub-001_ses-pre_task-rest_run-01_bold.json
Evidence: .CogAtlasID should match format “uri”
./sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-01_bold.json
Evidence: .CogAtlasID should match format “uri”
./sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-02_bold.json
Evidence: .CogAtlasID should match format “uri”
./sub-003/ses-pre/func/sub-003_ses-pre_task-nback_run-01_bold.json
Evidence: .CogAtlasID should match format “uri”
./sub-003/ses-pre/func/sub-003_ses-pre_task-nback_run-02_bold.json
Evidence: .CogAtlasID should match format “uri”
./sub-003/ses-pre/func/sub-003_ses-pre_task-rest_run-01_bold.json
Evidence: .CogAtlasID should match format “uri”
./sub-003/ses-pre/func/sub-003_ses-pre_task-sdmt_run-01_bold.json
Evidence: .CogAtlasID should match format “uri”
./sub-003/ses-pre/func/sub-003_ses-pre_task-sdmt_run-02_bold.json
Evidence: .CogAtlasID should match format “uri”
… and 328 more files having this issue (Use --verbose to see them all).

    Please visit https://neurostars.org/search?q=JSON_SCHEMA_VALIDATION_ERROR for existing conversations about this issue.

    2: [ERR] Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder. (code: 1 - NOT_INCLUDED)
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.json
                    Evidence: sub-031_ses-pre_run-01_T1w.json
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.nii.gz
                    Evidence: sub-031_ses-pre_run-01_T1w.nii.gz
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T2w.json
                    Evidence: sub-031_ses-pre_run-01_T2w.json
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T2w.nii.gz
                    Evidence: sub-031_ses-pre_run-01_T2w.nii.gz
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.bval
                    Evidence: sub-031_ses-pre_run-01_dwi.bval
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.bvec
                    Evidence: sub-031_ses-pre_run-01_dwi.bvec
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.json
                    Evidence: sub-031_ses-pre_run-01_dwi.json
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.nii.gz
                    Evidence: sub-031_ses-pre_run-01_dwi.nii.gz
            ./nback/sub-031/ses-pre/fmap/sub-031_ses-pre_run-01_magnitude1.json
                    Evidence: sub-031_ses-pre_run-01_magnitude1.json
            ./nback/sub-031/ses-pre/fmap/sub-031_ses-pre_run-01_magnitude1.nii.gz
                    Evidence: sub-031_ses-pre_run-01_magnitude1.nii.gz
            ... and 83 more files having this issue (Use --verbose to see them all).

    Please visit https://neurostars.org/search?q=NOT_INCLUDED for existing conversations about this issue.

    3: [ERR] 'IntendedFor' field needs to point to an existing file. (code: 37 - INTENDED_FOR)
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude1.nii.gz
                    Evidence: func/sub-003_ses-pre_task-nback_run-01_bold.nii.gz
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude1.nii.gz
                    Evidence: func/sub-003_ses-pre_task-nback_run-02_bold.nii.gz
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude1.nii.gz
                    Evidence: func/sub-003_ses-pre_task-rest_run-01_bold.nii.gz
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude1.nii.gz
                    Evidence: func/sub-003_ses-pre_task-sdmt_run-01_bold.nii.gz
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude1.nii.gz
                    Evidence: func/sub-003_ses-pre_task-sdmt_run-02_bold.nii.gz
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude2.nii.gz
                    Evidence: func/sub-003_ses-pre_task-nback_run-01_bold.nii.gz
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude2.nii.gz
                    Evidence: func/sub-003_ses-pre_task-nback_run-02_bold.nii.gz
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude2.nii.gz
                    Evidence: func/sub-003_ses-pre_task-rest_run-01_bold.nii.gz
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude2.nii.gz
                    Evidence: func/sub-003_ses-pre_task-sdmt_run-01_bold.nii.gz
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude2.nii.gz
                    Evidence: func/sub-003_ses-pre_task-sdmt_run-02_bold.nii.gz
            ... and 990 more files having this issue (Use --verbose to see them all).

    Please visit https://neurostars.org/search?q=INTENDED_FOR for existing conversations about this issue.

    4: [ERR] Ses label contain an Illegal Character hyphen or underscore. Please edit the filename as per BIDS spec. (code: 63 - SESSION_VALUE_CONTAINS_ILLEGAL_CHARACTER)
            ./sub-001/ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-001/ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii
            ./sub-003/ses-pre/anat/sub-003_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-003/ses-pre/anat/sub-003_ses-pre_T1w_label-lesion_roi.nii
            ./sub-004/ses-pre/anat/sub-004_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-004/ses-pre/anat/sub-004_ses-pre_T1w_label-lesion_roi.nii
            ./sub-006/ses-pre/anat/sub-006_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-006/ses-pre/anat/sub-006_ses-pre_T1w_label-lesion_roi.nii
            ./sub-007/ses-pre/anat/sub-007_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-007/ses-pre/anat/sub-007_ses-pre_T1w_label-lesion_roi.nii
            ./sub-008/ses-pre/anat/sub-008_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-008/ses-pre/anat/sub-008_ses-pre_T1w_label-lesion_roi.nii
            ./sub-009/ses-pre/anat/sub-009_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-009/ses-pre/anat/sub-009_ses-pre_T1w_label-lesion_roi.nii
            ./sub-010/ses-pre/anat/sub-010_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-010/ses-pre/anat/sub-010_ses-pre_T1w_label-lesion_roi.nii
            ./sub-012/ses-pre/anat/sub-012_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-012/ses-pre/anat/sub-012_ses-pre_T1w_label-lesion_roi.nii
            ./sub-020/ses-pre/anat/sub-020_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-020/ses-pre/anat/sub-020_ses-pre_T1w_label-lesion_roi.nii
            ... and 58 more files having this issue (Use --verbose to see them all).

    Please visit https://neurostars.org/search?q=SESSION_VALUE_CONTAINS_ILLEGAL_CHARACTER for existing conversations about this issue.

    5: [ERR] Subject label in the filename doesn't match with the path of the file. File seems to be saved in incorrect subject directory. (code: 64 - SUBJECT_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY)
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.json
                    Evidence: File: /nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.json is saved in incorrect subject directory as per sub-id in filename.
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.nii.gz
                    Evidence: File: /nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.nii.gz is saved in incorrect subject directory as per sub-id in filename.
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T2w.json
                    Evidence: File: /nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T2w.json is saved in incorrect subject directory as per sub-id in filename.
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T2w.nii.gz
                    Evidence: File: /nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T2w.nii.gz is saved in incorrect subject directory as per sub-id in filename.
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.bval
                    Evidence: File: /nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.bval is saved in incorrect subject directory as per sub-id in filename.
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.bvec
                    Evidence: File: /nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.bvec is saved in incorrect subject directory as per sub-id in filename.
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.json
                    Evidence: File: /nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.json is saved in incorrect subject directory as per sub-id in filename.
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.nii.gz
                    Evidence: File: /nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.nii.gz is saved in incorrect subject directory as per sub-id in filename.
            ./nback/sub-031/ses-pre/fmap/sub-031_ses-pre_run-01_magnitude1.json
                    Evidence: File: /nback/sub-031/ses-pre/fmap/sub-031_ses-pre_run-01_magnitude1.json is saved in incorrect subject directory as per sub-id in filename.
            ./nback/sub-031/ses-pre/fmap/sub-031_ses-pre_run-01_magnitude1.nii.gz
                    Evidence: File: /nback/sub-031/ses-pre/fmap/sub-031_ses-pre_run-01_magnitude1.nii.gz is saved in incorrect subject directory as per sub-id in filename.
            ... and 15 more files having this issue (Use --verbose to see them all).

    Please visit https://neurostars.org/search?q=SUBJECT_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY for existing conversations about this issue.

    6: [ERR] Session label in the filename doesn't match with the path of the file. File seems to be saved in incorrect session directory. (code: 65 - SESSION_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY)
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.json
                    Evidence: File: /nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.json is saved in incorrect session directory as per ses-id in filename.
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.nii.gz
                    Evidence: File: /nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.nii.gz is saved in incorrect session directory as per ses-id in filename.
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T2w.json
                    Evidence: File: /nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T2w.json is saved in incorrect session directory as per ses-id in filename.
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T2w.nii.gz
                    Evidence: File: /nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T2w.nii.gz is saved in incorrect session directory as per ses-id in filename.
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.bval
                    Evidence: File: /nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.bval is saved in incorrect session directory as per ses-id in filename.
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.bvec
                    Evidence: File: /nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.bvec is saved in incorrect session directory as per ses-id in filename.
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.json
                    Evidence: File: /nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.json is saved in incorrect session directory as per ses-id in filename.
            ./nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.nii.gz
                    Evidence: File: /nback/sub-031/ses-pre/dwi/sub-031_ses-pre_run-01_dwi.nii.gz is saved in incorrect session directory as per ses-id in filename.
            ./nback/sub-031/ses-pre/fmap/sub-031_ses-pre_run-01_magnitude1.json
                    Evidence: File: /nback/sub-031/ses-pre/fmap/sub-031_ses-pre_run-01_magnitude1.json is saved in incorrect session directory as per ses-id in filename.
            ./nback/sub-031/ses-pre/fmap/sub-031_ses-pre_run-01_magnitude1.nii.gz
                    Evidence: File: /nback/sub-031/ses-pre/fmap/sub-031_ses-pre_run-01_magnitude1.nii.gz is saved in incorrect session directory as per ses-id in filename.
            ... and 14 more files having this issue (Use --verbose to see them all).

    Please visit https://neurostars.org/search?q=SESSION_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY for existing conversations about this issue.

    7: [ERR] The filename in scans.tsv file does not match what is present in the BIDS dataset. (code: 129 - SCANS_FILENAME_NOT_MATCH_DATASET)
            ./sub-006/ses-pre/sub-006_ses-pre_scans.tsv
                    @ line: 12
                    Evidence: Filename: /sub-006/ses-pre/anat/sub-006_ses-pre_run-02_T2w.nii.gz
            ./sub-008/ses-pre/sub-008_ses-pre_scans.tsv
                    @ line: 5
                    Evidence: Filename: /sub-008/ses-pre/anat/sub-008_ses-pre_run-02_T1w.nii.gz
            ./sub-012/ses-pre/sub-012_ses-pre_scans.tsv
                    @ line: 12
                    Evidence: Filename: /sub-012/ses-pre/anat/sub-012_ses-pre_run-02_T2w.nii.gz
            ./sub-020/ses-pre/sub-020_ses-pre_scans.tsv
                    @ line: 12
                    Evidence: Filename: /sub-020/ses-pre/anat/sub-020_ses-pre_run-02_T2w.nii.gz
            ./sub-022/ses-pre/sub-022_ses-pre_scans.tsv
                    @ line: 12
                    Evidence: Filename: /sub-022/ses-pre/anat/sub-022_ses-pre_run-02_T2w.nii.gz
            ./sub-025/ses-pre/sub-025_ses-pre_scans.tsv
                    @ line: 12
                    Evidence: Filename: /sub-025/ses-pre/anat/sub-025_ses-pre_run-02_T2w.nii.gz
            ./sub-029/ses-pre/sub-029_ses-pre_scans.tsv
                    @ line: 12
                    Evidence: Filename: /sub-029/ses-pre/anat/sub-029_ses-pre_run-02_T2w.nii.gz
            ./sub-029/ses-pre/sub-029_ses-pre_scans.tsv
                    @ line: 13
                    Evidence: Filename: /sub-029/ses-pre/anat/sub-029_ses-pre_run-02_T1w.nii.gz
            ./sub-036/ses-pre/sub-036_ses-pre_scans.tsv
                    @ line: 12
                    Evidence: Filename: /sub-036/ses-pre/anat/sub-036_ses-pre_run-02_T2w.nii.gz
            ./sub-042/ses-pre/sub-042_ses-pre_scans.tsv
                    @ line: 9
                    Evidence: Filename: /sub-042/ses-pre/fmap/sub-042_ses-pre_run-02_magnitude1.nii.gz
            ... and 32 more files having this issue (Use --verbose to see them all).

    Please visit https://neurostars.org/search?q=SCANS_FILENAME_NOT_MATCH_DATASET for existing conversations about this issue.

    1: [WARN] Not all subjects contain the same files. Each subject should contain the same number of files with the same naming unless some files are known to be missing. (code: 38 - INCONSISTENT_SUBJECTS)
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-03_bold.json
                    Evidence: Subject: sub-001; Missing file: sub-001_ses-pre_task-nback_run-03_bold.json
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-03_bold.nii.gz
                    Evidence: Subject: sub-001; Missing file: sub-001_ses-pre_task-nback_run-03_bold.nii.gz
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-03_events.tsv
                    Evidence: Subject: sub-001; Missing file: sub-001_ses-pre_task-nback_run-03_events.tsv
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-rest_run-02_bold.json
                    Evidence: Subject: sub-001; Missing file: sub-001_ses-pre_task-rest_run-02_bold.json
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-rest_run-02_bold.nii.gz
                    Evidence: Subject: sub-001; Missing file: sub-001_ses-pre_task-rest_run-02_bold.nii.gz
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-rest_run-02_events.tsv
                    Evidence: Subject: sub-001; Missing file: sub-001_ses-pre_task-rest_run-02_events.tsv
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-04_bold.json
                    Evidence: Subject: sub-001; Missing file: sub-001_ses-pre_task-sdmt_run-04_bold.json
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-04_bold.nii.gz
                    Evidence: Subject: sub-001; Missing file: sub-001_ses-pre_task-sdmt_run-04_bold.nii.gz
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-04_events.tsv
                    Evidence: Subject: sub-001; Missing file: sub-001_ses-pre_task-sdmt_run-04_events.tsv
            ./sub-003/ses-pre/func/sub-003_ses-pre_task-nback_run-03_bold.json
                    Evidence: Subject: sub-003; Missing file: sub-003_ses-pre_task-nback_run-03_bold.json
            ... and 622 more files having this issue (Use --verbose to see them all).

    Please visit https://neurostars.org/search?q=INCONSISTENT_SUBJECTS for existing conversations about this issue.

    2: [WARN] Tabular file contains custom columns not described in a data dictionary (code: 82 - CUSTOM_COLUMN_WITHOUT_DESCRIPTION)
            ./participants.tsv
                    Evidence: Columns: age, sex, group not defined, please define in: /participants.json
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-01_events.tsv
                    Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-nback_events.json,/run-01_events.json,/task-nback_run-01_events.json,/sub-001/sub-001_events.json,/sub-001/sub-001_task-nback_events.json,/sub-001/sub-001_run-01_events.json,/sub-001/sub-001_task-nback_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-nback_events.json,/sub-001/ses-pre/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-nback_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-01_events.json
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-02_events.tsv
                    Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-nback_events.json,/run-02_events.json,/task-nback_run-02_events.json,/sub-001/sub-001_events.json,/sub-001/sub-001_task-nback_events.json,/sub-001/sub-001_run-02_events.json,/sub-001/sub-001_task-nback_run-02_events.json,/sub-001/ses-pre/sub-001_ses-pre_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-nback_events.json,/sub-001/ses-pre/sub-001_ses-pre_run-02_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-nback_run-02_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_run-02_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-02_events.json
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-rest_run-01_events.tsv

s.json,/sub-001/ses-pre/func/sub-001_ses-pre_run-02_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-02_events.json
Evidence: Columns: TODO – fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-rest_events.json,/run-01_events.json,/task-rest_run-01_events.json,/sub-0
01/sub-001_events.json,/sub-001/sub-001_task-rest_events.json,/sub-001/sub-001_run-01_events.json,/sub-001/sub-001_task-rest_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-rest_ev
ents.json,/sub-001/ses-pre/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-rest_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-rest_events.json,
/sub-001/ses-pre/func/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-rest_run-01_events.json
./sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-01_events.tsv
Evidence: Columns: TODO – fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-sdmt_events.json,/run-01_events.json,/task-sdmt_run-01_events.json,/sub-0
01/sub-001_events.json,/sub-001/sub-001_task-sdmt_events.json,/sub-001/sub-001_run-01_events.json,/sub-001/sub-001_task-sdmt_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-sdmt_ev
ents.json,/sub-001/ses-pre/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-sdmt_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_events.json,
/sub-001/ses-pre/func/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-01_events.json
./sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-02_events.tsv
Evidence: Columns: TODO – fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-sdmt_events.json,/run-02_events.json,/task-sdmt_run-02_events.json,/sub-0
01/sub-001_events.json,/sub-001/sub-001_task-sdmt_events.json,/sub-001/sub-001_run-02_events.json,/sub-001/sub-001_task-sdmt_run-02_events.json,/sub-001/ses-pre/sub-001_ses-pre_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-sdmt_ev
ents.json,/sub-001/ses-pre/sub-001_ses-pre_run-02_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-sdmt_run-02_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_events.json,
/sub-001/ses-pre/func/sub-001_ses-pre_run-02_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-02_events.json
./sub-001/ses-pre/sub-001_ses-pre_scans.tsv
Evidence: Columns: operator, randstr not defined, please define in: /scans.json, /sub-001/sub-001_scans.json,/sub-001/ses-pre/sub-001_ses-pre_scans.json
./sub-003/ses-pre/func/sub-003_ses-pre_task-nback_run-01_events.tsv
Evidence: Columns: TODO – fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-nback_events.json,/run-01_events.json,/task-nback_run-01_events.json,/sub
-003/sub-003_events.json,/sub-003/sub-003_task-nback_events.json,/sub-003/sub-003_run-01_events.json,/sub-003/sub-003_task-nback_run-01_events.json,/sub-003/ses-pre/sub-003_ses-pre_events.json,/sub-003/ses-pre/sub-003_ses-pre_task-nba
ck_events.json,/sub-003/ses-pre/sub-003_ses-pre_run-01_events.json,/sub-003/ses-pre/sub-003_ses-pre_task-nback_run-01_events.json,/sub-003/ses-pre/func/sub-003_ses-pre_events.json,/sub-003/ses-pre/func/sub-003_ses-pre_task-nback_event
s.json,/sub-003/ses-pre/func/sub-003_ses-pre_run-01_events.json,/sub-003/ses-pre/func/sub-003_ses-pre_task-nback_run-01_events.json
./sub-003/ses-pre/func/sub-003_ses-pre_task-nback_run-02_events.tsv
Evidence: Columns: TODO – fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-nback_events.json,/run-02_events.json,/task-nback_run-02_events.json,/sub
-003/sub-003_events.json,/sub-003/sub-003_task-nback_events.json,/sub-003/sub-003_run-02_events.json,/sub-003/sub-003_task-nback_run-02_events.json,/sub-003/ses-pre/sub-003_ses-pre_events.json,/sub-003/ses-pre/sub-003_ses-pre_task-nba
ck_events.json,/sub-003/ses-pre/sub-003_ses-pre_run-02_events.json,/sub-003/ses-pre/sub-003_ses-pre_task-nback_run-02_events.json,/sub-003/ses-pre/func/sub-003_ses-pre_events.json,/sub-003/ses-pre/func/sub-003_ses-pre_task-nback_event
s.json,/sub-003/ses-pre/func/sub-003_ses-pre_run-02_events.json,/sub-003/ses-pre/func/sub-003_ses-pre_task-nback_run-02_events.json
./sub-003/ses-pre/func/sub-003_ses-pre_task-rest_run-01_events.tsv
Evidence: Columns: TODO – fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-rest_events.json,/run-01_events.json,/task-rest_run-01_events.json,/sub-0
03/sub-003_events.json,/sub-003/sub-003_task-rest_events.json,/sub-003/sub-003_run-01_events.json,/sub-003/sub-003_task-rest_run-01_events.json,/sub-003/ses-pre/sub-003_ses-pre_events.json,/sub-003/ses-pre/sub-003_ses-pre_task-rest_ev
ents.json,/sub-003/ses-pre/sub-003_ses-pre_run-01_events.json,/sub-003/ses-pre/sub-003_ses-pre_task-rest_run-01_events.json,/sub-003/ses-pre/func/sub-003_ses-pre_events.json,/sub-003/ses-pre/func/sub-003_ses-pre_task-rest_events.json,
/sub-003/ses-pre/func/sub-003_ses-pre_run-01_events.json,/sub-003/ses-pre/func/sub-003_ses-pre_task-rest_run-01_events.json
… and 393 more files having this issue (Use --verbose to see them all).
Please visit Search results for 'CUSTOM_COLUMN_WITHOUT_DESCRIPTION' - Neurostars for existing conversations about this issue.
3: [WARN] The recommended file /README is very small. Please consider expanding it with additional information about the dataset. (code: 213 - README_FILE_SMALL)
Please visit Search results for 'README_FILE_SMALL' - Neurostars for existing conversations about this issue.
Summary: Available Tasks: Available Modalities:
2120 Files, 63.97GB nback MRI
68 - Subjects sdmt
1 - Session rest
TODO: full task name for nback
TODO: full task name for rest
TODO: full task name for sdmt

I changed the IntendedFor to include the session but still getting the same error:
“IntendedFor”:[
“ses-pre/func/sub-001_ses-pre_task-nback_run-01_bold.nii.gz”,
“ses-pre/func/sub-001_ses-pre_task-nback_run-02_bold.nii.gz”,
“ses-pre/func/sub-001_ses-pre_task-rest_run-01_bold.nii.gz”,
“ses-pre/func/sub-001_ses-pre_task-sdmt_run-01_bold.nii.gz”,
“ses-pre/func/sub-001_ses-pre_task-sdmt_run-02_bold.nii.gz”
],

No single-band-reference found for sub-001_ses-pre_task-nback_run-01_bold.nii.gz.
220108-21:14:05,882 nipype.workflow INFO:
Found usable B0-map (fieldmap) estimator(s) <auto_00000> to correct </data/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-01_bold.nii.gz> for susceptibility-derived distortions.

Steven · January 9, 2022, 2:24pm

There’s a lot to unpack there… to make debugging easier, let’s try working with just a single subject. Can you create a new directory and copy over just dataset_description.json and the sub-001 folder? Then paste the validation outputs of this temporary directory.

heenam · January 9, 2022, 3:13pm

Here’s the validation outputs (pasted below and screenshots) from sub-001 + dataset_description.json:

bash-4.2$ ./validator_latest.sif /fs/scratch/PAS0457/trac/1_bids/nifti/test_sub-001/ --verbose
bids-validator@1.8.9
bids-specification@disable
(node:20845) Warning: Closing directory handle on garbage collection
(Use node --trace-warnings ... to show where the warning was created)
1: [ERR] Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a “.bidsignore” file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder. (code: 1 - NOT_INCLUDED)
./sub-001/ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii
Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a “.bidsignore” file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
Evidence: sub-001_ses-pre_T1w_label-lesion_roi.nii

    Please visit https://neurostars.org/search?q=NOT_INCLUDED for existing conversations about this issue.

    2: [ERR] Ses label contain an Illegal Character hyphen or underscore. Please edit the filename as per BIDS spec. (code: 63 - SESSION_VALUE_CONTAINS_ILLEGAL_CHARACTER)
            ./sub-001/ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii
                    Ses label contain an Illegal Character hyphen or underscore. Please edit the filename as per BIDS spec.
                    Evidence: ses name contains illegal character:/sub-001/ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii

    Please visit https://neurostars.org/search?q=SESSION_VALUE_CONTAINS_ILLEGAL_CHARACTER for existing conversations about this issue.
    1: [WARN] Tabular file contains custom columns not described in a data dictionary (code: 82 - CUSTOM_COLUMN_WITHOUT_DESCRIPTION)
            ./sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-01_events.tsv
                    Tabular file contains custom columns not described in a data dictionary
                    Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-nback_events.json,/run-01_events.json,/task-nback_run-01_events.json,/sub

-001/sub-001_events.json,/sub-001/sub-001_task-nback_events.json,/sub-001/sub-001_run-01_events.json,/sub-001/sub-001_task-nback_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-nba
ck_events.json,/sub-001/ses-pre/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-nback_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_event
s.json,/sub-001/ses-pre/func/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-01_events.json
./sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-02_events.tsv
Tabular file contains custom columns not described in a data dictionary
Evidence: Columns: TODO – fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-nback_events.json,/run-02_events.json,/task-nback_run-02_events.json,/sub
-001/sub-001_events.json,/sub-001/sub-001_task-nback_events.json,/sub-001/sub-001_run-02_events.json,/sub-001/sub-001_task-nback_run-02_events.json,/sub-001/ses-pre/sub-001_ses-pre_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-nba
ck_events.json,/sub-001/ses-pre/sub-001_ses-pre_run-02_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-nback_run-02_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_event
s.json,/sub-001/ses-pre/func/sub-001_ses-pre_run-02_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-nback_run-02_events.json
./sub-001/ses-pre/func/sub-001_ses-pre_task-rest_run-01_events.tsv
Tabular file contains custom columns not described in a data dictionary
Evidence: Columns: TODO – fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-rest_events.json,/run-01_events.json,/task-rest_run-01_events.json,/sub-0
01/sub-001_events.json,/sub-001/sub-001_task-rest_events.json,/sub-001/sub-001_run-01_events.json,/sub-001/sub-001_task-rest_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-rest_ev
ents.json,/sub-001/ses-pre/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-rest_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-rest_events.json,
/sub-001/ses-pre/func/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-rest_run-01_events.json
./sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-01_events.tsv
Tabular file contains custom columns not described in a data dictionary
Evidence: Columns: TODO – fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-sdmt_events.json,/run-01_events.json,/task-sdmt_run-01_events.json,/sub-0
01/sub-001_events.json,/sub-001/sub-001_task-sdmt_events.json,/sub-001/sub-001_run-01_events.json,/sub-001/sub-001_task-sdmt_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-sdmt_ev
ents.json,/sub-001/ses-pre/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-sdmt_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_events.json,
/sub-001/ses-pre/func/sub-001_ses-pre_run-01_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-01_events.json
./sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-02_events.tsv
Tabular file contains custom columns not described in a data dictionary
Evidence: Columns: TODO – fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-sdmt_events.json,/run-02_events.json,/task-sdmt_run-02_events.json,/sub-0
01/sub-001_events.json,/sub-001/sub-001_task-sdmt_events.json,/sub-001/sub-001_run-02_events.json,/sub-001/sub-001_task-sdmt_run-02_events.json,/sub-001/ses-pre/sub-001_ses-pre_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-sdmt_ev
ents.json,/sub-001/ses-pre/sub-001_ses-pre_run-02_events.json,/sub-001/ses-pre/sub-001_ses-pre_task-sdmt_run-02_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_events.json,
/sub-001/ses-pre/func/sub-001_ses-pre_run-02_events.json,/sub-001/ses-pre/func/sub-001_ses-pre_task-sdmt_run-02_events.json
./sub-001/ses-pre/sub-001_ses-pre_scans.tsv
Tabular file contains custom columns not described in a data dictionary
Evidence: Columns: operator, randstr not defined, please define in: /scans.json, /sub-001/sub-001_scans.json,/sub-001/ses-pre/sub-001_ses-pre_scans.json
Please visit Search results for 'CUSTOM_COLUMN_WITHOUT_DESCRIPTION' - Neurostars for existing conversations about this issue.
2: [WARN] The recommended file /README is missing. See Section 03 (Modality agnostic files) of the BIDS specification. (code: 101 - README_FILE_MISSING)
Please visit Search results for 'README_FILE_MISSING' - Neurostars for existing conversations about this issue.
Summary: Available Tasks: Available Modalities:
32 Files, 941.87MB nback MRI
1 - Subject sdmt
1 - Session rest

Thanks Steven for helping me with this so promptly!

Steven · January 9, 2022, 3:34pm

Besides the lesion file, this subject looks fine. Try deleting that from the temporary directory and let fMRIPrep run to completion, passing in this temporary directory as the BIDS root.

Steven · January 9, 2022, 3:45pm

While it’s running we can work on the rest of the errors:

This one doesn’t actually matter. Just be sure to --skip-bids-validation when running fmriprep so this doesn’t halt your run.

heenam:

2: [ERR] Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder. (code: 1 - NOT_INCLUDED)
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.json
                    Evidence: sub-031_ses-pre_run-01_T1w.json

Each folder in the bids root has to be either code,derivatives,sourcedata, or a subject folder. Subject folders cannot be nested in another folder, such as it is here in nback

heenam:

 3: [ERR] 'IntendedFor' field needs to point to an existing file. (code: 37 - INTENDED_FOR)
            ./sub-003/ses-pre/fmap/sub-003_ses-pre_run-01_magnitude1.nii.gz
                    Evidence: func/sub-003_ses-pre_task-nback_run-01_bold.nii.gz

I think this is the same error that I pointed out a few posts ago (having to include session level in the intended for field). The fact that I do not see sub-001 here is promising!

heenam:

 4: [ERR] Ses label contain an Illegal Character hyphen or underscore. Please edit the filename as per BIDS spec. (code: 63 - SESSION_VALUE_CONTAINS_ILLEGAL_CHARACTER)
            ./sub-001/ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii
                    Evidence: ses name contains illegal character:/sub-001/ses-pre/anat/sub-001_ses-pre_T1w_label-lesion_roi.nii

With not being able to see all files this error applies to, it is hard to know if it is specific to these lesion files, or something else. I, for the time being, might move these lesion files to the BIDS sourcedata folder, so the validator will ignore them. I think there is active discussion about where lesion masks should go, but no ruling yet.

Another possibility is that there is some weird text encoding in these files. This can happen if you copy text from a PC to a supercomputer. It is most often apparent with quotes (e.g. curly vs straight apostrophes), but I guess it can happen with hypens or underscores too? Try moving those lesion files and see if this error persists.

heenam:

5: [ERR] Subject label in the filename doesn't match with the path of the file. File seems to be saved in incorrect subject directory. (code: 64 - SUBJECT_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY)
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.json
                    Evidence: File: /nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.json is saved in incorrect subject directory as per sub-id in filename.

This relates to error 2 already discussed above

heenam:

 6: [ERR] Session label in the filename doesn't match with the path of the file. File seems to be saved in incorrect session directory. (code: 65 - SESSION_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY)
            ./nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.json
                    Evidence: File: /nback/sub-031/ses-pre/anat/sub-031_ses-pre_run-01_T1w.json is saved in incorrect session directory as per ses-id in filename.

Also relates to error 2 already discussed above

heenam:

    7: [ERR] The filename in scans.tsv file does not match what is present in the BIDS dataset. (code: 129 - SCANS_FILENAME_NOT_MATCH_DATASET)
            ./sub-006/ses-pre/sub-006_ses-pre_scans.tsv

Hard to help with this one since I cannot see what is in those tsv files. But fMRIPrep as far as I know does not read them, so similar to the first error, this should be fine as long as you skip bids validation.

heenam · January 9, 2022, 4:52pm

It’s running and here is the initial output: I’ve changed --skip_bids_validation to --skip-bids-validation in my fmriprep script. Hopefully that helps!

What does this mean? Specifically, what is the <auto_00000>?

No single-band-reference found for sub-001_ses-pre_task-rest_run-01_bold.nii.gz. 
Found usable B0-map (fieldmap) estimator(s) <auto_00000> to correct </data/sub-001/ses-pre/func/sub-001_ses-pre_task-rest_run-01_bold.nii.gz> for susceptibility-derived distortions.

To clarify, should I not have a ses-pre folder inside the sub folder even if I include that in IntendedFor?

Each folder in the bids root has to be either `code` ,`derivatives` ,`sourcedata` , or a subject folder. 
Subject folders cannot be nested in another folder, such as it is here in `nback`

For sub-031, we collected data over 2 days so one set of anat, fmap, func files are for nback and another set are for the other functional task (sdmt). Maybe I should create a new subject folder and move those files there so nback is not lurking in the root dir?

With not being able to see all files this error applies to, it is hard to know if it is specific to these lesion files, or something else. I, for the time being, might move these lesion files to the BIDS `sourcedata` folder, so the validator will ignore them. I think there is active discussion about where lesion masks should go, but no ruling yet.

Steven · January 9, 2022, 5:08pm

Either syntax is valid (at least in fMRIPrep, not all softwares are that flexible with argument names)

Looks like it is working as intended! That is just a default name used for fieldmap estimation in the susceptibility distortion correction workflow. See here for more details: sdcflows.utils.wrangler module - sdcflows 2.0.10 documentation

No, what I was saying that, besides the subject folders (sub-XXXX) you should only have folders for code, sourcedata, and derivatives in the root directory. The subject folders should contain the ses-pre folders, and that should be reflected in IntendedFor.

heenam:

For sub-031, we collected data over 2 days so one set of anat, fmap, func files are for nback and another set are for the other functional task (sdmt). Maybe I should create a new subject folder and move those files there so nback is not lurking in the root dir?

No, if data were collected over two days for one subject, then the data should be put into two different session levels within the same subject.

heenam · January 9, 2022, 7:31pm

Thanks so much for clarifying, Kevin! That all makes sense! I ran bids validation one more time before rerunning fmriprep again for all subs.

Is it ok to have extra file names in the scans.tsv file than in the folders? We collected extra T2w Flairs for some subjects or had extra T1s which have been deleted since so only 1 is retained but the original files are still listed in the tsv:

I’d welcome any code to delete the names of those files that are no longer in their respective (anat/func/dwi/fmap) directories!

Thanks!

Steven · January 9, 2022, 7:52pm

Like I said earlier, fMRIprep does not need these, so if you skip bids validation in fmriprep it should still run fine. In fact, BIDS itself doesn’t need it, but if you do have them and they are incorrect, validator will throw this error. Just to be safe, let’s keep these original TSVs somewhere (like in sourcedata for example), and do one of two alternative solutions.

Simply do not have a scans tsv file (easy!)
Use python to make a tsv ifle (more informative)

For the second solution, you can import the glob module and use glob.glob('/path/to/bids/sub-XXX/ses-pre/*/*') to get all the files for sub XXX in ses-pre. Then you can add those to a text/tsv file. There are several routes one can use for this, but this tutorial might be helpful.

If I were in your position, I would just move the incorrect scan TSV files (or all of them) into sourcedata for the time being and then run fMRIPrep while skipping bids validation.

Dianne_Patterson · April 12, 2023, 4:12am

Dear group,

I was playing with BIDScoin and saw that it produced Intendedfor fields in a format I was unfamiliar with.

{
   "IntendedFor": [
        "bids::sub-01/ses-pre/func/sub-01_ses-pre_task-motor_run-1_bold.nii.gz",
        "bids::sub-01/ses-pre/func/sub-01_ses-pre_task-motor_run-2_bold.nii.gz"
    ]
}

fMRIprep 23.0.1 was not happy with this new format:

	3: [ERR] 'IntendedFor' field needs to point to an existing file. (code: 37 - INTENDED_FOR)
		./sub-CAM003/ses-01brain/fmap/sub-CAM003_ses-01brain_acq-FieldMap_run-1_phasediff.nii.gz
	Evidence: bids::sub-CAM003/ses-01brain/func/sub-CAM003_ses-01brain_task-RSFMRIMBhighresolution_run-1_echo-1_bold.nii.gz

fMRIprep was happy with the deprecated format when I revised the JSON files manually, e.g.,

ses-01brain/fmap/sub-CAM003_ses-01brain_acq-FieldMap_run-1_phasediff.nii.gz

When might fmriprep start using the new format and when might they abandon the old format! I would appreciate any insights into this changing landscape!

Thank you,

Dianne Patterson

Steven · April 12, 2023, 1:19pm

@Dianne_Patterson It is being worked on: Support BIDS-URIs in `IntendedFor` · Issue #336 · nipreps/sdcflows · GitHub

effigies · April 12, 2023, 2:26pm

We should upgrade the BIDS validator bundled with fMRIPrep… For now --skip-bids-validation will be needed.