BIDS Help - Validator (anat data and quantifying # of issues)

Hello all,

Pardon my lack of experience - I’m new to running heudiconv and I’m using a wrapper on the BIDS Validator called fw-heudiconv. fw-heudiconv runs on an Imaging cloud service called Flywheel and takes acquisitions that have been curated that is stored on Flywheel and input it into the standard BIDS Validator.

I’ve acquired the log from running the wrapper, and there are somethings that I am confused about and would appreciate some extra pairs of eyes.

  1. I have sorted any anatomical images (all of which are either pure T1/saggital, sometimes including T2) into /anat/. I looked within BIDS documentation to see if I can change it to anything else within my keys, but I couldn’t find any other solutions. What could I change about this?

  2. In ERR 2, it seems to specify the scans that are being named incorrectly by my heuristic, but then in ERR 3, it says there is ‘no compatible data’ underneath this subject. I think to better my understanding of what ERR 2 is trying to tell me, what is the difference between the first and the second scan listed in ‘Evidence’?

My log is down below, and the BIDS Structure is included here:

Gear Name: fw-heudiconv, Gear Version: 0.2.4_0.1.9
Gear starting...

/flywheel/v0/fw_heudiconv/query.py:4: UserWarning: The DICOM readers are highly experimental, unstable, and only work for Siemens time-series at the moment
Please use with caution.  We would be grateful for your help in improving them
  from nibabel.nicom.dicomwrappers import wrapper_from_data
INFO: ==============: fw-heudiconv gear manager starting up :===============

INFO: Calling fw-heudiconv with the following settings:
INFO: Project: Sandbox_HUP3TLegacy
INFO: Subject(s): ['121078']
INFO: Session(s): ['20160902-141444.953000']
INFO: Heuristic: 
INFO: Action: Validate
INFO: Dry run: False
INFO:fw-heudiconv-validator:================: fw-heudiconv validator starting up :================

INFO:fw-heudiconv-validator:Launching fw-heudiconv-export...
INFO:fw-heudiconv-validator:bids_directory/
    .bidsignore
    dataset_description.json
    sub-121078/
        ses-20160902.1414/
            dwi/
                sub-121078_ses-20160902.1414_axial_dti_ap.bval
                sub-121078_ses-20160902.1414_axial_dti_ap.bvec
                sub-121078_ses-20160902.1414_axial_dti_ap.json
                sub-121078_ses-20160902.1414_axial_dti_ap.nii.gz
            func/
                sub-121078_ses-20160902.1414_resting_bold.nii.gz
                sub-121078_ses-20160902.1414_resting_bold.json
            anat/
                sub-121078_ses-20160902.1414_T2.nii.gz
                sub-121078_ses-20160902.1414_T2.json
                sub-121078_ses-20160902.1414_T1.nii.gz
                sub-121078_ses-20160902.1414_T1.json

INFO:fw-heudiconv-validator:Launching bids-validator...
INFO:fw-heudiconv-validator:bids-validator@1.3.12

	1: [ERR] Invalid JSON file. The file is not formatted according the schema. (code: 55 - JSON_SCHEMA_VALIDATION_ERROR)
		./dataset_description.json
			Invalid JSON file. The file is not formatted according the schema.
			Evidence: .Funding should be array

	Please visit https://neurostars.org/search?q=JSON_SCHEMA_VALIDATION_ERROR for existing conversations about this issue.

	2: [ERR] Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder. (code: 1 - NOT_INCLUDED)
		./sub-121078/ses-20160902.1414/anat/sub-121078_ses-20160902.1414_T1.json
			Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
			Evidence: sub-121078_ses-20160902.1414_T1.json
		./sub-121078/ses-20160902.1414/anat/sub-121078_ses-20160902.1414_T1.nii.gz
			Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
			Evidence: sub-121078_ses-20160902.1414_T1.nii.gz
		./sub-121078/ses-20160902.1414/anat/sub-121078_ses-20160902.1414_T2.json
			Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
			Evidence: sub-121078_ses-20160902.1414_T2.json
		./sub-121078/ses-20160902.1414/anat/sub-121078_ses-20160902.1414_T2.nii.gz
			Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
			Evidence: sub-121078_ses-20160902.1414_T2.nii.gz
		./sub-121078/ses-20160902.1414/dwi/sub-121078_ses-20160902.1414_axial_dti_ap.bval
			Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
			Evidence: sub-121078_ses-20160902.1414_axial_dti_ap.bval
		./sub-121078/ses-20160902.1414/dwi/sub-121078_ses-20160902.1414_axial_dti_ap.bvec
			Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
			Evidence: sub-121078_ses-20160902.1414_axial_dti_ap.bvec
		./sub-121078/ses-20160902.1414/dwi/sub-121078_ses-20160902.1414_axial_dti_ap.json
			Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
			Evidence: sub-121078_ses-20160902.1414_axial_dti_ap.json
		./sub-121078/ses-20160902.1414/dwi/sub-121078_ses-20160902.1414_axial_dti_ap.nii.gz
			Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
			Evidence: sub-121078_ses-20160902.1414_axial_dti_ap.nii.gz
		./sub-121078/ses-20160902.1414/func/sub-121078_ses-20160902.1414_resting_bold.json
			Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
			Evidence: sub-121078_ses-20160902.1414_resting_bold.json
		./sub-121078/ses-20160902.1414/func/sub-121078_ses-20160902.1414_resting_bold.nii.gz
			Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
			Evidence: sub-121078_ses-20160902.1414_resting_bold.nii.gz

	Please visit https://neurostars.org/search?q=NOT_INCLUDED for existing conversations about this issue.

	3: [ERR] No BIDS compatible data found for at least one subject. (code: 67 - NO_VALID_DATA_FOUND_FOR_SUBJECT)
		./sub-121078
			No BIDS compatible data found for subject 121078

	Please visit https://neurostars.org/search?q=NO_VALID_DATA_FOUND_FOR_SUBJECT for existing conversations about this issue.

	1: [WARN] The recommended file /README is missing. See Section 03 (Modality agnostic files) of the BIDS specification. (code: 101 - README_FILE_MISSING)

	Please visit https://neurostars.org/search?q=README_FILE_MISSING for existing conversations about this issue.

	2: [WARN] The Authors field of dataset_description.json should contain an array of fields - with one author per field. This was triggered because there are no authors, which will make DOI registration from dataset metadata impossible. (code: 113 - NO_AUTHORS)

	Please visit https://neurostars.org/search?q=NO_AUTHORS for existing conversations about this issue.


        Summary:                 Available Tasks:        Available Modalities: 
        11 Files, 75.81MB                                                      
        0 - Subjects                                                           
        1 - Session                                                            


	If you have any questions, please post on https://neurostars.org/tags/bids.


INFO:fw-heudiconv-validator:
INFO:fw-heudiconv-validator:Done!
INFO:fw-heudiconv-validator:==================: Exiting fw-heudiconv validator :==================
INFO: Done!
INFO: ================: Exiting fw-heudiconv gear manager :=================


Gear completed successfully after running for 20.579s.
Uploading results...
Job complete.

Hi @isabelannwingert

Thank you for your message and welcome to NeuroStars! Digging into your questions:

Regarding the scans (error 2) the validator is getting stuck on the period between your dates for ses and on the modality scans (slightly different reasons for each). Breaking down into the ses and modality specific errors.

Regarding the session - this can be resolved by dropping the period between your two numbers. This would also need to be changed at your session level because if the ses in the filename and ses folder are different that will raise an error.

Regarding the anat - The suffix is close! Adding ‘w’ to make it T1w. An example for the path to the T1w file can look like:
sub-121078/ses-201609021414/anat/sub-121078_ses-201609021414_T1w.nii.gz

Regarding the functional - is adding the task key to the file name. An example, the path to the func file can look like:
sub-121078/ses-201609021414/func/sub-121078_ses-201609021414_task-rest_bold.nii.gz (with the same filename for the accompanying json file)

Regarding the diffusion - the direction can be preserved. An example, the path to the dwi file can look like:
sub-121078/ses-201609021414/dwi/sub-121078_ses-201609021414_dir-ap_dwi.nii.gz (with the same filename for the accompanying json, bval and bvec files)

I referenced the specification for comparing the file names with the templates specified.

When the filenames are resolved this will resolve error 3 because it currently does not see any compatible (validated) files.

Thank you,
Franklin

I am running into a similar error, but can not identtify why data is not being identified.

        {
                    "key": "NO_VALID_DATA_FOUND_FOR_SUBJECT",
                    "code": 67,
                    "file": {
                        "relativePath": "/sub-000410",
                        "webkitRelativePath": "/sub-000410",
                        "name": "/sub-000410",
                        "path": "/sub-000410"
                    },
                    "evidence": null,
                    "line": null,
                    "character": null,
                    "severity": "error",
                    "reason": "No BIDS compatible data found for subject 000410",
                    "helpUrl": "https://neurostars.org/search?q=NO_VALID_DATA_FOUND_FOR_SUBJECT"
                },

======================
find sub-000410 -type f
sub-000410/ses-43317/anat/sub-000410_ses-43317_run-003_T1w.json
sub-000410/ses-43317/anat/sub-000410_ses-43317_run-003_T1w.nii.gz

=====================
Of my 1500+ subjects only 18 are reported to have the NO_VALID_DATA_FOUND_FOR_SUBJECT problem.

Any assistance on why this issue persists would be greatly appreciated.

hi @Hans_Johnson

Thank you for your message! Perhaps to clarify - were there other associated errors for these subjects? The previous post had a few nested issues that resulted in the odd validator summary. By chance, do you have a large warning output?

May you please also share the version of the validator you are utilizing?

Thank you,
Franklin