Hi I was using the docker bids validator container with this command:
docker run -ti --rm -v C:\Users\sento\test_dir\physio_bids:/data bids/validator /data
However the quick validation error appears:
1: [ERR] Quick validation failed - the general folder structure does not resemble a BIDS dataset. Have you chosen the right folder (with “sub-*/” subfolders)? Check for structural/naming issues and presence of at least one subject. (code: 61 - QUICK_VALIDATION_FAILED)
This is the structure of my directory, there is only physiological data information:
Thank you for your message and welcome to NeuroStars! Perhaps can you try to validate your dataset against the web validator? That’ll resolve if there are docker mounting issues and help further evaluate this bug.
Hello @franklin,
Let me give you a bit more context:
@vinferrer is trying to set up a test for phys2bids, a python library that converts physiological signal data (e.g. cardiac pulse, respiratory belt, etc.) into BIDS format.
What he’s doing here is to test the output of our pipeline to see if we respect BIDS formats. ATM, we specifically work with data recorded during (f)MRI experiments, and that’s why we save _physio files in this structure.
@vinferrer raised the issue that the validation might be failing because, despite having a func folder, we only have _physio files inside and no _bold. Could that be the reason?
Alternatively, could the problem be the code/conversion folder, especially now that with BIDS 1.4 we have another hierarchical level on top of the sub-... folders? To better explain the question, could it be that since we produce the code folder we should be moving all the sub-... folders into a raw folder?
I know we could add the code folder to a .bidsignore file, but in our opinion it’s important to provide the information about the data conversion to BIDS. Is there any chance we can keep it?
Hello @smoia and @franklin
Thinking about the _bold file issue @smoia pointed out, I add a dummy file and now the bids validator seems be executing properly:
docker run -ti --rm -v C:\Users\sento\test_dir\physio_bids:/data bids/validator /data
bids-validator@1.5.7
1: [ERR] This file is too small to contain the minimal NIfTI header. (code: 36 - NIFTI_TOO_SMALL)
./sub-006/ses-01/func/sub-006_ses-01_task-test_rec-labchart_run-01_bold.nii.gz
Please visit https://neurostars.org/search?q=NIFTI_TOO_SMALL for existing conversations about this issue.
2: [ERR] Empty files not allowed. (code: 99 - EMPTY_FILE)
./sub-006/ses-01/func/sub-006_ses-01_task-test_rec-labchart_run-01_bold.nii.gz
Please visit https://neurostars.org/search?q=EMPTY_FILE for existing conversations about this issue.
1: [WARN] Tabular file contains custom columns not described in a data dictionary (code: 82 - CUSTOM_COLUMN_WITHOUT_DESCRIPTION)
./participants.tsv
Evidence: Columns: age, sex, handedness not defined, please define in: /participants.json
Please visit https://neurostars.org/search?q=CUSTOM_COLUMN_WITHOUT_DESCRIPTION for existing conversations about this issue.
2: [WARN] The Authors field of dataset_description.json should contain an array of fields - with one author per field. This was triggered because there are no authors, which will make DOI registration from dataset metadata impossible. (code: 113 - NO_AUTHORS)
Please visit https://neurostars.org/search?q=NO_AUTHORS for existing conversations about this issue.
Summary: Available Tasks: Available Modalities:
6 Files, 1.79MB bold
1 - Subject physio
1 - Session
If you have any questions, please post on https://neurostars.org/tags/bids.
Do you think that there’s a way we can escape the issue without using an empty file?
Nice, thank you @rwblair. I would love to see that implemented. However, will the absence of bold.nii.gz file affect the validation after the quicktest?
Looks like it should validate with no nifti files with the mentioned PR:
~/datasets $ /home/rwblair/projects/bids-validator/bids-validator/bin/bids-validator --ignoreWarnings physio/
bids-validator@1.5.8-dev.0
This dataset appears to be BIDS compatible.
Summary: Available Tasks: Available Modalities:
5 Files, 1.05KB physio
1 - Subject
1 - Session
If you have any questions, please post on https://neurostars.org/tags/bids.
(base) (rwblair@afterhours280) :) (09:20)
~/datasets $ find physio/
physio/
physio/dataset_description.json
physio/sub-006
physio/sub-006/ses-01
physio/sub-006/ses-01/func
physio/sub-006/ses-01/func/sub-006_ses-01_task-test_rec-labchart_run-01_physio.json
physio/sub-006/ses-01/func/sub-006_ses-01_task-test_rec-labchart_run-01_physio.tsv.gz
physio/README
physio/participants.tsv
@vinferrer Would there be any issue dumping these to beh modality instead of func, it seems to be the natural location from the specs definition, even if behavioral is a bit of a misnomer:
As you can see in the example the physio files are inside func, I think that is the most natural location to physio files that later on are used for bold denoising, for that reason I would not recommend to put them in beh modality
If they are supposed to be associated with BOLD series, then to validate them I would suggest creating some dummy BOLD series. Here’s some quick code to generate a filename
import numpy as np
import nibabel as nb
def gen_bold(nvols, TR, fname):
img = nb.Nifti1Image(np.zeros((2, 2, 2, nvols)), np.eye(4))
img.header.set_zooms((1, TR))
img.to_filename(fname)
(You’ll need to also create a JSON file with the mandatory TaskName and RepetitionTime metadata fields.)
BIDS is about having conceptually coherent datasets rather than validating that individual files meet a naming criterion. So functional data without a functional MR series doesn’t really make sense, even if all you’re trying to do is to validate related data. So you can either test by putting things in beh/, which can be treated as func/ (or meg/, eeg/ or ieeg/) without the neural data, or by creating some dummy data.
Hi @effigies, I tried your solution making a big dummy dataset and it seems to work. If you think physio files shouldn’t be allowed without a correspondent bold file I understand the logic and therefore I see why you don’t want to modify the quick validation test for this case. Thank you
