I wonder whether this is a problem on the BIDS structure of the output runs. Can you post a tree of your original dataset (one subject showing this issue)?
Additionally, if you want to run the extra mile, my hypothesis is that there is some change in pyBIDS 0.9.2 we’ve missed in the latest update and the selection of reportlets is not working.
The process is as follows:
-
Given a subject, generate a
Reportobject. -
Initialize a BIDS Layout, which is based on this config file.
-
Run queries scripted in the report template.
├── ses-01
│ ├── anat
│ │ ├── sub-23017_ses-01_acq-mprage_rec-ORIG_T1w.json
│ │ ├── sub-23017_ses-01_acq-mprage_rec-ORIG_T1w.nii.gz -> ../../../../edb/subj_internal_id/7/40008/subj_internal_id_7_40008_00001.nii.gz
│ │ ├── sub-23017_ses-01_rec-ORIG_T2w.json
│ │ └── sub-23017_ses-01_rec-ORIG_T2w.nii.gz -> ../../../../edb/subj_internal_id/7/40006/subj_internal_id_7_40006_00001.nii.gz
│ ├── fmap
│ │ ├── sub-23017_ses-01_acq-rest_dir-match_epi.json
│ │ ├── sub-23017_ses-01_acq-rest_dir-match_epi.nii.gz -> ../../../../edb/subj_internal_id/7/11/subj_internal_id_7_11_00001.nii.gz
│ │ ├── sub-23017_ses-01_acq-rest_dir-opp_epi.json
│ │ ├── sub-23017_ses-01_acq-rest_dir-opp_epi.nii.gz -> ../../../../edb/subj_internal_id/7/10/subj_internal_id_7_10_00001.nii.gz
│ │ ├── sub-23017_ses-01_acq-task_dir-match_epi.json
│ │ ├── sub-23017_ses-01_acq-task_dir-match_epi.nii.gz -> ../../../../edb/subj_internal_id/7/14/subj_internal_id_7_14_00001.nii.gz
│ │ ├── sub-23017_ses-01_acq-task_dir-opp_epi.json
│ │ └── sub-23017_ses-01_acq-task_dir-opp_epi.nii.gz -> ../../../../edb/subj_internal_id/7/13/subj_internal_id_7_13_00001.nii.gz
│ └── func
│ ├── sub-23017_ses-01_task-mid_bold.json
│ ├── sub-23017_ses-01_task-mid_bold.nii.gz -> ../../../../edb/subj_internal_id/7/12/subj_internal_id_7_12_00001.nii.gz
│ ├── sub-23017_ses-01_task-mid_events.json
│ ├── sub-23017_ses-01_task-mid_events.tsv
│ ├── sub-23017_ses-01_task-rest_echo-1_bold.json
│ ├── sub-23017_ses-01_task-rest_echo-1_bold.nii.gz -> ../../../../edb/subj_internal_id/7/9/subj_internal_id_7_9_00001.nii.gz
│ ├── sub-23017_ses-01_task-rest_echo-2_bold.json
│ ├── sub-23017_ses-01_task-rest_echo-2_bold.nii.gz -> ../../../../edb/subj_internal_id/7/9/subj_internal_id_7_9_00002.nii.gz
│ ├── sub-23017_ses-01_task-rest_echo-3_bold.json
│ ├── sub-23017_ses-01_task-rest_echo-3_bold.nii.gz -> ../../../../edb/subj_internal_id/7/9/subj_internal_id_7_9_00003.nii.gz
│ ├── sub-23017_ses-01_task-rest_echo-4_bold.json
│ ├── sub-23017_ses-01_task-rest_echo-4_bold.nii.gz -> ../../../../edb/subj_internal_id/7/9/subj_internal_id_7_9_00004.nii.gz
│ ├── sub-23017_ses-01_task-rest_physio.json
│ └── sub-23017_ses-01_task-rest_physio.tsv.gz
├── ses-02
│ ├── anat
│ │ ├── sub-23017_ses-02_acq-mprage_rec-ORIG_T1w.json
│ │ └── sub-23017_ses-02_acq-mprage_rec-ORIG_T1w.nii.gz -> ../../../../edb/subj_internal_id/10/40009/subj_internal_id_10_40009_00001.nii.gz
│ ├── fmap
│ │ ├── sub-23017_ses-02_acq-rest_dir-match_epi.json
│ │ ├── sub-23017_ses-02_acq-rest_dir-match_epi.nii.gz -> ../../../../edb/subj_internal_id/10/4/subj_internal_id_10_4_00001.nii.gz
│ │ ├── sub-23017_ses-02_acq-rest_dir-opp_epi.json
│ │ ├── sub-23017_ses-02_acq-rest_dir-opp_epi.nii.gz -> ../../../../edb/subj_internal_id/10/3/subj_internal_id_10_3_00001.nii.gz
│ │ ├── sub-23017_ses-02_acq-task_dir-match_epi.json
│ │ ├── sub-23017_ses-02_acq-task_dir-match_epi.nii.gz -> ../../../../edb/subj_internal_id/10/7/subj_internal_id_10_7_00001.nii.gz
│ │ ├── sub-23017_ses-02_acq-task_dir-opp_epi.json
│ │ └── sub-23017_ses-02_acq-task_dir-opp_epi.nii.gz -> ../../../../edb/subj_internal_id/10/6/subj_internal_id_10_6_00001.nii.gz
│ └── func
│ ├── sub-23017_ses-02_task-mid_bold.json
│ ├── sub-23017_ses-02_task-mid_bold.nii.gz -> ../../../../edb/subj_internal_id/10/8/subj_internal_id_10_8_00001.nii.gz
│ ├── sub-23017_ses-02_task-mid_events.json
│ ├── sub-23017_ses-02_task-mid_events.tsv
│ ├── sub-23017_ses-02_task-rest_echo-1_bold.json
│ ├── sub-23017_ses-02_task-rest_echo-1_bold.nii.gz -> ../../../../edb/subj_internal_id/10/5/subj_internal_id_10_5_00001.nii.gz
│ ├── sub-23017_ses-02_task-rest_echo-2_bold.json
│ ├── sub-23017_ses-02_task-rest_echo-2_bold.nii.gz -> ../../../../edb/subj_internal_id/10/5/subj_internal_id_10_5_00002.nii.gz
│ ├── sub-23017_ses-02_task-rest_echo-3_bold.json
│ ├── sub-23017_ses-02_task-rest_echo-3_bold.nii.gz -> ../../../../edb/subj_internal_id/10/5/subj_internal_id_10_5_00003.nii.gz
│ ├── sub-23017_ses-02_task-rest_echo-4_bold.json
│ ├── sub-23017_ses-02_task-rest_echo-4_bold.nii.gz -> ../../../../edb/subj_internal_id/10/5/subj_internal_id_10_5_00004.nii.gz
│ ├── sub-23017_ses-02_task-rest_physio.json
│ └── sub-23017_ses-02_task-rest_physio.tsv.gz
├── ses-03
│ ├── anat
│ │ ├── sub-23017_ses-03_acq-mprage_rec-ORIG_T1w.json
│ │ ├── sub-23017_ses-03_acq-mprage_rec-ORIG_T1w.nii.gz -> ../../../../edb/subj_internal_id/11/40005/subj_internal_id_11_40005_00001.nii.gz
│ │ ├── sub-23017_ses-03_rec-ORIG_T2w.json
│ │ └── sub-23017_ses-03_rec-ORIG_T2w.nii.gz -> ../../../../edb/subj_internal_id/11/40003/subj_internal_id_11_40003_00001.nii.gz
│ ├── fmap
│ │ ├── sub-23017_ses-03_acq-rest_dir-match_epi.json
│ │ ├── sub-23017_ses-03_acq-rest_dir-match_epi.nii.gz -> ../../../../edb/subj_internal_id/11/8/subj_internal_id_11_8_00001.nii.gz
│ │ ├── sub-23017_ses-03_acq-rest_dir-opp_epi.json
│ │ ├── sub-23017_ses-03_acq-rest_dir-opp_epi.nii.gz -> ../../../../edb/subj_internal_id/11/7/subj_internal_id_11_7_00001.nii.gz
│ │ ├── sub-23017_ses-03_acq-task_dir-match_epi.json
│ │ ├── sub-23017_ses-03_acq-task_dir-match_epi.nii.gz -> ../../../../edb/subj_internal_id/11/12/subj_internal_id_11_12_00001.nii.gz
│ │ ├── sub-23017_ses-03_acq-task_dir-opp_epi.json
│ │ └── sub-23017_ses-03_acq-task_dir-opp_epi.nii.gz -> ../../../../edb/subj_internal_id/11/11/subj_internal_id_11_11_00001.nii.gz
│ └── func
│ ├── sub-23017_ses-03_task-mid_run-2_bold.json
│ ├── sub-23017_ses-03_task-mid_run-2_bold.nii.gz -> ../../../../edb/subj_internal_id/11/10/subj_internal_id_11_10_00001.nii.gz
│ ├── sub-23017_ses-03_task-mid_run-2_events.json
│ ├── sub-23017_ses-03_task-mid_run-2_events.tsv
│ ├── sub-23017_ses-03_task-rest_echo-1_bold.json
│ ├── sub-23017_ses-03_task-rest_echo-1_bold.nii.gz -> ../../../../edb/subj_internal_id/11/6/subj_internal_id_11_6_00001.nii.gz
│ ├── sub-23017_ses-03_task-rest_echo-2_bold.json
│ ├── sub-23017_ses-03_task-rest_echo-2_bold.nii.gz -> ../../../../edb/subj_internal_id/11/6/subj_internal_id_11_6_00002.nii.gz
│ ├── sub-23017_ses-03_task-rest_echo-3_bold.json
│ ├── sub-23017_ses-03_task-rest_echo-3_bold.nii.gz -> ../../../../edb/subj_internal_id/11/6/subj_internal_id_11_6_00003.nii.gz
│ ├── sub-23017_ses-03_task-rest_echo-4_bold.json
│ ├── sub-23017_ses-03_task-rest_echo-4_bold.nii.gz -> ../../../../edb/subj_internal_id/11/6/subj_internal_id_11_6_00004.nii.gz
│ ├── sub-23017_ses-03_task-rest_physio.json
│ └── sub-23017_ses-03_task-rest_physio.tsv.gz
├── sub-23017_sessions.json
└── sub-23017_sessions.tsvI’m not seeing any echos on your outputs (above). Also the run indexes don’t seem consistent between subjects (not sure that could be a problem, just to note).
Are you missing only the echos in the reports?
For that particular subject, the only functional run in the report is session 3 task mid run 2. In other reports the multiecho rest does seem to have worked fine.
BTW - this other thread could be related Fmriprep html summary report does not include functional figures unless fieldmaps are ignored
How the IntendedFor metadata of the fieldmaps look like?
Here’s sub-23017_ses-01_acq-rest_dir-match_epi.json’s indented for field (this coreesponds to a functional run that doesn’t show up in the report):
“IntendedFor”: [
“ses-01/func/sub-23017_ses-01_task-rest_echo-1_bold.nii.gz”,
“ses-01/func/sub-23017_ses-01_task-rest_echo-2_bold.nii.gz”,
“ses-01/func/sub-23017_ses-01_task-rest_echo-3_bold.nii.gz”,
“ses-01/func/sub-23017_ses-01_task-rest_echo-4_bold.nii.gz”
]
sub-23017_ses-03_acq-task_dir-match_epi.json (corresponding to the one that does show up in the report:
“IntendedFor”: [
“ses-03/func/sub-23017_ses-03_task-mid_run-2_bold.nii.gz”
]
@effigies suggested I look at the Report object from niworkflows. There’s definitely something screwy with pybids here:
broke_rplt_dir = Path('/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets')
rpt = reports.core.Report(broke_rplt_dir, '/data/MBDU/midla/notebooks/test', '1', subject_id='sub-23017', packagename='fmriprep')
frpt = [rs for rs in rpt.sections if rs.name == 'Functional'][0]
[frpt_rplt.name for frpt_rplt in frpt.reportlets]
> [frpt_rplt.name for frpt_rplt in frpt.reportlets]
['datatype-func_desc-summary_suffix-bold_session-03_task-mid_run-2',
'datatype-func_desc-validation_suffix-bold_session-03_task-mid_run-2',
'datatype-func_desc-sdc_suffix-bold_session-03_task-mid_run-2',
'datatype-func_desc-bbregister_suffix-bold_session-03_task-mid_run-2',
'datatype-func_desc-rois_suffix-bold_session-03_task-mid_run-2',
'datatype-func_desc-compcorvar_suffix-bold_session-03_task-mid_run-2',
'datatype-func_desc-carpetplot_suffix-bold_session-03_task-mid_run-2',
'datatype-func_desc-confoundcorr_suffix-bold_session-03_task-mid_run-2',
'datatype-func_desc-aroma_suffix-bold_session-03_task-mid_run-2']
rpt.layout
> BIDS Layout: .../reportlets/fmriprep/sub-23017 | Subjects: 1 | Sessions: 3 | Runs: 1
rpt.layout.get()
> [<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/anat/sub-23017_desc-about_T1w.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/anat/sub-23017_desc-conform_T1w.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/anat/sub-23017_desc-reconall_T1w.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/anat/sub-23017_desc-summary_T1w.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/anat/sub-23017_dseg.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/anat/sub-23017_space-MNI152NLin6Asym_T1w.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/anat/sub-23017_space-MNI152NLin2009cAsym_T1w.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-mid_desc-aroma_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-mid_desc-bbregister_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-mid_desc-carpetplot_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-mid_desc-compcorvar_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-mid_desc-confoundcorr_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-mid_desc-rois_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-mid_desc-sdc_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-mid_desc-summary_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-mid_desc-validation_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-rest_echo-1_desc-aroma_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-rest_echo-1_desc-bbregister_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-rest_echo-1_desc-carpetplot_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-rest_echo-1_desc-compcorvar_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-rest_echo-1_desc-confoundcorr_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-rest_echo-1_desc-rois_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-rest_echo-1_desc-sdc_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-rest_echo-1_desc-summary_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-01/func/sub-23017_ses-01_task-rest_echo-1_desc-validation_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-mid_desc-aroma_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-mid_desc-bbregister_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-mid_desc-carpetplot_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-mid_desc-compcorvar_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-mid_desc-confoundcorr_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-mid_desc-rois_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-mid_desc-sdc_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-mid_desc-summary_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-mid_desc-validation_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-rest_echo-1_desc-aroma_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-rest_echo-1_desc-bbregister_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-rest_echo-1_desc-carpetplot_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-rest_echo-1_desc-compcorvar_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-rest_echo-1_desc-confoundcorr_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-rest_echo-1_desc-rois_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-rest_echo-1_desc-sdc_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-rest_echo-1_desc-summary_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-02/func/sub-23017_ses-02_task-rest_echo-1_desc-validation_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/._sub-23017_ses-03_task-mid_run-2_desc-summary_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-mid_run-2_desc-aroma_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-mid_run-2_desc-bbregister_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-mid_run-2_desc-carpetplot_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-mid_run-2_desc-compcorvar_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-mid_run-2_desc-confoundcorr_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-mid_run-2_desc-rois_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-mid_run-2_desc-sdc_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-mid_run-2_desc-summary_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-mid_run-2_desc-validation_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-rest_echo-1_desc-aroma_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-rest_echo-1_desc-bbregister_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-rest_echo-1_desc-carpetplot_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-rest_echo-1_desc-compcorvar_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-rest_echo-1_desc-confoundcorr_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-rest_echo-1_desc-rois_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-rest_echo-1_desc-sdc_bold.svg'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-rest_echo-1_desc-summary_bold.html'>,
<BIDSFile filename='/data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017/ses-03/func/sub-23017_ses-03_task-rest_echo-1_desc-validation_bold.html'>]
!tree {rpt.root}
> /data/MBDU/midla/data/derivatives/fmriprep/rerun_aroma_2/sub-23017/wrk/wrk/reportlets/fmriprep/sub-23017
├── anat
│ ├── sub-23017_desc-about_T1w.html
│ ├── sub-23017_desc-conform_T1w.html
│ ├── sub-23017_desc-reconall_T1w.svg
│ ├── sub-23017_desc-summary_T1w.html
│ ├── sub-23017_dseg.svg
│ ├── sub-23017_space-MNI152NLin2009cAsym_T1w.svg
│ └── sub-23017_space-MNI152NLin6Asym_T1w.svg
├── ses-01
│ └── func
│ ├── sub-23017_ses-01_task-mid_desc-aroma_bold.svg
│ ├── sub-23017_ses-01_task-mid_desc-bbregister_bold.svg
│ ├── sub-23017_ses-01_task-mid_desc-carpetplot_bold.svg
│ ├── sub-23017_ses-01_task-mid_desc-compcorvar_bold.svg
│ ├── sub-23017_ses-01_task-mid_desc-confoundcorr_bold.svg
│ ├── sub-23017_ses-01_task-mid_desc-rois_bold.svg
│ ├── sub-23017_ses-01_task-mid_desc-sdc_bold.svg
│ ├── sub-23017_ses-01_task-mid_desc-summary_bold.html
│ ├── sub-23017_ses-01_task-mid_desc-validation_bold.html
│ ├── sub-23017_ses-01_task-rest_echo-1_desc-aroma_bold.svg
│ ├── sub-23017_ses-01_task-rest_echo-1_desc-bbregister_bold.svg
│ ├── sub-23017_ses-01_task-rest_echo-1_desc-carpetplot_bold.svg
│ ├── sub-23017_ses-01_task-rest_echo-1_desc-compcorvar_bold.svg
│ ├── sub-23017_ses-01_task-rest_echo-1_desc-confoundcorr_bold.svg
│ ├── sub-23017_ses-01_task-rest_echo-1_desc-rois_bold.svg
│ ├── sub-23017_ses-01_task-rest_echo-1_desc-sdc_bold.svg
│ ├── sub-23017_ses-01_task-rest_echo-1_desc-summary_bold.html
│ └── sub-23017_ses-01_task-rest_echo-1_desc-validation_bold.html
├── ses-02
│ └── func
│ ├── sub-23017_ses-02_task-mid_desc-aroma_bold.svg
│ ├── sub-23017_ses-02_task-mid_desc-bbregister_bold.svg
│ ├── sub-23017_ses-02_task-mid_desc-carpetplot_bold.svg
│ ├── sub-23017_ses-02_task-mid_desc-compcorvar_bold.svg
│ ├── sub-23017_ses-02_task-mid_desc-confoundcorr_bold.svg
│ ├── sub-23017_ses-02_task-mid_desc-rois_bold.svg
│ ├── sub-23017_ses-02_task-mid_desc-sdc_bold.svg
│ ├── sub-23017_ses-02_task-mid_desc-summary_bold.html
│ ├── sub-23017_ses-02_task-mid_desc-validation_bold.html
│ ├── sub-23017_ses-02_task-rest_echo-1_desc-aroma_bold.svg
│ ├── sub-23017_ses-02_task-rest_echo-1_desc-bbregister_bold.svg
│ ├── sub-23017_ses-02_task-rest_echo-1_desc-carpetplot_bold.svg
│ ├── sub-23017_ses-02_task-rest_echo-1_desc-compcorvar_bold.svg
│ ├── sub-23017_ses-02_task-rest_echo-1_desc-confoundcorr_bold.svg
│ ├── sub-23017_ses-02_task-rest_echo-1_desc-rois_bold.svg
│ ├── sub-23017_ses-02_task-rest_echo-1_desc-sdc_bold.svg
│ ├── sub-23017_ses-02_task-rest_echo-1_desc-summary_bold.html
│ └── sub-23017_ses-02_task-rest_echo-1_desc-validation_bold.html
└── ses-03
└── func
├── sub-23017_ses-03_task-mid_run-2_desc-aroma_bold.svg
├── sub-23017_ses-03_task-mid_run-2_desc-bbregister_bold.svg
├── sub-23017_ses-03_task-mid_run-2_desc-carpetplot_bold.svg
├── sub-23017_ses-03_task-mid_run-2_desc-compcorvar_bold.svg
├── sub-23017_ses-03_task-mid_run-2_desc-confoundcorr_bold.svg
├── sub-23017_ses-03_task-mid_run-2_desc-rois_bold.svg
├── sub-23017_ses-03_task-mid_run-2_desc-sdc_bold.svg
├── sub-23017_ses-03_task-mid_run-2_desc-summary_bold.html
├── sub-23017_ses-03_task-mid_run-2_desc-validation_bold.html
├── sub-23017_ses-03_task-rest_echo-1_desc-aroma_bold.svg
├── sub-23017_ses-03_task-rest_echo-1_desc-bbregister_bold.svg
├── sub-23017_ses-03_task-rest_echo-1_desc-carpetplot_bold.svg
├── sub-23017_ses-03_task-rest_echo-1_desc-compcorvar_bold.svg
├── sub-23017_ses-03_task-rest_echo-1_desc-confoundcorr_bold.svg
├── sub-23017_ses-03_task-rest_echo-1_desc-rois_bold.svg
├── sub-23017_ses-03_task-rest_echo-1_desc-sdc_bold.svg
├── sub-23017_ses-03_task-rest_echo-1_desc-summary_bold.html
└── sub-23017_ses-03_task-rest_echo-1_desc-validation_bold.html
7 directories, 61 files@tal Do you have any ideas why pybids isn’t correctly parsing out the number of runs here? Is there anything I can do to keep debugging this, or should I just zip up the directory and send it to you?
Can you describe the screwyness in more detail? Are files missing, or something else? And if the former, can you give examples of filenames that aren’t being returned from .get()? (May be better to open an issue on the PyBIDS repo, and also see this issue, which may be related).
.get returns all the files, but .layout says there is only 1 run, and the reports generated only have reportlets from that run
To give some context, we are using PyBIDS here to query a “reportlets” root folder, which should be compliant with this PyBIDS confing - https://github.com/poldracklab/niworkflows/blob/master/niworkflows/reports/figures.json
So it is very likely there are errors on the config file.
@Shotgunosine, can you post a tree of $TMPDIR/wrk/reportlets so that @tal can see the root being indexed?
That’s at the bottom of the output I posted above:
!tree {rpt.root}
Is the output above listing all the files in the BIDS project, or just the outputs from the reportlet? If the former, then unless I’m misunderstanding the output, this looks to me like a potential problem with the file naming, not with PyBIDS. If the listing above is accurate, only some of the files have run included, and they’re all in ses-03. It’s not clear how PyBIDS is supposed to infer the run name automatically, given that you can have multiple runs per session (or not).
If this is just showing the files that have report output, could you produce a listing for all the files, including ones that aren’t showing up? If the issue is the same as above (i.e., no explicit run in the file name), then the obvious fix is to add explicit run key-pair values to all filenames, as there’s no safe way to automatically infer them.
Look back over what has worked and what hasn’t, it seems that my file naming may in fact be to blame. I only included a run tag in instance where there were more than one run of a particular task. Most of the time that didn’t happen, but some subjects had multiple runs in a single session. It seems like it has worked fine in cases where nothing has a run label, but if any scan has a run label, then only scans with run labels are included in the report.
Comparing side-by-side our config vs. current default config there seems to be differences for some keys, e.g.:
{
"name": "subject",
"pattern": "[/\\\\]sub-([a-zA-Z0-9]+)",
"directory": "{{root}}{subject}"
},
and “official” config:
{
"name": "subject",
"pattern": "[/\\\\]+sub-([a-zA-Z0-9]+)",
"directory": "{subject}"
},
we are using PyBIDS 0.9.2 - @tal should we get that config [fmriprep’s] updated ASAP?
I think the lack of explicit run labeling is the issue here. It may not be required by the spec, but in practice, if you have more than one run across your project, I think it’s pretty much mandatory for disambiguation. I’d maybe consider making that explicit in the spec if it isn’t already there.
If you update the filenames to include run and still have problems, feel free to open an issue on the repo!
@oesteban yes, that would probably be a good idea. I suspect this might prevent you from using .get(return_type='dir') properly (though everything else should work fine).
1 Like
Alright, I think I’ve figured out why I was still getting complete reports in cases where there were no run labels at all. Pybids still doesn’t find any runs in that case, as @tal described
These lines in reports.core pull combinations of all the levels of sessions, tasks, and runs. In the case where there are no runs, then it’s just all combinations of sessions and tasks, so all of the reportlets get returned and are added to the final report. If there is a single run that has a run key and value, then all of the combinations produced have that run value as part of the query, so only the runs with a run key and value get included in the report. The solution then, as Tal said, is to put a run number on everything.
Thanks for your help resolving this everyone.
1 Like
I’ll have a look into whether we should require that explicit run labeling.