Summary of what happened:
Hi, I am trying to run the xcp_d pipeline on fmriprep-preprocessed data within the babs framework. However, I ran into this error: "xcp_d: error: One or more participant labels were not found in the BIDS directory: sub-01.".
As input, I put the output_ria from the previous babs project (as suggested in the documentation). Judging by the log file, the dataset is successfully accessed and unzipped. I also tried feeding the pipeline the unzipped or zipped folder of the fmriprep output, but ran into the same error. Before running fmriprep, data has undergone BIDS structure validation (via CuBIDS). I am working on a HPC without internet access. I added […] wherever I think lines are not relevant to the issue.
Command used (and if a helper script was used, a link to the helper script or the command generated):
My yaml config file, to initialize the fmriprep babs project:
# Arguments in `singularity run`:
singularity_run:
-w: "$BABS_TMPDIR"
--mode: "none"
--nthreads: "40"
--omp-nthreads: "40"
--input-type: "fmriprep"
[...]
--stop-on-first-crash: ""
--fs-license-file: "/home/gabridele/license.txt"
-vvv: ""
# Output foldername(s) to be zipped, and the BIDS App version to be included in the zip filename(s):
zip_foldernames:
xcp_d: "0-10-0"
# How much cluster resources it needs:
cluster_resources:
interpreting_shell: /bin/bash
hard_memory_limit: 200G
hard_runtime_limit: '48:00:00'
# Necessary commands to be run first:
script_preamble: |
module load Anaconda3/2022.05
source /sw/easybuild_milan/software/Anaconda3/2022.05/bin/activate ~/.conda/envs/babs_28_11
module load GCC/12.2.0
module load OpenMPI/4.1.4
module load ANTs/2.5.0
module --ignore_cache load "FreeSurfer/7.3.2-centos8_x86_64"
# Where to run the jobs:
job_compute_space: "$SNIC_TMP" # COSMOS-SENS TEMP FOLDER
the babs-init --input argument is --input fmriprep ria+file://${PWD}/toydset_project2/output_ria#~data
Version:
singularity: 1.2.5-1.e19; singularity image xcp_d: 0.10.0; babs: 0.0.8; bids-validator 2.0.0
Environment (Docker, Singularity / Apptainer, custom installation):
I used singularity, through the babs framework
Data formatted according to a validatable standard? Please provide the output of the validator:
I performed bids validation via the bids-validator 2.0.0. The output raised some warnings about missing recommended parameters, but the fmriprep pipeline ran smoothly. So I don’t think it’s relevant here
Relevant log outputs (up to 20 lines):
+ dssource=ria+file:///home/gabridele/backup/cubids_test/xcp_toydset2/input_ria#c972c943-ed5d-46d8-9d89-c8019ae16cbf
+ pushgitremote=/home/gabridele/backup/cubids_test/xcp_toydset2/output_ria/c97/2c943-ed5d-46d8-9d89-c8019ae16cbf
+ subid=sub-01
+ sesid=ses-1
+ cd /local/slurmtmp.150646
+ BRANCH=job-150646-sub-01-ses-1
+ mkdir job-150646-sub-01-ses-1
+ cd job-150646-sub-01-ses-1
+ datalad clone ria+file:///home/gabridele/backup/cubids_test/xcp_toydset2/input_ria#c972c943-ed5d-46d8-9d89-c8019ae16cbf ds
[INFO] Attempting a clone into /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds
[INFO] Attempting to clone from file:///home/gabridele/backup/cubids_test/xcp_toydset2/input_ria/c97/2c943-ed5d-46d8-9d89-c8019ae16cbf to /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds
[INFO] Completed clone attempts for Dataset(/local/slurmtmp.150646/job-150646-sub-01-ses-1/ds)
+ cd ds
+ git remote add outputstore /home/gabridele/backup/cubids_test/xcp_toydset2/output_ria/c97/2c943-ed5d-46d8-9d89-c8019ae16cbf
+ git checkout -b job-150646-sub-01-ses-1
Switched to a new branch 'job-150646-sub-01-ses-1'
+ datalad get -n inputs/data/fmriprep
[INFO] Attempting a clone into /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/inputs/data/fmriprep
[INFO] Attempting to clone from file:///home/gabridele/backup/cubids_test/xcp_toydset2/input_ria/08d/92431-12cb-4bdb-a28b-5c47a0234301 to /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/inputs/data/fmriprep
[INFO] Attempting to clone from file:///home/gabridele/backup/cubids_test/toydset_project2/output_ria/alias/data to /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/inputs/data/fmriprep
[INFO] Completed clone attempts for Dataset(/local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/inputs/data/fmriprep)
[INFO] Configure additional publication dependency on "output-storage"
+ cd inputs/data/fmriprep
++ grep -v sub-01
++ ls sub-01_ses-1_fmriprep-24-1-1.zip sub-01_ses-2_fmriprep-24-1-1.zip
+ rm -f
++ cut -d @ -f 1
++ ls inputs/data/fmriprep/sub-01_ses-1_fmriprep-24-1-1.zip
+ FMRIPREP_ZIP=inputs/data/fmriprep/sub-01_ses-1_fmriprep-24-1-1.zip
+ echo 'found fmriprep zipfile:'
+ echo inputs/data/fmriprep/sub-01_ses-1_fmriprep-24-1-1.zip
+ '[' -z inputs/data/fmriprep/sub-01_ses-1_fmriprep-24-1-1.zip ']'
+ array=($FMRIPREP_ZIP)
+ '[' 1 -gt 1 ']'
+ datalad run -i code/xcp-d-0-10-0_zip.sh -i inputs/data/fmriprep/sub-01_ses-1_fmriprep-24-1-1.zip -i containers/.datalad/environments/xcp-d-0-10-0/image --explicit -o sub-01_ses-1_xcp_d-0-10-0.zip -m 'xcp-d-0-10-0 sub-01 ses-1' 'bash ./code/xcp-d-0-10-0_zip.sh sub-01 ses-1 inputs/data/fmriprep/sub-01_ses-1_fmriprep-24-1-1.zip'
[INFO] Making sure inputs are available (this may take some time)
[INFO] Attempting a clone into /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/containers
[INFO] Attempting to clone from file:///home/gabridele/backup/cubids_test/xcp_toydset2/input_ria/99f/1d775-06fb-4daf-8efb-fe859b4a80c4 to /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/containers
[INFO] Attempting to clone from xcp-sing to /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/containers
[INFO] Attempting to clone from /home/gabridele/backup/cubids_test/xcp-sing to /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/containers
[INFO] Completed clone attempts for Dataset(/local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/containers)
[INFO] == Command start (output follows) =====
+ subid=sub-01
+ sesid=ses-1
+ FMRIPREP_ZIP=inputs/data/fmriprep/sub-01_ses-1_fmriprep-24-1-1.zip
+ wd=/local/slurmtmp.150646/job-150646-sub-01-ses-1/ds
+ cd inputs/data/fmriprep
++ basename inputs/data/fmriprep/sub-01_ses-1_fmriprep-24-1-1.zip
+ 7z x sub-01_ses-1_fmriprep-24-1-1.zip
+ cd /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds
+ mkdir -p /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/.git/tmp/wkdir
+ singularity run --cleanenv -B /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds -B /home/gabridele/backup/templateflow:/SGLR/TEMPLATEFLOW_HOME -B /home/gabridele/license.txt:/SGLR/FREESURFER_HOME/license.txt --env TEMPLATEFLOW_HOME=/SGLR/TEMPLATEFLOW_HOME containers/.datalad/environments/xcp-d-0-10-0/image inputs/data/fmriprep/fmriprep outputs participant -w /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/.git/tmp/wkdir --mode none --nthreads 40 --omp-nthreads 40 --input-type fmriprep --smoothing 4 --file-format nifti --abcc-qc y --nuisance-regressors 36P --despike n --linc-qc n --motion-filter-type none --output-type censored --warp-surfaces-native2std n --combine-runs n --min_coverage 0 -f 0.5 --stop-on-first-crash --fs-license-file /SGLR/FREESURFER_HOME/license.txt -vvv --participant-label sub-01
[...]
2024-12-04 10:24:16,594 [ DEBUG] open file: /local/slurmtmp.150646/job-150646-sub-01-ses-1/ds/inputs/data/fmriprep/fmriprep/dataset_description.json
usage: xcp_d [-h] --mode {abcd,hbcd,linc,nichart,none} [...]
[--warp-surfaces-native2std [{y,n}]]
fmri_dir output_dir {participant}
xcp_d: error: One or more participant labels were not found in the BIDS directory: sub-01.
[INFO] == Command exit (modification check follows) =====
Screenshots / relevant information:
Here’s a screenshot of the directory structure where the xcp_d pipeline draws the input
