Summary of what happened:
Hello everyone, this is my first post. Any help with this is greatly appreciated!
When trying to run xcp_d with fmriprep derivatives as inputs (niftis, so in nichart mode) i run into the errors below.
I have read in this indexed_gzip.indexed_gzip.ZranError: zran_read returned error: ZRAN_READ_FAIL thread (although this is concerning an fmriprep run), that it might have to do with niftis not being compressed despite the .nii.gz file ending, so I
verified the compression with gzip
file "/Volumes/X9 Pro/Deprest/sample_bids_control_derivatives_test9/sub-001/ses-01/func/sub-001_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz"
/Volumes/X9 Pro/Deprest/sample_bids_control_derivatives_test9/sub-001/ses-01/func/sub-001_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz: gzip compressed data, max compression, original size modulo 2^32 600182
I also tried loading the image with nibabel, which seemed to work:
python3 -c "import nibabel as nib; img = nib.load('/Volumes/X9 Pro/Deprest/sample_bids_control_derivatives_test9/sub-001/ses-01/func/sub-001_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz'); print('Successfully loaded')"
Successfully loaded
One extra piece of information is frmiprep crashed in multiple runs and non-determinalistically for me, which is why I had to re-run multiple subjects multiple times until they finished successfully. I unsure whether this is related to the xcp_d crashes, since for this test sample data, the dataset completed fully on first run and there are no log outputs in the fmriprep logs folder.
Thank you for your help and best regards!
- Michael
Command used (and if a helper script was used, a link to the helper script or the command generated):
sbatch --array=0-2 /home/jferres/test_xcp_d_jobscript.sh
I also ran the command for a single subject (i. e. --array=0-0 too), to avoid conflicting processes, resulting in the same errors though.
job script:
#!/bin/bash
#
#SBATCH --job-name=sample_xcp_d-%a
#SBATCH --output=sample_xcp_d-%A-%a.txt
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=8
#SBATCH --partition=long
#SBATCH --time=48:00:00
#SBATCH --mem-per-cpu=5G
#SBATCH --mail-type=START,FAIL,END
# set the path to the dataset directory. This is the parent to the BIDS-formatted nifti directory.
dataset_dir=$HOME
# set the path to the BIDS-formatted nifti directory
bids_dir=$dataset_dir/bids_control
# set the path to the fmriprep derivatives directory
fmriprep_derivatives_dir=$dataset_dir/sample_bids_control_derivatives_test9
# set the path to the xcp_d output directory
denoised_derivatives_dir=$dataset_dir/sample_control_denoised
# create the derivatives directory if it doesn't exist
mkdir -p $denoised_derivatives_dir
# get participant ID
pid=$(awk "NR==$(($SLURM_ARRAY_TASK_ID+2)){print;exit}" $bids_dir/participants.tsv | cut -f 1)
container_img=$dataset_dir/my_images/xcp_d-0.13.0.simg
# run xcp_d via Apptainer
apptainer run --cleanenv \
--bind $fmriprep_derivatives_dir:$fmriprep_derivatives_dir \
--bind $denoised_derivatives_dir:$denoised_derivatives_dir \
--bind $TMPDIR:$TMPDIR \
$container_img \
$fmriprep_derivatives_dir \
$denoised_derivatives_dir \
participant \
--mode nichart \
--participant-label $pid \
--work-dir $TMPDIR \
--n-cpus 8 \
--omp-nthreads 8 \
--mem-mb 40000 \
--dummy-scans auto \
--smoothing 6 \
--head-radius auto \
--fd-thresh 0.3
Version:
xcp_d-0.13.0.simg
Environment (Docker, Singularity / Apptainer, custom installation):
Apptainer on HPC
Data formatted according to a validatable standard? Please provide the output of the validator:
PASTE VALIDATOR OUTPUT HERE
Relevant log outputs (up to 20 lines):
Traceback:
Traceback (most recent call last):
File "/usr/local/miniconda/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 401, in run
runtime = self._run_interface(runtime)
File "/usr/local/miniconda/lib/python3.10/site-packages/xcp_d/interfaces/censoring.py", line 183, in _run_interface
dropped_image = _drop_dummy_scans(self.inputs.bold_file, dummy_scans=dummy_scans)
File "/usr/local/miniconda/lib/python3.10/site-packages/xcp_d/utils/modified_data.py", line 69, in _drop_dummy_scans
data = bold_image.get_fdata()
File "/usr/local/miniconda/lib/python3.10/site-packages/nibabel/dataobj_images.py", line 374, in get_fdata
data = np.asanyarray(self._dataobj, dtype=dtype)
File "/usr/local/miniconda/lib/python3.10/site-packages/nibabel/arrayproxy.py", line 454, in __array__
arr = self._get_scaled(dtype=dtype, slicer=())
File "/usr/local/miniconda/lib/python3.10/site-packages/nibabel/arrayproxy.py", line 421, in _get_scaled
scaled = apply_read_scaling(self._get_unscaled(slicer=slicer), scl_slope, scl_inter)
File "/usr/local/miniconda/lib/python3.10/site-packages/nibabel/arrayproxy.py", line 391, in _get_unscaled
return array_from_file(
File "/usr/local/miniconda/lib/python3.10/site-packages/nibabel/volumeutils.py", line 467, in array_from_file
n_read = infile.readinto(data_bytes)
File "indexed_gzip/indexed_gzip.pyx", line 851, in indexed_gzip.indexed_gzip._IndexedGzipFile.readinto
indexed_gzip.indexed_gzip.ZranError: zran_read returned error: ZRAN_READ_FAIL (file: /home/jferres/sample_bids_control_derivatives_test9/sub-003/ses-02/func/sub-003_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz)
Or it is this similar error
Traceback (most recent call last):
File "/usr/local/miniconda/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/miniconda/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/miniconda/lib/python3.10/site-packages/xcp_d/cli/workflow.py", line 94, in build_workflow
retval['workflow'] = init_xcpd_wf()
File "/usr/local/miniconda/lib/python3.10/site-packages/xcp_d/workflows/base.py", line 82, in init_xcpd_wf
single_subject_wf = init_single_subject_wf(subject_id, anat_session, func_sessions)
File "/usr/local/miniconda/lib/python3.10/site-packages/xcp_d/workflows/base.py", line 487, in init_single_subject_wf
TR = _get_tr(nb.load(task_files[0]))
File "/usr/local/miniconda/lib/python3.10/site-packages/nibabel/loadsave.py", line 118, in load
raise ImageFileError(f'Cannot work out file type of "{filename}"')
nibabel.filebasedimages.ImageFileError: Cannot work out file type of "/home/jferres/sample_bids_control_derivatives_test9/sub-001/ses-02/func/sub-001_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz"