XCP_D processing multiple sessions

Heechberri · July 3, 2023, 4:53am

Summary of what happened:

Hi Experts,

I am trying to post-process fmriprep outputs with multiple sessions with XCP_D, and am running into several issues.

Command used (and if a helper script was used, a link to the helper script or the command generated):

Here is the command that I am using:

disk="/home/jiayi/Desktop"
project_dir="Intervention"
data_dir="data_fmriprep_fMRI"
output_dir="data_fmri_XCPD"
bids_filter_file="/home/jiayi/Desktop/Intervention/Codes/MRI/Linux/3_fMRI_linux/bids_filter_file.json"
scratch_dir=$output_dir/"scratch"

sudo docker run -ti \
-v $disk/$project_dir/$data_dir:/data:ro \
-v $disk/$project_dir/$output_dir:/out \
-v $disk/$project_dir/$scratch_dir:/scratch \
-v $bids_filter_file:/bids_filter_file.json \
pennlinc/xcp_d:0.3.2 \
/data /out participant \
--task rest \
--bids-filter-file /bids_filter_file.json \
-w /scratch \
--smoothing 6 \
--despike \
--head_radius 50 --input fmriprep \
--nuisance-regressors acompcor \
--lower-bpf 0.01 --upper-bpf 0.08 \
--omp-nthreads 16 \
--mem_gb 64 \
--verbose \
--resource-monitor

and here is the bids filter file that i am using

{
   "bold": {
      "session": ["1","2"],
      "task": "rest",
      "space":"MNI152NLin6Asym",
      "res":["2"]
   }
}

Version:

0.3.2

Environment (Docker, Singularity, custom installation):

Docker

Data formatted according to a validatable standard? Please provide the output of the validator:

Relevant log outputs (up to 20 lines):

I have been running into this error with XCP_D


ValueError: More than one surface found.
Surfaces found:
	/data/sub-EXCI0001/ses-1/anat/sub-EXCI0001_ses-1_hemi-L_pial.surf.gii
	/data/sub-EXCI0001/ses-2/anat/sub-EXCI0001_ses-2_hemi-L_pial.surf.gii
Query: {'hemi': 'L', 'desc': None, 'suffix': 'pial'}

Screenshots / relevant information:

My fmriprep output directory follows the usual output except that some subjects have 1 session (session 2) and some subject has 2 sessions (both session 1 or 2). This is because some of the session 1s has already been processed prior. So I processed some session 2 seperately and combined the directories. each session is organised in their own ses-folder, so I am not sure why I am still getting the error above.

Also, I have limited space on the server which I am running fMRIprep and XCP_D, so i often move files to a larger drive and delete unnecessary files on the processing server. My second question is, is the freesufer files needed for XCP_D, or all the anat files needed are already under the anat folder?

Thank you!
Regards,
Vae

Steven · July 3, 2023, 1:11pm

Hi @Heechberri,

I have relabeled your post as Software Support and reorganized your post according to the software support post template. Please post software related issues under this category in the future.

Before continuing, can you confirm the error persists under the most recent XCP_D version (0.4.0 at this time)?

All of the anat files needed should be available from the fMRIPrep derivatives. Making those files requires Freesurfer, but once fMRIPrep finishes you should have everything you need to run XCP_D.

Best,
Steven

sreeniva · August 4, 2023, 8:23pm

I encountered a similar issue and stumbled upon this post while browsing the forum. I thought it would be helpful to share my input here rather than create a new post. If this approach is not acceptable, please forgive me and I’ll create a new post if necessary.

Command used (and if a helper script was used, a link to the helper script or the command generated):

FS_LICENSE=/path/to/freesurfer/license.txt
TEST_DIR=/path/to/test/folder
XCP_d_VER=xcp_d_050rc2

singularity run --cleanenv -B /projects,$FS_LICENSE:/opt/freesurfer/license.txt /path/to/test/folder/${XCP_d_VER}.simg ${TEST_DIR}/derivatives/fmriprep ${TEST_DIR}/derivatives/${XCP_d_VER} participant --verbose --participant_label SUBJ01 --input-type fmriprep --nthreads 16 --mem_gb 48 --smoothing 3 --despike --nuisance-regressors acompcor_gsr --dummy-scans 4 --bpf-order 2 --motion-filter-type notch --band-stop-min 10 --band-stop-max 30 --lower-bpf 0.01 --upper-bpf 0.1 --fd-thresh 0.4 --dcan-qc --cifti --work_dir /path/to/work/folder --bids-filter-file ${TEST_DIR}/session_filter.json

Version:

0.4.1rc2 and 0.5.0rc2

Environment (Docker, Singularity, custom installation):

Singularity

Data formatted according to a validatable standard? Please provide the output of the validator:

Relevant log outputs (up to 20 lines):

ver 0.5.0rc2 log

230804-14:21:11,223 nipype.utils INFO:
	 No standard-space surfaces found.
230804-14:21:11,273 nipype.utils INFO:
	 No standard-space surfaces found.
230804-14:21:11,444 nipype.utils INFO:
	 No standard-space surfaces found.
230804-14:21:11,496 nipype.utils INFO:
	 No standard-space surfaces found.
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/cli/run.py", line 990, in build_workflow
    retval["workflow"] = init_xcpd_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflows/base.py", line 199, in init_xcpd_wf
    single_subj_wf = init_subject_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflows/base.py", line 384, in init_subject_wf
    mesh_available, standard_space_mesh, mesh_files = collect_mesh_data(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/utils/bids.py", line 445, in collect_mesh_data
    "More than one surface found.\n"
KeyError: 'lh_pial_surf'

Below is the session_filter.json file that I used

{
   "bold": {
       "session": ["01"],
       "task": ["rest"]
    }
 }

04.0 and 0.4.1rc2 errors were similar. (log much more similar to that shared by OP)

Screenshots / relevant information:

I successfully ran postprocessing for ses-01 alone by removing ses-02 from the …/derivatives/fmriprep folder.

singularity run --cleanenv -B /projects,$FS_LICENSE:/opt/freesurfer/license.txt /path/to/test/folder/${XCP_d_VER}.simg ${TEST_DIR}/derivatives/fmriprep ${TEST_DIR}/derivatives/${XCP_d_VER} participant --verbose --participant_label SUBJ01 --input-type fmriprep --nthreads 16 --mem_gb 48 --smoothing 3 --despike --nuisance-regressors acompcor_gsr --dummy-scans 4 --bpf-order 2 --motion-filter-type notch --band-stop-min 10 --band-stop-max 30 --lower-bpf 0.01 --upper-bpf 0.1 --fd-thresh 0.4 --dcan-qc --cifti --work_dir /path/to/work/folder

However, when I attempted to use the filter JSON file after reintroducing ses-02, I encountered the error mentioned above.

I am unsure of what I may be missing or doing incorrectly. Any assistance would be greatly appreciated. Thank you.

Steven · August 5, 2023, 9:04pm

Hi @sreeniva, and welcome to Neurostars!

This sounds like a separate issue, but let’s see if we can address it here. Can you share the fMRIPrep command you used to make the data? I am especially curious if you included the --fs-no-reconall flag (which is not good, you should always use the FS workflow when you can), which would perhaps lead to XCP_D not finding surface correctly.

Best,
Steven

sreeniva · August 7, 2023, 4:02pm

Hello Steven, thank you

The "More than one surface found.\n" line and the error only showing up when I was trying to select one session from multiple sessions made me think they were potentially similar errors, my apologies.

Below is the fMRIPrep command I used, and the --fs-no-reconall flag was not included.

singularity run --cleanenv -B /projects -B $Scratch_space $Singularity_path/fmriprep-23.0.2.simg 
$BIDS_DIR $BIDS_DIR/derivatives/fmriprep-23.0.2/ participant --participant-label SUBJ01 
-w $Scratch_space/workdir --nthreads 16 --omp-nthreads 12 --mem_mb 48000 
--fs-license-file $FSLICENSEFILE --fs-subjects-dir $FSDIR --output-spaces anat fsnative 
MNI152NLin2009cAsym fsLR fsaverage --cifti-output 91k --fd-spike-threshold 0.5 --dvars-spike-
threshold 1.5 --dummy-scans 4 --skip-bids-validation --bids-filter-file $SESFILTER

I added full op logs using both 0.4.1rc2 and 0.5.0rc2 for reference.

Version: 0.4.1rc2

230807-10:31:29,901 nipype.workflow IMPORTANT:
	 Running xcp_d version 0.4.1rc2:
    * fMRI directory path: /path/to/test/folder/derivatives/fmriprep.
    * Participant list: ['SUBJ01'].
    * Run identifier: 20230807-103127_2a970825-d75b-47b2-8c05-bba902388959.


230807-10:31:29,998 nipype.workflow INFO:
	 Beginning the xcpd_wf workflow
230807-10:31:30,784 nipype.utils WARNING:
	 Multiple files found for query 't1w':
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_desc-preproc_T1w.nii.gz
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_desc-preproc_T1w.nii.gz
230807-10:31:30,784 nipype.utils WARNING:
	 Multiple files found for query 'anat_dseg':
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_dseg.nii.gz
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_dseg.nii.gz
230807-10:31:30,784 nipype.utils WARNING:
	 Multiple files found for query 'template_to_anat_xfm':
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5
230807-10:31:30,784 nipype.utils WARNING:
	 Multiple files found for query 'anat_brainmask':
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_desc-brain_mask.nii.gz
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_desc-brain_mask.nii.gz
230807-10:31:30,784 nipype.utils WARNING:
	 Multiple files found for query 'anat_to_template_xfm':
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5
230807-10:31:30,785 nipype.utils IMPORTANT:
	 Collected data:
anat_brainmask: /path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_desc-brain_mask.nii.gz
anat_dseg: /path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_dseg.nii.gz
anat_to_template_xfm: /path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5
bold:
- /path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/func/sub-SUBJ01_ses-01_task-rest_dir-AP_space-fsLR_den-91k_bold.dtseries.nii
t1w: /path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_desc-preproc_T1w.nii.gz
t2w: /path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/anat/sub-SUBJ01_desc-preproc_T2w.nii.gz
template_to_anat_xfm: /path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5

230807-10:31:30,822 nipype.utils INFO:
	 No standard-space surfaces found.
230807-10:31:30,858 nipype.utils INFO:
	 No standard-space surfaces found.
230807-10:31:30,895 nipype.utils INFO:
	 No standard-space surfaces found.
230807-10:31:30,931 nipype.utils INFO:
	 No standard-space surfaces found.
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/cli/run.py", line 937, in build_workflow
    retval["workflow"] = init_xcpd_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflows/base.py", line 192, in init_xcpd_wf
    single_subj_wf = init_subject_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflows/base.py", line 369, in init_subject_wf
    mesh_available, morphometry_files, standard_space_mesh, surface_data = collect_surface_data(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/utils/bids.py", line 489, in collect_surface_data
    mesh_available, standard_space_mesh, mesh_files = _find_standard_space_surfaces(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/utils/bids.py", line 422, in _find_standard_space_surfaces
    raise ValueError(
ValueError: More than one surface found.
Surfaces found:
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_hemi-L_pial.surf.gii
	/path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_hemi-L_pial.surf.gii
Query: {'hemi': 'L', 'desc': None, 'suffix': 'pial', 'extension': '.surf.gii'}

Version: 0.5.0rc2

230804-14:21:09,908 nipype.workflow IMPORTANT:
	 Running xcp_d version 0.5.0rc2:
    * fMRI directory path: /path/to/test/folder/derivatives/fmriprep.
    * Participant list: ['SUBJ01'].
    * Run identifier: 20230804-142106_e0b61b5b-2ae3-4175-b051-a33c90eb66ad.


230804-14:21:10,34 nipype.workflow INFO:
	 Beginning the xcpd_wf workflow
230804-14:21:11,169 nipype.utils WARNING:
	 Multiple files found for query 't1w':
	path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_desc-preproc_T1w.nii.gz
	path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_desc-preproc_T1w.nii.gz
230804-14:21:11,170 nipype.utils WARNING:
	 Multiple files found for query 'anat_dseg':
	path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_dseg.nii.gz
	path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_dseg.nii.gz
230804-14:21:11,170 nipype.utils WARNING:
	 Multiple files found for query 'template_to_anat_xfm':
	path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5
	path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5
230804-14:21:11,170 nipype.utils WARNING:
	 Multiple files found for query 'anat_brainmask':
	path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_desc-brain_mask.nii.gz
	path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_desc-brain_mask.nii.gz
230804-14:21:11,170 nipype.utils WARNING:
	 Multiple files found for query 'anat_to_template_xfm':
	path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5
	path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-02/anat/sub-SUBJ01_ses-02_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5
230804-14:21:11,172 nipype.utils IMPORTANT:
	 Collected data:
anat_brainmask: path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_desc-brain_mask.nii.gz
anat_dseg: path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_dseg.nii.gz
anat_to_template_xfm: path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5
bold:
- path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/func/sub-SUBJ01_ses-01_task-rest_dir-AP_space-fsLR_den-91k_bold.dtseries.nii
t1w: path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_desc-preproc_T1w.nii.gz
t2w: path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/anat/sub-SUBJ01_desc-preproc_T2w.nii.gz
template_to_anat_xfm: path/to/test/folder/derivatives/fmriprep/sub-SUBJ01/ses-01/anat/sub-SUBJ01_ses-01_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5

230804-14:21:11,223 nipype.utils INFO:
	 No standard-space surfaces found.
230804-14:21:11,273 nipype.utils INFO:
	 No standard-space surfaces found.
230804-14:21:11,444 nipype.utils INFO:
	 No standard-space surfaces found.
230804-14:21:11,496 nipype.utils INFO:
	 No standard-space surfaces found.
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/cli/run.py", line 990, in build_workflow
    retval["workflow"] = init_xcpd_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflows/base.py", line 199, in init_xcpd_wf
    single_subj_wf = init_subject_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflows/base.py", line 384, in init_subject_wf
    mesh_available, standard_space_mesh, mesh_files = collect_mesh_data(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/utils/bids.py", line 445, in collect_mesh_data
    "More than one surface found.\n"
KeyError: 'lh_pial_surf'

Thank you.

Steven · August 7, 2023, 4:08pm

Did you process sessions separately? I thought fMRIPrep makes a single subject template based on all available T1s.

sreeniva · August 7, 2023, 4:33pm

Yes, that is true it does create a single template on all available T1s in the input BIDS path. I followed one of your suggestions (late thank you ) #4644 to process the sessions independently.

By using a filter JSON file for selecting the session and linking the corresponding FS dir to the sub-ses specific folder name and using the -fs-subjs-dir flag I was able to process the sessions independently.

Steven · August 7, 2023, 4:36pm

Haha, no problem!

Can you try introducing a filter file for XCP to similarly limit the sessions analyzed in each run? Running XCP-D — xcp_d 0.5.0rc2.dev1+ge7d9bab documentation

Best,
Steven

sreeniva · August 7, 2023, 4:48pm

I actually used a similar filter file, but unfortunately, it did not solve the issue. Interestingly, the error remains the same whether a filter file is used or not.

Below is the session_filter.json file that I used.

{
   "bold": {
       "session": ["01"],
       "task": ["rest"]
    }
 }

This made me think if the JSON file is not being passed along so I changed “bold” to “fmap” and from the error, it seems like the JSON file is being input (please see below).

{
   "fmap": {
       "session": ["01"],
       "task": ["rest"]
    }
 }

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/cli/run.py", line 990, in build_workflow
    retval["workflow"] = init_xcpd_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflows/base.py", line 199, in init_xcpd_wf
    single_subj_wf = init_subject_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflows/base.py", line 370, in init_subject_wf
    layout, subj_data = collect_data(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/utils/bids.py", line 250, in collect_data
    queries[acq].update(entities)
KeyError: 'fmap'

Steven · August 7, 2023, 4:51pm

You do not have any anatomical filters in your file.

Another workaround would be to, as part of your processing script, symlink (i.e., make shortcuts) only data from a single session to a new directory and have XCP run on that symlinked data.

sreeniva · August 7, 2023, 5:05pm

I based mine off the example bids filter file after initially trying to use the same filters as fMRIPrep (modified after I realized that this filter was different and specific to xcp-d). The anat filters listed I thought were optional. The anat however still would be present in both sessions and the error above shows that it is actually picking both sessions. I will re-run again including all flags to see if they solve the issue.

The sym-link workaround was my first go-to option and working on it in parallel. There are other errors that pop up regarding missing data and I am running checks to make sure I include all the folders needed.