Summary of what happened:
I am trying to process the HCP data using XCP-D, however, XCP-D is only recruiting one core at a time.
I have set XCP-D’s flag --ncpus 27 (I have 28 cores), as well as the apptainer run flag --cpus 27. According to this Apptainer help page, my Ubuntu 22.04 system should be correctly configured to allow Apptainer to use multiple cores.
I have messed around with Apptainer --cpus-set and XCP-D --omp-nthreads, although neither enabled parallelization.
Am I missing something?
Command used (and if a helper script was used, a link to the helper script or the command generated):
#!/bin/bash
hcpya=/mnt/astrid_remote/HCPYA/
fmri_dir=${hcpya}/src/
output_dir=${hcpya}/clean/
participants="$(echo "${1}" | tr '[:space:]' ' ')"
# symlink to latest xcpd version
xcpd=/home/tkmday/trevor/xcpd.sif
mkdir -p /tmp/xcpd_workdir/
now=$(date '+%y%m%d_%H%M')
# $participants is supposed to be word-separated
# shellcheck disable=SC2086
apptainer run \
-B ${fmri_dir}:/fmri_dir/:ro \
-B ${output_dir}:/output/ \
-B /tmp/xcpd_workdir/:/wkdir/ \
-B /tmp:/scrth \
--cpus 27 \
${xcpd} \
/fmri_dir/ /output/ participant \
--work-dir /wkdir/ \
--clean-workdir \
--participant-label ${participants} \
--mode none \
--file-format cifti \
--input-type hcp \
--nuisance-regressors gsr_only \
--motion-filter-type none \
--despike y \
--fd-thresh 0.2 \
--atlases Glasser \
--min-coverage 0.5 \
--min-time 60 \
--combine-runs n \
--output-type censored \
--warp-surfaces-native2std n \
--smoothing 6 \
--abcc-qc n --linc-qc n \
--nprocs 27 \
tee -a "date-${now}_log.txt"
Version:
- XCP-D version
v0.10.5 - Ubuntu release:
22.04 - Apptainer version:
1.3.4
Environment (Docker, Singularity / Apptainer, custom installation):
Apptainer
Data formatted according to a validatable standard? Please provide the output of the validator:
Using XCP-D’s built-in HCP to BIDS function.
Screenshots / relevant information:
Concluded it was only recruiting one core watching the Ubuntu System Monitor GUI.