Hi neurostars friends (and especially probably hi Steven, lol)!!
First and foremost, I genuinely and sincerely apologize if this is a stupid error or I completely missed something in a read the docs site or another neurostars post or something. Also I apologize if I formatted something incorrectly/not standard. Anyway, I’m hoping someone might have some thoughts and might be able to help me! Also if you need any more information please let me know.
Summary of what happened:
I have N=12 typically developing adults, with 2 MRI sessions per person (pre and post intervention) - 24 total. I have, among other images, a T1, a multishell (30 directions at b=1000 and 60 directions at b=2000) DW image, and a reverse phase encoded image (PA direction? original image is AP direction?) for each MRI session. I ran QSIprep version 1.0.0 on all of them, and got no error messages, all the expected outputs, and the .html files in the qsiprep outputs folder look as expected. So 24/24 success. I then ran QSIrecon version 1.0.0 (using recon spec pyAFQ specifically – but a slightly edited yaml, to get RD and AD in addition to the defaults FA and MD) on all of them, and got no error messages (all of the html files say everything ran successfully!), but did not get all the expected outputs. Some participants (16/24) successfully created and populated both the /derivatives/…/figures/ and /derivatives/…/dwi/, but many (8/24, which feels like a lot?!) only have /derivatives/…/figures/ and no /derivatives/…/dwi/. But again there are no error messages!? So I’m not sure what happened. Again, I genuinely and sincerely apologize if this is a stupid question or I missed something somewhere.
Command used (and if a helper script was used, a link to the helper script or the command generated):
Looking at the QSIrecon html file for one MRI session from one participant, sub-261269, this is the command used:
QSIPrep:
/opt/conda/envs/qsiprep/bin/qsiprep /data /output participant --participant-label sub-261269 -w /work --fs-license-file /resources/license.txt --output-resolution 1.0
But I actually ran all 24 participants at once using this slurm script:
#!/bin/bash
#SBATCH --mail-user=<emily.m.harriott@vanderbilt.edu>
#SBATCH --mail-type=ALL
#SBATCH --nodes=1
#SBATCH --cpus-per-task=4
#SBATCH --mem-per-cpu=32G
#SBATCH --time=24:00:00
#SBATCH --output=/data/cutting_lab/users/harrioem/tms_summer2025/logs/qsiprep_%A_%a.log
#SBATCH --error=/data/cutting_lab/users/harrioem/tms_summer2025/logs/qsiprep_%A_%a.err
#SBATCH --array=1-24
# Parameters
participants=(sub-261814 sub-161741 sub-261729 sub-261711 sub-161147 sub-161231 sub-161323 sub-161512 sub-261103 sub-261126 sub-261190 sub-261195 sub-261234 sub-261269 sub-261283 sub-261325 sub-261382 sub-261414 sub-261421 sub-261422 sub-261441 sub-261497 sub-261501 sub-261571)
PARTICIPANT_LABEL=${participants[(${SLURM_ARRAY_TASK_ID} - 1)]}
BIDS_DIR=/data/cutting_lab/users/harrioem/tms_summer2025/qsiprep_input/
OUTPUT_DIR=/data/cutting_lab/users/harrioem/tms_summer2025/qsiprep_output/
WORK_DIR=/data/cutting_lab/users/harrioem/tms_summer2025/qsiprep_workdir/
QSIPREP_RESOURCES_PATH=/data/cutting_lab/users/harrioem/QSIprep/admin/
echo "array id: " ${SLURM_ARRAY_TASK_ID} "subject id: " ${PARTICIPANT_LABEL}
singularity run \
--cleanenv \
-B ${QSIPREP_RESOURCES_PATH}:/resources \
-B ${BIDS_DIR}:/data \
-B ${WORK_DIR}:/work \
-B ${OUTPUT_DIR}:/output \
/panfs/accrepfs.vampire/data/cutting_lab/users/harrioem/QSIprep/qsiprep_latest.sif \
/data /output \
participant --participant-label $PARTICIPANT_LABEL \
-w /work \
--fs-license-file /resources/license.txt \
--output-resolution 1.0 \
QSIRecon:
/opt/conda/envs/qsiprep/bin/qsirecon /data /output participant --participant-label sub-261269 -w /work --fs-license-file /resources/license.txt --recon-spec /resources/pyafq_tractometry_edit.yaml --output-resolution 1.0 --skip-odf-reports --input-type qsiprep
But I actually ran all 24 participants at once using this slurm script:
#!/bin/bash
#SBATCH --mail-user=<emily.m.harriott@vanderbilt.edu>
#SBATCH --mail-type=ALL
#SBATCH --nodes=1
#SBATCH --cpus-per-task=4
#SBATCH --mem-per-cpu=32G
#SBATCH --time=24:00:00
#SBATCH --output=/data/cutting_lab/users/harrioem/tms_summer2025/logs/qsiprep_%A_%a.log
#SBATCH --error=/data/cutting_lab/users/harrioem/tms_summer2025/logs/qsiprep_%A_%a.err
#SBATCH --array=1-24
# Parameters
participants=(sub-261814 sub-161741 sub-261729 sub-261711 sub-161147 sub-161231 sub-161323 sub-161512 sub-261103 sub-261126 sub-261190 sub-261195 sub-261234 sub-261269 sub-261283 sub-261325 sub-261382 sub-261414 sub-261421 sub-261422 sub-261441 sub-261497 sub-261501 sub-261571)
PARTICIPANT_LABEL=${participants[(${SLURM_ARRAY_TASK_ID} - 1)]}
INPUTS=/data/cutting_lab/users/harrioem/tms_summer2025/qsiprep_output/
RESULTS=/data/cutting_lab/users/harrioem/tms_summer2025/qsirecon_output/
WORK_DIR=/data/cutting_lab/users/harrioem/tms_summer2025/qsirecon_workdir/
QSIPREP_RESOURCES_PATH=/data/cutting_lab/users/harrioem/QSIprep/admin/
echo "array id: " ${SLURM_ARRAY_TASK_ID} "subject id: " ${PARTICIPANT_LABEL}
singularity run \
--cleanenv \
-B ${QSIPREP_RESOURCES_PATH}:/resources \
-B ${INPUTS}:/data \
-B ${WORK_DIR}:/work \
-B ${RESULTS}:/output \
/panfs/accrepfs.vampire/data/cutting_lab/users/harrioem/QSIprep/qsirecon_latest.sif \
/data /output \
participant --participant-label $PARTICIPANT_LABEL \
-w /work \
--fs-license-file /resources/license.txt \
--recon-spec /resources/pyafq_tractometry_edit.yaml \
--output-resolution 1.0 \
--skip-odf-reports \
--input-type qsiprep \
Version:
QSIPrep version: 1.0.0rc2.dev0+g789be41.d20241119
QSIRecon version: 1.0.0rc3.dev0+g619341b.d20241121
Environment (Docker, Singularity / Apptainer, custom installation):
I’m using a singularity/apptainer run on Vanderbilt’s ACCRE computing cluster.
Data formatted according to a validatable standard? Please provide the output of the validator:
It is BIDS valid because QSIprep ran successfully and I didn’t include the ignore BIDS flag!
Relevant log outputs (up to 20 lines – this is more than 20 lines b/c I’m not sure what is relevant):
[environment]
cpu_count = 256
exec_env = "singularity"
free_mem = 706.0
overcommit_policy = "heuristic"
overcommit_limit = "50%"
nipype_version = "1.9.1"
templateflow_version = "24.2.2"
version = "1.0.0rc3.dev0+g619341b.d20241121"
[execution]
bids_dir = "/data"
input_dir = "PosixPath('/data')"
atlases = []
bids_database_dir = "/work/20250917-141745_5fca6c23-72a1-4f87-953e-d0fd5c9a34bd/bids_db"
bids_description_hash = "52f468dbb9682e7b91465f7760d8a59306f62a8bbd4129aa41a70071c5676260"
boilerplate_only = false
sloppy = false
debug = []
fs_license_file = "/resources/license.txt"
layout = "BIDS Layout: .../data | Subjects: 1 | Sessions: 0 | Runs: 0"
log_dir = "/output/logs"
log_level = 25
low_mem = false
notrack = false
output_dir = "/output"
reports_only = false
run_uuid = "20250917-141745_5fca6c23-72a1-4f87-953e-d0fd5c9a34bd"
report_output_level = "root"
skip_odf_reports = true
participant_label = [ "261414",]
processing_list = [ "sub-261414/dwi/sub-261414_space-ACPC_desc-preproc_dwi.nii.gz:sub-261414/anat/sub-261414_space-ACPC_desc-preproc_T1w.nii.gz",]
templateflow_home = "/home/harrioem/.cache/templateflow"
work_dir = "/work"
write_graph = false
[workflow]
b0_threshold = 100
infant = false
input_type = "qsiprep"
recon_spec = "/resources/pyafq_tractometry_edit.yaml"
output_resolution = 1.0
qsirecon_suffixes = [ "PYAFQ",]
spaces = "MNI152NLin2009cAsym"
[nipype]
crashfile_format = "txt"
get_linked_libs = false
nprocs = 256
omp_nthreads = 8
plugin = "MultiProc"
remove_unnecessary_outputs = true
resource_monitor = false
stop_on_first_crash = false
[seeds]
master = 62155
ants = 3246
numpy = 17755
[execution.datasets]
[execution.dataset_links]
preprocessed = "/data"
templateflow = "/home/harrioem/.cache/templateflow"
[nipype.plugin_args]
maxtasksperchild = 1
raise_insufficient = false
Screenshots / relevant information:
Edited recon spec file (just added RD and AD)
description: Use pyAFQ to perform the full Tractometry pipeline
name: pyafq_tractometry
nodes:
- action: pyafq_tractometry
input: qsirecon
name: pyafq_tractometry
parameters:
b0_threshold: 50
brain_mask_definition: ''
bundle_info: null
clean_rounds: 5
clip_edges: false
csd_lambda_: 1
csd_response: ''
csd_sh_order: ''
csd_tau: 0.1
directions: prob
dist_to_atlas: 4
dist_to_waypoint: ''
distance_threshold: 3
export: all
filter_b: true
filter_by_endpoints: true
greater_than: 50
gtol: 0.01
import_tract: ''
length_threshold: 4
mapping_definition: ''
max_angle: 30.0
max_bval: ''
max_length: 250
min_bval: ''
min_length: 50
min_sl: 20
model_clust_thr: 1.25
n_points: 100
n_points_bundles: 40
n_points_indiv: 40
n_seeds: 1
nb_points: false
nb_streamlines: false
odf_model: CSD
parallel_segmentation: '{''n_jobs'': -1, ''engine'': ''joblib'', ''backend'':
''loky''}'
presegment_bundle_dict: null
presegment_kwargs: '{}'
prob_threshold: 0
profile_weights: gauss
progressive: true
pruning_thr: 12
random_seeds: false
reduction_thr: 25
refine: false
reg_algo: ''
reg_subject_spec: power_map
reg_template_spec: mni_T1
return_idx: false
rm_small_clusters: 50
rng: ''
rng_seed: ''
robust_tensor_fitting: false
roi_dist_tie_break: false
save_intermediates: ''
sbv_lims_bundles: '[None, None]'
sbv_lims_indiv: '[None, None]'
scalars: '[''dti_fa'', ''dti_md'', ''dti_ad'', ''dti_rd'']'
seed_mask: ''
seed_threshold: 0
seg_algo: AFQ
sphere: ''
stat: mean
step_size: 0.5
stop_mask: ''
stop_threshold: 0
tracker: local
use_external_tracking: false
virtual_frame_buffer: false
viz_backend_spec: plotly_no_gif
volume_opacity_bundles: 0.3
volume_opacity_indiv: 0.3
qsirecon_suffix: PYAFQ
software: pyAFQ
space: T1w
Here is my screenshot of ‘no errors to report’ from the html for this example participant, even though there’s no /dwi/ folder in their /derivatives/ folder 
It also might be worth mentioning that images were acquired on a 3T philips scanner.
If there is anything else I can include or you have any questions please of course let me know. And for the third time I do apologize if this is a stupid question or I missed something.
Thank you so very (very very!) much in advance for any assistance you might be able to provide!! 
Cheers,
Emily