Summary of what happened:
Good morning Qsiprep’s team and thank you for making this great software!
I have a problem that I can’t figure out… I ran the preprocessing of qsiprep and everything worked great. I am now trying to create connectivity matrices for my subjects using the Desikan Killiany atlas, which I registered to the MNI152NLin2009cAsym T1 template in LPS provided by qsiprep. I copied the atlases provided in the singularity image into a folder on my PC and added there the desikan name, along with its filepath, node ids and names, to the list of atlases in the atlas_config.json. I also copied the mrtrix_singleshell_ss3t_ACT-hsvs json provided by qsiprep to the directory where I run my script and added the desikan atlas. Then I run the code below. The process works smoothly and no errors are reported. The exit code status is 0 and the html files are correctly generated, as well as the figures containing the SC matrices. The problem is that, although I also have an anat and figures folder for my subject, I do not have any dwi folder nor any type of reconstructions saved in the folder for any of the atlases (I first ran it only with the desikan and also did not work)! If I run using the default --recon-spec everything works as expected. I really don’t know what I am doing wrong 
Thank you for your help!
Command used (and if a helper script was used, a link to the helper script or the command generated):
Commandline: singularity run --cleanenv
-B /home/xxx/Projects/TADAD/data/derivatives_ses-00/qsiprep:/data:ro \
-B /home/xxx/Projects/TADAD/data/derivatives_ses-00:/out -B \
/home/xxx/Projects/TADAD/data/temp_dir/sub-ADNI003S4441:/work \
-B /home/xxx/Projects/TADAD/data/utils/qsirecon_atlases:/atlas/qsirecon_atlases \
-B /home/xxx/Projects/TADAD/documents/freesurfer/license.txt:/freesurfer_license.txt:ro \
home/xxx/Projects/TADAD/singims/qsiprep_0.19.1.sif \
/data /out participant \
--participant-label sub-ADNI003S4441 --output_resolution 1 \
--recon-only --recon-spec mrtrix_singleshell_ss3_hsvst_dkt_atlas.json \
--recon_input /out/qsiprep --freesurfer-input /out/freesurfer --fs-license-file \
/home/xxx/Projects/TADAD/documents/freesurfer/license.txt \
--nthreads 48 --mem_mb 131072 --stop-on-first-crash -w /work
This is the start of the mrtrix_singleshell_ss3_hsvst_dkt_atlas.json (same as standard with desikan added):
{
"name": "mrtrix_singleshell_ss3_hsvst",
"space": "T1w",
"atlases": ["schaefer100", "schaefer200", "schaefer400", "brainnetome246", "aicha384", "gordon333", "aal116", "desikan"],
"anatomical": ["mrtrix_5tt_hsvs"],
"nodes": [{
"name": "ss3t_csd",
"software": "MRTrix3", ....
Version:
Qsiprep 0.19.1
Environment (Docker, Singularity / Apptainer, custom installation):
Singularity
Data formatted according to a validatable standard? Please provide the output of the validator:
My dataset is divided into different sessions (because I used fMRIPREP and I expected the longitudinal structural data to change between sesssions), but the standard recon workflow works fine.
Relevant log outputs (up to 20 lines):
None… all seems to work well but no connectivity matrices files are generated
Screenshots / relevant information:
As a sanity check I ran the reconstruction using the default --recon-spec mrtrix_singleshell_ss3t_ACT-hsvs and everything works fine with the expected connectivity matrices produced.
Hi Ricardo,
I have the same issue using the 4S156 atlas from AtlasPack (GitHub - PennLINC/AtlasPack: Combined cortical/subcortical atlases for xcp-d and qsiprep) and QSIPrep v0.19.1 via Singularity. The anat hsvs output and the connectivity matrix plots (that do contain the custom atlas) and FOD plots are present for the participant in the qsirecon folder, but no dwi folder. Similarly, no errors are reported by QSIPrep despite the missing output.
I originally ran this reconstruction using a pipeline configuration JSON that did not include the default QSIPrep atlases, but I think this discussion suggests that the custom atlases should be added to the atlases list as you have done in your custom mrtrix_singleshell_ss3t_ACT-hsvs.json. I am now testing whether including the default atlases in my recon spec JSON makes a difference.
I will give an update if I find a fix for this issue. Please let me know if you find a fix as well!
As an update, adding both the custom atlases and the default atlases to the workflow JSON did not resolve the error of missing dwi outputs - even though all connectivity matrices are plotted for all included atlases in the HTML report.
Any debugging suggestions or fixes would be greatly appreciated!
Thanks!
Hi @pcamach2, are the files present in the working directory? I wonder if there is a bug in the datasink. If the figures are being created, the process must have run successfully
Hi @mattcieslak, the working directory qsirecon_wf folders contain the <atlas>_<edge_weight>_connectivity.mat files, the exemplar .tck files, and the other pieces that I would expect from this reconstruction pipeline. I think the bug may be in the datasink as you suggest.
I am running with a fresh working directory now just in case, but I will update when that is done.
Thanks!
Hello @pcamach2 and @mattcieslak,
I can confirm that the files are in the working directory and do not get copied to the final directory when running the custom reconstruction.
I also tried the following workaround. I set the default --recon-spec mrtrix_singleshell_ss3t_ACT-hsvs, but when mounting the qsirecon_atlases folder, I changed the name of my Desikan atlas to Brainnettome and I also updated the atlas_config.json for labels and indices accordingly, but left the old “brainnettome…nii.gz” filename. Doing this, the files get copied to the final directory… So I guess, although not very elegant, this might be a solution in case one does not need one of the standard atlases, the other being explicitly copying from the working directory! Thanks!
Hi @mattcieslak, I can confirm that the outputs are in the working directory, but no qsirecon/<participant_id>/<session_id>/dwi is created in the bids derivatives folder. Does the derivatives datasink check for new custom atlases or is there a hardcoded list somewhere besides atlas_config.json that is causing the issue?
Hi @Riccardo_Leone, thank you for the quick workaround! Since the atlases that I am adding are very similar to the schaefer100 and schaefer200 atlases bundled with QSIPrep, I could switch them out as you did with the brainnettome parcellation. This would limit the size of outputs, which is always good. I am concerned that this could make interpretation and documentation a bit muddled in the long run, though.
Cheers!
Due to time constraints for this project, I ended up running all 50 participant’s data through QSIPrep with the temp fix that you suggested, @Riccardo_Leone. Thank you!
I linked this post to an issue on the QSIPrep GitHub (Reconstruction dwi outputs not included in derivatives when using a custom atlas · Issue #721 · PennLINC/qsiprep · GitHub), so hopefully we can get this derivatives datasink fixed for custom workflows.
Cheers!