i’m establishing a protocol in the beginning of a study’s analysis so i have the luxury of fiddling around before i settle on something that works well. i had some issues with QSIprep 0.23.0 so i backtracked to 0.22.0 which worked for a single session of a subject. now that i’ve added a “–longitudinal” option (2 sessions) it’s breaking with a very similar error to this.
seeing @mattcieslak reply there, looking at “ShimSetting”: in ses-1/dwi/sub-SF005_ses-1_dir-AP_dwi.json
shows that the shim was identical between AP/PA runs for session 1.
and looking at “ShimSetting”: in ses-2/dwi/sub-SF005_ses-2_dir-AP_dwi.json
shows that the shim had changed between AP/PA runs for session 2.
so my question is in 2 parts:
for future knowledge, if i want to see if my errors go away by trying new versions of QSIprep, how would i know where the “stable” version lies in the ranking at the top of the QSIprep version numbering? i can’t find anything but upload dates for “unstable” in the docker website and the “What’s new” page hasn’t been updated recently, either.
if anyone can tell right away that updating my QSIprep still won’t fix my “different FOV” issue then i’ll open a different and focused ticket/question for that specifically. otherwise i’ll start trying different versions and can report back with what i’ve observed.
The stable release should be the highest numbered release (besides release candidates which will have a “rc” label) on dockerhub.
The upcoming release is going to have lots of changes, some relating to longitudinal processing like you have. I’d personally recommend waiting for that. But 0.24.0 was also released recently and you can try that.
thanks again for your help. i’ve built ver 0.24.0 into a .sif and have been trying that. based on a series of its error recommendations (e.g. removing “–longitudinal” and replacing with “–subject-anatomical-reference unbiased”) my qsiprep command on my 2-session dataset is now:
and it runs for a little while and then breaks. below i’m including a snippet of the output including line numbers so you have a sense of how far it got. it seems like it’s downloading a template to our HPC but then can’t use it afterwards? can i download separately and utilize “–anatomical-template” to reference its absolute path?
finally, and related to all this, i see some very recent movement on github to the tune of “Prepare for 1.0.0rc1 release” which I think is the upcoming release you mentioned. would an approximate ETA be days, weeks or months? I have a deadline in approx a month and so i’m not sure how to strategize my analysis plan if i’m literally in between versions that’ll deal well with longitudinal processing. so, insider recommendations would be appreciated
231 [Node] Executing "get_template_image" <qsiprep.interfaces.anatomical.GetTemplate>
232 Downloading https://templateflow.s3.amazonaws.com/tpl-MNI152NLin2009cAsym/tpl-MNI152NLin2009cAsym_res-01_T1w.nii.gz
233 241119-12:22:03,773 nipype.workflow INFO:
234 [Node] Finished "get_template_image", elapsed time 0.027784s.
235 241119-12:22:03,773 nipype.workflow WARNING:
236 Storing result file without outputs
237 241119-12:22:03,786 nipype.workflow WARNING:
238 [Node] Error on "qsiprep_0_24_wf.sub_SF005_ses_01_02_wf.anat_preproc_wf.get_template_image" (/wynton/protected/home/rad-wynton-only/storrisi/sf005_wd/qsiprep_0_24_wf/sub_SF005_ses_01_02_wf/anat_preproc_wf/get_template_image)
239 241119-12:22:03,853 nipype.interface INFO:
240 Not applying reorientation to /wynton/protected/home/rad-wynton-only/storrisi/misc_dont_delete/motSF005/sub-SF005/ses-01/dwi/sub-SF005_ses-01_dir-PA_dwi.nii.gz: already in LAS
241 241119-12:22:03,859 nipype.workflow INFO:
242 [Node] Finished "conform_dwis01", elapsed time 0.146712s.
243 241119-12:22:04,283 nipype.workflow INFO:
244 [Job 0] Completed (qsiprep_0_24_wf.sub_SF005_ses_01_02_wf.bidssrc).
245 241119-12:22:04,292 nipype.workflow ERROR:
246 Node get_template_image failed to run on host msg-id25.wynton.ucsf.edu.
247 241119-12:22:04,301 nipype.workflow ERROR:
248 Saving crash info to /wynton/protected/home/rad-wynton-only/storrisi/sub-SF005/log/20241119-122015_dd36dd95-fcbd-4683-bdb4-c8221df1b3fb/crash-20241119-122204-storrisi-get_template_image-db33a87e-7b51-4161-ba38-a51d70c728d1.txt
249 Traceback (most recent call last):
250 File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
251 result["result"] = node.run(updatehash=updatehash)
252 File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
253 result = self._run_interface(execute=True)
254 File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
255 return self._run_command(execute)
256 File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
257 raise NodeExecutionError(msg)
258 nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node get_template_image.
Can you include the full traceback? It seems like you cut it off before it includes the specific error message. Does your cluster compute nodes have internet access? It also is not clear from your command what your singularity/apptainer preamble looks like, which is important knowledge.
I think the full 1.0.0 release will take a few weeks since we want users to test it before making the official release. Your input would be appreciated if you have time.
You can also try the distortion group merge in 0.24.0+ if you are using the same dataset as your previous issue (--combine-all-dwis not recognized).
i could include the entire traceback but i think i’ve identified the issue.
i just double checked with the sys admins of this HPC… indeed these compute nodes do not have internet access. hmm, this arguably most-used template is only 13MB and could be included inside future QSIprep releases? in the meantime i directly downloaded it and tried adding it as an absolute path after “–anatomical-template” but that doesn’t work because it expects keyword “MNI152NLin2009cAsym”. trying that then breaks because it’s still trying to access AWS. what to do?
ok a few weeks; i think i’ll continue to plow ahead with 0.24.0 if you still think it’s acceptable for longitudinal processing and future input into QSIrecon. i’d be happy to test future versions if i’m able to, but can’t promise anything if i’m in the throes of deadline-pushed-analyses (which is only sometimes and temporary).
i will try distortion group merge again later, too, but for now i really would like to get a longitudinally-appropriate preprocessing pipeline working. thanks again!
Yes, it may soon be updated to allow for different spaces, but even when it does, it is not a path input, but rather a templateflow template name.
Download templateflow set with datalad (GitHub - templateflow/templateflow: The TemplateFlow Archive - A DataLad super-dataset), datalad get the tpl-MNI152NLin2009cAsym folder (there may be other folders too, but at least that one to start), set export SINGULARITYENV_TEMPLATEFLOW_HOME='/your/downloaded/templateflow. Make sure it is mounted tin the container too.
from templateflow.api import get as get_template
anatomical_contrast = "T1w"
template_name = “MNI152NLin2009cAsym”
cohort = None
if "+" in template_name:
template_name, cohort = template_name.split("+")
template_path = get_template(
template_name,
cohort=cohort,
resolution="1",
desc=None,
suffix=anatomical_contrast,
extension=".nii.gz",
)
mask_path = get_template(
template_name,
cohort=cohort,
resolution="1",
desc="brain",
suffix="mask",
extension=".nii.gz",
)
QSIRecon will now consider any < 1.0.0rc1 outputs legacy which are supported for now, but can’t guarantee for how long it will be supported for. Many of the changes in 1.0.0+ (prep and recon) deal with session wise longitudinal data. I don’t know your dataset, planned recon, or analysis goals, so I can’t say if it will be a big deal for you.
This shouldn’t impact longitudinal workflows, if it does, let us know!
i was trying to avoid your suggested tricks for now so was still using 0.24.0 and added “–skip-anat-based-spatial-normalization” to my QSIprep command. nonetheless it’s still trying to pull from AWS on our compute nodes with no internet access. i will try what you suggest if there’s no other way to turn off pulling from AWS… so just checking that there isn’t? and will this be different in 1.0.0rc2 and beyond? thanks!
