Hi all,
I’m trying to run TractoFlow on a GPU server using Singularity (Apptainer) with scilus_1.6.0. I’ve followed the documentation and installed everything successfully, but I keep encountering the following error:
NXF_VER=21.10.6 nextflow run tractoflow -r 2.4.3 --input /home/ra/tsangk9/tests/data/tractoflow_ready -with-singularity scilus_1.6.0.sif -resume
N E X T F L O W ~ version 21.10.6
Launching `scilus/tractoflow` [mighty_spence] - revision: 2ca9dad434 [2.4.3]
TractoFlow pipeline
===================
Start time: 2025-04-03T14:20:52.092243574Z
Max DTI shell extracted: 1200
Min FODF shell extracted: 700
Input: /home/ra/tsangk9/tests/data/tractoflow_ready
executor > local (10)
executor > local (15)
[cf/65253e] process > README (README) [ 50%] 1 of 2, failed: 1, retries: 1
[5c/af5c9c] process > Bet_Prelim_DWI (O_2) [ 50%] 3 of 6, failed: 3, retries: 3
[b0/ebb5c8] process > Denoise_DWI (O_2) [ 50%] 3 of 6, failed: 3, retries: 3
[- ] process > Gibbs_correction -
[- ] process > Prepare_for_Topup -
[- ] process > Topup -
[- ] process > Prepare_dwi_for_eddy -
[- ] process > Eddy_Topup -
[- ] process > Eddy -
[- ] process > Bet_DWI -
executor > local (20)
[cf/65253e] process > README (README) [ 50%] 1 of 2, failed: 1, retries: 1
[86/dba888] process > Bet_Prelim_DWI (O_3) [ 50%] 3 of 6, failed: 3, retries: 3
[a3/1a1df9] process > Denoise_DWI (O_1) [ 50%] 3 of 6, failed: 3, retries: 3
[- ] process > Gibbs_correction -
[- ] process > Prepare_for_Topup -
[- ] process > Topup -
[- ] process > Prepare_dwi_for_eddy -
[- ] process > Eddy_Topup -
[- ] process > Eddy -
[- ] process > Bet_DWI -
executor > local (30)
[f3/a599ef] process > README (README) [ 66%] 2 of 3, failed: 2, retries: 2
[44/ef4d2d] process > Bet_Prelim_DWI (O_3) [ 66%] 6 of 9, failed: 6, retries: 6
[96/5588af] process > Denoise_DWI (O_1) [ 66%] 6 of 9, failed: 6, retries: 6
[- ] process > Gibbs_correction -
[- ] process > Prepare_for_Topup -
[- ] process > Topup -
[- ] process > Prepare_dwi_for_eddy -
[- ] process > Eddy_Topup -
[- ] process > Eddy -
[- ] process > Bet_DWI -
[- ] process > N4_DWI -
[- ] process > Crop_DWI -
[- ] process > Denoise_T1 -
[3a/5d5b88] process > N4_T1 (O_1) [ 66%] 6 of 9, failed: 6, retries: 6
[- ] process > Resample_T1 -
[- ] process > Bet_T1 -
[- ] process > Crop_T1 -
[- ] process > Normalize_DWI -
[- ] process > Resample_DWI -
[- ] process > Extract_B0 -
executor > local (39)
[76/ffeee7] process > README (README) [ 75%] 3 of 4, failed: 3, retries: 3
[24/f172d2] process > Bet_Prelim_DWI (O_2) [ 72%] 8 of 11, failed: 8, retries: 8
[c4/e94328] process > Denoise_DWI (O_1) [ 75%] 9 of 12, failed: 9, retries: 9
[- ] process > Gibbs_correction -
[- ] process > Prepare_for_Topup -
[- ] process > Topup -
[- ] process > Prepare_dwi_for_eddy -
[- ] process > Eddy_Topup -
[- ] process > Eddy -
[- ] process > Bet_DWI -
[- ] process > N4_DWI -
[- ] process > Crop_DWI -
[- ] process > Denoise_T1 -
[d9/949075] process > N4_T1 (O_1) [ 75%] 9 of 12, failed: 9, retries: 9
[- ] process > Resample_T1 -
[- ] process > Bet_T1 -
[- ] process > Crop_T1 -
[- ] process > Normalize_DWI -
[- ] process > Resample_DWI -
[- ] process > Extract_B0 -
[- ] process > Extract_SH_Fitting_Shell -
[- ] process > SH_Fitting -
[- ] process > Extract_DTI_Shell -
[- ] process > DTI_Metrics -
[- ] process > Extract_FODF_Shell -
[- ] process > Register_T1 -
[- ] process > Register_Freesurfer -
[- ] process > Segment_Freesurfer -
[- ] process > Segment_Tissues -
executor > local (40)
[76/ffeee7] process > README (README) [ 75%] 3 of 4, failed: 3, retries: 3
[11/088497] process > Bet_Prelim_DWI (O_3) [ 75%] 9 of 12, failed: 9, retries: 9
[c4/e94328] process > Denoise_DWI (O_1) [ 75%] 9 of 12, failed: 9, retries: 9
[- ] process > Gibbs_correction -
[- ] process > Prepare_for_Topup -
[- ] process > Topup -
[- ] process > Prepare_dwi_for_eddy -
[- ] process > Eddy_Topup -
[- ] process > Eddy -
[- ] process > Bet_DWI -
[- ] process > N4_DWI -
[- ] process > Crop_DWI -
[- ] process > Denoise_T1 -
[d9/949075] process > N4_T1 (O_1) [ 75%] 9 of 12, failed: 9, retries: 9
[- ] process > Resample_T1 -
[- ] process > Bet_T1 -
[- ] process > Crop_T1 -
[- ] process > Normalize_DWI -
[- ] process > Resample_DWI -
[- ] process > Extract_B0 -
[- ] process > Extract_SH_Fitting_Shell -
[- ] process > SH_Fitting -
[- ] process > Extract_DTI_Shell -
[- ] process > DTI_Metrics -
[- ] process > Extract_FODF_Shell -
[- ] process > Register_T1 -
[- ] process > Register_Freesurfer -
[- ] process > Segment_Freesurfer -
[- ] process > Segment_Tissues -
[- ] process > Compute_FRF -
executor > local (40)
[76/ffeee7] process > README (README) [100%] 4 of 4, failed: 4, retries: 3 ✔
[11/088497] process > Bet_Prelim_DWI (O_3) [ 75%] 9 of 12, failed: 9, retries: 9
[c4/e94328] process > Denoise_DWI (O_1) [ 75%] 9 of 12, failed: 9, retries: 9
[- ] process > Gibbs_correction -
[- ] process > Prepare_for_Topup -
[- ] process > Topup -
[- ] process > Prepare_dwi_for_eddy -
[- ] process > Eddy_Topup -
[- ] process > Eddy -
[- ] process > Bet_DWI -
[- ] process > N4_DWI -
[- ] process > Crop_DWI -
[- ] process > Denoise_T1 -
[d9/949075] process > N4_T1 (O_1) [ 75%] 9 of 12, failed: 9, retries: 9
[- ] process > Resample_T1 -
[- ] process > Bet_T1 -
[- ] process > Crop_T1 -
[- ] process > Normalize_DWI -
[- ] process > Resample_DWI -
[- ] process > Extract_B0 -
[- ] process > Extract_SH_Fitting_Shell -
[- ] process > SH_Fitting -
[- ] process > Extract_DTI_Shell -
[- ] process > DTI_Metrics -
[- ] process > Extract_FODF_Shell -
[- ] process > Register_T1 -
[- ] process > Register_Freesurfer -
[- ] process > Segment_Freesurfer -
[- ] process > Segment_Tissues -
[- ] process > Compute_FRF -
[- ] process > Mean_FRF -
executor > local (40)
[76/ffeee7] process > README (README) [100%] 4 of 4, failed: 4, retries: 3 ✔
[11/088497] process > Bet_Prelim_DWI (O_3) [100%] 12 of 12, failed: 12, retries: 9 ✔
[c4/e94328] process > Denoise_DWI (O_1) [100%] 12 of 12, failed: 12, retries: 9 ✔
[- ] process > Gibbs_correction -
[- ] process > Prepare_for_Topup -
[- ] process > Topup -
[- ] process > Prepare_dwi_for_eddy -
[- ] process > Eddy_Topup -
[- ] process > Eddy -
[- ] process > Bet_DWI -
[- ] process > N4_DWI -
[- ] process > Crop_DWI -
[- ] process > Denoise_T1 -
[d9/949075] process > N4_T1 (O_1) [100%] 12 of 12, failed: 12, retries: 9 ✔
[- ] process > Resample_T1 -
[- ] process > Bet_T1 -
[- ] process > Crop_T1 -
[- ] process > Normalize_DWI -
[- ] process > Resample_DWI -
[- ] process > Extract_B0 -
[- ] process > Extract_SH_Fitting_Shell -
[- ] process > SH_Fitting -
[- ] process > Extract_DTI_Shell -
[- ] process > DTI_Metrics -
[- ] process > Extract_FODF_Shell -
[- ] process > Register_T1 -
[- ] process > Register_Freesurfer -
[- ] process > Segment_Freesurfer -
[- ] process > Segment_Tissues -
[- ] process > Compute_FRF -
[- ] process > Mean_FRF -
[- ] process > FODF_Metrics -
[- ] process > PFT_Tracking_Maps -
[- ] process > PFT_Seeding_Mask -
[- ] process > PFT_Tracking -
[- ] process > Local_Tracking_Mask -
[- ] process > Local_Seeding_Mask -
[- ] process > Local_Tracking -
Pipeline completed at: 2025-04-03T14:20:58.297808058Z
Execution status: OK
Execution duration: 6.2s
[cf/65253e] NOTE: Process `README (README)` terminated with an error exit status (1) -- Execution is retried (1)
[d8/ee457a] NOTE: Process `Bet_Prelim_DWI (O_1)` terminated with an error exit status (1) -- Execution is retried (1)
[26/ace33c] NOTE: Process `Denoise_DWI (O_3)` terminated with an error exit status (1) -- Execution is retried (1)
[f8/21342e] NOTE: Process `Denoise_DWI (O_2)` terminated with an error exit status (1) -- Execution is retried (1)
[d1/5fa9b1] NOTE: Process `Bet_Prelim_DWI (O_2)` terminated with an error exit status (1) -- Execution is retried (1)
[dc/ce6b9c] NOTE: Process `Denoise_DWI (O_1)` terminated with an error exit status (1) -- Execution is retried (1)
[f2/bd340a] NOTE: Process `Bet_Prelim_DWI (O_3)` terminated with an error exit status (1) -- Execution is retried (1)
[78/458c4e] NOTE: Process `N4_T1 (O_2)` terminated with an error exit status (1) -- Execution is retried (1)
[72/85f55d] NOTE: Process `N4_T1 (O_3)` terminated with an error exit status (1) -- Execution is retried (1)
[d8/b23e52] NOTE: Process `N4_T1 (O_1)` terminated with an error exit status (1) -- Execution is retried (1)
[f6/63f178] NOTE: Process `README (README)` terminated with an error exit status (1) -- Execution is retried (2)
[36/e3307e] NOTE: Process `Denoise_DWI (O_3)` terminated with an error exit status (1) -- Execution is retried (2)
[e2/53ef2a] NOTE: Process `Bet_Prelim_DWI (O_1)` terminated with an error exit status (1) -- Execution is retried (2)
[b0/ebb5c8] NOTE: Process `Denoise_DWI (O_2)` terminated with an error exit status (1) -- Execution is retried (2)
[69/24a7ea] NOTE: Process `N4_T1 (O_2)` terminated with an error exit status (1) -- Execution is retried (2)
[5c/af5c9c] NOTE: Process `Bet_Prelim_DWI (O_2)` terminated with an error exit status (1) -- Execution is retried (2)
[a3/1a1df9] NOTE: Process `Denoise_DWI (O_1)` terminated with an error exit status (1) -- Execution is retried (2)
[49/d81cdf] NOTE: Process `N4_T1 (O_3)` terminated with an error exit status (1) -- Execution is retried (2)
[86/dba888] NOTE: Process `Bet_Prelim_DWI (O_3)` terminated with an error exit status (1) -- Execution is retried (2)
[40/017b94] NOTE: Process `N4_T1 (O_1)` terminated with an error exit status (1) -- Execution is retried (2)
[f3/a599ef] NOTE: Process `README (README)` terminated with an error exit status (1) -- Execution is retried (3)
[0b/52fa7d] NOTE: Process `N4_T1 (O_2)` terminated with an error exit status (1) -- Execution is retried (3)
[fd/64b32f] NOTE: Process `Denoise_DWI (O_3)` terminated with an error exit status (1) -- Execution is retried (3)
[8c/6f00b7] NOTE: Process `Denoise_DWI (O_2)` terminated with an error exit status (1) -- Execution is retried (3)
[cf/7ea9ef] NOTE: Process `Bet_Prelim_DWI (O_1)` terminated with an error exit status (1) -- Execution is retried (3)
[3c/c69962] NOTE: Process `N4_T1 (O_3)` terminated with an error exit status (1) -- Execution is retried (3)
[96/5588af] NOTE: Process `Denoise_DWI (O_1)` terminated with an error exit status (1) -- Execution is retried (3)
[b0/ecb287] NOTE: Process `Bet_Prelim_DWI (O_2)` terminated with an error exit status (1) -- Execution is retried (3)
[3a/5d5b88] NOTE: Process `N4_T1 (O_1)` terminated with an error exit status (1) -- Execution is retried (3)
[44/ef4d2d] NOTE: Process `Bet_Prelim_DWI (O_3)` terminated with an error exit status (1) -- Execution is retried (3)
[76/ffeee7] NOTE: Process `README (README)` terminated with an error exit status (1) -- Error is ignored
[f5/22f268] NOTE: Process `N4_T1 (O_2)` terminated with an error exit status (1) -- Error is ignored
[45/be8be5] NOTE: Process `N4_T1 (O_3)` terminated with an error exit status (1) -- Error is ignored
[d3/b25f4a] NOTE: Process `Denoise_DWI (O_2)` terminated with an error exit status (1) -- Error is ignored
[08/b382ca] NOTE: Process `Denoise_DWI (O_3)` terminated with an error exit status (1) -- Error is ignored
[31/b8564d] NOTE: Process `Bet_Prelim_DWI (O_1)` terminated with an error exit status (1) -- Error is ignored
[c4/e94328] NOTE: Process `Denoise_DWI (O_1)` terminated with an error exit status (1) -- Error is ignored
[d9/949075] NOTE: Process `N4_T1 (O_1)` terminated with an error exit status (1) -- Error is ignored
[24/f172d2] NOTE: Process `Bet_Prelim_DWI (O_2)` terminated with an error exit status (1) -- Error is ignored
[11/088497] NOTE: Process `Bet_Prelim_DWI (O_3)` terminated with an error exit status (1) -- Error is ignored
This same pipeline runs successfully on my local machine using Docker, but now I am trying to execute it on a more powerful CPU server running zsh. Has anyone encountered a similar issue, or have suggestions for troubleshooting?
Thanks in advance!