BrokenProcessPool Error in fmriprep-docker & Child Process Termination

Software Support
,
1

Hey everyone,

I’m new to using fmriprep-docker for fMRI data preprocessing, and I’m facing an issue during execution. When I run the container, I encounter the following error message:

concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
250124-05:39:59,655 nipype.workflow CRITICAL:
	 fMRIPrep failed: A child process terminated abruptly, the process pool is not usable anymore
2025-01-24 05:39:59,655 [CRITICAL] fMRIPrep failed: A child process terminated abruptly, the process pool is not usable anymore
Sentry is attempting to send 1 pending events
Waiting up to 2 seconds
Press Ctrl-C to quit
fMRIPrep: Please report errors to https://github.com/nipreps/fmriprep/issues

Command used

fmriprep-docker /home/iamakp7/Desktop/data /home/iamakp7/Desktop/data/derivatives participant --participant-label 'sub'-DBS02 --fs-license-file /home/iamakp7/Desktop/license/license.txt

Version:

fMRIPrep version 24.1.1

Environment (Docker, Singularity / Apptainer, custom installation):

I’m running the software using the Docker container.

Relevant log outputs (up to 20 lines):

2025-01-24 05:39:57,528 [    INFO] [Node] Executing "boldref_bold" <fmriprep.interfaces.resampling.ResampleSeries>
2025-01-24 05:39:57,679 [ WARNING] WARNING: Reference space not set
2025-01-24 05:39:57,721 [ WARNING] WARNING: Reference space not set
2025-01-24 05:39:57,998 [ WARNING] WARNING: The behavior of affine_transform with a 1-D array supplied for the matrix parameter has changed in SciPy 0.18.0.
250124-05:39:58,26 nipype.workflow INFO:
	 [Node] Finished "gen_ref", elapsed time 3.327699s.
2025-01-24 05:39:58,026 [    INFO] [Node] Finished "gen_ref", elapsed time 3.327699s.
2025-01-24 05:39:59,382 [   ERROR] exception calling callback for <Future at 0x7f353972aa90 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
2025-01-24 05:39:59,409 [   ERROR] exception calling callback for <Future at 0x7f354c9da8d0 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
2025-01-24 05:39:59,423 [   ERROR] exception calling callback for <Future at 0x7f35397e8c50 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
2025-01-24 05:39:59,437 [   ERROR] exception calling callback for <Future at 0x7f354cad8fd0 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
2025-01-24 05:39:59,446 [   ERROR] exception calling callback for <Future at 0x7f35398fa4d0 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
2025-01-24 05:39:59,453 [   ERROR] exception calling callback for <Future at 0x7f3539b62ad0 state=finished raised BrokenProcessPool>
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 159, in _async_callback
    result = args.result()
             ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
250124-05:39:59,655 nipype.workflow CRITICAL:
	 fMRIPrep failed: A child process terminated abruptly, the process pool is not usable anymore
2025-01-24 05:39:59,655 [CRITICAL] fMRIPrep failed: A child process terminated abruptly, the process pool is not usable anymore
Sentry is attempting to send 1 pending events
Waiting up to 2 seconds
Press Ctrl-C to quit
fMRIPrep: Please report errors to https://github.com/nipreps/fmriprep/issues

Screenshots / relevant information:

Screenshot_2025-01-24_11-42-28
Screenshot_2025-01-24_11-42-281923×895 148 KB

I’d appreciate any insights or suggestions on how to troubleshoot this process pool termination error.

  • Could it be related to insufficient system resources (memory, CPU)?
  • Are there any specific checks I should perform within the Docker container?
  • Would providing more details about my data or Docker setup be helpful?

Thank you for your time and support!

Best,
Ashish Singh Panchal

2

Hi @iamakp7 and welcome to neurostars!

Likely an out of memory issue. How many resources are you devoting to the job?

Best,
Steven

3

Hi @Steven ,

Thank you for your response and for welcoming me to NeuroStars! You were right—the issue was indeed related to memory. I had initially allocated 8 GB of RAM, but after increasing it to 50 GB, the memory-related errors were resolved.

However, I now encounter BIDS validation errors when running fMRIPrep. Specifically:

  1. A NOT_INCLUDED error for a fmriprep.toml file, which I’ve addressed by adding it to a .bidsignore file.
  2. A SLICE_TIMING_NOT_DEFINED warning for several functional MRI files. I’m working on adding the SliceTiming metadata to the corresponding JSON files.
  3. A README_FILE_MISSING warning, which I’ve resolved by adding a README file to the dataset.

Once these issues are resolved, I’ll re-run fMRIPrep and let you know if I encounter any further problems.

Thank you again for your help!

Best regards,
Ashish Singh Panchal