FMRIPREP error: OSError: handle is closed

Dear experts,

I am trying to run fmriprep-docker with the following command:

fmriprep-docker /Users/noelkuo/Desktop/Data/Nifti /Users/noelkuo/Desktop/Data/derivatives participant --participant-label NC080001 --fs-license-file /Users/noelkuo/freesurfer.txt

The following error message:

 [Node] Finished "fmriprep_wf.single_subject_ADNI0001_wf.func_preproc_task_rest_acq_multiband_wf.bold_mni_trans_wf.bold_reference_wf.enhance_and_skullstrip_bold_wf.apply_mask".
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/concurrent/futures/process.py", line 101, in _python_exit
    thread_wakeup.wakeup()
  File "/usr/local/miniconda/lib/python3.7/concurrent/futures/process.py", line 89, in wakeup
    self._writer.send_bytes(b"")
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 183, in send_bytes
    self._check_closed()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed
Sentry is attempting to send 0 pending error messages
Waiting up to 2.0 seconds
Press Ctrl-C to quit

However, pre-processed data seems to have been completed

Any help would be greatly appreciated,
Noel

Hi @Noel, could you post the full output log? I suspect that the error is generated right at the end of execution, after all the processing is done. If that is the case, I don’t think you’ll need to worry.

Hi, Oscar
In sub-NC080001.html, there was no error detected.
I will try fmriprep again. Would I use the “–verbose” or “-vvv” ? Which output log could help u to check the error information?

Hi @Noel, as far as processing goes, the last thing of consequence that fMRIPrep does is generate the reports, so if no errors have occurred up to that point, you’re fine. In this case, it appears that there is a problem with how we exit Python.

Note the Error in atexit._run_exitfuncs:. This means that it’s clean-up code. I’m not sure that there’s a good way for us to catch and prevent this output. If we can reproduce it, we can try to make sure that the multiprocessing engine is shut down correctly (though this may be a Python bug, and not actually something we can directly do anything about).

1 Like

Hi, @effigies @oesteban
I share the output log. I guess the error is generated at the end of execution, but still, need your help to check the logs.logs.txt (654.6 KB)
logs_detail.txt (2.4 MB)

Hi @noel, these logs look fine.

1 Like

Hi @effigies @oesteban ,
Thank you for your help and the explanation about the error.

@effigies But how to avoid this error inside docker? i have the issue running fmriprep inside docker, after it raise an error docker container shuts down.

This is a minor bug that exhibits at the end of processing, and should have no effect on the processing itself. It is expected that the container would shut down, as the process terminates.

Are you seeing a situation where processing does not complete?

I use it inside my own script, inside docker:

        echo `time fmriprep /input/BIDS /tmp/preproc participant --skip-bids-validation --fs-license-file /opt/freesurfer/license.txt --ignore fieldmaps --fs-no-reconall --use-aroma --force-bbr --participant_label $subject_id -w /tmp/preproc` && \
        cp -r /tmp/preproc /input &
    else
        mkdir -p /tmp/preproc/fmriprep
        cp -r /input/preproc/fmriprep/sub-${subject_id}/ /tmp/preproc/fmriprep
    fi && \

and it terminates, after cp -r /tmp/preproc /input how sentry error could be ignored by docker?

I see. You could use || true to ensure that the failure doesn’t cascade.

(echo `time fmriprep /input/BIDS /tmp/preproc participant \
     --skip-bids-validation --fs-license-file /opt/freesurfer/license.txt \
     --ignore fieldmaps --fs-no-reconall --use-aroma --force-bbr \
     --participant_label $subject_id -w /tmp/preproc || true`) && \
cp -r /tmp/preproc /input &

I’m getting this error now with 1-4-1rc5, but am assuming it’s pretty okay, except that I’m missing carpetplots. Sometimes it generates errors in the .html report, which are always in Node Name: fmriprep_wf.single_subject_NDARINV0CCEN5K2_wf.func_preproc_task_rest_run_[xx]_wf.carpetplot_wf.select_std, and sometimes not. An example of when it does:

	 [Node] Finished "fmriprep_wf.single_subject_NDARINV0CF1U8X8_wf.func_preproc_task_rest_run_01_wf.gen_cifti".
/usr/local/miniconda/lib/python3.7/site-packages/scipy/fftpack/basic.py:160: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  z[index] = x
/usr/local/miniconda/lib/python3.7/site-packages/scipy/fftpack/basic.py:160: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  z[index] = x
/usr/local/miniconda/lib/python3.7/site-packages/scipy/fftpack/basic.py:160: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  z[index] = x
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
Preprocessing did not finish successfully. Errors occurred while processing data from participants: NDARINV0CF1U8X8 (3). Check the HTML reports for details.
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/concurrent/futures/process.py", line 101, in _python_exit
    thread_wakeup.wakeup()
  File "/usr/local/miniconda/lib/python3.7/concurrent/futures/process.py", line 89, in wakeup
    self._writer.send_bytes(b"")
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 183, in send_bytes
    self._check_closed()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

and when it doesn’t:

	 [Node] Finished "fmriprep_wf.single_subject_NDARINV0P4XZMZA_wf.func_preproc_task_rest_run_01_wf.bold_std_trans_wf.bold_reference_wf.enhance_and_skullstrip_bold_wf.apply_mask".
/usr/local/miniconda/lib/python3.7/site-packages/scipy/fftpack/basic.py:160: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  z[index] = x
/usr/local/miniconda/lib/python3.7/site-packages/scipy/fftpack/basic.py:160: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  z[index] = x
/usr/local/miniconda/lib/python3.7/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/scipy/fftpack/basic.py:160: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  z[index] = x
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
/usr/local/miniconda/lib/python3.7/site-packages/scipy/fftpack/basic.py:160: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  z[index] = x
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/nipype/algorithms/confounds.py:1099: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  betas = np.linalg.lstsq(X, data.T)[0]
/usr/local/miniconda/lib/python3.7/site-packages/matplotlib/contour.py:1173: UserWarning: No contour levels were found within the data range.
  warnings.warn("No contour levels were found"
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/concurrent/futures/process.py", line 101, in _python_exit
    thread_wakeup.wakeup()
  File "/usr/local/miniconda/lib/python3.7/concurrent/futures/process.py", line 89, in wakeup
    self._writer.send_bytes(b"")
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 183, in send_bytes
    self._check_closed()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

Preprocessing seems to finish fine for both, even in the second case where it looks like it stopped at rest_run_01_wf.bold_std_trans_wf.bold_reference_wf.enhance_and_skullstrip_bold_wf.apply_mask, but I don’t get any nice carpetplots in the .html file.

Edit: I just realized I reduced the amount of memory from 25 GBs down to 18-19 GBs…is this just a memory issue or something?