Fmriprep 1.0.7 container no longer working on cluster

fmriprep
#1

Hi all,

Apologies if this is a redundant issue, but while this topic has been covered fairly extensively, none of the previous solutions have resolved my issue.
I ran fmriprep version 1.0.7 from a singularity image on a HPC a while ago, and am now trying to run the same image on some new data. At the time, all was fine, but now I get a recurring error:

File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/util
s/filemanip.py”, line 605, in loadpkl
unpkl = pickle.load(pkl_file)
ModuleNotFoundError: No module named ‘niworkflows.interfaces.images’

I assumed this meant my Python environment was leaking into the container, but nothing I do to try to fix that has alleviated the issue. I have tried unset PYTHONPATH, I have tried PYTHONNOUSEREXPORT=1, PYTHONPATH = " ", I have tried using --no-home, -e, and -H to set the home as something that doesn’t have a Python path in the Singularity run command, and anything else I can find suggested, all to no avail. I have rebuilt the image and the problem persists.
It’s worth noting that I get this error for version 1.0.7 and 1.0.3, but I do not get this error using the newer versions of fmriprep we have images for (1.2.6, 1.3.0, 1.3.2). However, I do have other issues while running those (that I don’t need to get into now), but ideally I would like to use the same processing for all subjects, and analyses have already been run for the previously analyzed subjects, so if at all possible, I’d like to use 1.0.7.

Anyone have any quick thoughts on what else I can try? Do early versions have an issue with nipype that I’m not aware of? I’ve exhausted the ideas of everyone in my lab, as well as the ITS people working on the HPC.
Anything at all would be greatly appreciated. Thanks!

Also, if it’s helpful, the command I’m submitting to the cluster is this:

singularity run -e --bind /{singularityimg_path}/poldracklab_fmriprep_1.0.7-2018-02-14-a45d0782cdfc.img /{bids_path}/data_BIDS/ /{bids_path}/derivatives participant --participant_label 0954 -t GNGs --longitudinal --fs-license-file /{fs_path}/license.txt -w /{workingdir_path}/fmriprep_workingdir/ --nthreads 8 --use-aroma

#2