Problems installing FSL

km345 · February 26, 2024, 10:06am

Hi NeuroStars,

I’m trying to install fsl on a HPC but it fails few minutes into the installation. Could you advise which version to choose and if there’s any specific flags to use to compensate for no root privileges?

Command used

python fslinstaller.py # installed into scratch
# also tried with python2 and --conda

Part of the log file (with 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: [stderr]: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:56 fslinstaller.py:1084: 09:23:58 fslinstaller.py:1084: [stderr]: error 09:23:58 fslinstaller.py:1084: 09:23:58 fslinstaller.py:1084: [stderr]: 09:24:00 fslinstaller.py: 206:
ERROR occurred during installation!
09:24:00 fslinstaller.py: 206: the error):
[stderr]: # >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
[stderr]:
Traceback (most recent call last):
File “/mnt/shared/scratch/kmarians/apps/fsl/lib/python3.10/site-packages/conda/exception_handler.py”, line 17, in call
return func(*args, **kwargs)
File “/mnt/shared/scratch/kmarians/apps/fsl/lib/python3.10/site-packages/conda_env/cli/main.py”, line 56, in do_call
exit_code = getattr(module, func_name)(arguments, parser)
File “/mnt/shared/scratch/kmarians/apps/fsl/lib/python3.10/site-packages/conda/notices/core.py”, line 124, in wrapper
return func(*args, **kwargs)
File “/mnt/shared/scratch/kmarians/apps/fsl/lib/python3.10/site-packages/conda_env/cli/main_update.py”, line 143, in execute
result[installer_type] = installer.install(prefix, specs, args, env)
File “/mnt/shared/scratch/kmarians/apps/fsl/lib/python3.10/site-packages/mamba/mamba_env.py”, line 155, in mamba_install
transaction.fetch_extract_packages()
RuntimeError: Resource temporarily unavailable
[stderr]:
[stderr]: $ /home/kmarians/scratch/apps/fsl/bin/mamba update -p /home/kmarians/scratch/apps/fsl -f /tmp/tmpypxrahjp/fsl-6.0.7.7_linux-64.yml
[stderr]:
environment variables:
CIO_TEST=
CONDA_AUTO_UPDATE_CONDA=false
CONDA_ROOT=/mnt/shared/scratch/kmarians/apps/fsl
CURL_CA_BUNDLE=
LD_LIBRARY_PATH=/opt/slurm/latest/lib64
LD_PRELOAD=
MANPATH=/opt/slurm/latest/share/man/:/opt/slurm/latest/share/man/:/opt/slurm/l
atest/share/man/::
MODULEPATH=/etc/scl/modulefiles:/etc/scl/modulefiles:/etc/scl/modulefiles:/usr/sh
are/Modules/modulefiles:/etc/modulefiles:/usr/share/modulefiles
PATH=/opt/openmpi/latest/bin:/opt/slurm/latest/bin:/mnt/shared/apps/guppy/4
.2.2:/mnt/shared/apps/aspera/3.9.2/bin:/mnt/cluster/scripts:/mnt/share
d/scratch/kmarians/apps/conda/bin:/opt/openmpi/latest/bin:/opt/slurm/l
atest/bin:/mnt/shared/apps/guppy/4.2.2:/mnt/shared/apps/aspera/3.9.2/b
in:/mnt/cluster/scripts:/mnt/shared/scratch/kmarians/apps/conda/bin:/h
ome/kmarians/.local/bin:/home/kmarians/bin:/opt/openmpi/latest/bin:/op
t/slurm/latest/bin:/usr/share/Modules/bin:/mnt/shared/apps/guppy/4.2.2
:/mnt/shared/apps/aspera/3.9.2/bin:/mnt/cluster/scripts:/mnt/shared/sc
ratch/kmarians/apps/conda/condabin:/mnt/shared/scratch/kmarians/apps/c
onda/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
REQUESTS_CA_BUNDLE=
SSL_CERT_FILE=
[stderr]:
active environment : None
user config file : /home/kmarians/.condarc
populated config files : /mnt/shared/scratch/kmarians/apps/fsl/.condarc
/home/kmarians/.condarc
/home/kmarians/scratch/apps/fsl/.condarc
conda version : 23.11.0
conda-build version : not installed
python version : 3.10.13.final.0
solver : libmamba (default)
virtual packages : __archspec=1=skylake_avx512
__conda=23.11.0=0
__glibc=2.28=0
__linux=4.18.0=0
__unix=0=0
base environment : /mnt/shared/scratch/kmarians/apps/fsl (writable)
conda av data dir : /mnt/shared/scratch/kmarians/apps/fsl/etc/conda
conda av metadata url : None
channel URLs : public/linux-64
public/noarch
conda-forge/linux-64
conda-forge/noarch
package cache : /home/kmarians/scratch/apps/fsl/pkgs
envs directories : /mnt/shared/scratch/kmarians/apps/fsl/envs
/home/kmarians/.conda/envs
platform : linux-64
user-agent : conda/23.11.0 requests/2.31.0 CPython/3.10.13 Linux/4.18.0-513.11.1.el8_9.0.1.x86_64 rocky/8.9 glibc/2.28 solver/libmamba conda-libmamba-solver/23.12.0 libmambapy/1.5.5
UID:GID : 896600222:896600222
netrc file : None
offline mode : False
[stderr]:
[stderr]:
[stderr]: An unexpected error has occurred. Conda has prepared the above report.
[stderr]: If you suspect this error is being caused by a malfunctioning plugin,
[stderr]: consider using the --no-plugins option to turn off plugins.
[stderr]:
[stderr]: Example: conda --no-plugins install
[stderr]:
[stderr]: Alternatively, you can set the CONDA_NO_PLUGINS environment variable on
[stderr]: the command line to run the command without plugins enabled.
[stderr]:
[stderr]: Example: CONDA_NO_PLUGINS=true conda install
[stderr]:
libmamba Error when extracting package: std::bad_alloc
[stderr]: terminate called after throwing an instance of ‘std::bad_alloc’
what(): std::bad_alloc
This command returned an error: /home/kmarians/scratch/apps/fsl/bin/mamba env update -p /home/kmarians/scratch/apps/fsl -f /tmp/tmpypxrahjp/fsl-6.0.7.7_linux-64.yml

09:24:00 fslinstaller.py:2768: File “/mnt/shared/scratch/kmarians/apps/FSL/fslinstaller.py”, line 2760, in handle_error
yield
File “/mnt/shared/scratch/kmarians/apps/FSL/fslinstaller.py”, line 2875, in main
install_fsl(ctx)
File “/mnt/shared/scratch/kmarians/apps/FSL/fslinstaller.py”, line 2138, in install_fsl
ctx.run(Process.monitor_progress, cmd,
File “/mnt/shared/scratch/kmarians/apps/FSL/fslinstaller.py”, line 1586, in run
return process_func(admin=self.need_admin,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/mnt/shared/scratch/kmarians/apps/FSL/fslinstaller.py”, line 1056, in monitor_progress
raise RuntimeError('This command returned ’ …

(happy to include the entire log file if that helps)

paulmccarthy · February 26, 2024, 2:06pm

Hi @km345, this looks like some sort of a system issue - how much RAM does your system have, and how much free disk space do you have in your scratch folder? FSL needs over 10GB of space when installed.

km345 · February 26, 2024, 4:24pm

Hi @paulmccarthy, the head node has 6 Gb of RAM but I now requested a worker node with 32 Gb of RAM. Not sure how much disk space there is but it should not be a problem on a HPC. With 32 Gb RAM the installer fails at 6% with an error:
ERROR occurred during installation! [Errno 2] No such file or directory: '/home/kmarians/scratch/apps/fsl/pkgs/fsl-data_first_models_317_bin-2006.0-0.tar.bz2'

paulmccarthy · February 28, 2024, 8:20am

Can you share the log file?

I would confirm with your IT team how much storage space you have available - on a HPC it is common for per-user/group storage limits to be enforced.

km345 · February 28, 2024, 9:55am

Thanks @paulmccarthy, please see the log file. I am pretty certain that there is no storage limit in my scratch - I was always able to install software and download large files (including most recently many thousands fMRIs to be used with FSL).
fsl_installation_20240226172345.txt (184.6 KB)

paulmccarthy · February 28, 2024, 10:00am

Ok, I’ll trust that you have enough space. We have just released a new version of FSL - would you mind trying again (you should be able to simply re-run the same fslinstaller.py script)?

What kind of file system are you installing FSL to? It appears to be a network file system of some sort.

Jigi_Moudgil-Joshi · March 28, 2024, 8:54am

Hi Paul,

For some reason I am also having issues downloading FSL? I have attached the log file

Any help much appreciated! I downloaded but cannot seem to get the installer to work

paulmccarthy · March 28, 2024, 9:58am

Hi @Jigi_Moudgil-Joshi, it looks like the miniconda installer file is not being downloaded correctly (this seems to be the case in both of the log files that you uploaded). I’m not sure why, as going by the download timings in the log file, your internet connection appears to be quite fast - how are you connecting to the internet?

In any case, could you try downloading this file:

https://repo.anaconda.com/miniconda/Miniconda3-py311_23.11.0-2-MacOSX-arm64.sh

And then in a terminal run this command to calculate its checksum:

shasum -a 256 ~/Downloads/Miniconda3-py311_23.11.0-2-MacOSX-arm64.sh

The result should be:

5694c382e6056d62ed874f22692224c4f53bca22e8135b6f069111e081be07aa

If the checksum matches, you can try running the fslinstaller.py with this miniconda installer:

python fslinstaller.py --miniconda ~/Downloads/Miniconda3-py311_23.11.0-2-MacOSX-arm64.sh

Jigi_Moudgil-Joshi · March 28, 2024, 10:14am

Thank you so much - will give this a try later today

Using a 5G hub I have at home… would this impact anything

Jigi_Moudgil-Joshi · March 28, 2024, 9:47pm

All sorted! Thank you so so much

Jigi_Moudgil-Joshi · April 6, 2024, 3:07pm

Hi Paul,

When I went to go and download the unix dataset for practice from the website, the response was this:

David_F · May 14, 2024, 6:06am

Hello, I recently purchased a new MacBook Air (M3, Sonoma 14.4.1) and have tried to install FSL with no success (I have installed FSL on previous machines with no problem). The “Installing FSL into …” step keeps stalling at 93%, and then I get the following error:

ERROR occurred during installation!
This command returned an error: /Users/User/fsl/bin/conda env
update -p /Users/User/fsl -f /private/var/folders/hh/p779zgr17tb91xm578k
k0lwc0000gn/T/tmpij_787a2/fsl-6.0.7.11_macos-64.yml
Removing failed installation directory /Users/User/fsl
FSL installation failed!
Please check the log file - it may contain some more information to
help you diagnose the problem:
/Users/User/fsl_installation_20240514010933.log

I’ve tried downloading the installer from the link recently supplied, and ran the command that seem to work for others (python fslinstaller.py --miniconda https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh), but no success. Any suggestions?

paulmccarthy · May 14, 2024, 8:54am

Hi @David_F, have you checked the contents of the log file, as the error message suggests?

David_F · May 14, 2024, 9:06am

I did, but unfortunately don’t understand it. Can access the log here in case it’s helpful (won’t let me paste)?: Dropbox - fsl_installation_20240514010933.log - Simplify your life

paulmccarthy · May 14, 2024, 9:17am

Are you able to share a copy of the log file where you ran fslinstaller.py without any additional arguments? Passing an alternate miniconda installer should no longer be necessary.

paulmccarthy · May 14, 2024, 9:50am

Hi @David_F, actually I think there has been a change to conda which has broken FSL installations. Can you try re-running the fslinstaller.py script like this (don’t specify an alternative miniconda installer):

CONDA_AUTO_UPDATE_CONDA=false ./fslinstaller.py

and see if that allows the installation to succeed?

David_F · May 14, 2024, 10:12am

I get a permissions issue with that:

zsh: permission denied: ./fslinstaller.py

Am I missing something simple?

paulmccarthy · May 14, 2024, 10:18am

Sorry, I meant something like the following (using whatever python you used to run the installer previously):

CONDA_AUTO_UPDATE_CONDA=false python ./fslinstaller.py

David_F · May 14, 2024, 8:43pm

That works, thank you!

Shivani_Rajeshree · May 21, 2024, 2:18pm

keep getting this error…tried solving it with some suggestions I found online but nothing seems to work