Nilearn development environment fails with memory error

JohnAtl · May 9, 2022, 7:21pm

When setting up the nilearn development environment according to the instructions, the process fails when I run the tests using:

pytest nilearn

Here’s the essence of the error:

E           ValueError: Memory profiler measured an untrustable memory consumption (0.00 MiB). The expected memory limit was 100.00 MiB. Try to bench with larger objects (at least 100MiB in memory).

So far, I haven’t found a way to ‘bench with larger objects’.
Any suggestions?

Steven · May 9, 2022, 7:28pm

What prompted the error?

JohnAtl · May 9, 2022, 7:38pm

Thanks!
Great question! I didn’t realize I left out all the context
Updated the question.

bthirion · May 10, 2022, 6:58am

Hi,
Can you provide more context (envrionment, full logs) ?
Thx,
Bertrand

JohnAtl · May 10, 2022, 2:29pm

Thanks!
This is what I know how to do. Happy to run other commands per your suggestions.

gist.github.com

https://gist.github.com/JohnAtl/6f95710caff3d71cd422edc1b92a1425

my_env.yml

name: nilearn
channels:
  - conda-forge
  - defaults
prefix: /usr/local/Caskroom/miniconda/base/envs/nilearn

pip_freeze.txt

alabaster==0.7.12
astor==0.8.1
attrs==21.4.0
Babel==2.9.1
backports.entry-points-selectable==1.1.1
beautifulsoup4==4.10.0
bids-validator==1.9.2
black==21.12b0
certifi==2021.10.8
cfgv==3.3.1

This file has been truncated. show original

pip_packages.txt

Package                           Version            Editable project location
--------------------------------- ------------------ --------------------------------------------
alabaster                         0.7.12
astor                             0.8.1
attrs                             21.4.0
Babel                             2.9.1
backports.entry-points-selectable 1.1.1
beautifulsoup4                    4.10.0
bids-validator                    1.9.2
black                             21.12b0

This file has been truncated. show original

There are more than three files. show original

bthirion · May 10, 2022, 7:35pm

@NicolasGensollen do you have any clue on that one ?
Best,
Bertrand

ymzayek · May 13, 2022, 1:12pm

Hi, I had a look but could not reproduce the error. It may be related to the specific operating system and/or special configurations of your machine. Can you provide some information about your operating system?
I am running:

  Operating System: Ubuntu 20.04.4 LTS
            Kernel: Linux 5.13.0-40-generic
      Architecture: x86-64

tested with:

python==3.9.12
nilearn==0.9.2.dev0

JohnAtl · May 13, 2022, 1:41pm

macOS Monterey 12.3.1 (21E258)
Python 3.9.12

(nilearn) Johns-iMac-Pro :: Research/fmri/nilearn » conda -V
conda 4.12.0
(nilearn) Johns-iMac-Pro :: Research/fmri/nilearn » python -m pip -V
pip 22.0.4 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

Darwin Johns-iMac-Pro.local 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18 00:45:05 PDT 2022; root:xnu-8020.101.4~15/RELEASE_X86_64 x86_64

platform darwin -- Python 3.9.12, pytest-7.0.1, pluggy-1.0.0 -- /usr/local/opt/python@3.9/bin/python3.9

In my fork, this is the latest tag that I have:

0.9.1  …
Verified
Release `0.9.1` (nilearn#3219)
      * REL 0.9.1

NicolasGensollen · May 16, 2022, 9:27am

Hi @JohnAtl

Thanks for reporting ! The issue happens in this test:

github.com

nilearn/nilearn/blob/e10a146c2a51e01ae609d2b55c7e6d764fa3c90e/nilearn/tests/test_testing.py#L31-L34

      
        
            # Ensure ValueError is raised if memory used is above expected memory
            # limit.
            with pytest.raises(ValueError, match="Memory consumption measured"):
                assert_memory_less_than(100, 0.1, create_object, 200 * 1024 ** 2)

which is basically creating an object of size 200 MiB and trying to assert that the memory used is less than 100 MiB, in order to verify that this error is correctly raised:

github.com

nilearn/nilearn/blob/e10a146c2a51e01ae609d2b55c7e6d764fa3c90e/nilearn/_utils/testing.py#L88-L92

      
        
            if mem_used > memory_limit * (1 + tolerance):
                raise ValueError("Memory consumption measured ({0:.2f} MiB) is "
                                 "greater than required memory limit ({1} MiB) within "
                                 "accepted tolerance ({2:.2f}%)."
                                 "".format(mem_used, memory_limit, tolerance * 100))

In your case, for some reason, the computed memory used (men_used ) seems to be smaller than 50 MiB, which is triggering this error instead:

github.com

nilearn/nilearn/blob/e10a146c2a51e01ae609d2b55c7e6d764fa3c90e/nilearn/_utils/testing.py#L94-L102

      
        
            # We are confident in memory_profiler measures above 100MiB.
            # We raise an error if the measure is below the limit of 50MiB to avoid
            # false positive.
            if mem_used < 50:
                raise ValueError("Memory profiler measured an untrustable memory "
                                 "consumption ({0:.2f} MiB). The expected memory "
                                 "limit was {1:.2f} MiB. Try to bench with larger "
                                 "objects (at least 100MiB in memory).".
                                 format(mem_used, memory_limit))

I don’t know yet why you get this result and I cannot reproduce the bug on my machine (M1 chip macOS Monterey 12.2.1, python 3.9.12).

One quick and dirty way to get rid of this error is to uninstall memory_profiler, which will skip this test.

Otherwise, I’d be curious to know what value of mem_used you have. You can add a breakpoint() above line 97 above and run pytest -v nilearn/tests/test_testing.py.

Hope this helps a little bit, and please let me know how it goes!
Best,
Nicolas