Error while building the singularity image of fMRIprep 20.2.1

,
1

The build of the singularity image of fMRIprep 20.2.1 didn’t succeed on HPC.

After launching the following line :

singularity build /my_images/fmriprep-20.2.1.simg docker://nipreps/fmriprep:20.2.1

I got the following error :

FATAL: While performing build: while creating SIF: while creating container: container file creation failed: open /my_images/fmriprep-20.2.1.simg: no such file or directory

I verified that the version of singularity that is installed on the HPC is higher than 2.5, so that’s not the problem.

What can I do to solve this ?

Thank you for your help.

PS Here is the full log :

$ singularity build /my_images/fmriprep-20.2.1.simg docker://nipreps/fmriprep:20.2.1
INFO: Starting build…
Getting image source signatures
Copying blob 0a01a72a686c done
Copying blob cc899a5544da done
Copying blob 19197c550755 done
Copying blob 716d454e56b6 done
Copying blob b5bf898e214a done
Copying blob 42da0942cc0e done
Copying blob 14f5757104e9 done
Copying blob 611fe4f705a5 done
Copying blob ac0b78389510 done
Copying blob 9e499ad45d56 done
Copying blob d9fe4a797d63 done
Copying blob 0096e55d11c8 done
Copying blob 8edcf0ca7093 done
Copying blob 4d1c726a5a26 done
Copying blob 6a13c4f0f499 done
Copying blob 4041fab309df done
Copying blob 76cb4738a4de done
Copying blob 9f64cb20b123 done
Copying blob 59feaf5d0228 done
Copying blob bf2e5cea3f0a done
Copying blob 0ecfa573ec6f done
Copying blob dca99f6f89e1 done
Copying blob 4ccd3fd697fd done
Copying blob ad3cb2354786 done
Copying blob c037d6ea6ece done
Copying blob 470e0d163955 done
Copying blob bff4925a9775 done
Copying config b8f6bbbbda done
Writing manifest to image destination
Storing signatures
2021/04/27 15:33:51 info unpack layer: sha256:0a01a72a686c389637334de1e2d0012da298960366f6d8f358b8e10dc3b5e330
2021/04/27 15:33:51 warn rootless{dev/agpgart} creating empty file in place of device 10:175
2021/04/27 15:33:51 warn rootless{dev/audio} creating empty file in place of device 14:4
2021/04/27 15:33:51 warn rootless{dev/audio1} creating empty file in place of device 14:20
2021/04/27 15:33:51 warn rootless{dev/audio2} creating empty file in place of device 14:36
2021/04/27 15:33:51 warn rootless{dev/audio3} creating empty file in place of device 14:52
2021/04/27 15:33:51 warn rootless{dev/audioctl} creating empty file in place of device 14:7
2021/04/27 15:33:51 warn rootless{dev/console} creating empty file in place of device 5:1
2021/04/27 15:33:51 warn rootless{dev/dsp} creating empty file in place of device 14:3
2021/04/27 15:33:51 warn rootless{dev/dsp1} creating empty file in place of device 14:19
2021/04/27 15:33:51 warn rootless{dev/dsp2} creating empty file in place of device 14:35
2021/04/27 15:33:51 warn rootless{dev/dsp3} creating empty file in place of device 14:51
2021/04/27 15:33:51 warn rootless{dev/full} creating empty file in place of device 1:7
2021/04/27 15:33:51 warn rootless{dev/kmem} creating empty file in place of device 1:2
2021/04/27 15:33:51 warn rootless{dev/loop0} creating empty file in place of device 7:0
2021/04/27 15:33:51 warn rootless{dev/loop1} creating empty file in place of device 7:1
2021/04/27 15:33:51 warn rootless{dev/loop2} creating empty file in place of device 7:2
2021/04/27 15:33:51 warn rootless{dev/loop3} creating empty file in place of device 7:3
2021/04/27 15:33:51 warn rootless{dev/loop4} creating empty file in place of device 7:4
2021/04/27 15:33:52 warn rootless{dev/loop5} creating empty file in place of device 7:5
2021/04/27 15:33:52 warn rootless{dev/loop6} creating empty file in place of device 7:6
2021/04/27 15:33:52 warn rootless{dev/loop7} creating empty file in place of device 7:7
2021/04/27 15:33:52 warn rootless{dev/mem} creating empty file in place of device 1:1
2021/04/27 15:33:52 warn rootless{dev/midi0} creating empty file in place of device 35:0
2021/04/27 15:33:52 warn rootless{dev/midi00} creating empty file in place of device 14:2
2021/04/27 15:33:52 warn rootless{dev/midi01} creating empty file in place of device 14:18
2021/04/27 15:33:52 warn rootless{dev/midi02} creating empty file in place of device 14:34
2021/04/27 15:33:52 warn rootless{dev/midi03} creating empty file in place of device 14:50
2021/04/27 15:33:52 warn rootless{dev/midi1} creating empty file in place of device 35:1
2021/04/27 15:33:52 warn rootless{dev/midi2} creating empty file in place of device 35:2
2021/04/27 15:33:52 warn rootless{dev/midi3} creating empty file in place of device 35:3
2021/04/27 15:33:52 warn rootless{dev/mixer} creating empty file in place of device 14:0
2021/04/27 15:33:52 warn rootless{dev/mixer1} creating empty file in place of device 14:16
2021/04/27 15:33:52 warn rootless{dev/mixer2} creating empty file in place of device 14:32
2021/04/27 15:33:52 warn rootless{dev/mixer3} creating empty file in place of device 14:48
2021/04/27 15:33:52 warn rootless{dev/mpu401data} creating empty file in place of device 31:0
2021/04/27 15:33:52 warn rootless{dev/mpu401stat} creating empty file in place of device 31:1
2021/04/27 15:33:52 warn rootless{dev/null} creating empty file in place of device 1:3
2021/04/27 15:33:52 warn rootless{dev/port} creating empty file in place of device 1:4
2021/04/27 15:33:52 warn rootless{dev/ram0} creating empty file in place of device 1:0
2021/04/27 15:33:52 warn rootless{dev/ram1} creating empty file in place of device 1:1
2021/04/27 15:33:52 warn rootless{dev/ram10} creating empty file in place of device 1:10
2021/04/27 15:33:52 warn rootless{dev/ram11} creating empty file in place of device 1:11
2021/04/27 15:33:52 warn rootless{dev/ram12} creating empty file in place of device 1:12
2021/04/27 15:33:52 warn rootless{dev/ram13} creating empty file in place of device 1:13
2021/04/27 15:33:52 warn rootless{dev/ram14} creating empty file in place of device 1:14
2021/04/27 15:33:52 warn rootless{dev/ram15} creating empty file in place of device 1:15
2021/04/27 15:33:52 warn rootless{dev/ram16} creating empty file in place of device 1:16
2021/04/27 15:33:52 warn rootless{dev/ram2} creating empty file in place of device 1:2
2021/04/27 15:33:52 warn rootless{dev/ram3} creating empty file in place of device 1:3
2021/04/27 15:33:52 warn rootless{dev/ram4} creating empty file in place of device 1:4
2021/04/27 15:33:52 warn rootless{dev/ram5} creating empty file in place of device 1:5
2021/04/27 15:33:52 warn rootless{dev/ram6} creating empty file in place of device 1:6
2021/04/27 15:33:52 warn rootless{dev/ram7} creating empty file in place of device 1:7
2021/04/27 15:33:52 warn rootless{dev/ram8} creating empty file in place of device 1:8
2021/04/27 15:33:52 warn rootless{dev/ram9} creating empty file in place of device 1:9
2021/04/27 15:33:52 warn rootless{dev/random} creating empty file in place of device 1:8
2021/04/27 15:33:52 warn rootless{dev/rmidi0} creating empty file in place of device 35:64
2021/04/27 15:33:52 warn rootless{dev/rmidi1} creating empty file in place of device 35:65
2021/04/27 15:33:52 warn rootless{dev/rmidi2} creating empty file in place of device 35:66
2021/04/27 15:33:52 warn rootless{dev/rmidi3} creating empty file in place of device 35:67
2021/04/27 15:33:52 warn rootless{dev/sequencer} creating empty file in place of device 14:1
2021/04/27 15:33:52 warn rootless{dev/smpte0} creating empty file in place of device 35:128
2021/04/27 15:33:52 warn rootless{dev/smpte1} creating empty file in place of device 35:129
2021/04/27 15:33:52 warn rootless{dev/smpte2} creating empty file in place of device 35:130
2021/04/27 15:33:52 warn rootless{dev/smpte3} creating empty file in place of device 35:131
2021/04/27 15:33:52 warn rootless{dev/sndstat} creating empty file in place of device 14:6
2021/04/27 15:33:52 warn rootless{dev/tty} creating empty file in place of device 5:0
2021/04/27 15:33:52 warn rootless{dev/tty0} creating empty file in place of device 4:0
2021/04/27 15:33:52 warn rootless{dev/tty1} creating empty file in place of device 4:1
2021/04/27 15:33:52 warn rootless{dev/tty2} creating empty file in place of device 4:2
2021/04/27 15:33:52 warn rootless{dev/tty3} creating empty file in place of device 4:3
2021/04/27 15:33:52 warn rootless{dev/tty4} creating empty file in place of device 4:4
2021/04/27 15:33:52 warn rootless{dev/tty5} creating empty file in place of device 4:5
2021/04/27 15:33:52 warn rootless{dev/tty6} creating empty file in place of device 4:6
2021/04/27 15:33:52 warn rootless{dev/tty7} creating empty file in place of device 4:7
2021/04/27 15:33:52 warn rootless{dev/tty8} creating empty file in place of device 4:8
2021/04/27 15:33:52 warn rootless{dev/tty9} creating empty file in place of device 4:9
2021/04/27 15:33:52 warn rootless{dev/urandom} creating empty file in place of device 1:9
2021/04/27 15:33:52 warn rootless{dev/zero} creating empty file in place of device 1:5
2021/04/27 15:34:02 info unpack layer: sha256:cc899a5544da1a6cfb970d2484d32c063f8df26a430d92f39c98e72261e226f2
2021/04/27 15:34:02 info unpack layer: sha256:19197c55075519928dd2ff059745665a2c9b72f4e8af6f7a1ce662e696d339bd
2021/04/27 15:34:02 info unpack layer: sha256:716d454e56b61d1343a01f3b1829574333e2e3df20e77d1958d7b0b939ea1b61
2021/04/27 15:34:02 info unpack layer: sha256:b5bf898e214a893171c1e1ab287fc7f1d3573e414869f21f06e3468fea43add3
2021/04/27 15:34:02 info unpack layer: sha256:42da0942cc0e3de8b86ba649f506f3a0c87533451756c51d7b8bf181ba94a2eb
2021/04/27 15:34:37 info unpack layer: sha256:14f5757104e98f3e93ab8930b4afd4eb21146c829f97cb5f942f31f67419dfc1
2021/04/27 15:34:39 info unpack layer: sha256:611fe4f705a58c51fd740b790122f3f5900386b72f3a4c5bed2f8bd6b5c28a19
2021/04/27 15:38:05 info unpack layer: sha256:ac0b78389510ddc5f05baf66cce3660bd59dcc6c40f868b039805e42f61755fb
2021/04/27 15:38:05 info unpack layer: sha256:9e499ad45d567b53c6c6b5a72f4b2d7b37da99751cbc4368d244f517c8951e4d
2021/04/27 15:38:41 info unpack layer: sha256:d9fe4a797d63bba1aae0765133adacf3a5facb66bdddb26bc87925308bfeeb82
2021/04/27 15:39:24 info unpack layer: sha256:0096e55d11c829debe30a66f703997ce80ea037ff805383bd044a964282a3c43
2021/04/27 15:39:25 info unpack layer: sha256:8edcf0ca7093145f01d7f3f491f684f94c344746494592fe2dcda225b3a1af0e
2021/04/27 15:39:37 info unpack layer: sha256:4d1c726a5a26b9313034bbc817ef3f068419e77d624685996c12b0c66a3b5df1
2021/04/27 15:39:43 info unpack layer: sha256:6a13c4f0f4998804e8f87d30cdaf819b2aa6c8832616bc5f8d3eda4473c09b66
2021/04/27 15:40:52 info unpack layer: sha256:4041fab309df595df3c73db5496fe891a545dcaeb157b97917d4768ed71289df
2021/04/27 15:40:52 info unpack layer: sha256:76cb4738a4de0a90bebd2feeffa33c2c924db77a15faa8c337b2f7243414059e
2021/04/27 15:41:30 info unpack layer: sha256:9f64cb20b1233c3f0145d0bbb0f7b21eb0ea379dcac88ffb7a6c2d33583bd55e
2021/04/27 15:43:56 info unpack layer: sha256:59feaf5d02284126e1b93f4f950e7dc713fd58c52d19fe9b77e2fefca93c87e2
2021/04/27 15:43:56 info unpack layer: sha256:bf2e5cea3f0a1120a050975921eeb2ca8f55a118d41915ecaff4f4178f05ed60
2021/04/27 15:43:56 info unpack layer: sha256:0ecfa573ec6ff0a2c316dc5d25a1933f80262e83aa53dafeda94f7ff6f017ca4
2021/04/27 15:43:56 info unpack layer: sha256:dca99f6f89e1c2a2c7c66d4f0897088bfc6b67984948c3ae6abd034bdec2e514
2021/04/27 15:44:18 info unpack layer: sha256:4ccd3fd697fdd50b44a023c6e1f937b528d885a3f87cd377fc29f9fc9928ce98
2021/04/27 15:44:19 info unpack layer: sha256:ad3cb2354786ee6a093167c7506c9b0382dabb513a9f8183a7dc69abad5592ec
2021/04/27 15:45:02 info unpack layer: sha256:c037d6ea6ece4d087069142e882f963003746a3aee1c19de007963dac465687f
2021/04/27 15:45:02 info unpack layer: sha256:470e0d163955845c8fdbe419c23361b31d65aea2c6885ff4f25c549ccd95d220
2021/04/27 15:45:07 info unpack layer: sha256:bff4925a97752c91821e817c10f4df35d169197b10426b159042927a157e7e15
INFO: Creating SIF file…
FATAL: While performing build: while creating SIF: while creating container: container file creation failed: open /my_images/fmriprep-20.2.1.simg: no such file or directory

2

Does the folder /my_images/ exist?

1 Like
3

If you are already in the directory containing the my_images folder, then you should leave off the leading “/”, since the leading “/” indicates an absolute path, not a relative path.

1 Like
4

@tsalo Indeed the folder /my_images/ doesn’t exist

5

The inexistence of the folder /my_images/ was indeed part of the problem. But even once this was solved, the build was not succesfull.

The problem, we suspect, is that to build the docker image using singularity, docker needs to be installed on the machine with root privileges which is not the case on our HPC.
The solution would be to build the image locally on my machine and upload it to the cluster.
I hope this might help someone encountering the same problem in the future.

6

What’s the error? You shouldn’t need Docker to install docker images with Singularity. But your workaround also is fine.

Best,
Steven

2 Likes
7

The terminal displayed no error ; the process INFO: Creating SIF file… never got accomplished and the job timed out. Note however that I didn’t run singularity in debug mode, so that may be why I couldn’t see an error message.

8

Building large images like that do take a while. Maybe you can run it as a batch job and devote some memory while explicitly making the length till timeout large.

1 Like
9

I devoted 6GB in total to this batch job and it got stuck for more than 24h.
Do you think it could take even more time than that or that the memory allocated was not enough ?

10

I would imagine 6GB is not enough. Considering the final image itself is ~5GB and memory is also devoted to collect the individual container components, I am surprised there wasn’t an out of memory notice (I guess singularity is efficient about memory storage). To be safe, I would try to devote at least 2X the memory of the container itself (so since fmriprep is about 5GB, try 10GB or ideally yourself some buffer with 12).

2 Likes
11

Thanks a lot @Steven for your help. Running the job with more memory allocated resulted in a successfull build indeed.

1 Like
12

Great, glad to hear it!

1 Like