How to run a single Mindboggle function in Docker? Tutorial jupyter notebook does not launch

kalemaria · June 5, 2019, 9:53pm

Dear Arno, dear all,

I was trying to run the function CurvatureMain from the Mindboggle Docker container, but the command was not found, neither “curvature”. Only “mindboggle” was found and seemed to partially work. However, I don’t need to run the whole workflow. Is it possible to run a single Mindboggle or FreeSurfer function in the Docker container, or do I have to install Mindboggle / FreeSurfer differently?
I have installed Docker on an Ubuntu 18.04.2 LTS virtual machine (Docker version 18.09.6, build 481bc77) and followed all the Mindboggle Docker installation instructions. If more details are needed please let me know.

First, I wanted to try to find an answer in the Mindboggle jupyter notebook tutorial. However, when I tried to run jupyter notebook from the Docker container as described in “Getting help” on the Mindboggle (github) page, an error occurred:
OSError: [Errno 99] Cannot assign requested address
I found here and here (issue 168) that one has to call jupyter notebook with options “–ip=0.0.0.0 –port=8080 --allow-root”, then jupyter notebook launched in the terminal, but not in a browser, and a NotebookApp error occurred:
No web browser found: could not locate runnable browser.
Copying the URL with the token to a browser did not work. Also “port=8888” and “–ip=127.0.0.1” (as advised here) and different combinations of the options did not help.
The issue 168 with a related problem to mine has latest answer: “We forgot to update this doc after switching to neurodocker. The command mindboggle123 has to be specified after the docker image name.” But I could not understand where one is supposed to specify “mindboggle123”.

I am looking forward to any help. Thanks in advance!
Ria

kalemaria · June 6, 2019, 1:12pm

Update: after trying desperately, the following worked to get to the tutorial:
First and I think it was the most important step, I had to run the docker and jupyter notebook in a single line:

docker run --rm -ti -v $HOST:/home/jovyan/work -p 8888:8888 nipy/mindboggle jupyter notebook /opt/mindboggle/docs/mindboggle_tutorial.ipynb --ip=0.0.0.0 –allow-root

Then jupyter started in the terminal, showing anyway the warning:

[W 12:44:16.191 NotebookApp] No web browser found: could not locate runnable browser.

Then, I copied and pasted the URL to my browser, but leaving only 127.0.0.1 in the option (what I already tried before, but it did not work when first entering docker and then running jupyter as described on github):
shown: http://(4a600b2d44ca or 127.0.0.1):8888/?token=
change to: http://127.0.0.1:8888/?token=

Hope this helps to someone else. Since it’s my first time using Docker, it took me really long to find the solution. Dear Mindboggle developers, please update this in your documentation! Now I will do the tutorial and try to solve my initial question.

binarybottle · June 9, 2019, 5:17pm

Thank you very much for writing, @kalemaria! I’m sorry you were running into difficulties and am happy you found a solution to get the jupyter notebook running. I have updated Mindboggle’s documentation to include the command you used:

docker run --rm -ti -v $HOST:/home/jovyan/work -p 8888:8888 nipy/mindboggle jupyter notebook /opt/mindboggle/docs/mindboggle_tutorial.ipynb --ip=0.0.0.0 –allow-root

and am trying to see if I can build a new docker image that doesn’t require you to run as root (https://github.com/nipy/mindboggle/issues/168).

As for your original problem, I don’t know why you wouldn’t be able to run CurvatureMain, or any of the surface shape measurement C++ code. Perhaps you are not running the full path, as in?:

/opt/vtk_cpp_tools/travel_depth/TravelDepthMain

You should also be able to run the python code that calls these C++ functions – take a look at the jupyter notebook tutorial for examples. NOTE: I just discovered that the version of VTK that conda installs is breaking the C++ code, so I am creating a new docker image with VTK=8.2, which seems to work. I just submitted an issue (https://github.com/nipy/mindboggle/issues/171) and am testing…

binarybottle · June 10, 2019, 12:57am

Installing VTK 8.2 resolved issue 171.

Please tell me if you are able to run CurvatureMain and other C++/python functions from within the docker container (or jupyter notebook tutorial).

kalemaria · June 10, 2019, 5:49pm

Dear Arno,

Thank you so much for your elaborate reply and for taking care of this issue!

You are right, I was not calling the function with the full path from the command line, sorry! I managed to run the example now.

I have already managed to run the curvature function using python code, after solving a tiny bug affecting methods 0 and 1:
when setting the output file names in mindboggle.shapes.surface_shapes.py lines 291-296, the variable stem was not defined, I solved it by defining it beforehand:

stem = os.path.join(os.getcwd(), basename)

(it was used for mean_curvature_file, but not as stem) I can also add it as an issue on github, if it’s easier.
I just found it a bit confusing that the mean curvature output file is not to be set by user as in the command line version, but it is written by default…

I will try to get your fixed Docker version and will rerun the tests.

binarybottle · June 10, 2019, 6:27pm

Thank you for pointing this out!

The default (mean) curvature file name was appropriately named, but not the non-default (gaussian, max, min) curvature files. Fixed in issue #180 and pushed to github. I will make a new docker container tonight.

If you find more issues, please submit via github. Thank you again!

kalemaria · June 10, 2019, 8:11pm

Great, thank you, too! I think it were enough issues for tonight. Just the last question here, because I’m not sure I’m doing it correctly: To update my docker image, should I run:

docker pull nipy/mindboggle

right? I think it worked last week, but when I’m trying this now for some reason I get the following error:

Using default tag: latest
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

Thanks again in advance!

binarybottle · June 10, 2019, 8:29pm

That is the correct command. My guess is that you’ve just got a bad connection.

kalemaria · June 11, 2019, 11:52am

I have just downloaded the latest image (from work, probably I had a bad connection at home yesterday evening).
However, to run the jupyter notebook tutorial I still needed the arguments --ip=0.0.0.0 and --allow-root, without both I got the initial error:

INFO: /root/matlab/startup.m does not exist … creating
[I 11:37:55.055 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
Traceback (most recent call last):
File “/opt/miniconda-latest/envs/mb/bin/jupyter-notebook”, line 11, in
sys.exit(main())
File “/opt/miniconda-latest/envs/mb/lib/python3.6/site-packages/jupyter_core/application.py”, line 266, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File “/opt/miniconda-latest/envs/mb/lib/python3.6/site-packages/traitlets/config/application.py”, line 657, in launch_instance
app.initialize(argv)
File “</opt/miniconda-latest/envs/mb/lib/python3.6/site-packages/decorator.py:decorator-gen-7>”, line 2, in initialize
File “/opt/miniconda-latest/envs/mb/lib/python3.6/site-packages/traitlets/config/application.py”, line 87, in catch_config_error
return method(app, *args, **kwargs)
File “/opt/miniconda-latest/envs/mb/lib/python3.6/site-packages/notebook/notebookapp.py”, line 1630, in initialize
self.init_webapp()
File “/opt/miniconda-latest/envs/mb/lib/python3.6/site-packages/notebook/notebookapp.py”, line 1407, in init_webapp
self.http_server.listen(port, self.ip)
File “/opt/miniconda-latest/envs/mb/lib/python3.6/site-packages/tornado/tcpserver.py”, line 151, in listen
sockets = bind_sockets(port, address=address)
File “/opt/miniconda-latest/envs/mb/lib/python3.6/site-packages/tornado/netutil.py”, line 174, in bind_sockets
sock.bind(sockaddr)
OSError: [Errno 99] Cannot assign requested address

and with --ip=0.0.0.0 but without --allow-root I got this output:

INFO: /root/matlab/startup.m does not exist … creating
[I 11:45:36.407 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
[C 11:45:37.223 NotebookApp] Running as root is not recommended. Use --allow-root to bypass.

UPDATE
Sorry, I was too quick, apparently I tried a few minutes before you updated the docker. I’ll try it again.

UPDATE 2
I have just seen that the issue #168 was not included yet in the new image, sorry for the misunderstanding! I reloaded the tutorial as currently described on the webpage, but when I click on “mindboggle_tutorial.ipynb” and it opens in the next tab, a new error comes:

Unreadable Notebook: /opt/mindboggle/docs/mindboggle_tutorial.ipynb NotJSONError('Notebook does not appear to be JSON: ‘{\n “cells”: [\n {\n “cell_type”: "m…’,)

I have submitted it as an issue#181 in github.

But curvature runs now like a charm, both from command line and with a python script, thank you for the quick fix!

binarybottle · June 11, 2019, 3:43pm

Fixed! Please docker pull nipy/mindboggle for 1.3.5