Using ABCD-Repronim Jupyterhub container locally via docker

Courses ABCD ReproNim
1

Students have access to an online version of Jupyterhub if they have an NDA DUC. All students and others can get the same environment locally via docker, including the ability to use singularity within the docker container. For installing docker see Docker Desktop overview | Docker Documentation.

mkdir -p abcd-repronim
docker run -it --rm --privileged -p 8888:8888 -v $(pwd)/abcd-repronim:/home/jovyan \
  satra/abcd-repronim:<tag> start.sh jupyter lab
  • Note that the specific tag <tag> will change throughout the course as we update the container. The <tag> in use will be updated here with the variable singleuser_image_tag: https://github.com/ABCD-ReproNim/reprohub/blob/reprohub/dandi-info/group_vars/all#L13
  • When this initializes you will notice an output that ends with http://127.0.0.1:8888/?token=43ea5391332129e84e6e87566089c88e429afe20a62d15fe (the token will be different each time you start). Paste this in your browser to access jupyterlab. This is the same interface that we are using in the online version.
  • Once you are done, you can hit Ctrl+C in the terminal and then y for yes to close down the session. if you have mounted the directory in the docker command and saved notebooks and files in the lab, your information will be preserved.
  • The abcd-repronim directory will be mounted inside and will preserve any changes you make in your home directory. The above command uses the full path. This is necessary.
  • The --privileged flag is necessary if you want to run singularity inside the container.

Linux specific issue

The user inside the docker container is jovyan (userid 1000). Thus this user is unable to write to the home directory since that linux directory has a different outside user id. One way to avoid this issue is to allow anyone to write to that directory.

mkdir test
chmod 700 test
mkdir test/home
chmod 777 test/home
docker ... -v $(pwd)/test/home:/home/jovyan ...

note the above command makes the first local directory inaccessible to group or others and makes the internal directory open to everyone. thus if you are on a shared system you will want to be very careful about who can see that path.

On a shared system with docker, please ensure that no one knows the full path. instead of calling the directory test you could use a uuid to minimize guessing.

On a personal machine this is not an issue if you are the only user.

Once you run the above command the files and directories created inside test/home will belong to some arbitrary user with uid:gid === 1000:1000. unless you can become that user or have root privileges on the system you will not be able to edit or delete the test/home directory. you will have to use the docker container and mount the parent directory (i.e. test) to some location (e.g. /outside_test) to be able to delete the contents.

5 Likes
2

Note: that container could be used as a reusable execution environment beyond just running jupyter lab. E.g. if you would like to use that container to not start jupyterlab but rather to run your script (e.g. let’s call it script.py) which you have developed while working on the hub, it should be possible to use that container but with a different entrypoint (e.g. could be your script), so with

$> cd abcd-repronim 
$> cat scripts/myscript.py 
#!/usr/bin/env python3
import datalad
print(f"I have imported datalad {datalad.__version__}")

I can do

$> docker run -it --rm --privileged --entrypoint scripts/myscript.py -v $(pwd):/home/jovyan/ satra/abcd-repronim:f6fc0ec5
It is highly recommended to configure Git before using DataLad. Set both 'user.name' and 'user.email' configuration variables.
It is highly recommended to configure Git before using DataLad. Set both 'user.name' and 'user.email' configuration variables.
I have imported datalad 0.14.0