Jupyter

From 2024 we provide options to either use the EasyBuild/lmod installed JupyterLab module, or install jupyter yourself using a Python virtual environment or a conda environment.

Using self-installed Python or Conda environments

By providing your own environment with Jupyter installed, you have full control over versions and dependencies. This can be useful if you for example require a different Python version than what we have available.

The environment must be created in a terminal before starting the Ondemand job. The procedure is straight forward:

• Create a new environment from scratch or use an existing one
• Install the jupyterlab package in the environment (This package also includes Notebook should you prefer that to Lab)

You can now start the Ondemand job.

This also makes it easy for a project to share an environment. Simply create it as required and copy it to the project folder. Ensure that file permissions are set appropriately.

Using the EasyBuild module

Leave "Use self-installed Jupyter" unchecked and select the JupyterLab module you prefer.

The JupyterLab module is dependent on several other software modulesin the Easybuild toolchain (compilers, libraries and so on). In many cases, this will cause version conflicts and other errors if care is not taken to maintain a clean environment.

One way of dealing with this is to create a new jupyter kernel using ipykernel in combination with tools like miniconda.

Example use case: Create a custom environment from scratch

In this example we will attempt to install and run a novel machine learning algorithm we found on GitHub. It doesn't use the same python version as we have installed, so we will create a fresh conda environment.

• Begin by launching a new Jupyter session, ask for at least a couple of hours of runtime since the installation can take some time.

• Once the session has started, unload the current IPython module clicking the jupyterlmod menu item om the left side. Then find IPython under "Loaded modules" and click "Unload".

This is necessary because certain python packages that are included in the module will cause conflicts with the environment we are installing. You will have to unload this module every time you want to run the newly created jupyter kernel.

• Open a terminal from the Jupyter launcher.
• Run the following sequence of commands:

## Clean out all modules to avoid conflicts
module purge

## Load in the Miniconda3 module to use for the installation

module load Miniconda3/22.11.1-1

## Fetch the source code and follow the installation instructions

git clone https://github.com/facebookresearch/ImageBind
cd ImageBind

conda create -n imagebind python=3.8
conda activate imagebind
pip install -r requirements.txt

## Now to create a new Jupyter kernel that has this environment:
pip install ipykernel
python -m ipykernel install --user --name imagebind-kernel 

## After testing, we see a few error messages regarding outdated jupyter libraries, so let's update:
pip install jupyter --upgrade

If you wait a minute (or reload Jupyter) the new kernel will turn up either on the launcher in Lab or under the "New" pulldown menu in Notebook. Start it to run the example code from the github page. For this code, you will need to change the working directory of the notebook so that the data files are found. To do this, use the following code:

import os
os.chdir("path/to/cloned_repository")

The you can run the example code and verify that the installation was successful.

Note that you must unload the IPython module again if you start a new Jupyter session, otherwise the new kernel will not be able to start due to version conflicts.