This guide provides a straightforward approach to launching a Jupyter Lab server from a compute node of an HPC cluster (I am using the Leonardo for this tutorial). This tutorial is tested on
- Leonardo Cineca supercomputer
- SISSA Ulysses
- Moroccan Marwan cluster
the same steps are valid and should work for all the other clusters.
- Prior to proceeding, confirm that Jupyter Notebook is installed in your home directory on the login node. Once verified, follow these steps:
- Generate a configuration file for jupyter:s
(ZHPC) [zdahbi00@login01 jupyter-server]$ jupyter lab --generate-config
- Generate a hashed password that you will use to login to jupyter:
Enter password:
Verify password:
[JupyterPasswordApp] Wrote hashed password to /leonardo/home/userexternal/zdahbi00/.jupyter/jupyter_server_config.json
- Modify the configuration file
jupyter_lab_config.py
:
(ZHPC) [zdahbi00@login01 jupyter-server]$ vim ~/.jupyter/jupyter_lab_config.py
(ZHPC) [zdahbi00@login01 jupyter-server]$ head ~/.jupyter/jupyter_lab_config.py
# Configuration file for lab.
c = get_config() #noqa
c.NotebookApp.open_browser = False
c.NotebookApp.port = 6666
where 6666
is the port (you can change it: choose any 4 numbers!).
- Choose the resources you want to allocate by modifying the job script
jupy_leonardo.job
and execute the job.
(ZHPC) [zdahbi00@login01 jupyter-server]$ sbatch jupyter.job
- Run
squeue -u your_username
and get the hostname of the allocated node. For leonardo supercomputer, the hostnames of the compute nodes are something likelrdnxxxx
.
- Modify line 3 of the script
run_jupy.sh
by changing username with the username you use to login to the cluster as well as the domain name used to access the cluster.
#!/bin/sh
ssh -L 6666:$1:6666 -t username@login01-ext.leonardo.cineca.it 'jupyter lab'
- From the local terminal, run the script by passing the hostname of the node from the command line
(base) ➜ ~ ./run_jupy.sh lrdnxxxx
- If you have followed the steps correctly. You will be able to access jupyter notebook in the compute node from the browser through:
http://127.0.0.1:6666
You will see a case where you should put the password you set before and press Enter to access Jupyter Lab kernel of the compute nodes: