Many professional data scientists use Jupyter Notebook to accomplish their daily tasks, from preliminary data exploration to model prototyping. Notebooks' interactivity is particularly convenient for data-centric programming and their self-documenting nature provides excellent support for the communication of analytical results.
Nevertheless, Jupyter Notebook has been often criticized for inducing bad programming habits and scarcely supporting Software Engineering best practices. To really benefit from notebooks, users should be aware of their common pitfalls and learn how to prevent them.
In previous work (see "Eliciting Best Practices for Collaboration with Computational Notebooks" [1]), we introduced a catalog of 17 empirically-validated guidelines for the collaborative use of notebooks in a professional context.
To foster the adoption of these best practices, we have created Pynblint, a static analysis tool for Jupyter notebooks written in Python. Pynblint reveals potential notebook defects and recommends corrective actions. It can be operated either as a standalone CLI application or as part of a CI/CD pipeline.
The core linting rules of Pynblint have been derived as operationalizations of the best practices from our catalog. Nonetheless, the plug-in architecture of Pynblint enables its users to easily extend the core set of checks with their own linting rules.
Python 3.7+.
Pynblint can be installed with pip
or another PyPI package manager:
pip install pynblint
After installation, we recommend exploring the command-line interface of the tool:
pynblint --help
Pynblint can be used to analyze:
-
a standalone notebook:
pynblint path/to/the/notebook.ipynb
-
a code repository containing notebooks:
pynblint path/to/the/project/dir/
-
(possibly also compressed as a
.zip
archive):pynblint path/to/the/compressed/archive.zip
-
-
a public GitHub repository containing notebooks (support for private repositories is on our roadmap 🙂):
pynblint --from-github https://github.com/collab-uniba/pynblint
For further information on the available options, please refer to the project documentation.
Pynblint currently implements 22 linting rules:
- 17 rules applicable to standalone notebooks;
- 5 rules applicable to entire data science repositories.
These rules are derived from our comprehensive catalog of best practices for collaborative notebook development [1]. Below is the current implementation state of linting rules associated with each best practice:
- ✅ Complete: Fully implemented
- 🧩 Partial: Partially implemented with planned improvements
- ⏳ In Progress: Implementation planned for future releases
- ❌ Not Planned: No implementation planned
Best Practice from [1] | Status | Details |
---|---|---|
1. Use version control | ✅ Complete | Fully implemented |
2. Manage project dependencies | ✅ Complete | Fully implemented |
3. Use self-contained environments | ⏳ In Progress: | Planned: detection of - Python virtual environments (e.g., created with venv , pyenv , or conda )- containerized environments (e.g., Docker). |
4. Put imports at the beginning | ✅ Complete | Fully implemented |
5. Ensure re-executability (re-run notebooks top to bottom) | ✅ Complete | Fully implemented |
6. Modularize your code | 🧩 Partial | Current: detection of modularization constructs in notebooks. Planned: modularization-focused refactoring recommendations based on detected patterns |
7. Test your code | 🧩 Partial | Current: repository-level detection of coverage data. Planned: detection of test modules/functions independent of coverage tools |
8. Name your notebooks consistently | ✅ Complete | Fully implemented |
9. Stick to coding standards | 🧩 Partial | Current: detection of cells with invalid Python syntax. Planned: Integration of Python linters such as pylint, flake8, and ruff |
10. Use relative paths | ⏳ In Progress: | Planned: identification of absolute paths instances in notebooks and recommendation of corresponding relative paths |
11. Document your analysis | ✅ Complete | Fully implemented |
12. Leverage Markdown headings to structure your notebook | ✅ Complete | Fully implemented |
13. Keep your notebook clean | ✅ Complete | Fully implemented |
14. Keep your notebook concise | ✅ Complete | Fully implemented |
15. Distinguish production and development artifacts | ❌ Not Planned | Implementation deemed infeasible (see the note below) |
16. Make your notebooks available | ⏳ In Progress: | Planned: Pynblint will ensure that notebooks marked as published in its configuration are available online |
17. Make your data available | ✅ Complete | Fully implemented |
Note: Best practice #15 (Distinguishing Production/Development Artifacts) will not be implemented as there are no objective criteria for determining this distinction through static analysis of notebooks or repositories.
This project is licensed under the terms of the MIT license.
[1] Luigi Quaranta, Fabio Calefato, and Filippo Lanubile. 2022. Eliciting Best Practices for Collaboration with Computational Notebooks. Proc. ACM Hum.-Comput. Interact. 6, CSCW1, Article 87 (April 2022), 41 pages. https://doi.org/10.1145/3512934