Prep for Release 2.3.0

- Added release notes and freeze file. - Updated the README with the new Release number and changed the list of releases to Previous Releases. - Bumped the dependency versions. - Updated the year for requirements.in - Updated release_checklist.md with some simpler steps for committing release notes and changes in the development environment. Signed-off-by: Nisha K <nishak@vmware.com>
tern-tools · Nov 20, 2020 · 924f748 · 924f748
1 parent bb38e14
commit 924f748
Show file tree

Hide file tree

Showing 6 changed files with 233 additions and 21 deletions.
diff --git a/README.md b/README.md
@@ -288,11 +288,11 @@ $ python tests/<test file>.py
 ```
 
 ## Project Status<a name="project-status"/>
-Release 2.2.0 is out! See the [release notes](docs/releases/v2_2_0.md) for more information.
+Release 2.3.0 is out! See the [release notes](docs/releases/v2_3_0.md) for more information.
 
 We try to keep the [project roadmap](./docs/project-roadmap.md) as up to date as possible. We are currently working on Release 3.0.0.
 
-## Releases
+## Previous Releases
 * [v2.2.0](docs/releases/v2_2_0.md)
 * [v2.1.0](docs/releases/v2_1_0.md)
 * [v2.0.0](docs/releases/v2_0_0.md)

diff --git a/docs/releases/release_checklist.md b/docs/releases/release_checklist.md
@@ -5,20 +5,21 @@ This is a checklist for cutting a release
 - [ ] Prepare Release PR.
     * Freeze development on master.
     * Prepare your local development environment by committing or stashing your changes. Work at the tip of master.
+    * Create a branch for the release: `git checkout -b <release branch name>`.
     * In a separate folder, create a fresh environment and activate it.
     * Clone the `tern/master` repository by running `git clone --single-branch git@github.com:tern-tools/tern.git` and `cd` into it.
-    * Create a branch for the release: `git checkout -b <release branch name>`.
 
 - [ ] Update direct dependencies and run tests.
-    * Run `pip install wheel pip-tools twine`.
+    * In the fresh environment, run `pip install wheel pip-tools twine`.
     * Run `pip-compile --upgrade --output-file upgrade.txt`.
-    * Compare the dependency versions from the output of the pip-compile command to the current dependency versions listed in the `requirements.txt` file. Upgrade `requirements.txt` if necessary.
-    * Run `pip install .` to install tern.
+    * Compare the module versions in upgrade.txt with requirements.txt in the development environment. Bump up versions if needed.
+    * In the fresh environment, run `pip install .` to install tern.
     * Run appropriate tests. Roll back requirements if necessary.
     * When satisfied, run `pip-compile --generate-hashes --output-file v<release>-requirements.txt` where <release> is of the form `major_minor_patch`.
+    * Copy this file to the `docs/releases/` folder in the development environment.
 
 - [ ] Write release notes.
-    * Create a new file for the release notes: `docs/releases/v<release>.md`
+    * In the development environment, create a new file for the release notes: `docs/releases/v<release>.md`
     * If you are writing release notes for a patched release, only include:
       - A link to the primary release notes.
       - A brief summary of what the patched release changes do.
@@ -39,15 +40,8 @@ This is a checklist for cutting a release
 
     * Update the Project Status part of the README.md to reflect this release and add it to the list of releases.
 
-- [ ] Commit release notes and create patch for your changes
-    * `git add` and `git commit` any changes. This will likely include`v<release>-requirements.txt`, any changes to `requirements.txt` and `v<release>.md`. **Do not push these changes to master!**
-    * Run `git format-patch -n1`. This will create a patch file of the release changes you just committed called `0001-<commit_title>.patch`.
-    * Open a new terminal and `cd` into a development virtual environment that contains your forked version of the Tern repo. `cd` into the forked Tern repo directory.
-    * Create a new branch. You will use this branch to submit a PR for the release changes.
-    * Copy the patch file you just created into your new forked repo environment.
-    * Run `git am 0001-<commit_message_title>.patch`.
-    * Run `git push origin <branch-you-created>` to push the changes to your forked repo.
-    * The changes are now available in your forked repo. You can verify this by running `git log` and looking at the top commit from the output.
+- [ ] Commit release notes and submit a PR
+    * `git add` and `git commit` any changes. This will likely include`v<release>-requirements.txt`, any changes to `requirements.txt` and `v<release>.md`.
     * Open a pull request in the Tern project repository for your release changes.
     * Request a review from another maintainer. Update PR as needed based on feedback. Merge the PR. This commit is where the release will be tagged.
 

diff --git a/docs/releases/v2_3_0-requirements.txt b/docs/releases/v2_3_0-requirements.txt
@@ -0,0 +1,116 @@
+#
+# This file is autogenerated by pip-compile
+# To update, run:
+#
+#    pip-compile --generate-hashes --output-file=v2_3_0-requirements.txt
+#
+attrs==20.3.0 \
+    --hash=sha256:31b2eced602aa8423c2aea9c76a724617ed67cf9513173fd3a4f03e3a929c7e6 \
+    --hash=sha256:832aa3cde19744e49938b91fea06d69ecb9e649c93ba974535d08ad92164f700 \
+    # via debut
+certifi==2020.11.8 \
+    --hash=sha256:1f422849db327d534e3d0c5f02a263458c3955ec0aae4ff09b95f195c59f4edd \
+    --hash=sha256:f05def092c44fbf25834a51509ef6e631dc19765ab8a57b4e7ab85531f0a9cf4 \
+    # via requests
+chardet==3.0.4 \
+    --hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae \
+    --hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691 \
+    # via debut, requests
+debut==0.9.8 \
+    --hash=sha256:b353e1d826d0be80a7268762efd99ba05f9d1df1aef0553fb7ea17c670bee85c \
+    --hash=sha256:edd4ff3d265ca5bf645c73d6863a886d34743152d215a5de094c4d31fa6943e3 \
+    # via -r requirements.in
+docker==4.3.1 \
+    --hash=sha256:13966471e8bc23b36bfb3a6fb4ab75043a5ef1dac86516274777576bed3b9828 \
+    --hash=sha256:bad94b8dd001a8a4af19ce4becc17f41b09f228173ffe6a4e0355389eef142f2 \
+    # via -r requirements.in
+dockerfile-parse==1.1.0 \
+    --hash=sha256:80ea4b88694ab014001e39e62335aa2f4feb695b80de751377e994a344fa5952 \
+    --hash=sha256:f37bfa327fada7fad6833aebfaac4a3aaf705e4cf813b737175feded306109e8 \
+    # via -r requirements.in
+idna==2.10 \
+    --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \
+    --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 \
+    # via requests
+pbr==5.5.1 \
+    --hash=sha256:5fad80b613c402d5b7df7bd84812548b2a61e9977387a80a5fc5c396492b13c9 \
+    --hash=sha256:b236cde0ac9a6aedd5e3c34517b423cd4fd97ef723849da6b0d2231142d89c00 \
+    # via -r requirements.in, stevedore
+pyyaml==5.3.1 \
+    --hash=sha256:06a0d7ba600ce0b2d2fe2e78453a470b5a6e000a985dd4a4e54e436cc36b0e97 \
+    --hash=sha256:240097ff019d7c70a4922b6869d8a86407758333f02203e0fc6ff79c5dcede76 \
+    --hash=sha256:4f4b913ca1a7319b33cfb1369e91e50354d6f07a135f3b901aca02aa95940bd2 \
+    --hash=sha256:6034f55dab5fea9e53f436aa68fa3ace2634918e8b5994d82f3621c04ff5ed2e \
+    --hash=sha256:69f00dca373f240f842b2931fb2c7e14ddbacd1397d57157a9b005a6a9942648 \
+    --hash=sha256:73f099454b799e05e5ab51423c7bcf361c58d3206fa7b0d555426b1f4d9a3eaf \
+    --hash=sha256:74809a57b329d6cc0fdccee6318f44b9b8649961fa73144a98735b0aaf029f1f \
+    --hash=sha256:7739fc0fa8205b3ee8808aea45e968bc90082c10aef6ea95e855e10abf4a37b2 \
+    --hash=sha256:95f71d2af0ff4227885f7a6605c37fd53d3a106fcab511b8860ecca9fcf400ee \
+    --hash=sha256:ad9c67312c84def58f3c04504727ca879cb0013b2517c85a9a253f0cb6380c0a \
+    --hash=sha256:b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d \
+    --hash=sha256:cc8955cfbfc7a115fa81d85284ee61147059a753344bc51098f3ccd69b0d7e0c \
+    --hash=sha256:d13155f591e6fcc1ec3b30685d50bf0711574e2c0dfffd7644babf8b5102ca1a \
+    # via -r requirements.in
+regex==2020.11.13 \
+    --hash=sha256:02951b7dacb123d8ea6da44fe45ddd084aa6777d4b2454fa0da61d569c6fa538 \
+    --hash=sha256:0d08e71e70c0237883d0bef12cad5145b84c3705e9c6a588b2a9c7080e5af2a4 \
+    --hash=sha256:1862a9d9194fae76a7aaf0150d5f2a8ec1da89e8b55890b1786b8f88a0f619dc \
+    --hash=sha256:1ab79fcb02b930de09c76d024d279686ec5d532eb814fd0ed1e0051eb8bd2daa \
+    --hash=sha256:1fa7ee9c2a0e30405e21031d07d7ba8617bc590d391adfc2b7f1e8b99f46f444 \
+    --hash=sha256:262c6825b309e6485ec2493ffc7e62a13cf13fb2a8b6d212f72bd53ad34118f1 \
+    --hash=sha256:2a11a3e90bd9901d70a5b31d7dd85114755a581a5da3fc996abfefa48aee78af \
+    --hash=sha256:2c99e97d388cd0a8d30f7c514d67887d8021541b875baf09791a3baad48bb4f8 \
+    --hash=sha256:3128e30d83f2e70b0bed9b2a34e92707d0877e460b402faca908c6667092ada9 \
+    --hash=sha256:38c8fd190db64f513fe4e1baa59fed086ae71fa45083b6936b52d34df8f86a88 \
+    --hash=sha256:3bddc701bdd1efa0d5264d2649588cbfda549b2899dc8d50417e47a82e1387ba \
+    --hash=sha256:4902e6aa086cbb224241adbc2f06235927d5cdacffb2425c73e6570e8d862364 \
+    --hash=sha256:49cae022fa13f09be91b2c880e58e14b6da5d10639ed45ca69b85faf039f7a4e \
+    --hash=sha256:56e01daca75eae420bce184edd8bb341c8eebb19dd3bce7266332258f9fb9dd7 \
+    --hash=sha256:5862975b45d451b6db51c2e654990c1820523a5b07100fc6903e9c86575202a0 \
+    --hash=sha256:6a8ce43923c518c24a2579fda49f093f1397dad5d18346211e46f134fc624e31 \
+    --hash=sha256:6c54ce4b5d61a7129bad5c5dc279e222afd00e721bf92f9ef09e4fae28755683 \
+    --hash=sha256:6e4b08c6f8daca7d8f07c8d24e4331ae7953333dbd09c648ed6ebd24db5a10ee \
+    --hash=sha256:717881211f46de3ab130b58ec0908267961fadc06e44f974466d1887f865bd5b \
+    --hash=sha256:749078d1eb89484db5f34b4012092ad14b327944ee7f1c4f74d6279a6e4d1884 \
+    --hash=sha256:7913bd25f4ab274ba37bc97ad0e21c31004224ccb02765ad984eef43e04acc6c \
+    --hash=sha256:7a25fcbeae08f96a754b45bdc050e1fb94b95cab046bf56b016c25e9ab127b3e \
+    --hash=sha256:83d6b356e116ca119db8e7c6fc2983289d87b27b3fac238cfe5dca529d884562 \
+    --hash=sha256:8b882a78c320478b12ff024e81dc7d43c1462aa4a3341c754ee65d857a521f85 \
+    --hash=sha256:8f6a2229e8ad946e36815f2a03386bb8353d4bde368fdf8ca5f0cb97264d3b5c \
+    --hash=sha256:9801c4c1d9ae6a70aeb2128e5b4b68c45d4f0af0d1535500884d644fa9b768c6 \
+    --hash=sha256:a15f64ae3a027b64496a71ab1f722355e570c3fac5ba2801cafce846bf5af01d \
+    --hash=sha256:a3d748383762e56337c39ab35c6ed4deb88df5326f97a38946ddd19028ecce6b \
+    --hash=sha256:a63f1a07932c9686d2d416fb295ec2c01ab246e89b4d58e5fa468089cab44b70 \
+    --hash=sha256:b2b1a5ddae3677d89b686e5c625fc5547c6e492bd755b520de5332773a8af06b \
+    --hash=sha256:b2f4007bff007c96a173e24dcda236e5e83bde4358a557f9ccf5e014439eae4b \
+    --hash=sha256:baf378ba6151f6e272824b86a774326f692bc2ef4cc5ce8d5bc76e38c813a55f \
+    --hash=sha256:bafb01b4688833e099d79e7efd23f99172f501a15c44f21ea2118681473fdba0 \
+    --hash=sha256:bba349276b126947b014e50ab3316c027cac1495992f10e5682dc677b3dfa0c5 \
+    --hash=sha256:c084582d4215593f2f1d28b65d2a2f3aceff8342aa85afd7be23a9cad74a0de5 \
+    --hash=sha256:d1ebb090a426db66dd80df8ca85adc4abfcbad8a7c2e9a5ec7513ede522e0a8f \
+    --hash=sha256:d2d8ce12b7c12c87e41123997ebaf1a5767a5be3ec545f64675388970f415e2e \
+    --hash=sha256:e32f5f3d1b1c663af7f9c4c1e72e6ffe9a78c03a31e149259f531e0fed826512 \
+    --hash=sha256:e3faaf10a0d1e8e23a9b51d1900b72e1635c2d5b0e1bea1c18022486a8e2e52d \
+    --hash=sha256:f7d29a6fc4760300f86ae329e3b6ca28ea9c20823df123a2ea8693e967b29917 \
+    --hash=sha256:f8f295db00ef5f8bae530fc39af0b40486ca6068733fb860b42115052206466f \
+    # via -r requirements.in
+requests==2.25.0 \
+    --hash=sha256:7f1a0b932f4a60a1a65caa4263921bb7d9ee911957e0ae4a23a6dd08185ad5f8 \
+    --hash=sha256:e786fa28d8c9154e6a4de5d46a1d921b8749f8b74e28bde23768e5e16eece998 \
+    # via -r requirements.in, docker
+six==1.15.0 \
+    --hash=sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259 \
+    --hash=sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced \
+    # via docker, dockerfile-parse
+stevedore==3.2.2 \
+    --hash=sha256:5e1ab03eaae06ef6ce23859402de785f08d97780ed774948ef16c4652c41bc62 \
+    --hash=sha256:f845868b3a3a77a2489d226568abe7328b5c2d4f6a011cc759dfa99144a521f0 \
+    # via -r requirements.in
+urllib3==1.26.2 \
+    --hash=sha256:19188f96923873c92ccb987120ec4acaa12f0461fa9ce5d3d0772bc965a39e08 \
+    --hash=sha256:d8ff90d979214d7b4f8ce956e80f4028fc6860e4431f731ea4a8c08f23f99473 \
+    # via requests
+websocket-client==0.57.0 \
+    --hash=sha256:0fc45c961324d79c781bab301359d5a1b00b13ad1b10415a4780229ef71a5549 \
+    --hash=sha256:d735b91d6d1692a6a181f2a8c9e0238e5f6373356f561bb9dc4c7af36f452010 \
+    # via docker
diff --git a/docs/releases/v2_3_0.md b/docs/releases/v2_3_0.md
@@ -0,0 +1,102 @@
+# Release 2.3.0
+
+## Summary
+This release contains a big code refactor which fixed a good number of technical debt issues. It also delivers support for [multistage Dockerfiles](https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds), which is valuable as Docker removes the intermediate stages leaving only the final deploy container image to analyze. Teams building applications using this method can now get a Sofware Bill of Materials for each stage. A special thanks to Junlai Wang (@ForgetMe17 on GitHub) for laying the groundwork to finally implement this feature. We also have a Dockerfile for building Tern with Scancode-Toolkit. To build this image, simply run `docker built -t ternscancode -f docker/Dockerfile.scancode .` and then `docker_run.sh ternscancode "report -x scancode -i <image:tag>"`. Thanks to Jeroen Knoops (@JeroenKnoops on GitHub) for contributing this Dockerfile.
+
+A note about this release: Although this is a minor version bump, the short `-d` for `--driver` is now `-dr` to prevent confusion between `-d` for passing a Dockerfile.
+
+As always, we would like to thank our community for contributing to this release.
+
+## New Features
+* [Preliminary support for multistage Dockerfiles](https://github.com/tern-tools/tern/issues/612): Tern can now generate reports in HTML, JSON, YAML and human-readable formats for multistage Dockerfiles. Note that this is the case only for Dockerfiles, not container images that may have been built using Dockerfiles. We think this is pretty cool!
+
+## Bug Fixes
+* [Fix crash when an image is not found by the Docker API](https://github.com/tern-tools/tern/issues/828)
+* [Fix crash when a script invocation fails](https://github.com/tern-tools/tern/issues/822)
+* [Fix parsing tabs in a Docker image's created_by value](https://github.com/tern-tools/tern/issues/812)
+* Many bugs were fixed as a result of the code refactor.
+
+
+## Resolved Technical Debt
+* Parts of a larger code refactor:
+  * [Move container pull and dump operations to a new module](https://github.com/tern-tools/tern/issues/802)
+  * [Move setup and teardown checks into a new module](https://github.com/tern-tools/tern/issues/808)
+  * [Re-organize tern/analyze folder](https://github.com/tern-tools/tern/issues/803)
+* [Resolving all code complexity debt](https://github.com/tern-tools/tern/issues/789)
+
+## Future Work
+* A "step" subcommand to step through container image layers and analyze them individually.
+* Analysis for OCI style images.
+* Continuing code cleanup
+
+The next release will be a Beta release 3.0.0. Since it will be the first in 2021, and the US holidays are upon us, expect the next release by March or April. Watch the [Beta Release Milestone](https://github.com/tern-tools/tern/milestone/13) for progress. We're really excited about this release!
+
+## Changelog
+
+Note: This changelog will not include these release notes
+
+Changelog generated by command: `git log --pretty=format:"%h %s" v2.1.0..master`
+
+```
+bb38e14 merge: Enable analysis for multistage Dockerfiles
+906edac Fix ci build for locking a Dockerfile
+24b4e51 Fixes for reading and writing Dockerfiles
+daab1d4 Fix Dockerfile build with context
+142c74e Enable multistage Dockerfile analysis
+453fad6 Replace the short driver option with -dr
+4ca9b88 Add subroutine to analyze multistage Dockerfiles
+3e2325e Update code navigation document
+a8ec222 Add Dockerfile for scancode
+ad2b97c Add 'apt' Snippet In Command Library
+e420355 Fix crash when a chroot command fails
+e33357d Fix Dockerfile analysis if no base image is found
+1621437 Gracefully exit if there is no image to analyze
+222a138 Fix unbound local error when repo digest is given
+cfb8d10 Recognize assignments before command in script
+14c2dca merge: Organize code under tern/analyze
+85bbd09 Fix tests after refactor
+e7b3b6a Shorten fill_package_metadata function
+0c0d587 Re-enable Dockerfile lock
+f0ff818 Fix operation errors after refactor
+fe1de25 Refactor functions with too many branches
+716b1e0 Complete Dockerfile analysis
+a991b0f Fix multi-layer container analysis
+c2e8dfa Fix single layer analysis
+5f24e3e More moving of code into logical places
+43f64af Organized code in the analyze folder
+2f5f4c6 Move multi-layer analysis to default
+e8a8228 Move command_lib into default and organize
+4b67c87 Create new folder for default operation
+9b181d3 merge: Move external interactions to load directory
+21156d0 Remove container.py and some deprecated functions
+5681dac Fix checksum parsing and Dockerfile building
+5f4b0f5 Fixed tests and linting for common.py and Package
+90cd6cb Fix loading package files from cache
+5706b2b Hook up docker_api to setup and teardown
+70fdc09 classes: Use load functions in DockerImage
+c5cc233 load: New code section for external interactions
+338fde3 merge: Map layer files to packages
+056c309 Fix error caused by tabs in ENV
+bebbb18 Add file info for packages
+2d29c8d Extract file info for packages
+d561fce docs: Add GitHub Action link in README
+1139109 ci: Update python version for GHA
+7f6ab45 Refactor Dockerfiles
+```
+
+## Contributors
+
+```
+asifjoardar mrsparrow04@gmail.com
+HeroicHitesh hiteshkumar_1mv17cs042@sirmvit.edu
+Isac Sund isac@isacsund.com
+Jeroen Knoops jeroen.knoops@philips.com
+PrajwalM2212 prajwalmmath@gmail.com
+WangJL hazard15020@gmail.com
+Yann Jorelle yann.jorelle@nokia.com
+```
+
+## Contact the Maintainers
+
+Nisha Kumar: nishak@vmware.com
+Rose Judge: rjudge@vmware.com
diff --git a/requirements.in b/requirements.in
@@ -1,4 +1,4 @@
-# Copyright (c) 2019 VMware, Inc. All Rights Reserved.
+# Copyright (c) 2019-2020 VMware, Inc. All Rights Reserved.
 # SPDX-License-Identifier: BSD-2-Clause
 #
 # This file is used by pip-tools for release management

diff --git a/requirements.txt b/requirements.txt
@@ -8,9 +8,9 @@
 
 PyYAML>=5.3
 docker~=4.3
-dockerfile-parse~=1.0
-requests~=2.24
+dockerfile-parse~=1.1
+requests~=2.25
 stevedore>=3.2
-pbr>=5.4
+pbr>=5.5
 debut>=0.9
-regex>=2020.7
+regex>=2020.11