Prep for Release 2.2.0

- Added release notes and freeze file. - Updated README with new Release. - Bumped dependency versions. - Added some clarity to the release checklist. Signed-off-by: Nisha K <nishak@vmware.com>
tern-tools · Aug 27, 2020 · abbd1bb · abbd1bb
1 parent a0dd8cd
commit abbd1bb
Show file tree

Hide file tree

Showing 5 changed files with 239 additions and 13 deletions.
diff --git a/README.md b/README.md
@@ -284,11 +284,12 @@ $ python tests/<test file>.py
 ```
 
 ## Project Status<a name="project-status"/>
-Release 2.1.0 is out! See the [release notes](docs/releases/v2_1_0.md) for more information.
+Release 2.2.0 is out! See the [release notes](docs/releases/v2_2_0.md) for more information.
 
-We try to keep the [project roadmap](./docs/project-roadmap.md) as up to date as possible. We are currently working on Release 2.2.0.
+We try to keep the [project roadmap](./docs/project-roadmap.md) as up to date as possible. We are currently working on Release 3.0.0.
 
 ## Releases
+* [v2.2.0](docs/releases/v2_2_0.md)
 * [v2.1.0](docs/releases/v2_1_0.md)
 * [v2.0.0](docs/releases/v2_0_0.md)
 * [v1.0.1](docs/releases/v1_0_1.md)

diff --git a/docs/releases/release_checklist.md b/docs/releases/release_checklist.md
@@ -4,17 +4,18 @@ This is a checklist for cutting a release
 
 - [ ] Prepare Release PR.
     * Freeze development on master.
-    * Create a fresh environment and activate it.
-    * Clone the `tern/master` repository and `cd` into it.
-    * Create a branch for the release.
+    * Prepare your local development environment by committing or stashing your changes. Work at the tip of master.
+    * In a separate folder, create a fresh environment and activate it.
+    * Clone the `tern/master` repository by running `git clone --single-branch git@github.com:tern-tools/tern.git` and `cd` into it.
+    * Create a branch for the release: `git checkout -b <release branch name>`.
 
 - [ ] Update direct dependencies and run tests.
     * Run `pip install wheel pip-tools twine`.
-    * Run `pip-compile --upgrade`.
+    * Run `pip-compile --upgrade --output-file upgrade.txt`.
     * Compare the dependency versions from the output of the pip-compile command to the current dependency versions listed in the `requirements.txt` file. Upgrade `requirements.txt` if necessary.
     * Run `pip install .` to install tern.
     * Run appropriate tests. Roll back requirements if necessary.
-    * When satisfied, run `pip-compile --generate-hashes --output-file docs/releases/v<release>-requirements.txt`.
+    * When satisfied, run `pip-compile --generate-hashes --output-file v<release>-requirements.txt` where <release> is of the form `major_minor_patch`.
 
 - [ ] Write release notes.
     * Create a new file for the release notes: `docs/releases/v<release>.md`

diff --git a/docs/releases/v2_2_0-requirements.txt b/docs/releases/v2_2_0-requirements.txt
@@ -0,0 +1,102 @@
+#
+# This file is autogenerated by pip-compile
+# To update, run:
+#
+#    pip-compile --generate-hashes --output-file=v2_2_0-requirements.txt
+#
+attrs==20.1.0 \
+    --hash=sha256:0ef97238856430dcf9228e07f316aefc17e8939fc8507e18c6501b761ef1a42a \
+    --hash=sha256:2867b7b9f8326499ab5b0e2d12801fa5c98842d2cbd22b35112ae04bf85b4dff \
+    # via debut
+certifi==2020.6.20 \
+    --hash=sha256:5930595817496dd21bb8dc35dad090f1c2cd0adfaf21204bf6732ca5d8ee34d3 \
+    --hash=sha256:8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41 \
+    # via requests
+chardet==3.0.4 \
+    --hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae \
+    --hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691 \
+    # via debut, requests
+debut==0.9.8 \
+    --hash=sha256:b353e1d826d0be80a7268762efd99ba05f9d1df1aef0553fb7ea17c670bee85c \
+    --hash=sha256:edd4ff3d265ca5bf645c73d6863a886d34743152d215a5de094c4d31fa6943e3 \
+    # via -r requirements.in
+docker==4.3.1 \
+    --hash=sha256:13966471e8bc23b36bfb3a6fb4ab75043a5ef1dac86516274777576bed3b9828 \
+    --hash=sha256:bad94b8dd001a8a4af19ce4becc17f41b09f228173ffe6a4e0355389eef142f2 \
+    # via -r requirements.in
+dockerfile-parse==1.0.0 \
+    --hash=sha256:9ed92ede29a646094b52b8b302e477f08e63465b6ee524f5750810280143712e \
+    --hash=sha256:f04920c573d980904ce99abc70e31d28140d9195fb10f4d50c2dee1b6f45ebed \
+    # via -r requirements.in
+idna==2.10 \
+    --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \
+    --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 \
+    # via requests
+importlib-metadata==1.7.0 \
+    --hash=sha256:90bb658cdbbf6d1735b6341ce708fc7024a3e14e99ffdc5783edea9f9b077f83 \
+    --hash=sha256:dc15b2969b4ce36305c51eebe62d418ac7791e9a157911d58bfb1f9ccd8e2070 \
+    # via stevedore
+pbr==5.4.5 \
+    --hash=sha256:07f558fece33b05caf857474a366dfcc00562bca13dd8b47b2b3e22d9f9bf55c \
+    --hash=sha256:579170e23f8e0c2f24b0de612f71f648eccb79fb1322c814ae6b3c07b5ba23e8 \
+    # via -r requirements.in, stevedore
+pyyaml==5.3.1 \
+    --hash=sha256:06a0d7ba600ce0b2d2fe2e78453a470b5a6e000a985dd4a4e54e436cc36b0e97 \
+    --hash=sha256:240097ff019d7c70a4922b6869d8a86407758333f02203e0fc6ff79c5dcede76 \
+    --hash=sha256:4f4b913ca1a7319b33cfb1369e91e50354d6f07a135f3b901aca02aa95940bd2 \
+    --hash=sha256:69f00dca373f240f842b2931fb2c7e14ddbacd1397d57157a9b005a6a9942648 \
+    --hash=sha256:73f099454b799e05e5ab51423c7bcf361c58d3206fa7b0d555426b1f4d9a3eaf \
+    --hash=sha256:74809a57b329d6cc0fdccee6318f44b9b8649961fa73144a98735b0aaf029f1f \
+    --hash=sha256:7739fc0fa8205b3ee8808aea45e968bc90082c10aef6ea95e855e10abf4a37b2 \
+    --hash=sha256:95f71d2af0ff4227885f7a6605c37fd53d3a106fcab511b8860ecca9fcf400ee \
+    --hash=sha256:b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d \
+    --hash=sha256:cc8955cfbfc7a115fa81d85284ee61147059a753344bc51098f3ccd69b0d7e0c \
+    --hash=sha256:d13155f591e6fcc1ec3b30685d50bf0711574e2c0dfffd7644babf8b5102ca1a \
+    # via -r requirements.in
+regex==2020.7.14 \
+    --hash=sha256:0dc64ee3f33cd7899f79a8d788abfbec168410be356ed9bd30bbd3f0a23a7204 \
+    --hash=sha256:1269fef3167bb52631ad4fa7dd27bf635d5a0790b8e6222065d42e91bede4162 \
+    --hash=sha256:14a53646369157baa0499513f96091eb70382eb50b2c82393d17d7ec81b7b85f \
+    --hash=sha256:3a3af27a8d23143c49a3420efe5b3f8cf1a48c6fc8bc6856b03f638abc1833bb \
+    --hash=sha256:46bac5ca10fb748d6c55843a931855e2727a7a22584f302dd9bb1506e69f83f6 \
+    --hash=sha256:4c037fd14c5f4e308b8370b447b469ca10e69427966527edcab07f52d88388f7 \
+    --hash=sha256:51178c738d559a2d1071ce0b0f56e57eb315bcf8f7d4cf127674b533e3101f88 \
+    --hash=sha256:5ea81ea3dbd6767873c611687141ec7b06ed8bab43f68fad5b7be184a920dc99 \
+    --hash=sha256:6961548bba529cac7c07af2fd4d527c5b91bb8fe18995fed6044ac22b3d14644 \
+    --hash=sha256:75aaa27aa521a182824d89e5ab0a1d16ca207318a6b65042b046053cfc8ed07a \
+    --hash=sha256:7a2dd66d2d4df34fa82c9dc85657c5e019b87932019947faece7983f2089a840 \
+    --hash=sha256:8a51f2c6d1f884e98846a0a9021ff6861bdb98457879f412fdc2b42d14494067 \
+    --hash=sha256:9c568495e35599625f7b999774e29e8d6b01a6fb684d77dee1f56d41b11b40cd \
+    --hash=sha256:9eddaafb3c48e0900690c1727fba226c4804b8e6127ea409689c3bb492d06de4 \
+    --hash=sha256:bbb332d45b32df41200380fff14712cb6093b61bd142272a10b16778c418e98e \
+    --hash=sha256:bc3d98f621898b4a9bc7fecc00513eec8f40b5b83913d74ccb445f037d58cd89 \
+    --hash=sha256:c11d6033115dc4887c456565303f540c44197f4fc1a2bfb192224a301534888e \
+    --hash=sha256:c50a724d136ec10d920661f1442e4a8b010a4fe5aebd65e0c2241ea41dbe93dc \
+    --hash=sha256:d0a5095d52b90ff38592bbdc2644f17c6d495762edf47d876049cfd2968fbccf \
+    --hash=sha256:d6cff2276e502b86a25fd10c2a96973fdb45c7a977dca2138d661417f3728341 \
+    --hash=sha256:e46d13f38cfcbb79bfdb2964b0fe12561fe633caf964a77a5f8d4e45fe5d2ef7 \
+    # via -r requirements.in
+requests==2.24.0 \
+    --hash=sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b \
+    --hash=sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898 \
+    # via -r requirements.in, docker
+six==1.15.0 \
+    --hash=sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259 \
+    --hash=sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced \
+    # via docker, dockerfile-parse, websocket-client
+stevedore==3.2.0 \
+    --hash=sha256:38791aa5bed922b0a844513c5f9ed37774b68edc609e5ab8ab8d8fe0ce4315e5 \
+    --hash=sha256:c8f4f0ebbc394e52ddf49de8bcc3cf8ad2b4425ebac494106bbc5e3661ac7633 \
+    # via -r requirements.in
+urllib3==1.25.10 \
+    --hash=sha256:91056c15fa70756691db97756772bb1eb9678fa585d9184f24534b100dc60f4a \
+    --hash=sha256:e7983572181f5e1522d9c98453462384ee92a0be7fac5f1413a1e35c56cc0461 \
+    # via requests
+websocket-client==0.57.0 \
+    --hash=sha256:0fc45c961324d79c781bab301359d5a1b00b13ad1b10415a4780229ef71a5549 \
+    --hash=sha256:d735b91d6d1692a6a181f2a8c9e0238e5f6373356f561bb9dc4c7af36f452010 \
+    # via docker
+zipp==3.1.0 \
+    --hash=sha256:aa36550ff0c0b7ef7fa639055d797116ee891440eac1a56f378e2d3179e0320b \
+    --hash=sha256:c599e4d75c98f6798c509911d08a22e6c021d074469042177c8c86fb92eefd96 \
+    # via importlib-metadata
diff --git a/docs/releases/v2_2_0.md b/docs/releases/v2_2_0.md
@@ -0,0 +1,122 @@
+# Release 2.2.0
+
+## Summary
+This release features work done by our GSoC 2020 interns: Abhay Katheria and Junlai Wang. Abhay created a new HTML format which greatly helps the readability of all the container image metadata Tern supports. He also made a significant effort in inventorying go applications that use go modules. Junlai greatly improved the parsing of RUN instructions, increasing Tern's accuracy in detecting binaries used to install packages. He also added initial functionality to analyze multistage Dockerfiles. Thanks Abhay and Junlai for your work towards this release!
+
+In addition, we have enabled Tern to run in a container without the need for a Linux host. This feature allows Windows and Mac OS X users to run Tern natively if they have Docker installed.
+
+At this point, we think that we are on our way to making the next release a Beta release. For this to happen, we need to do a few housekeeping tasks:
+* Refactor the code to allow our community to make changes more easily.
+* Allow our CI/CD pipeline to test our currently supported extensions.
+* Increase our code coverage.
+
+As always, we would like to thank our community for contributing to this release.
+
+## New Features
+* [HTML format](https://github.com/tern-tools/tern/issues/614): You can now create a browsable HTML document using `tern report -f html -i <image> -o image.html`. To view this document, just open it in your browser.
+* [Run Tern in a Docker container on a non-Linux host](https://github.com/tern-tools/tern/issues/679): You can now run Tern in a Docker container on Windows and Mac OS X development environments that have Docker installed. Build the container image using `docker build -t tern:v2.2.0 -f Dockerfile .` to build the Docker container with Tern installed and run `docker run --privileged --device /dev/fuse -v /var/run/docker.sock:/var/run/docker.sock` or use the `docker_run.sh` script.
+* [Better RUN instruction parser](https://github.com/tern-tools/tern/issues/521): Tern will now report any non-deterministic branching statements for which it is not known which branch was executed during container build. In general, the new shell script parser improves Tern's accuracy to determine what gets installed in complicated shell scripts used in the RUN instruction of a Dockerfile.
+* [Go application support](https://github.com/tern-tools/tern/issues/695): Tern can now inventory Go applications which use Go modules. At this time, Tern does not support previous versions of Go dependency management like the vendor directory.
+* [Collect package information using Scancode](https://github.com/tern-tools/tern/issues/790): Tern can now collect package information using the Scancode extension.
+* [Support for openSUSE](https://github.com/tern-tools/tern/issues/693): Tern now supports openSUSE based images.
+* [Support for microdnf](https://github.com/tern-tools/tern/issues/724): Tern now supports inventorying packages installed with microdnf.
+
+## Deprecated Features
+* [Instantiate only with image and tag or image and digest](https://github.com/tern-tools/tern/issues/747): Images on registries are either identified by the image name and tag or the image name and manifest digest. As this is the standard way of referencing images, the instatiation with image ID is not necessary.
+
+## Bug Fixes
+* [Replace YAML file cache with JSON for faster parsing](https://github.com/tern-tools/tern/issues/627)
+* [Skip empty layers rather than exiting](https://github.com/tern-tools/tern/issues/686)
+* [Fix absolute symlink to busybox in Alpine based images](https://github.com/tern-tools/issues/769)
+* Fix SPDX tag-value validation errors
+* [Fix inventorying images in a raw tarball format](https://github.com/tern-tools/issues/719)
+* [Fix property name in output reporting](https://github.com/tern-tools/tern/issues/741)
+* Some CI/CD fixes
+* [Fix package version string reporting for Alpine](https://github.com/tern-tools/tern/issues/758)
+* [Fix exit if the RUN instruction is too long](https://github.com/tern-tools/tern/issues/772)
+
+## Resolved Technical Debt
+* Moved license collection and printing to the report module.
+* [Remove raise immediately statements](https://github.com/tern-tools/tern/issues/201)
+
+## Future Work
+* A code refactor to make it easy to make future changes.
+* A "step" subcommand to step through container image layers and analyze them individually.
+* Multistage Dockerfile analysis.
+* Analysis for OCI style images.
+
+The next release is slated for the end of November 2020. February 2021's release will be a small one due to winter holidays in the US. We will try to create more good-first-issues and hacktoberfest issues this time around. Watch the [project roadmap](/docs/project-roadmap.md) for updates.
+
+## Changelog
+
+Note: This changelog will not include these release notes
+
+Changelog generated by command: `git log --pretty=format:"%h %s" v2.1.0..master`
+
+```
+a0dd8cd merge: Integrate package data from scancode
+dfcacca Reconcile scancode packages with existing ones
+f845ac3 Include packages collected by scancode in reports
+888fb74 Split multistage dockerfile for building images
+bd94e91 Enable golang package listing
+b425819 Set environment vars before package collection
+e291986 Add logic to find shell in Alpine images
+f752cde Remove warning to Mac users in README
+420fa38 Skip empty layers during analysis
+5259e56 merge: Run in container on a non-linux host
+cfb86a0 Update utilities to set working directory
+e5bc4f8 Enable Terns run in a container for non-Linux host
+cf1c707 Amend Dockerfile and docker_run for fuse-overlayfs
+b6f353a Add support to execute commands from WORKDIR
+ecad1cd Fixes exit with complicated RUN statement
+92bca0e merge: Improve branch statement reporting
+5d69043 Add test for get_shell_commands()
+4ab945d Add report for branch statement
+044dc47 Fix package version collection for Alpine
+56a9a7a Update functions for shell script parser
+6f9c73c Bump up prospector and bandit versions
+3444740 github actions: Use a supported python version
+f3f54fa Add docker APIError to list of possible exceptions
+abf6e91 Removing multiple options to instantiate Image class
+fd147bf Remove raise immediately statements
+3d598b3 Use container image layer index
+ad35588 Remove extra underscore in Image class properties
+d8960bb Add CI test for HTML format
+21fd581 Print report to console if no output file provided
+465c1bc Remove failing CircleCI badge from README
+6bb9179 Add HTML report format
+aca49c6 Add new HTML report format
+352e037 Move license collection reporting to content.py
+dea53a8 Update GH Actions for better UI presentation
+e7e3c80 Remove circleci from running on pull requests
+cdf8f70 merge: Initial RUN shell script parser
+c1f7e33 Quick fix on pipe symbol and export cmd
+9a81523 Generate report.txt when no output file specified
+c090e49 Add parse loop and use clean_command()
+df38d0c Replace yaml file cache
+d8a86c5 utils: Use JSON instead of YAML as cache
+3151796 extensions: scancode: Store headers as a list
+07279ca formats: spdxtagvalue: Fix missing LicenseRef
+179df2e Using Regex to split shell script
+535802e Adding microdnf package type
+4c7ef6b formats: spdxtagvalue: Fix 2.2 validation errors
+e463831 Don't set digest type for raw image tarballs
+f3e4425 Adding zypper to analyze openSUSE images
+e228afa Add test dockerfiles for split shell script.
+fc5cc50 Allow for multiple snippet install and remove cmds
+3ec6a98 Update README with support Docker image info
+```
+
+## Contributors
+
+```
+Abhay Katheria abhay.katheria1998@gmail.com
+mukultaneja mtaneja@vmware.com
+WangJL hazard15020@gmail.com
+Yann Jorelle yann.jorelle@nokia.com
+```
+
+## Contact the Maintainers
+
+Nisha Kumar: nishak@vmware.com
+Rose Judge: rjudge@vmware.com
diff --git a/requirements.txt b/requirements.txt
@@ -1,4 +1,4 @@
-# Copyright (c) 2017-2019 VMware, Inc. All Rights Reserved.
+# Copyright (c) 2017-2020 VMware, Inc. All Rights Reserved.
 # SPDX-License-Identifier: BSD-2-Clause
 #
 # Please only add direct dependencies here, i.e., do not update with the
@@ -7,10 +7,10 @@
 # what should be updated.
 
 PyYAML>=5.3
-docker~=4.2
-dockerfile-parse~=0.0
-requests~=2.23
-stevedore>=1.32
+docker~=4.3
+dockerfile-parse~=1.0
+requests~=2.24
+stevedore>=3.2
 pbr>=5.4
 debut>=0.9
-regex>=2020.5.14
+regex>=2020.7