diff --git a/.github/workflows/release-3-master-into-dev.yml b/.github/workflows/release-3-master-into-dev.yml index 621c180ad5..465a8fc2e9 100644 --- a/.github/workflows/release-3-master-into-dev.yml +++ b/.github/workflows/release-3-master-into-dev.yml @@ -55,7 +55,23 @@ jobs: grep version dojo/__init__.py grep appVersion helm/defectdojo/Chart.yaml grep version components/package.json - + + - name: Create upgrade notes to documentation + run: | + minorv=$(echo ${{ github.event.inputs.release_number_dev }} | cut -d '.' -f -2) + patchv=$(echo ${{ github.event.inputs.release_number_dev }} | cut -d '-' -f -1) + weight=$(date +%Y%m%d) + echo -n "--- + title: 'Upgrading to DefectDojo Version $minorv.x' + toc_hide: true + weight: -$weight + description: No special instructions. + --- + There are no special instructions for upgrading to $minorv.x. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/$patchv) for the contents of the release. + " > docs/content/en/getting_started/upgrading/$minorv.md + git add docs/content/en/getting_started/upgrading/$minorv.md + if: endsWith(github.event.inputs.release_number_dev, '.0-dev') + - name: Push version changes uses: stefanzweifel/git-auto-commit-action@v5.0.0 with: diff --git a/README.md b/README.md index 3eb26774ce..296288dbe6 100644 --- a/README.md +++ b/README.md @@ -1,64 +1,70 @@ # DefectDojo - - - - + + + +
- Open Source Security Index - Fastest Growing Open Source Security Projects - -

OWASP Flagship GitHub release YouTube Subscribe Twitter Follow -

-

Unit TestsIntegration Tests CII Best Practices

-
+ + Open Source Security Index - Fastest Growing Open Source Security Projects + + +

+ OWASP Flagship + GitHub release + YouTube Subscribe + Twitter Follow +

+

+ Unit Tests + Integration Tests + CII Best Practices +

+
![Screenshot of DefectDojo](https://raw.githubusercontent.com/DefectDojo/django-DefectDojo/dev/docs/static/images/screenshot1.png) -[DefectDojo](https://www.defectdojo.com/) is a security orchestration and -vulnerability management platform. -DefectDojo allows you to manage your application security program, maintain -product and application information, triage vulnerabilities and -push findings to systems like JIRA and Slack. DefectDojo enriches and -refines vulnerability data using a number of heuristic algorithms that -improve with the more you use the platform. +[DefectDojo](https://www.defectdojo.com/) is a DevSecOps, ASPM (application security posture management), and +vulnerability management tool. DefectDojo orchestrates end-to-end security testing, vulnerability tracking, +deduplication, remediation, and reporting. ## Demo -Try out the demo server at [demo.defectdojo.org](https://demo.defectdojo.org) +Try out DefectDojo on our demo server at [demo.defectdojo.org](https://demo.defectdojo.org) -Log in with `admin / 1Defectdojo@demo#appsec`. Please note that the demo is publicly accessible and regularly reset. Do not put sensitive data in the demo. +Log in with username `admin` and password `1Defectdojo@demo#appsec`. Please note that the demo is publicly accessible +and regularly reset. Do not put sensitive data in the demo. ## Quick Start for Compose V2 + From July 2023 Compose V1 [stopped receiving updates](https://docs.docker.com/compose/reference/). -Compose V2 integrates compose functions into the Docker platform, continuing to support most of the previous docker-compose features and flags. You can run Compose V2 by replacing the hyphen (-) with a space, using `docker compose`, instead of `docker-compose`. +Compose V2 integrates compose functions into the Docker platform, continuing to support most of the previous +docker-compose features and flags. You can run Compose V2 by replacing the hyphen (-) with a space, using +`docker compose` instead of `docker-compose`. ```sh +# Clone the project git clone https://github.com/DefectDojo/django-DefectDojo cd django-DefectDojo -# building + +# Building Docker images ./dc-build.sh -# running (for other profiles besides postgres-redis look at https://github.com/DefectDojo/django-DefectDojo/blob/dev/readme-docs/DOCKER.md) + +# Run the application (for other profiles besides postgres-redis see +# https://github.com/DefectDojo/django-DefectDojo/blob/dev/readme-docs/DOCKER.md) ./dc-up.sh postgres-redis -# obtain admin credentials. the initializer can take up to 3 minutes to run -# use docker-compose logs -f initializer to track progress + +# Obtain admin credentials. The initializer can take up to 3 minutes to run. +# Use docker compose logs -f initializer to track its progress. docker compose logs initializer | grep "Admin password:" ``` + ## For Docker Compose V1 -You can run Compose V1 by editing the below files to add the hyphen (-) between `docker compose`. + +You can run Compose V1 by editing the files below to add the hyphen (-) between `docker compose`. ```sh dc-build.sh dc-down.sh @@ -71,17 +77,18 @@ You can run Compose V1 by editing the below files to add the hyphen (-) between docker/setEnv.sh ``` - -Navigate to . - +Navigate to `http://localhost:8080` to see your new instance! ## Documentation -- [Official Docs](https://documentation.defectdojo.com/) ([latest](https://documentation.defectdojo.com/) | [dev](https://documentation.defectdojo.com/dev)) -- [REST APIs](https://documentation.defectdojo.com/integrations/api-v2-docs/) -- [Client APIs and Wrappers](https://documentation.defectdojo.com/integrations/api-v2-docs/#clients--api-wrappers) -- [Authentication Options](readme-docs/AVAILABLE-PLUGINS.md) -- [Parsers](https://documentation.defectdojo.com/integrations/parsers/) +* [Official Docs](https://documentation.defectdojo.com/) + * [Docs for our `dev` branch](https://documentation.defectdojo.com/dev/) +* [REST APIs](https://documentation.defectdojo.com/integrations/api-v2-docs/) +* [Client APIs and Wrappers](https://documentation.defectdojo.com/integrations/api-v2-docs/#clients--api-wrappers) +* Authentication options: + * [OAuth2/SAML2](https://documentation.defectdojo.com/integrations/social-authentication/) + * [LDAP](https://documentation.defectdojo.com/integrations/ldap-authentication/) +* [Supported tools](https://documentation.defectdojo.com/integrations/parsers/) ## Supported Installation Options @@ -91,47 +98,62 @@ Navigate to . ## Community, Getting Involved, and Updates -[Slack](https://owasp-slack.herokuapp.com/) +[Slack](https://owasp.org/slack/invite) [LinkedIn](https://www.linkedin.com/company/defectdojo) [Twitter](https://twitter.com/defectdojo) [Youtube](https://www.youtube.com/channel/UCWw9qzqptiIvTqSqhOFuCuQ) -[Join the slack community](https://owasp.org/slack/invite) and discussion! Realtime discussion is done in the OWASP Slack Channel, #defectdojo. -Follow DefectDojo on [Twitter](https://twitter.com/defectdojo), [Linkedin](https://www.linkedin.com/company/defectdojo), and [YouTube](https://www.youtube.com/channel/UCWw9qzqptiIvTqSqhOFuCuQ) for project updates! +[Join the OWASP Slack community](https://owasp.org/slack/invite) and participate in the discussion! You can find us in +our channel there, [#defectdojo](https://owasp.slack.com/channels/defectdojo). Follow DefectDojo on +[Twitter](https://twitter.com/defectdojo), [LinkedIn](https://www.linkedin.com/company/defectdojo), and +[YouTube](https://www.youtube.com/channel/UCWw9qzqptiIvTqSqhOFuCuQ) for project updates! ## Contributing -:warning: Please note that DefectDojo will soon stop accepting new features to stabilize the API and data model for a -forthcoming v3 release. See the contributing guidelines below for more details. :warning: +:warning: We have instituted a [feature freeze](https://github.com/DefectDojo/django-DefectDojo/discussions/8002) on v2 +of DefectDojo as we begin work on v3. Please see our [contributing guidelines](readme-docs/CONTRIBUTING.md) for more +information. Check out our latest update on v3 [here](https://github.com/DefectDojo/django-DefectDojo/discussions/8974). -See our [Contributing guidelines](readme-docs/CONTRIBUTING.md) +## Pro Edition +[Upgrade to DefectDojo Pro](https://www.defectdojo.com/pricing) today to take your DevSecOps to 11. DefectDojo Pro is +designed to meet you wherever you are on your security journey and help you scale, with enhanced dashboards, additional +smart features, tunable deduplication, and support from DevSecOps experts. -## Commercial Support and Training -[Commercial support and training is availaible.](https://www.defectdojo.com/) For information please email info@defectdojo.com. +Alternatively, for information please email info@defectdojo.com ## About Us DefectDojo is maintained by: -* Greg Anderson ([@devGregA](https://github.com/devgrega) | [linkedin](https://www.linkedin.com/in/g-anderson/)) -* Matt Tesauro ([@mtesauro](https://github.com/mtesauro) | [linkedin](https://www.linkedin.com/in/matttesauro/) | [@matt_tesauro](https://twitter.com/matt_tesauro)) +* Greg Anderson ([@devGregA](https://github.com/devgrega) | [LinkedIn](https://www.linkedin.com/in/g-anderson/)) +* Matt Tesauro ([@mtesauro](https://github.com/mtesauro) | [LinkedIn](https://www.linkedin.com/in/matttesauro/) | + [@matt_tesauro](https://twitter.com/matt_tesauro)) Core Moderators can help you with pull requests or feedback on dev ideas: -* Cody Maffucci ([@Maffooch](https://github.com/maffooch) | [linkedin](https://www.linkedin.com/in/cody-maffucci)) +* Cody Maffucci ([@Maffooch](https://github.com/maffooch) | [LinkedIn](https://www.linkedin.com/in/cody-maffucci)) Moderators can help you with pull requests or feedback on dev ideas: -* Damien Carol ([@damnielcarol](https://github.com/damiencarol) | [linkedin](https://www.linkedin.com/in/damien-carol/)) +* Damien Carol ([@damiencarol](https://github.com/damiencarol) | [LinkedIn](https://www.linkedin.com/in/damien-carol/)) * Jannik Jürgens ([@alles-klar](https://github.com/alles-klar)) * Dubravko Sever ([@dsever](https://github.com/dsever)) - +* Charles Neill ([@cneill](https://github.com/cneill) | [@ccneill](https://twitter.com/ccneill)) +* Jay Paz ([@jjpaz](https://twitter.com/jjpaz)) +* Blake Owens ([@blakeaowens](https://github.com/blakeaowens)) ## Hall of Fame -* Valentijn Scholten ([@valentijnscholten](https://github.com/valentijnscholten) | [sponsor](https://github.com/sponsors/valentijnscholten) | [linkedin](https://www.linkedin.com/in/valentijn-scholten/)) - Valentijn served as a core moderator for 3 years. Valentijn’s contributions were numerous and extensive. He overhauled, improved, and optimized many parts of the codebase. He consistently fielded questions, provided feedback on pull requests, and provided a helping hand wherever it was needed. -* Fred Blaise ([@madchap](https://github.com/madchap) | [linkedin](https://www.linkedin.com/in/fredblaise/)) - Fred served as a core moderator during a critical time for DefectDojo. He contributed code, helped the team stay organized, and architected important policies and procedures. -* Charles Neill ([@ccneill](https://twitter.com/ccneill)) – Charles served as a - DefectDojo Maintainer for years and wrote some of Dojo's core functionality. -* Jay Paz ([@jjpaz](https://twitter.com/jjpaz)) – Jay was a DefectDojo - maintainer for years. He performed Dojo's first UI overhaul, optimized code structure/features, and added numerous enhancements. +* Valentijn Scholten ([@valentijnscholten](https://github.com/valentijnscholten) | + [Sponsor](https://github.com/sponsors/valentijnscholten) | + [LinkedIn](https://www.linkedin.com/in/valentijn-scholten/)) - Valentijn served as a core moderator for 3 years. + Valentijn’s contributions were numerous and extensive. He overhauled, improved, and optimized many parts of the + codebase. He consistently fielded questions, provided feedback on pull requests, and provided a helping hand wherever + it was needed. +* Fred Blaise ([@madchap](https://github.com/madchap) | [LinkedIn](https://www.linkedin.com/in/fredblaise/)) - Fred + served as a core moderator during a critical time for DefectDojo. He contributed code, helped the team stay organized, + and architected important policies and procedures. +* Aaron Weaver ([@aaronweaver](https://github.com/aaronweaver) | [LinkedIn](https://www.linkedin.com/in/aweaver/)) - + Aaron has been a long time contributor and user of DefectDojo. He did the second major UI overhaul and his + contributions include automation enhancements, CI/CD engagements, increased metadata at the product level, and many + more. ## Security @@ -139,4 +161,4 @@ Please report Security issues via our [disclosure policy](readme-docs/SECURITY.m ## License -DefectDojo is licensed under the [BSD-3-Clause License](LICENSE.md) +DefectDojo is licensed under the [BSD 3-Clause License](LICENSE.md) diff --git a/docker-compose.yml b/docker-compose.yml index 8f1d7606c3..8da8ddf250 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -149,7 +149,7 @@ services: volumes: - defectdojo_postgres:/var/lib/postgresql/data rabbitmq: - image: rabbitmq:3.12.8-alpine@sha256:f1a169ec5763caccdd05c35499c1441a7eacf0c8f442618ca15df4c2da96a735 + image: rabbitmq:3.12.9-alpine@sha256:801dbe7ad31edd693418cfd6adf5294773b140a76ac43fa27637b702b51b98a5 profiles: - mysql-rabbitmq - postgres-rabbitmq diff --git a/docs/content/en/getting_started/upgrading.md b/docs/content/en/getting_started/upgrading.md deleted file mode 100644 index 5e566a5998..0000000000 --- a/docs/content/en/getting_started/upgrading.md +++ /dev/null @@ -1,864 +0,0 @@ ---- -title: "Upgrading" -description: "Release specific upgrading instructions" -draft: false -weight: 5 ---- - -Docker-compose --------------- - -When you deploy a vanilla docker-compose, it will create a persistent -volume for your MySQL database. As long as your volume is there, you -should not lose any data. - -### Using docker images provided in DockerHub - -{{% alert title="Information" color="info" %}} -If you\'re using `latest`, then you need to pre pull the `latest` from -DockerHub to update. -{{% /alert %}} - - -The generic upgrade method for docker-compose are as follows: -- Pull the latest version - - ``` {.sourceCode .bash} - docker pull defectdojo/defectdojo-django:latest - docker pull defectdojo/defectdojo-nginx:latest - ``` - -- If you would like to use a version other than the latest, specify the version (tag) you want to upgrade to: - - ``` {.sourceCode .bash} - docker pull defectdojo/defectdojo-django:1.10.2 - docker pull defectdojo/defectdojo-nginx:1.10.2 - ``` - -- If you would like to use alpine based images, you specify the version (tag) you want to upgrade to: - - ``` {.sourceCode .bash} - docker pull defectdojo/defectdojo-django:1.10.2-alpine - docker pull defectdojo/defectdojo-nginx:1.10.2-alpine - ``` - -- Go to the directory where your docker-compose.yml file lives -- Stop DefectDojo: `./dc-stop.sh` -- Re-start DefectDojo, allowing for container recreation: - `./dc-up-d.sh` -- Database migrations will be run automatically by the initializer. - Check the output via `docker-compose logs initializer` or relevant k8s command -- If you have the initializer disabled (or if you want to be on the - safe side), run the migration command: - `docker-compose exec uwsgi /bin/bash -c "python manage.py migrate"` - -### Building your local images - -If you build your images locally and do not use the ones from DockerHub, -the instructions are the same, with the caveat that you must build your images -first. -- Pull the latest DefectDojo changes - - ``` {.sourceCode .bash} - git fetch - git pull - git merge origin/master - ``` - -Then replace the first step of the above generic upgrade method for docker-compose with: `docker-compose build` - -godojo installations --------------------- - -If you have installed DefectDojo on "iron" and wish to upgrade the installation, please see the [instructions in the repo](https://github.com/DefectDojo/godojo/blob/master/docs-and-scripts/upgrading.md). - -## Upgrading to DefectDojo Version 2.28.x. - -There are no special instruction for upgrading to 2.28.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.28.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.27.x. - -There are no special instruction for upgrading to 2.27.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.27.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.26.x. - -There are no special instruction for upgrading to 2.26.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.26.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.25.x. - -There are no special instruction for upgrading to 2.25.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.25.0) for the contents of the release. - -A few query parameters related to filtering object via API related to a products tags have been renamed to be more consistent with the other "related object tags": - -**Breaking Change** - - - Engagement - - `product__tags__name` -> `product__tags` - - `not_product__tags__name` -> `not_product__tags` - - Test - - `engagement__product__tags__name` -> `engagement__product__tags` - - `not_engagement__product__tags__name` -> `not_engagement__product__tags` - - Finding - - `test__engagement__product__tags__name` -> `test__engagement__product__tags` - - `not_test__engagement__product__tags__name` -> `not_test__engagement__product__tags` - -**Deprecation** - -The OpenAPI 2.0 Swagger API documentation is being deprecated in favor of the existing -OpenAPI 3.0 API documentation page. The OpenAPI 2.0 Swagger API documentation page is -slated for removal in version 2.30.0 - -*Note*: The API has not changed in any way and behaves the same between OAPI2 and OAPI3 - -For all other changes, check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.25.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.24.x. - -There are no special instruction for upgrading to 2.24.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.24.0) for the contents of the release. - - -## Upgrading to DefectDojo Version 2.23.x. - -There is a migration from the legacy Nessus and Nessus WAS parsers to a single Tenable parser. The updated Tenable parser simply merges existing support for Nessus and Nessus WAS without introducing new functionality that could create instability - -There is a migration process built into the upgrade that will automatically convert exiting Nessus and Nessus WAS findings and tests into Tenable findings and tests - -**Breaking Change** - - - If there is any use of the Nessus or Nessus WAS in automated fashion via the import and reimport API endpoints, the `scan-type` parameter needs to be updated to `Tenable Scan` - - The default containerized database will now be [PostgreSQL](https://www.postgresql.org/) rather than [MySQL](https://dev.mysql.com/) due to the use of case insensitivity on fields by default - - It is recommended to update the [database character set and collation](https://dev.mysql.com/doc/refman/5.7/en/charset-database.html) to use UTF encoding - - If your deployment uses the MySQL containerized database, please see the following updates to run DefectDojo: - - Use of the helper script "dc-up": `./dc-up.sh mysql-rabbitmq` or `./dc-up.sh mysql-redis` - - Use of the helper script "dc-up-d": `./dc-up-d.sh mysql-rabbitmq` or `./dc-up-d.sh mysql-redis` - - Use of Docker Compose directly: `docker-compose --profile mysql-rabbitmq --env-file ./docker/environments/mysql-rabbitmq.env up` or `docker-compose --profile mysql-redis --env-file ./docker/environments/mysql-redis.env up` - -For all other changes, check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.23.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.22.x. - -There are no special instruction for upgrading to 2.22.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.22.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.21.x. - -There are no special instruction for upgrading to 2.21.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.21.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.20.x. - -There are no special instruction for upgrading to 2.20.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.20.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.19.x - -There are new docker images based on alpine with fewer third party dependencies. Related to the new images the current docker files had to be renamed and have a "-debian" or the new images a "-alpine" at the end. Furthermore there are new docker tags [DefectdojoVersion]-[OS]. For example 2.19.0-alpine or 2.19.0-debian. The currend tags (latest and [DefectdojoVersion]) are still based on the "old" images. Be aware that the new alpine images are not heavily tested and may contain bugs. - -**Breaking Change** - -In version 2.19.3, the GitHub OAuth integration has been removed to prevent configurations that may allow more access than intended. - -[DefectDojo Security Advisory: Severity Medium | Potential GitHub Authentication Misconfiguration](https://github.com/DefectDojo/django-DefectDojo/security/advisories/GHSA-hfp4-q5pg-2p7r) - -## Upgrading to DefectDojo Version 2.18.x - -**Upgrade instructions for helm chart with rabbitMQ enabled**: The rabbitMQ uses a statefulset by default. Before upgrading the helm chart we have to ensure that all queues are empty: - -```bash -kubectl exec -i -- rabbitmqctl list_queues -``` - -Next step is to delete rabbitMQ pvc: - -```bash -kubectl delete pvc -l app.kubernetes.io/name=rabbitmq -``` - -Last step is to perform the upgrade. - -For more information: https://artifacthub.io/packages/helm/bitnami/rabbitmq/11.2.0 - - - -## Upgrading to DefectDojo Version 2.17.x. - -There are no special instruction for upgrading to 2.17.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.17.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.16.x. - -There are no special instruction for upgrading to 2.16.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.16.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.15.x. - -There are no special instruction for upgrading to 2.15.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.15.0) for the contents of the release. - -## Upgrading to DefectDojo Version 2.13.x. - -The last release implemented the search for vulnerability ids, but the search database was not initialized. To populate the database table of the vulnerability ids, execute this django command from the defect dojo installation directory or from a shell of the Docker container or Kubernetes pod: - -`./manage.py migrate_cve` - -Additionally this requires a one-time rebuild of the Django-Watson search index. Execute this django command from the defect dojo installation directory or from a shell of the Docker container or Kubernetes pod: - -`./manage.py buildwatson` - -**Upgrade instructions for helm chart with postgres enabled**: The postgres database uses a statefulset by default. Before upgrading the helm chart we have to delete the statefullset and ensure that the pvc is reused, to keep the data. For more information: https://docs.bitnami.com/kubernetes/infrastructure/postgresql/administration/upgrade/ . - -```bash -helm repo update -helm dependency update ./helm/defectdojo - -# obtain name oft the postgres pvc -export POSTGRESQL_PVC=$(kubectl get pvc -l app.kubernetes.io/instance=defectdojo,role=primary -o jsonpath="{.items[0].metadata.name}") - -# delete postgres statefulset -kubectl delete statefulsets.apps defectdojo-postgresql --namespace default --cascade=orphan - -# upgrade -helm upgrade \ - defectdojo \ - ./helm/defectdojo/ \ - --set primary.persistence.existingClaim=$POSTGRESQL_PVC \ - ... # add your custom settings -``` - -**Further changes:** - -Legacy authorization for changing configurations based on staff users has been removed. - -## Upgrading to DefectDojo Version 2.12.x. - -**Breaking change for search:** The field `cve` has been removed from the search index for Findings and the Vulnerability Ids have been added to the search index. With this the syntax to search explicitly for vulnerability ids have been changed from `cve:` to `vulnerability_id:`, e.g. `vulnerability_id:CVE-2020-27619`. - - -## Upgrading to DefectDojo Version 2.10.x. - -**Breaking change for Findings:** The field `cve` will be replaced by a list of Vulnerability Ids, which can store references to security advisories associated with this finding. These can be Common Vulnerabilities and Exposures (CVE) or from other sources, eg. GitHub Security Advisories. Although the field does still exist in the code, the API and the UI have already been changed to use the list of Vulnerability Ids. Other areas like hash code calculation, search and parsers will be migrated step by step in later stages. - -This change also causes an API change for the endpoint `/engagements/{id}/accept_risks/`. - - -## Upgrading to DefectDojo Version 2.9.x. - -**Breaking change for APIv2:** `configuration_url` was removed from API endpoint `/api/v2/tool_configurations/` due to redundancy. - - -## Upgrading to DefectDojo Version 2.8.x. - -**Breaking change for Docker Compose:** Starting DefectDojo with Docker Compose now supports 2 databases (MySQL and PostgreSQL) and 2 celery brokers (RabbitMQ and Redis). To make this possible, docker-compose needs to be started with the parameters `--profile` and `--env-file`. You can get more information in [Setup via Docker Compose - Profiles](https://github.com/DefectDojo/django-DefectDojo/blob/master/readme-docs/DOCKER.md#setup-via-docker-compose---profiles). The profile `mysql-rabbitmq` provides the same configuration as in previous releases. With this the prerequisites have changed as well: Docker requires at least version 19.03.0 and Docker Compose 1.28.0. - -**Breaking change for Helm Chart:** In one of the last releases we upgraded the redis dependency in our helm chart without renaming keys in our helm chart. We fixed this bug with this release, but you may want to check if all redis values are correct ([Pull Request](https://github.com/DefectDojo/django-DefectDojo/pull/5886)). - -The flexible permissions for the configuration of DefectDojo are now active by default. With this, the flag **Staff** for users is not relevant and not visible anymore. The old behaviour can still be activated by setting the parameter `FEATURE_CONFIGURATION_AUTHORIZATION` to `False`. If you haven't done so with the previous release, you can still run a migration script with `./manage.py migrate_staff_users`. This script: - -* creates a group for all staff users, -* sets all configuration permissions that staff users had and -* sets the global Owner role, if `AUTHORIZATION_STAFF_OVERRIDE` is set to `True`. - -## Upgrading to DefectDojo Version 2.7.x. - -This release is a breaking change regarding the Choctaw Hog parser. As the maintainers of this project unified multiple parsers under the RustyHog parser, we now support the parsing of Choctaw Hog JSON output files through the Rusty Hog parser. Furthermore, we also support Gottingen Hog and Essex Hog JSON output files with the RustyHog parser. - -There is another breaking change regarding the import of SSLyze scans. The parser has been renamed from `SSLyze 3 Scan (JSON)` to `SSLyze Scan (JSON)`. The data in the database is fixed by the initializer, but it may break scripted API calls. - -Release 2.7.0 contains a beta functionality to make permissions for the configuration of DefectDojo more flexible. When the settings parameter `FEATURE_CONFIGURATION_AUTHORIZATION` is set to `True`, many configuration dialogues and API endpoints can be enabled for users or groups of users, regardless of their **Superuser** or **Staff** status, see [Configuration Permissions]({{< ref "../usage/permissions/#configuration-permissions" >}}). - -The functionality using the flag `AUTHORIZATION_STAFF_OVERRIDE` has been removed. The same result can be achieved with giving the staff users a global Owner role. - -To support the transition for these 2 changes, you can run a migration script with ``./manage.py migrate_staff_users``. This script: - -* creates a group for all staff users, -* sets all configuration permissions that staff users had and -* sets the global Owner role, if `AUTHORIZATION_STAFF_OVERRIDE` is set to `True`. - -## Upgrading to DefectDojo Version 2.6.x. - -There are no special instruction for upgrading to 2.6.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.6.0) for the contents of the release. - -Please consult the security advisories [GHSA-f82x-m585-gj24](https://github.com/DefectDojo/django-DefectDojo/security/advisories/GHSA-f82x-m585-gj24) (moderate) and [GHSA-v7fv-g69g-x7p2](https://github.com/DefectDojo/django-DefectDojo/security/advisories/GHSA-v7fv-g69g-x7p2) (high) to see what security issues were fixed in this release. These will be published and become visible at January 18th, 2022. - -## Upgrading to DefectDojo Version 2.5.x. - -Legacy authorization has been completely removed with version 2.5.0. This includes removal of the migration of users -to the new authorization as described in https://documentation.defectdojo.com/getting_started/upgrading/#authorization. -If you are still using the legacy authorization, you should run the migration with ``./manage.py migrate_authorization_v2`` -before upgrading to version 2.5.0 - -This release introduces the "Forgot password" functionality (`DD_FORGOT_PASSWORD`: default `True`). The function -allows sending an e-mail with the reset password link. Missing configuration or misconfiguration of SMTP -(`DD_EMAIL_URL`) could raise an error (HTTP-500). Check and test (for example by resetting your own password) if you -configured SMTP correctly. If you want to avoid HTTP-500 and you don't want to set up SMTP, you can just simply switch -off the "Forgot password" functionality (`DD_FORGOT_PASSWORD=False`). - -Release renamed system setting `mail_notifications_from` to `email_from`. This value will not be used only for sending -notifications but also for sending the reset password emails. It is highly recommended to check the content of this -value if you are satisfied. If you installed DefectDojo earlier, you can expect `"from@example.com"` there. A fresh -installation will use `"no-reply@example.com"` - -This release [updates](https://github.com/DefectDojo/django-DefectDojo/pull/5450) our helm dependencies. There is a breaking change if you are using the mysql database from the helm chart because we replaced the deprecated chart from the stable repo with a chart from bitnami. If you have persistance enabled, ensure to backup your data before upgrading. All data get lost when replacing the mysql chart during the upgrade. For data migration take a look at the mysql backup and restore process. - -Furthermore we updated our kubernetes version. Current tests run on 1.18.16 and 1.22.0. - -## Upgrading to DefectDojo Version 2.4.x. (Security Release) - -This releases fixes a High severity vulnerability for which the details will be disclosed on November 16th in [GHSA-fwg9-752c-qh8w](https://github.com/DefectDojo/django-DefectDojo/security/advisories/GHSA-fwg9-752c-qh8w) - -There is a breaking change in the API for importing and re-importings scans with SonarQube API and Cobalt.io API. The [scan configurations -have been unified](https://github.com/DefectDojo/django-DefectDojo/pull/5289) and are set now with the attribute `api_scan_configuration`. -The existing configurations for SonarQube API and Cobalt.io API have been migrated. - -At the request of pyup.io, we had to remove the parser for Safety scans. - - -## Upgrading to DefectDojo Version 2.3.x. - -There are no special instruction for upgrading to 2.3.0. -In 2.3.0 we [changed the default password hashing algorithm to Argon2 (from PBKDF2)](https://github.com/DefectDojo/django-DefectDojo/pull/5205). -When logging in, exising hashes get replaced by an Argon2 hash. If you want to rehash password without users having to login, -please see the [Django password management docs](https://docs.djangoproject.com/en/3.2/topics/auth/passwords/). -The previous password hashing algorithm (PBKDF2) was not unsafe, but we wanted to follow the [OWASP guidelines](https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html). - - -## Upgrading to DefectDojo Version 2.2.x. - -Upgrade to 2.0.0 contained migration of endpoints. Some parts of migration haven't been done properly. This deficiency -may manifest as a doubled slash in endpoint URLs (like `http://foo.bar:8080//test`) or as a problem with deduplication -of the same endpoints. The mentioned bug was fixed in 2.2.0 and if you have seen these kinds of problems, just rerun -"Endpoint migration" as it is written in [Upgrading to DefectDojo Version 2.0.x.](#upgrading-to-defectdojo-version-20x). - - -## Upgrading to DefectDojo Version 2.0.x. - -Follow the usual steps to upgrade as described above. - -BEFORE UPGRADING -- If you are using SAML2 checkout the new [documentaion](https://documentation.defectdojo.com/integrations/social-authentication/#saml-20) and update you settings following the migration section. We replaced [django-saml2-auth](https://github.com/fangli/django-saml2-auth) with [djangosaml2](https://github.com/IdentityPython/djangosaml2). - -AFTER UPGRADING -- Usual migration process (`python manage.py migrate`) try to migrate all endpoints to new format and merge duplicates. -- All broken endpoints (which weren't possible to migrate) have red flag 🚩 in standard list of endpoints. -- Check if all your endpoints was migrated successfully, go to: https:///endpoint/migrate. -- Alternatively, this can be run as management command: `docker-compose exec uwsgi ./manage.py endpoint_migration --dry-run` -- When all endpoint will be fixed (there is not broken endpoint), press "Run migration" in https:///endpoint/migrate -- Or, you can run management command: `docker-compose exec uwsgi ./manage.py endpoint_migration` -- Details about endpoint migration / improvements in https://github.com/DefectDojo/django-DefectDojo/pull/4473 - -We decided to name this version 2.0.0 because we did some big cleanups in this release: - -- Remove API v1 ([#4413](https://github.com/DefectDojo/django-DefectDojo/pull/4413)) -- Remove setup.bash installation method ([#4417](https://github.com/DefectDojo/django-DefectDojo/pull/4417)) -- Rename Finding.is_Mitigated field to Finding.is_mitigated ([#3854](https://github.com/DefectDojo/django-DefectDojo/pull/4854)) -- Remove everything related to the old tagging library ([#4419](https://github.com/DefectDojo/django-DefectDojo/pull/4419)) -- Remove S0/S1/S2../S5 severity display option ([#4415](https://github.com/DefectDojo/django-DefectDojo/pull/4415)) -- Refactor EndPoint handling/formatting ([#4473](https://github.com/DefectDojo/django-DefectDojo/pull/4473)) -- Upgrade to Django 3.x ([#3632](https://github.com/DefectDojo/django-DefectDojo/pull/3632)) -- PDF Reports removed ([#4418](https://github.com/DefectDojo/django-DefectDojo/pull/4418)) -- Hashcode calculation logic has changed. To update existing findings run: - - `./manage.py dedupe --hash_code_only`. - -If you're using docker: - -`docker-compose exec uwsgi ./manage.py dedupe --hash_code_only`. - -This can take a while depending on your instance size. - -- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.0.0 - -### Endpoints - -- The usual migration process (`python manage.py migrate`) tries to migrate all endpoints to new format and merge duplicates. -- All broken endpoints (which weren't possible to migrate) have a red flag 🚩 in the standard list of endpoints. -- Check if all your endpoints were migrated successfully, go to: https:///endpoint/migrate. -- Alternatively, this can be run as management command: `docker-compose exec uwsgi ./manage.py endpoint_migration --dry-run` -- When all endpoint are fixed (there is not broken endpoint), press "Run migration" in https:///endpoint/migrate -- Or, you can run management command: `docker-compose exec uwsgi ./manage.py endpoint_migration` -- Details about endpoint migration / improvements in https://github.com/DefectDojo/django-DefectDojo/pull/4473 - -### Authorization - -The new authorization system for Products and Product Types based on roles is the default now. The fields for authorized users are not available anymore, but you can assign roles as described in [Permissions](../../usage/permissions). Users are migrated automatically, so that their permissions are as close as possible to the previous authorization: -- Superusers will still have all permissions on Products and Product Types, so they must not be changed. -- Staff users have had all permissions for all product types and products, so they will be get a global role as *Owner*. -- Product_Members and Product Type_Members will be added for authorized users according to the settings for the previous authorization: - - The *Reader* role is set as the default. - - If `AUTHORIZED_USERS_ALLOW_STAFF` is `True`, the user will get the *Owner* role for the respective Product or Product Type. - - If `AUTHORIZED_USERS_ALLOW_CHANGE` or `AUTHORIZED_USERS_ALLOW_DELETE` is `True`, the user will get the *Writer* role for the respective Product or Product Type. - -The new authorization is active for both UI and API. Permissions set via authorized users or via the Django Admin interface are no longer taken into account. - -Please review the roles for your users after the upgrade to avoid an unintended permissions creep. - - -## Upgrading to DefectDojo Version 1.15.x - -- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.15.0 -- If you have made changes to JIRA templates or the template config in the JIRA Project config for instances/products/engagements: -The jira template settings introduced in 1.13 have been changed. You now have to select a subfolder instead of a sinlge template file. If you have chosen a non-default template here, you have to reapply that to all products / engagements. Also you have to move your custom templates into the correct subfolder in `dojo/templates/issue-trackers/`. -- Hashcode calculation logic has changed in #4134, #4308 and #4310 to update existing findings run: - - `./manage.py dedupe --hash_code_only` - -If you're using docker: - -`docker-compose exec uwsgi ./manage.py dedupe --hash_code_only` - -This can take a while depending on your instance size. - - - -## Upgrading to DefectDojo Version 1.14.x - -- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.14.0 - -Note that the below fields are now optional without default value. They will not be filled anymore with values such as "No references given" when found empty while saving the findings -- mitigation -- references -- impact -- url - - - -## Upgrading to DefectDojo Version 1.13.x - -- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.13.0 -- Hashcode settings affecting deduplication have changed, to update existing findings run: - - `./manage.py dedupe` - -If you're using docker: - - docker-compose exec uwsgi ./manage.py dedupe - -This can take a while depeneding on your instance size. It might possible that new duplicates are detected among existing findings, so make a backup before running! - - -## Upgrading to DefectDojo Version 1.12.x - -- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.12.0 -- 1.12.1 is a security release https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.12.1 - -## Upgrading to DefectDojo Version 1.11.x - -- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.11.0 -- 1.11.1 is a security release https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.11.1 - -## Upgrading to DefectDojo Version 1.10.x - -**1.10.4 is a security release** - -- See the security advisory: - -- See release notes: - -- Version 1.10.4 replaces 1.10.3 as the latter contained an incomplete - fix - -**What\'s New:** - -- See release notes: - -- DefectDojo now provides a `settings.py` file - out-of-the-box. Custom settings need to go into - `local\_settings.py`. See - - and - -- A quickfix is to rename your own / customized - `settings.py` or `settings.dist.py` to - `local\_settings.py`. Details of that PR: - -- Major JIRA integration refactoring, for which you should at least - use 1.10.1 and not 1.10.0 for many bug fixes. - -**Breaking changes** - -Kubernetes/Helm users: we have moved away from the \"stable\" repository -to \"bitnami\" in this release. The bitnami postgresql chart required us -to add a new key to the postgresql secret, which will give you the error -`postgresql-postgres-password is missing` if you have -`createPostgresqlSecret: false`. In 1.10.1, a fix was also included to -allow your existing `postgresqlPassword` to be reused properly. - -Including in 1.10.1 were a couple fixes related to a rabbitMQ upgrade. -The path to access `password`, `erlangCookie` and -`existingPasswordSecret` changed from `rabbitmq` to `auth`. Furthermore, -as rabbitMQ is deployed as a StatefulSet, an in-place upgrade is not -possible and an error will likely be thrown such as -`Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden`. -After ensuring your rabbitMQ celery queue is empty, you will then want -to delete your rabbitMQ StatefulSet and PVC to allow them to get -re-created, or fully delete and recreate defectdojo. - -## Upgrading to DefectDojo Version 1.9.3 - -**This is a security release** - -- See the [security - advisory](https://github.com/DefectDojo/django-DefectDojo/security/advisories/GHSA-8q8j-7wc4-vjg5) -- See [release - notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.9.3) - -**What\'s New:** - -- See release notes: - - -**NOTE:** - -When upgrading from before 1.9.2, a corrective script may need to be ran - -`./manage.py create\_endpoint\_status` - -If you\'re using docker: - -`docker-compose exec uwsgi ./manage.py create\_endpoint\_status` - -This can take a while depending on your hardware and the number of -findings in your instance. - -- Search index tweaking index rebuild after upgrade: - -This requires a (one-time) rebuild of the Django-Watson search index. -Execute the django command from the defect dojo installation directory: - -`./manage.py buildwatson]` - -If you\'re using docker: - -`docker-compose exec uwsgi ./manage.py buildwatson` - -This can take a while depending on your hardware and the number of -findings in your instance. - -## Upgrading to DefectDojo Version 1.8.0 - -**What\'s New:** - -- See release notes: - -- Improved search, which requires an index rebuild - () - -This requires a (one-time) rebuild of the Django-Watson search index. -Execute the django command from the defect dojo installation directory: - -`./manage.py buildwatson` - -If you\'re using docker: - -`docker-compose exec uwsgi ./manage.py buildwatson` - -This can take a while depending on your hardware and the number of -findings in your instance. - -- **NOTE:** - -As a result of a breaking bug revolving around Endpoint\_status objects, -a corrective script will need to be ran after every dynamic scan -imported through either API version. - -The script can be found -[here](https://github.com/DefectDojo/django-DefectDojo/blob/dev/dojo/management/commands/create_endpoint_status.py) - -`./manage.py create\_endpoint\_status` - -If you\'re using docker: - -`docker-compose exec uwsgi ./manage.py create\_endpoint\_status` - -This can take a while depending on your hardware and the number of -findings in your instance. - -## Upgrading to DefectDojo Version 1.7.0 - -**What\'s New:** - -- Updated search, you can now search for CVE-XXXX-YYYY -- Updated search index, fields added to index: \'id\', \'title\', - \'cve\', \'url\', \'severity\', \'description\', \'mitigation\', - \'impact\', \'steps\_to\_reproduce\', \'severity\_justification\', - \'references\', \'sourcefilepath\', \'sourcefile\', \'hash\_code\', - \'file\_path\', \'component\_name\', \'component\_version\', - \'unique\_id\_from\_tool\' - -This requires a (one-time) rebuild of the Django-Watson search index. -Execute the django command from the defect dojo installation directory: - -`./manage.py buildwatson dojo.Finding` - -If you\'re using docker: - -`docker-compose exec uwsgi ./manage.py buildwatson dojo.Finding` - -Upgrading to DefectDojo Version 1.5.0 -------------------------------------- - -**What\'s New:** - -- Updated UI with a new DefectDojo logo, default colors and CSS. -- Updated Product views with tabs for Product Overview, Metrics, - Engagements, Endpoints, Benchmarks (ASVS), and Settings to make it - easier to navigate and manage your products. -- New Product Information fields: Regulations, Criticality, Platform, - Lifecycle, Origin, User Records, Revenue, External Audience, - Internet Accessible -- Languages pie chart on product overview, only supported through the - API and Django admin, integrates with cloc analyzer -- New Engagement type of CI/CD to support continual testing -- Engagement shortcuts and ability to import findings and auto-create - an engagement -- Engagement labels for overdue, no tests and findings -- New Contextual menus throughout DefectDojo and shortcuts to new - findings and critical findings -- Ability to merge a finding into a parent finding and either - inactivate or delete the merged findings. -- Report improvements and styling adjustment with the default option - of HTML reports -- SLA for remediation of severities based on finding criticality, for - example critical findings remediated within 7 days. Configurable in - System Settings. -- Engagement Auto-Close Days in System Settings. Automatically close - an engagement if open past the end date. -- Ability to apply remediation advice based on CWE. For example XSS - can be configured as a template so that it\'s consistent across all - findings. Enabled in system settings. -- Finding confidence field supported from scanners. First - implementation in the Burp importer. -- Goast importer for static analysis of Golang products -- Celery status check on System Settings -- Beta rules framework release for modifying findings on the fly -- DefectDojo 2.0 API with Swagger support -- Created and Modified fields on all major tables -- Various bug fixes reported on Github - -**Upgrading to 1.5.0 requirements:** - -1. Back up your database first, ideally take the backup from production - and test the upgrade on a staging server. -2. Edit the settings.py file which can be found in - `django-DefectDojo/dojo/settings/settings.py`. Copy in the rest - framework configuration after the CSRF\_COOKIE\_SECURE = True: - - REST_FRAMEWORK = { - 'DEFAULT_AUTHENTICATION_CLASSES': ( - 'rest_framework.authentication.TokenAuthentication', - 'rest_framework.authentication.BasicAuthentication', - ), - 'DEFAULT_PERMISSION_CLASSES': ( - 'rest_framework.permissions.DjangoModelPermissions', - ), - 'DEFAULT_RENDERER_CLASSES': ( - 'rest_framework.renderers.JSONRenderer', - ), - 'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.LimitOffsetPagination', - 'PAGE_SIZE': 25 - } - -Navigate to: LOGIN\_EXEMPT\_URLS and add the following after -r\'\^%sfinding/image/(?P\\[\^/\]+)\$\' % URL\_PREFIX: - - r'^%sfinding/image/(?P[^/]+)$' % URL_PREFIX, - r'^%sapi/v2/' % URL_PREFIX, - -Navigate to: INSTALLED\_APPS and add the following after: -\'multiselectfield\',: - - 'multiselectfield', - 'rest_framework', - 'rest_framework.authtoken', - 'rest_framework_swagger', - 'dbbackup', - -Navigate to: CELERY\_TASK\_IGNORE\_RESULT = True and add the following -after CELERY\_TASK\_IGNORE\_RESULT line: - - CELERY_RESULT_BACKEND = 'db+sqlite:///dojo.celeryresults.sqlite' - -Save your modified settings file. For reference the modified file should -look like the new 1.5.0 -\[settings\]() -file, minus the environmental configurations. As an alternative this -file can be used and the enviromental configurations from you -environment can be copied into this file. - -3. Activate your virtual environment and then upgrade the requirements: - -`pip install -r requirements.txt --upgrade` - -4. Upgrade the database: - - ./manage.py makemigrations - ./manage.py migrate - -5. Collect the static files (Javascript, Images, CSS): - - ./manage.py collectstatic --noinput - -6. Complete - -## Upgrading to DefectDojo Version 1.3.1 - -**What\'s New:** - -- New importers for Contrast, Nikto and TruffleHog (finding secrets in - git repos). -- Improved merging of findings for dynamic and static importers -- Markdown support for findings -- HTML report improvements including support of Markdown. -- System settings Celery status page to assist in debugging if Celery - is functional. - -**Upgrading to 1.3.1 requires:** - -1. pip install markdown pip install pandas -2. ./manage.py makemigrations ./manage.py migrate -3. ./manage.py collectstatic \--noinput -4. Complete - -## Upgrading to DefectDojo Version 1.2.9 - -**What\'s New:** New feature: Benchmarks (OWASP ASVS) - -**Upgrading to 1.2.9 requires:** - -1. ./manage.py makemigrations ./manage.py migrate ./manage.py loaddata - dojo/fixtures/benchmark\_type.json ./manage.py loaddata - dojo/fixtures/benchmark\_category.json ./manage.py loaddata - dojo/fixtures/benchmark\_requirement.json -2. ./manage.py collectstatic \--noinput -3. Complete - -## Upgrading to DefectDojo Version 1.2.8 - -New feature: Product Grading (Overall Product Health) Upgrading to 1.2.8 -requires: - -1. ./manage.py makemigrations ./manage.py migrate ./manage.py - system\_settings -2. ./manage.py collectstatic \--noinput -3. pip install asteval -4. pip install \--upgrade celery -5. Complete - -## Upgrading to DefectDojo Version 1.2.4 - -Upgrading to 1.2.4 requires: - -1. ./manage.py makemigrations ./manage.py migrate ./manage.py loaddata - dojo/fixtures/objects\_review.json - -## Upgrading to DefectDojo Version 1.2.3 - -Upgrading to 1.2.3 requires: - -1. ./manage.py makemigrations ./manage.py migrate ./manage.py loaddata - dojo/fixtures/language\_type.json -2. Currently languages and technologies can be updated via the API or - in the admin section of Django. - -## July 6th 2017 - New location for system settings - -Pull request \#313 moves a number of system settings previously located -in the application\'s settings.py to a model that can be used and -changed within the web application under \"Configuration -\> System -Settings\". - -If you\'re using a custom `URL_PREFIX` you will need to set this in the -model after upgrading by editing `dojo/fixtures/system_settings.json` -and setting your URL prefix in the `url_prefix` value there. Then issue -the command `./manage.py loaddata system_settings.json` to load your -settings into the database. - -If you\'re not using a custom `URL_PREFIX`, after upgrading simply go to -the System Settings page and review which values you want to set for -each setting, as they\'re not automatically migrated from settings.py. - -If you like you can then remove the following settings from settings.py -to avoid confusion: - -- `ENABLE_DEDUPLICATION` -- `ENABLE_JIRA` -- `S_FINDING_SEVERITY_NAMING` -- `URL_PREFIX` -- `TIME_ZONE` -- `TEAM_NAME` - -## Upgrading to DefectDojo Version 1.2.2 - -Upgrading to 1.2.2 requires: - -1. Copying settings.py to the settings/ folder. -2. If you have supervisor scripts change - DJANGO\_SETTINGS\_MODULE=dojo.settings.settings - -## Upgrading to Django 1.1.5 - -If you are upgrading an existing version of DefectDojo, you will need to -run the following commands manually: - -1. First install Yarn. Follow the instructions based on your OS: - -2. The following must be removed/commented out from `settings.py`: : - - 'djangobower.finders.BowerFinder', - - From the line that contains: - # where should bower install components - ... - - To the end of the bower declarations - 'justgage' - ) - -3. The following needs to be updated in `settings.py`: : - - STATICFILES_DIRS = ( - # Put strings here, like "/home/html/static" or "C:/www/django/static". - # Always use forward slashes, even on Windows. - # Don't forget to use absolute paths, not relative paths. - os.path.dirname(DOJO_ROOT) + "/components/yarn_components", - ) - -## Upgrading to Django 1.11 - -Pull request \#300 makes DefectDojo Django 1.11 ready. A fresh install -of DefectDojo can be done with the setup.bash script included - no -special steps are required. - -If you are upgrading an existing installation of DefectDojo, you will -need to run the following commands manually: : - - pip install django-tastypie --upgrade - pip install django-tastypie-swagger --upgrade - pip install django-filter --upgrade - pip install django-watson --upgrade - pip install django-polymorphic --upgrade - pip install django --upgrade - pip install pillow --upgrade - ./manage.py makemigrations - ./manage.py migrate - -The following must be removed/commented out from settings.py: : - - TEMPLATE_DIRS - TEMPLATE_DEBUG - TEMPLATE_LOADERS - TEMPLATE_CONTEXT_PROCESSORS - -The following needs to be added to settings.py: : - - TEMPLATES = [ - { - 'BACKEND': 'django.template.backends.django.DjangoTemplates', - 'APP_DIRS': True, - 'OPTIONS': { - 'context_processors': [ - 'django.template.context_processors.debug', - 'django.template.context_processors.request', - 'django.contrib.auth.context_processors.auth', - 'django.contrib.messages.context_processors.messages', - ], - }, - }, - ] - -Once all these steps are completed your installation of DefectDojo will -be running under Django 1.11 diff --git a/docs/content/en/getting_started/upgrading/1.10.md b/docs/content/en/getting_started/upgrading/1.10.md new file mode 100644 index 0000000000..540ec306aa --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.10.md @@ -0,0 +1,50 @@ +--- +title: "Upgrading to DefectDojo Version 1.10.x" +toc_hide: true +weight: -20201124 +description: security release + breaking changes +--- +**1.10.4 is a security release** + +- See the security advisory: + +- See release notes: + +- Version 1.10.4 replaces 1.10.3 as the latter contained an incomplete + fix + +**What\'s New:** + +- See release notes: + +- DefectDojo now provides a `settings.py` file + out-of-the-box. Custom settings need to go into + `local\_settings.py`. See + + and + +- A quickfix is to rename your own / customized + `settings.py` or `settings.dist.py` to + `local\_settings.py`. Details of that PR: + +- Major JIRA integration refactoring, for which you should at least + use 1.10.1 and not 1.10.0 for many bug fixes. + +**Breaking changes** + +Kubernetes/Helm users: we have moved away from the \"stable\" repository +to \"bitnami\" in this release. The bitnami postgresql chart required us +to add a new key to the postgresql secret, which will give you the error +`postgresql-postgres-password is missing` if you have +`createPostgresqlSecret: false`. In 1.10.1, a fix was also included to +allow your existing `postgresqlPassword` to be reused properly. + +Including in 1.10.1 were a couple fixes related to a rabbitMQ upgrade. +The path to access `password`, `erlangCookie` and +`existingPasswordSecret` changed from `rabbitmq` to `auth`. Furthermore, +as rabbitMQ is deployed as a StatefulSet, an in-place upgrade is not +possible and an error will likely be thrown such as +`Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden`. +After ensuring your rabbitMQ celery queue is empty, you will then want +to delete your rabbitMQ StatefulSet and PVC to allow them to get +re-created, or fully delete and recreate defectdojo. diff --git a/docs/content/en/getting_started/upgrading/1.11.md b/docs/content/en/getting_started/upgrading/1.11.md new file mode 100644 index 0000000000..9110d06f15 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.11.md @@ -0,0 +1,8 @@ +--- +title: "Upgrading to DefectDojo Version 1.11.x" +toc_hide: true +weight: -20201229 +description: security release +--- +- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.11.0 +- 1.11.1 is a security release https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.11.1 diff --git a/docs/content/en/getting_started/upgrading/1.12.md b/docs/content/en/getting_started/upgrading/1.12.md new file mode 100644 index 0000000000..39c8371d91 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.12.md @@ -0,0 +1,8 @@ +--- +title: "Upgrading to DefectDojo Version 1.12.x" +toc_hide: true +weight: -20210126 +description: security release +--- +- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.12.0 +- 1.12.1 is a security release https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.12.1 diff --git a/docs/content/en/getting_started/upgrading/1.13.md b/docs/content/en/getting_started/upgrading/1.13.md new file mode 100644 index 0000000000..b5948a91a6 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.13.md @@ -0,0 +1,17 @@ +--- +title: "Upgrading to DefectDojo Version 1.13.x" +toc_hide: true +weight: -20210223 +description: hashcode calculation logic has changed +--- +- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.13.0 +- Hashcode settings affecting deduplication have changed, to update existing findings run: + + `./manage.py dedupe` + +If you're using docker: + + docker-compose exec uwsgi ./manage.py dedupe + +This can take a while depeneding on your instance size. It might possible that new duplicates are detected among existing findings, so make a backup before running! + diff --git a/docs/content/en/getting_started/upgrading/1.14.md b/docs/content/en/getting_started/upgrading/1.14.md new file mode 100644 index 0000000000..4f7c72981e --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.14.md @@ -0,0 +1,15 @@ +--- +title: "Upgrading to DefectDojo Version 1.14.x" +toc_hide: true +weight: -20210330 +description: hashcode calculation logic has changed +--- +- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.14.0 + +Note that the below fields are now optional without default value. They will not be filled anymore with values such as "No references given" when found empty while saving the findings +- mitigation +- references +- impact +- url + + diff --git a/docs/content/en/getting_started/upgrading/1.15.md b/docs/content/en/getting_started/upgrading/1.15.md new file mode 100644 index 0000000000..4341ba5127 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.15.md @@ -0,0 +1,20 @@ +--- +title: "Upgrading to DefectDojo Version 1.15.x" +toc_hide: true +weight: -20210500 +description: hashcode calculation logic has changed +--- +- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.15.0 +- If you have made changes to JIRA templates or the template config in the JIRA Project config for instances/products/engagements: +The jira template settings introduced in 1.13 have been changed. You now have to select a subfolder instead of a sinlge template file. If you have chosen a non-default template here, you have to reapply that to all products / engagements. Also you have to move your custom templates into the correct subfolder in `dojo/templates/issue-trackers/`. +- Hashcode calculation logic has changed in #4134, #4308 and #4310 to update existing findings run: + + `./manage.py dedupe --hash_code_only` + +If you're using docker: + +`docker-compose exec uwsgi ./manage.py dedupe --hash_code_only` + +This can take a while depending on your instance size. + + diff --git a/docs/content/en/getting_started/upgrading/1.2.2.md b/docs/content/en/getting_started/upgrading/1.2.2.md new file mode 100644 index 0000000000..e536698281 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.2.2.md @@ -0,0 +1,11 @@ +--- +title: "Upgrading to DefectDojo Version 1.2.2" +toc_hide: true +weight: -20200202 +description: multiple instructions +--- +Upgrading to 1.2.2 requires: + +1. Copying settings.py to the settings/ folder. +2. If you have supervisor scripts change + DJANGO\_SETTINGS\_MODULE=dojo.settings.settings diff --git a/docs/content/en/getting_started/upgrading/1.2.3.md b/docs/content/en/getting_started/upgrading/1.2.3.md new file mode 100644 index 0000000000..ea2685271d --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.2.3.md @@ -0,0 +1,12 @@ +--- +title: "Upgrading to DefectDojo Version 1.2.3" +toc_hide: true +weight: -20200203 +description: multiple instructions +--- +Upgrading to 1.2.3 requires: + +1. ./manage.py makemigrations ./manage.py migrate ./manage.py loaddata + dojo/fixtures/language\_type.json +2. Currently languages and technologies can be updated via the API or + in the admin section of Django. diff --git a/docs/content/en/getting_started/upgrading/1.2.4.md b/docs/content/en/getting_started/upgrading/1.2.4.md new file mode 100644 index 0000000000..54ed3c196c --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.2.4.md @@ -0,0 +1,10 @@ +--- +title: "Upgrading to DefectDojo Version 1.2.4" +toc_hide: true +weight: -20200204 +description: multiple instructions +--- +Upgrading to 1.2.4 requires: + +1. ./manage.py makemigrations ./manage.py migrate ./manage.py loaddata + dojo/fixtures/objects\_review.json diff --git a/docs/content/en/getting_started/upgrading/1.2.8.md b/docs/content/en/getting_started/upgrading/1.2.8.md new file mode 100644 index 0000000000..d8fd7029f6 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.2.8.md @@ -0,0 +1,15 @@ +--- +title: "Upgrading to DefectDojo Version 1.2.8" +toc_hide: true +weight: -20200208 +description: multiple instructions +--- +New feature: Product Grading (Overall Product Health) Upgrading to 1.2.8 +requires: + +1. ./manage.py makemigrations ./manage.py migrate ./manage.py + system\_settings +2. ./manage.py collectstatic \--noinput +3. pip install asteval +4. pip install \--upgrade celery +5. Complete diff --git a/docs/content/en/getting_started/upgrading/1.2.9.md b/docs/content/en/getting_started/upgrading/1.2.9.md new file mode 100644 index 0000000000..ad798aa280 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.2.9.md @@ -0,0 +1,16 @@ +--- +title: "Upgrading to DefectDojo Version 1.2.9" +toc_hide: true +weight: -20200209 +description: multiple instructions +--- +**What\'s New:** New feature: Benchmarks (OWASP ASVS) + +**Upgrading to 1.2.9 requires:** + +1. ./manage.py makemigrations ./manage.py migrate ./manage.py loaddata + dojo/fixtures/benchmark\_type.json ./manage.py loaddata + dojo/fixtures/benchmark\_category.json ./manage.py loaddata + dojo/fixtures/benchmark\_requirement.json +2. ./manage.py collectstatic \--noinput +3. Complete diff --git a/docs/content/en/getting_started/upgrading/1.3.1.md b/docs/content/en/getting_started/upgrading/1.3.1.md new file mode 100644 index 0000000000..239463d664 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.3.1.md @@ -0,0 +1,22 @@ +--- +title: "Upgrading to DefectDojo Version 1.3.1" +toc_hide: true +weight: -20200301 +description: multiple instructions +--- +**What\'s New:** + +- New importers for Contrast, Nikto and TruffleHog (finding secrets in + git repos). +- Improved merging of findings for dynamic and static importers +- Markdown support for findings +- HTML report improvements including support of Markdown. +- System settings Celery status page to assist in debugging if Celery + is functional. + +**Upgrading to 1.3.1 requires:** + +1. pip install markdown pip install pandas +2. ./manage.py makemigrations ./manage.py migrate +3. ./manage.py collectstatic \--noinput +4. Complete diff --git a/docs/content/en/getting_started/upgrading/1.7.0.md b/docs/content/en/getting_started/upgrading/1.7.0.md new file mode 100644 index 0000000000..4c70df94ce --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.7.0.md @@ -0,0 +1,130 @@ +--- +title: "Upgrading to DefectDojo Version 1.7.0" +toc_hide: true +weight: -20200700 +description: multiple instructions +--- +**What\'s New:** + +- Updated search, you can now search for CVE-XXXX-YYYY +- Updated search index, fields added to index: \'id\', \'title\', + \'cve\', \'url\', \'severity\', \'description\', \'mitigation\', + \'impact\', \'steps\_to\_reproduce\', \'severity\_justification\', + \'references\', \'sourcefilepath\', \'sourcefile\', \'hash\_code\', + \'file\_path\', \'component\_name\', \'component\_version\', + \'unique\_id\_from\_tool\' + +This requires a (one-time) rebuild of the Django-Watson search index. +Execute the django command from the defect dojo installation directory: + +`./manage.py buildwatson dojo.Finding` + +If you\'re using docker: + +`docker-compose exec uwsgi ./manage.py buildwatson dojo.Finding` + +Upgrading to DefectDojo Version 1.5.0 +------------------------------------- + +**What\'s New:** + +- Updated UI with a new DefectDojo logo, default colors and CSS. +- Updated Product views with tabs for Product Overview, Metrics, + Engagements, Endpoints, Benchmarks (ASVS), and Settings to make it + easier to navigate and manage your products. +- New Product Information fields: Regulations, Criticality, Platform, + Lifecycle, Origin, User Records, Revenue, External Audience, + Internet Accessible +- Languages pie chart on product overview, only supported through the + API and Django admin, integrates with cloc analyzer +- New Engagement type of CI/CD to support continual testing +- Engagement shortcuts and ability to import findings and auto-create + an engagement +- Engagement labels for overdue, no tests and findings +- New Contextual menus throughout DefectDojo and shortcuts to new + findings and critical findings +- Ability to merge a finding into a parent finding and either + inactivate or delete the merged findings. +- Report improvements and styling adjustment with the default option + of HTML reports +- SLA for remediation of severities based on finding criticality, for + example critical findings remediated within 7 days. Configurable in + System Settings. +- Engagement Auto-Close Days in System Settings. Automatically close + an engagement if open past the end date. +- Ability to apply remediation advice based on CWE. For example XSS + can be configured as a template so that it\'s consistent across all + findings. Enabled in system settings. +- Finding confidence field supported from scanners. First + implementation in the Burp importer. +- Goast importer for static analysis of Golang products +- Celery status check on System Settings +- Beta rules framework release for modifying findings on the fly +- DefectDojo 2.0 API with Swagger support +- Created and Modified fields on all major tables +- Various bug fixes reported on Github + +**Upgrading to 1.5.0 requirements:** + +1. Back up your database first, ideally take the backup from production + and test the upgrade on a staging server. +2. Edit the settings.py file which can be found in + `django-DefectDojo/dojo/settings/settings.py`. Copy in the rest + framework configuration after the CSRF\_COOKIE\_SECURE = True: + + REST_FRAMEWORK = { + 'DEFAULT_AUTHENTICATION_CLASSES': ( + 'rest_framework.authentication.TokenAuthentication', + 'rest_framework.authentication.BasicAuthentication', + ), + 'DEFAULT_PERMISSION_CLASSES': ( + 'rest_framework.permissions.DjangoModelPermissions', + ), + 'DEFAULT_RENDERER_CLASSES': ( + 'rest_framework.renderers.JSONRenderer', + ), + 'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.LimitOffsetPagination', + 'PAGE_SIZE': 25 + } + +Navigate to: LOGIN\_EXEMPT\_URLS and add the following after +r\'\^%sfinding/image/(?P\\[\^/\]+)\$\' % URL\_PREFIX: + + r'^%sfinding/image/(?P[^/]+)$' % URL_PREFIX, + r'^%sapi/v2/' % URL_PREFIX, + +Navigate to: INSTALLED\_APPS and add the following after: +\'multiselectfield\',: + + 'multiselectfield', + 'rest_framework', + 'rest_framework.authtoken', + 'rest_framework_swagger', + 'dbbackup', + +Navigate to: CELERY\_TASK\_IGNORE\_RESULT = True and add the following +after CELERY\_TASK\_IGNORE\_RESULT line: + + CELERY_RESULT_BACKEND = 'db+sqlite:///dojo.celeryresults.sqlite' + +Save your modified settings file. For reference the modified file should +look like the new 1.5.0 +\[settings\]() +file, minus the environmental configurations. As an alternative this +file can be used and the enviromental configurations from you +environment can be copied into this file. + +3. Activate your virtual environment and then upgrade the requirements: + +`pip install -r requirements.txt --upgrade` + +4. Upgrade the database: + + ./manage.py makemigrations + ./manage.py migrate + +5. Collect the static files (Javascript, Images, CSS): + + ./manage.py collectstatic --noinput + +6. Complete diff --git a/docs/content/en/getting_started/upgrading/1.8.0.md b/docs/content/en/getting_started/upgrading/1.8.0.md new file mode 100644 index 0000000000..150d72d7be --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.8.0.md @@ -0,0 +1,42 @@ +--- +title: "Upgrading to DefectDojo Version 1.8.0" +toc_hide: true +weight: -20200800 +description: fix buildwatson create_endpoint_status +--- +**What\'s New:** + +- See release notes: + +- Improved search, which requires an index rebuild + () + +This requires a (one-time) rebuild of the Django-Watson search index. +Execute the django command from the defect dojo installation directory: + +`./manage.py buildwatson` + +If you\'re using docker: + +`docker-compose exec uwsgi ./manage.py buildwatson` + +This can take a while depending on your hardware and the number of +findings in your instance. + +- **NOTE:** + +As a result of a breaking bug revolving around Endpoint\_status objects, +a corrective script will need to be ran after every dynamic scan +imported through either API version. + +The script can be found +[here](https://github.com/DefectDojo/django-DefectDojo/blob/dev/dojo/management/commands/create_endpoint_status.py) + +`./manage.py create\_endpoint\_status` + +If you\'re using docker: + +`docker-compose exec uwsgi ./manage.py create\_endpoint\_status` + +This can take a while depending on your hardware and the number of +findings in your instance. diff --git a/docs/content/en/getting_started/upgrading/1.9.3.md b/docs/content/en/getting_started/upgrading/1.9.3.md new file mode 100644 index 0000000000..abbef40ab0 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/1.9.3.md @@ -0,0 +1,44 @@ +--- +title: "Upgrading to DefectDojo Version 1.9.3" +toc_hide: true +weight: -20201115 +description: security release +--- +**This is a security release** + +- See the [security + advisory](https://github.com/DefectDojo/django-DefectDojo/security/advisories/GHSA-8q8j-7wc4-vjg5) +- See [release + notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/1.9.3) + +**What\'s New:** + +- See release notes: + + +**NOTE:** + +When upgrading from before 1.9.2, a corrective script may need to be ran + +`./manage.py create\_endpoint\_status` + +If you\'re using docker: + +`docker-compose exec uwsgi ./manage.py create\_endpoint\_status` + +This can take a while depending on your hardware and the number of +findings in your instance. + +- Search index tweaking index rebuild after upgrade: + +This requires a (one-time) rebuild of the Django-Watson search index. +Execute the django command from the defect dojo installation directory: + +`./manage.py buildwatson]` + +If you\'re using docker: + +`docker-compose exec uwsgi ./manage.py buildwatson` + +This can take a while depending on your hardware and the number of +findings in your instance. diff --git a/docs/content/en/getting_started/upgrading/2.0.md b/docs/content/en/getting_started/upgrading/2.0.md new file mode 100644 index 0000000000..2f45f98e63 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.0.md @@ -0,0 +1,66 @@ +--- +title: "Upgrading to DefectDojo Version 2.0.x" +toc_hide: true +weight: -20210629 +description: breaking changes +--- +Follow the usual steps to upgrade as described above. + +BEFORE UPGRADING +- If you are using SAML2 checkout the new [documentaion](https://documentation.defectdojo.com/integrations/social-authentication/#saml-20) and update you settings following the migration section. We replaced [django-saml2-auth](https://github.com/fangli/django-saml2-auth) with [djangosaml2](https://github.com/IdentityPython/djangosaml2). + +AFTER UPGRADING +- Usual migration process (`python manage.py migrate`) try to migrate all endpoints to new format and merge duplicates. +- All broken endpoints (which weren't possible to migrate) have red flag 🚩 in standard list of endpoints. +- Check if all your endpoints was migrated successfully, go to: https:///endpoint/migrate. +- Alternatively, this can be run as management command: `docker-compose exec uwsgi ./manage.py endpoint_migration --dry-run` +- When all endpoint will be fixed (there is not broken endpoint), press "Run migration" in https:///endpoint/migrate +- Or, you can run management command: `docker-compose exec uwsgi ./manage.py endpoint_migration` +- Details about endpoint migration / improvements in https://github.com/DefectDojo/django-DefectDojo/pull/4473 + +We decided to name this version 2.0.0 because we did some big cleanups in this release: + +- Remove API v1 ([#4413](https://github.com/DefectDojo/django-DefectDojo/pull/4413)) +- Remove setup.bash installation method ([#4417](https://github.com/DefectDojo/django-DefectDojo/pull/4417)) +- Rename Finding.is_Mitigated field to Finding.is_mitigated ([#3854](https://github.com/DefectDojo/django-DefectDojo/pull/4854)) +- Remove everything related to the old tagging library ([#4419](https://github.com/DefectDojo/django-DefectDojo/pull/4419)) +- Remove S0/S1/S2../S5 severity display option ([#4415](https://github.com/DefectDojo/django-DefectDojo/pull/4415)) +- Refactor EndPoint handling/formatting ([#4473](https://github.com/DefectDojo/django-DefectDojo/pull/4473)) +- Upgrade to Django 3.x ([#3632](https://github.com/DefectDojo/django-DefectDojo/pull/3632)) +- PDF Reports removed ([#4418](https://github.com/DefectDojo/django-DefectDojo/pull/4418)) +- Hashcode calculation logic has changed. To update existing findings run: + + `./manage.py dedupe --hash_code_only`. + +If you're using docker: + +`docker-compose exec uwsgi ./manage.py dedupe --hash_code_only`. + +This can take a while depending on your instance size. + +- See release notes: https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.0.0 + +### Endpoints + +- The usual migration process (`python manage.py migrate`) tries to migrate all endpoints to new format and merge duplicates. +- All broken endpoints (which weren't possible to migrate) have a red flag 🚩 in the standard list of endpoints. +- Check if all your endpoints were migrated successfully, go to: https:///endpoint/migrate. +- Alternatively, this can be run as management command: `docker-compose exec uwsgi ./manage.py endpoint_migration --dry-run` +- When all endpoint are fixed (there is not broken endpoint), press "Run migration" in https:///endpoint/migrate +- Or, you can run management command: `docker-compose exec uwsgi ./manage.py endpoint_migration` +- Details about endpoint migration / improvements in https://github.com/DefectDojo/django-DefectDojo/pull/4473 + +### Authorization + +The new authorization system for Products and Product Types based on roles is the default now. The fields for authorized users are not available anymore, but you can assign roles as described in [Permissions](../../usage/permissions). Users are migrated automatically, so that their permissions are as close as possible to the previous authorization: +- Superusers will still have all permissions on Products and Product Types, so they must not be changed. +- Staff users have had all permissions for all product types and products, so they will be get a global role as *Owner*. +- Product_Members and Product Type_Members will be added for authorized users according to the settings for the previous authorization: + - The *Reader* role is set as the default. + - If `AUTHORIZED_USERS_ALLOW_STAFF` is `True`, the user will get the *Owner* role for the respective Product or Product Type. + - If `AUTHORIZED_USERS_ALLOW_CHANGE` or `AUTHORIZED_USERS_ALLOW_DELETE` is `True`, the user will get the *Writer* role for the respective Product or Product Type. + +The new authorization is active for both UI and API. Permissions set via authorized users or via the Django Admin interface are no longer taken into account. + +Please review the roles for your users after the upgrade to avoid an unintended permissions creep. + diff --git a/docs/content/en/getting_started/upgrading/2.10.md b/docs/content/en/getting_started/upgrading/2.10.md new file mode 100644 index 0000000000..cb5a19774e --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.10.md @@ -0,0 +1,10 @@ +--- +title: "Upgrading to DefectDojo Version 2.10.x" +toc_hide: true +weight: -20220503 +description: breaking change +--- +**Breaking change for Findings:** The field `cve` will be replaced by a list of Vulnerability Ids, which can store references to security advisories associated with this finding. These can be Common Vulnerabilities and Exposures (CVE) or from other sources, eg. GitHub Security Advisories. Although the field does still exist in the code, the API and the UI have already been changed to use the list of Vulnerability Ids. Other areas like hash code calculation, search and parsers will be migrated step by step in later stages. + +This change also causes an API change for the endpoint `/engagements/{id}/accept_risks/`. + diff --git a/docs/content/en/getting_started/upgrading/2.12.md b/docs/content/en/getting_started/upgrading/2.12.md new file mode 100644 index 0000000000..10bdec5369 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.12.md @@ -0,0 +1,8 @@ +--- +title: "Upgrading to DefectDojo Version 2.12.x" +toc_hide: true +weight: -20220705 +description: breaking change +--- +**Breaking change for search:** The field `cve` has been removed from the search index for Findings and the Vulnerability Ids have been added to the search index. With this the syntax to search explicitly for vulnerability ids have been changed from `cve:` to `vulnerability_id:`, e.g. `vulnerability_id:CVE-2020-27619`. + diff --git a/docs/content/en/getting_started/upgrading/2.13.md b/docs/content/en/getting_started/upgrading/2.13.md new file mode 100644 index 0000000000..24432b2475 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.13.md @@ -0,0 +1,37 @@ +--- +title: "Upgrading to DefectDojo Version 2.13.x" +toc_hide: true +weight: -20220802 +description: instructions for helm chart and others +--- +The last release implemented the search for vulnerability ids, but the search database was not initialized. To populate the database table of the vulnerability ids, execute this django command from the defect dojo installation directory or from a shell of the Docker container or Kubernetes pod: + +`./manage.py migrate_cve` + +Additionally this requires a one-time rebuild of the Django-Watson search index. Execute this django command from the defect dojo installation directory or from a shell of the Docker container or Kubernetes pod: + +`./manage.py buildwatson` + +**Upgrade instructions for helm chart with postgres enabled**: The postgres database uses a statefulset by default. Before upgrading the helm chart we have to delete the statefullset and ensure that the pvc is reused, to keep the data. For more information: https://docs.bitnami.com/kubernetes/infrastructure/postgresql/administration/upgrade/ . + +```bash +helm repo update +helm dependency update ./helm/defectdojo + +# obtain name oft the postgres pvc +export POSTGRESQL_PVC=$(kubectl get pvc -l app.kubernetes.io/instance=defectdojo,role=primary -o jsonpath="{.items[0].metadata.name}") + +# delete postgres statefulset +kubectl delete statefulsets.apps defectdojo-postgresql --namespace default --cascade=orphan + +# upgrade +helm upgrade \ + defectdojo \ + ./helm/defectdojo/ \ + --set primary.persistence.existingClaim=$POSTGRESQL_PVC \ + ... # add your custom settings +``` + +**Further changes:** + +Legacy authorization for changing configurations based on staff users has been removed. diff --git a/docs/content/en/getting_started/upgrading/2.15.md b/docs/content/en/getting_started/upgrading/2.15.md new file mode 100644 index 0000000000..7423ae3c8d --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.15.md @@ -0,0 +1,7 @@ +--- +title: "Upgrading to DefectDojo Version 2.15.x" +toc_hide: true +weight: -20221004 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.15.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.15.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.16.md b/docs/content/en/getting_started/upgrading/2.16.md new file mode 100644 index 0000000000..3432c7b775 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.16.md @@ -0,0 +1,7 @@ +--- +title: "Upgrading to DefectDojo Version 2.16.x" +toc_hide: true +weight: -20221102 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.16.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.16.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.17.md b/docs/content/en/getting_started/upgrading/2.17.md new file mode 100644 index 0000000000..ef872a49a6 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.17.md @@ -0,0 +1,7 @@ +--- +title: "Upgrading to DefectDojo Version 2.17.x" +toc_hide: true +weight: -20221206 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.17.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.17.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.18.md b/docs/content/en/getting_started/upgrading/2.18.md new file mode 100644 index 0000000000..e608581989 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.18.md @@ -0,0 +1,23 @@ +--- +title: "Upgrading to DefectDojo Version 2.18.x" +toc_hide: true +weight: -20230103 +description: instructions for helm chart +--- +**Upgrade instructions for helm chart with rabbitMQ enabled**: The rabbitMQ uses a statefulset by default. Before upgrading the helm chart we have to ensure that all queues are empty: + +```bash +kubectl exec -i -- rabbitmqctl list_queues +``` + +Next step is to delete rabbitMQ pvc: + +```bash +kubectl delete pvc -l app.kubernetes.io/name=rabbitmq +``` + +Last step is to perform the upgrade. + +For more information: https://artifacthub.io/packages/helm/bitnami/rabbitmq/11.2.0 + + diff --git a/docs/content/en/getting_started/upgrading/2.19.md b/docs/content/en/getting_started/upgrading/2.19.md new file mode 100644 index 0000000000..0b1f2da320 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.19.md @@ -0,0 +1,13 @@ +--- +title: "Upgrading to DefectDojo Version 2.19.x" +toc_hide: true +weight: -20230206 +description: breaking change +--- +There are new docker images based on alpine with fewer third party dependencies. Related to the new images the current docker files had to be renamed and have a "-debian" or the new images a "-alpine" at the end. Furthermore there are new docker tags [DefectdojoVersion]-[OS]. For example 2.19.0-alpine or 2.19.0-debian. The currend tags (latest and [DefectdojoVersion]) are still based on the "old" images. Be aware that the new alpine images are not heavily tested and may contain bugs. + +**Breaking Change** + +In version 2.19.3, the GitHub OAuth integration has been removed to prevent configurations that may allow more access than intended. + +[DefectDojo Security Advisory: Severity Medium | Potential GitHub Authentication Misconfiguration](https://github.com/DefectDojo/django-DefectDojo/security/advisories/GHSA-hfp4-q5pg-2p7r) diff --git a/docs/content/en/getting_started/upgrading/2.2.md b/docs/content/en/getting_started/upgrading/2.2.md new file mode 100644 index 0000000000..e81dd3f02f --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.2.md @@ -0,0 +1,11 @@ +--- +title: "Upgrading to DefectDojo Version 2.2.x" +toc_hide: true +weight: -20210831 +description: No special instructions. +--- +Upgrade to 2.0.0 contained migration of endpoints. Some parts of migration haven't been done properly. This deficiency +may manifest as a doubled slash in endpoint URLs (like `http://foo.bar:8080//test`) or as a problem with deduplication +of the same endpoints. The mentioned bug was fixed in 2.2.0 and if you have seen these kinds of problems, just rerun +"Endpoint migration" as it is written in [Upgrading to DefectDojo Version 2.0.x.](#upgrading-to-defectdojo-version-20x). + diff --git a/docs/content/en/getting_started/upgrading/2.20.md b/docs/content/en/getting_started/upgrading/2.20.md new file mode 100644 index 0000000000..a2033f00b8 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.20.md @@ -0,0 +1,7 @@ +--- +title: "Upgrading to DefectDojo Version 2.20.x" +toc_hide: true +weight: -20230306 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.20.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.20.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.21.md b/docs/content/en/getting_started/upgrading/2.21.md new file mode 100644 index 0000000000..2e5726c8f9 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.21.md @@ -0,0 +1,7 @@ +--- +title: "Upgrading to DefectDojo Version 2.21.x" +toc_hide: true +weight: -20230403 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.21.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.21.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.22.md b/docs/content/en/getting_started/upgrading/2.22.md new file mode 100644 index 0000000000..1da6368422 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.22.md @@ -0,0 +1,7 @@ +--- +title: "Upgrading to DefectDojo Version 2.22.x" +toc_hide: true +weight: -20230501 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.22.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.22.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.23.md b/docs/content/en/getting_started/upgrading/2.23.md new file mode 100644 index 0000000000..5ebcc4edc6 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.23.md @@ -0,0 +1,21 @@ +--- +title: "Upgrading to DefectDojo Version 2.23.x" +toc_hide: true +weight: -20230605 +description: breaking change +--- +There is a migration from the legacy Nessus and Nessus WAS parsers to a single Tenable parser. The updated Tenable parser simply merges existing support for Nessus and Nessus WAS without introducing new functionality that could create instability + +There is a migration process built into the upgrade that will automatically convert exiting Nessus and Nessus WAS findings and tests into Tenable findings and tests + +**Breaking Change** + + - If there is any use of the Nessus or Nessus WAS in automated fashion via the import and reimport API endpoints, the `scan-type` parameter needs to be updated to `Tenable Scan` + - The default containerized database will now be [PostgreSQL](https://www.postgresql.org/) rather than [MySQL](https://dev.mysql.com/) due to the use of case insensitivity on fields by default + - It is recommended to update the [database character set and collation](https://dev.mysql.com/doc/refman/5.7/en/charset-database.html) to use UTF encoding + - If your deployment uses the MySQL containerized database, please see the following updates to run DefectDojo: + - Use of the helper script "dc-up": `./dc-up.sh mysql-rabbitmq` or `./dc-up.sh mysql-redis` + - Use of the helper script "dc-up-d": `./dc-up-d.sh mysql-rabbitmq` or `./dc-up-d.sh mysql-redis` + - Use of Docker Compose directly: `docker-compose --profile mysql-rabbitmq --env-file ./docker/environments/mysql-rabbitmq.env up` or `docker-compose --profile mysql-redis --env-file ./docker/environments/mysql-redis.env up` + +For all other changes, check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.23.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.24.md b/docs/content/en/getting_started/upgrading/2.24.md new file mode 100644 index 0000000000..b5948678ac --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.24.md @@ -0,0 +1,8 @@ +--- +title: "Upgrading to DefectDojo Version 2.24.x" +toc_hide: true +weight: -20230703 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.24.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.24.0) for the contents of the release. + diff --git a/docs/content/en/getting_started/upgrading/2.25.md b/docs/content/en/getting_started/upgrading/2.25.md new file mode 100644 index 0000000000..43502f0f96 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.25.md @@ -0,0 +1,31 @@ +--- +title: "Upgrading to DefectDojo Version 2.25.x" +toc_hide: true +weight: -20230807 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.25.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.25.0) for the contents of the release. + +A few query parameters related to filtering object via API related to a products tags have been renamed to be more consistent with the other "related object tags": + +**Breaking Change** + + - Engagement + - `product__tags__name` -> `product__tags` + - `not_product__tags__name` -> `not_product__tags` + - Test + - `engagement__product__tags__name` -> `engagement__product__tags` + - `not_engagement__product__tags__name` -> `not_engagement__product__tags` + - Finding + - `test__engagement__product__tags__name` -> `test__engagement__product__tags` + - `not_test__engagement__product__tags__name` -> `not_test__engagement__product__tags` + +**Deprecation** + +The OpenAPI 2.0 Swagger API documentation is being deprecated in favor of the existing +OpenAPI 3.0 API documentation page. The OpenAPI 2.0 Swagger API documentation page is +slated for removal in version 2.30.0 + +*Note*: The API has not changed in any way and behaves the same between OAPI2 and OAPI3 + +For all other changes, check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.25.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.26.md b/docs/content/en/getting_started/upgrading/2.26.md new file mode 100644 index 0000000000..a89c77cbc2 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.26.md @@ -0,0 +1,7 @@ +--- +title: "Upgrading to DefectDojo Version 2.26.x" +toc_hide: true +weight: -20230905 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.26.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.26.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.27.md b/docs/content/en/getting_started/upgrading/2.27.md new file mode 100644 index 0000000000..581e02296f --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.27.md @@ -0,0 +1,7 @@ +--- +title: "Upgrading to DefectDojo Version 2.27.x" +toc_hide: true +weight: -20231002 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.27.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.27.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.28.md b/docs/content/en/getting_started/upgrading/2.28.md new file mode 100644 index 0000000000..b456837cbb --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.28.md @@ -0,0 +1,7 @@ +--- +title: "Upgrading to DefectDojo Version 2.28.x" +toc_hide: true +weight: -20231106 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.28.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.28.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.29.md b/docs/content/en/getting_started/upgrading/2.29.md new file mode 100644 index 0000000000..dfce7a5bbf --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.29.md @@ -0,0 +1,7 @@ +--- +title: "Upgrading to DefectDojo Version 2.29.x" +toc_hide: true +weight: -20231110 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.29.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.29.0) for the contents of the release. diff --git a/docs/content/en/getting_started/upgrading/2.3.md b/docs/content/en/getting_started/upgrading/2.3.md new file mode 100644 index 0000000000..73e8e0b25b --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.3.md @@ -0,0 +1,12 @@ +--- +title: "Upgrading to DefectDojo Version 2.3.x" +toc_hide: true +weight: -20211005 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.3.0. +In 2.3.0 we [changed the default password hashing algorithm to Argon2 (from PBKDF2)](https://github.com/DefectDojo/django-DefectDojo/pull/5205). +When logging in, exising hashes get replaced by an Argon2 hash. If you want to rehash password without users having to login, +please see the [Django password management docs](https://docs.djangoproject.com/en/3.2/topics/auth/passwords/). +The previous password hashing algorithm (PBKDF2) was not unsafe, but we wanted to follow the [OWASP guidelines](https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html). + diff --git a/docs/content/en/getting_started/upgrading/2.4.md b/docs/content/en/getting_started/upgrading/2.4.md new file mode 100644 index 0000000000..36bfd7b109 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.4.md @@ -0,0 +1,14 @@ +--- +title: "Upgrading to DefectDojo Version 2.4.x (Security Release)" +toc_hide: true +weight: -20211102 +description: security Release +--- +This releases fixes a High severity vulnerability for which the details will be disclosed on November 16th in [GHSA-fwg9-752c-qh8w](https://github.com/DefectDojo/django-DefectDojo/security/advisories/GHSA-fwg9-752c-qh8w) + +There is a breaking change in the API for importing and re-importings scans with SonarQube API and Cobalt.io API. The [scan configurations +have been unified](https://github.com/DefectDojo/django-DefectDojo/pull/5289) and are set now with the attribute `api_scan_configuration`. +The existing configurations for SonarQube API and Cobalt.io API have been migrated. + +At the request of pyup.io, we had to remove the parser for Safety scans. + diff --git a/docs/content/en/getting_started/upgrading/2.5.md b/docs/content/en/getting_started/upgrading/2.5.md new file mode 100644 index 0000000000..7d45b8995c --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.5.md @@ -0,0 +1,25 @@ +--- +title: "Upgrading to DefectDojo Version 2.5.x" +toc_hide: true +weight: -20211208 +description: legacy authorization removed +--- +Legacy authorization has been completely removed with version 2.5.0. This includes removal of the migration of users +to the new authorization as described in https://documentation.defectdojo.com/getting_started/upgrading/#authorization. +If you are still using the legacy authorization, you should run the migration with ``./manage.py migrate_authorization_v2`` +before upgrading to version 2.5.0 + +This release introduces the "Forgot password" functionality (`DD_FORGOT_PASSWORD`: default `True`). The function +allows sending an e-mail with the reset password link. Missing configuration or misconfiguration of SMTP +(`DD_EMAIL_URL`) could raise an error (HTTP-500). Check and test (for example by resetting your own password) if you +configured SMTP correctly. If you want to avoid HTTP-500 and you don't want to set up SMTP, you can just simply switch +off the "Forgot password" functionality (`DD_FORGOT_PASSWORD=False`). + +Release renamed system setting `mail_notifications_from` to `email_from`. This value will not be used only for sending +notifications but also for sending the reset password emails. It is highly recommended to check the content of this +value if you are satisfied. If you installed DefectDojo earlier, you can expect `"from@example.com"` there. A fresh +installation will use `"no-reply@example.com"` + +This release [updates](https://github.com/DefectDojo/django-DefectDojo/pull/5450) our helm dependencies. There is a breaking change if you are using the mysql database from the helm chart because we replaced the deprecated chart from the stable repo with a chart from bitnami. If you have persistance enabled, ensure to backup your data before upgrading. All data get lost when replacing the mysql chart during the upgrade. For data migration take a look at the mysql backup and restore process. + +Furthermore we updated our kubernetes version. Current tests run on 1.18.16 and 1.22.0. diff --git a/docs/content/en/getting_started/upgrading/2.6.md b/docs/content/en/getting_started/upgrading/2.6.md new file mode 100644 index 0000000000..22f4a38bb8 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.6.md @@ -0,0 +1,9 @@ +--- +title: "Upgrading to DefectDojo Version 2.6.x" +toc_hide: true +weight: -20220104 +description: No special instructions. +--- +There are no special instructions for upgrading to 2.6.0. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.6.0) for the contents of the release. + +Please consult the security advisories [GHSA-f82x-m585-gj24](https://github.com/DefectDojo/django-DefectDojo/security/advisories/GHSA-f82x-m585-gj24) (moderate) and [GHSA-v7fv-g69g-x7p2](https://github.com/DefectDojo/django-DefectDojo/security/advisories/GHSA-v7fv-g69g-x7p2) (high) to see what security issues were fixed in this release. These will be published and become visible at January 18th, 2022. diff --git a/docs/content/en/getting_started/upgrading/2.7.md b/docs/content/en/getting_started/upgrading/2.7.md new file mode 100644 index 0000000000..672c3a77a0 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.7.md @@ -0,0 +1,19 @@ +--- +title: "Upgrading to DefectDojo Version 2.7.x" +toc_hide: true +weight: -20220201 +description: breaking change +--- +This release is a breaking change regarding the Choctaw Hog parser. As the maintainers of this project unified multiple parsers under the RustyHog parser, we now support the parsing of Choctaw Hog JSON output files through the Rusty Hog parser. Furthermore, we also support Gottingen Hog and Essex Hog JSON output files with the RustyHog parser. + +There is another breaking change regarding the import of SSLyze scans. The parser has been renamed from `SSLyze 3 Scan (JSON)` to `SSLyze Scan (JSON)`. The data in the database is fixed by the initializer, but it may break scripted API calls. + +Release 2.7.0 contains a beta functionality to make permissions for the configuration of DefectDojo more flexible. When the settings parameter `FEATURE_CONFIGURATION_AUTHORIZATION` is set to `True`, many configuration dialogues and API endpoints can be enabled for users or groups of users, regardless of their **Superuser** or **Staff** status, see [Configuration Permissions]({{< ref "../../usage/permissions/#configuration-permissions" >}}). + +The functionality using the flag `AUTHORIZATION_STAFF_OVERRIDE` has been removed. The same result can be achieved with giving the staff users a global Owner role. + +To support the transition for these 2 changes, you can run a migration script with ``./manage.py migrate_staff_users``. This script: + +* creates a group for all staff users, +* sets all configuration permissions that staff users had and +* sets the global Owner role, if `AUTHORIZATION_STAFF_OVERRIDE` is set to `True`. diff --git a/docs/content/en/getting_started/upgrading/2.8.md b/docs/content/en/getting_started/upgrading/2.8.md new file mode 100644 index 0000000000..0667084706 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.8.md @@ -0,0 +1,15 @@ +--- +title: "Upgrading to DefectDojo Version 2.8.x" +toc_hide: true +weight: -20220301 +description: breaking changes +--- +**Breaking change for Docker Compose:** Starting DefectDojo with Docker Compose now supports 2 databases (MySQL and PostgreSQL) and 2 celery brokers (RabbitMQ and Redis). To make this possible, docker-compose needs to be started with the parameters `--profile` and `--env-file`. You can get more information in [Setup via Docker Compose - Profiles](https://github.com/DefectDojo/django-DefectDojo/blob/master/readme-docs/DOCKER.md#setup-via-docker-compose---profiles). The profile `mysql-rabbitmq` provides the same configuration as in previous releases. With this the prerequisites have changed as well: Docker requires at least version 19.03.0 and Docker Compose 1.28.0. + +**Breaking change for Helm Chart:** In one of the last releases we upgraded the redis dependency in our helm chart without renaming keys in our helm chart. We fixed this bug with this release, but you may want to check if all redis values are correct ([Pull Request](https://github.com/DefectDojo/django-DefectDojo/pull/5886)). + +The flexible permissions for the configuration of DefectDojo are now active by default. With this, the flag **Staff** for users is not relevant and not visible anymore. The old behaviour can still be activated by setting the parameter `FEATURE_CONFIGURATION_AUTHORIZATION` to `False`. If you haven't done so with the previous release, you can still run a migration script with `./manage.py migrate_staff_users`. This script: + +* creates a group for all staff users, +* sets all configuration permissions that staff users had and +* sets the global Owner role, if `AUTHORIZATION_STAFF_OVERRIDE` is set to `True`. diff --git a/docs/content/en/getting_started/upgrading/2.9.md b/docs/content/en/getting_started/upgrading/2.9.md new file mode 100644 index 0000000000..fb58a96a23 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/2.9.md @@ -0,0 +1,8 @@ +--- +title: "Upgrading to DefectDojo Version 2.9.x" +toc_hide: true +weight: -20220406 +description: breaking change for APIv2 +--- +**Breaking change for APIv2:** `configuration_url` was removed from API endpoint `/api/v2/tool_configurations/` due to redundancy. + diff --git a/docs/content/en/getting_started/upgrading/_index.md b/docs/content/en/getting_started/upgrading/_index.md new file mode 100644 index 0000000000..5a9398a0d8 --- /dev/null +++ b/docs/content/en/getting_started/upgrading/_index.md @@ -0,0 +1,73 @@ +--- +title: "Upgrading" +description: "Release specific upgrading instructions" +draft: false +weight: 5 +--- + +## Docker-compose + +When you deploy a vanilla docker-compose, it will create a persistent +volume for your MySQL database. As long as your volume is there, you +should not lose any data. + +### Using docker images provided in DockerHub + +{{% alert title="Information" color="info" %}} +If you\'re using `latest`, then you need to pre pull the `latest` from +DockerHub to update. +{{% /alert %}} + + +The generic upgrade method for docker-compose are as follows: +- Pull the latest version + + ``` {.sourceCode .bash} + docker pull defectdojo/defectdojo-django:latest + docker pull defectdojo/defectdojo-nginx:latest + ``` + +- If you would like to use a version other than the latest, specify the version (tag) you want to upgrade to: + + ``` {.sourceCode .bash} + docker pull defectdojo/defectdojo-django:1.10.2 + docker pull defectdojo/defectdojo-nginx:1.10.2 + ``` + +- If you would like to use alpine based images, you specify the version (tag) you want to upgrade to: + + ``` {.sourceCode .bash} + docker pull defectdojo/defectdojo-django:1.10.2-alpine + docker pull defectdojo/defectdojo-nginx:1.10.2-alpine + ``` + +- Go to the directory where your docker-compose.yml file lives +- Stop DefectDojo: `./dc-stop.sh` +- Re-start DefectDojo, allowing for container recreation: + `./dc-up-d.sh` +- Database migrations will be run automatically by the initializer. + Check the output via `docker-compose logs initializer` or relevant k8s command +- If you have the initializer disabled (or if you want to be on the + safe side), run the migration command: + `docker-compose exec uwsgi /bin/bash -c "python manage.py migrate"` + +### Building your local images + +If you build your images locally and do not use the ones from DockerHub, +the instructions are the same, with the caveat that you must build your images +first. +- Pull the latest DefectDojo changes + + ``` {.sourceCode .bash} + git fetch + git pull + git merge origin/master + ``` + +Then replace the first step of the above generic upgrade method for docker-compose with: `docker-compose build` + +## godojo installations + +If you have installed DefectDojo on "iron" and wish to upgrade the installation, please see the [instructions in the repo](https://github.com/DefectDojo/godojo/blob/master/docs-and-scripts/upgrading.md). + +## Upgrade notes for each release diff --git a/docs/content/en/integrations/parsers/file/trivy_operator.md b/docs/content/en/integrations/parsers/file/trivy_operator.md index 7fe417b81e..47a93f7ebd 100644 --- a/docs/content/en/integrations/parsers/file/trivy_operator.md +++ b/docs/content/en/integrations/parsers/file/trivy_operator.md @@ -3,3 +3,5 @@ title: "Trivy Operator" toc_hide: true --- JSON report of [trivy operator scanner](https://github.com/aquasecurity/trivy-operator). + +To import the generated Vulnerability Reports, you can also use the [trivy-dojo-report-operator](https://github.com/telekom-mms/trivy-dojo-report-operator). diff --git a/dojo/db_migrations/0192_notifications_scan_added_empty.py b/dojo/db_migrations/0192_notifications_scan_added_empty.py new file mode 100644 index 0000000000..bd444af76c --- /dev/null +++ b/dojo/db_migrations/0192_notifications_scan_added_empty.py @@ -0,0 +1,19 @@ +# Generated by Django 4.1.11 on 2023-11-08 20:33 + +from django.db import migrations +import multiselectfield.db.fields + + +class Migration(migrations.Migration): + + dependencies = [ + ('dojo', '0191_alter_notifications_risk_acceptance_expiration'), + ] + + operations = [ + migrations.AddField( + model_name='notifications', + name='scan_added_empty', + field=multiselectfield.db.fields.MultiSelectField(blank=True, choices=[('slack', 'slack'), ('msteams', 'msteams'), ('mail', 'mail'), ('alert', 'alert')], default=[], help_text='Triggered whenever an (re-)import has been done (even if that created/updated/closed no findings).', max_length=24), + ), + ] diff --git a/dojo/importers/importer/importer.py b/dojo/importers/importer/importer.py index b7f9bad84a..e9508d5ec1 100644 --- a/dojo/importers/importer/importer.py +++ b/dojo/importers/importer/importer.py @@ -367,8 +367,7 @@ def import_scan(self, scan, scan_type, engagement, lead, environment, active=Non logger.debug('IMPORT_SCAN: Generating notifications') notifications_helper.notify_test_created(test) updated_count = len(new_findings) + len(closed_findings) - if updated_count > 0: - notifications_helper.notify_scan_added(test, updated_count, new_findings=new_findings, findings_mitigated=closed_findings) + notifications_helper.notify_scan_added(test, updated_count, new_findings=new_findings, findings_mitigated=closed_findings) logger.debug('IMPORT_SCAN: Updating Test progress') importer_utils.update_test_progress(test) diff --git a/dojo/importers/reimporter/reimporter.py b/dojo/importers/reimporter/reimporter.py index b057fc6260..a31d3673c7 100644 --- a/dojo/importers/reimporter/reimporter.py +++ b/dojo/importers/reimporter/reimporter.py @@ -747,15 +747,14 @@ def reimport_scan( updated_count = ( len(closed_findings) + len(reactivated_findings) + len(new_findings) ) - if updated_count > 0: - notifications_helper.notify_scan_added( - test, - updated_count, - new_findings=new_findings, - findings_mitigated=closed_findings, - findings_reactivated=reactivated_findings, - findings_untouched=untouched_findings, - ) + notifications_helper.notify_scan_added( + test, + updated_count, + new_findings=new_findings, + findings_mitigated=closed_findings, + findings_reactivated=reactivated_findings, + findings_untouched=untouched_findings, + ) logger.debug("REIMPORT_SCAN: Done") diff --git a/dojo/models.py b/dojo/models.py index 3113197140..bfef74cfae 100755 --- a/dojo/models.py +++ b/dojo/models.py @@ -3791,12 +3791,19 @@ def __str__(self): return text + " | Jira Key: " + str(self.jira_key) +NOTIFICATION_CHOICE_SLACK = ("slack", "slack") +NOTIFICATION_CHOICE_MSTEAMS = ("msteams", "msteams") +NOTIFICATION_CHOICE_MAIL = ("mail", "mail") +NOTIFICATION_CHOICE_ALERT = ("alert", "alert") + NOTIFICATION_CHOICES = ( - ("slack", "slack"), ("msteams", "msteams"), ("mail", "mail"), - ("alert", "alert") + NOTIFICATION_CHOICE_SLACK, + NOTIFICATION_CHOICE_MSTEAMS, + NOTIFICATION_CHOICE_MAIL, + NOTIFICATION_CHOICE_ALERT, ) -DEFAULT_NOTIFICATION = ("alert", "alert") +DEFAULT_NOTIFICATION = NOTIFICATION_CHOICE_ALERT class Notifications(models.Model): @@ -3806,6 +3813,7 @@ class Notifications(models.Model): test_added = MultiSelectField(choices=NOTIFICATION_CHOICES, default=DEFAULT_NOTIFICATION, blank=True) scan_added = MultiSelectField(choices=NOTIFICATION_CHOICES, default=DEFAULT_NOTIFICATION, blank=True, help_text=_('Triggered whenever an (re-)import has been done that created/updated/closed findings.')) + scan_added_empty = MultiSelectField(choices=NOTIFICATION_CHOICES, default=[], blank=True, help_text=_('Triggered whenever an (re-)import has been done (even if that created/updated/closed no findings).')) jira_update = MultiSelectField(choices=NOTIFICATION_CHOICES, default=DEFAULT_NOTIFICATION, blank=True, verbose_name=_("JIRA problems"), help_text=_("JIRA sync happens in the background, errors will be shown as notifications/alerts so make sure to subscribe")) upcoming_engagement = MultiSelectField(choices=NOTIFICATION_CHOICES, default=DEFAULT_NOTIFICATION, blank=True) stale_engagement = MultiSelectField(choices=NOTIFICATION_CHOICES, default=DEFAULT_NOTIFICATION, blank=True) diff --git a/dojo/notifications/helper.py b/dojo/notifications/helper.py index e09cb5354e..9339e4cf9a 100644 --- a/dojo/notifications/helper.py +++ b/dojo/notifications/helper.py @@ -390,6 +390,12 @@ def notify_scan_added(test, updated_count, new_findings=[], findings_mitigated=[ findings_untouched = sorted(list(findings_untouched), key=lambda x: x.numerical_severity) title = 'Created/Updated ' + str(updated_count) + " findings for " + str(test.engagement.product) + ': ' + str(test.engagement.name) + ': ' + str(test) - create_notification(event='scan_added', title=title, findings_new=new_findings, findings_mitigated=findings_mitigated, findings_reactivated=findings_reactivated, + + if updated_count == 0: + event = 'scan_added_empty' + else: + event = 'scan_added' + + create_notification(event=event, title=title, findings_new=new_findings, findings_mitigated=findings_mitigated, findings_reactivated=findings_reactivated, finding_count=updated_count, test=test, engagement=test.engagement, product=test.engagement.product, findings_untouched=findings_untouched, url=reverse('view_test', args=(test.id,))) diff --git a/dojo/settings/settings.dist.py b/dojo/settings/settings.dist.py index 4f0716a554..5a8054a557 100644 --- a/dojo/settings/settings.dist.py +++ b/dojo/settings/settings.dist.py @@ -6,6 +6,9 @@ import environ from netaddr import IPNetwork, IPSet import json +import logging + +logger = logging.getLogger(__name__) # See https://documentation.defectdojo.com/getting_started/configuration/ for options # how to tune the configuration to your needs. @@ -1267,9 +1270,13 @@ def saml2_attrib_map_format(dict): env_hashcode_fields_per_scanner = json.loads(env('DD_HASHCODE_FIELDS_PER_SCANNER')) for key, value in env_hashcode_fields_per_scanner.items(): if key in HASHCODE_FIELDS_PER_SCANNER: - print("Replacing {} with value {} from env var DD_HASHCODE_FIELDS_PER_SCANNER".format(key, value)) + logger.info("Replacing {} with value {} (previously set to {}) from env var DD_HASHCODE_FIELDS_PER_SCANNER".format(key, value, HASHCODE_FIELDS_PER_SCANNER[key])) + HASHCODE_FIELDS_PER_SCANNER[key] = value + if key not in HASHCODE_FIELDS_PER_SCANNER: + logger.info("Adding {} with value {} from env var DD_HASHCODE_FIELDS_PER_SCANNER".format(key, value)) HASHCODE_FIELDS_PER_SCANNER[key] = value + # This tells if we should accept cwe=0 when computing hash_code with a configurable list of fields from HASHCODE_FIELDS_PER_SCANNER (this setting doesn't apply to legacy algorithm) # If False and cwe = 0, then the hash_code computation will fallback to legacy algorithm for the concerned finding # Default is True (if scanner is not configured here but is configured in HASHCODE_FIELDS_PER_SCANNER, it allows null cwe) @@ -1432,7 +1439,7 @@ def saml2_attrib_map_format(dict): 'Gitleaks Scan': DEDUPE_ALGO_HASH_CODE, 'pip-audit Scan': DEDUPE_ALGO_HASH_CODE, 'Edgescan Scan': DEDUPE_ALGO_HASH_CODE, - 'Bugcrowd API': DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL, + 'Bugcrowd API Import': DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL, 'Rubocop Scan': DEDUPE_ALGO_HASH_CODE, 'JFrog Xray Scan': DEDUPE_ALGO_HASH_CODE, 'CycloneDX Scan': DEDUPE_ALGO_HASH_CODE, @@ -1464,7 +1471,10 @@ def saml2_attrib_map_format(dict): env_dedup_algorithm_per_parser = json.loads(env('DD_DEDUPLICATION_ALGORITHM_PER_PARSER')) for key, value in env_dedup_algorithm_per_parser.items(): if key in DEDUPLICATION_ALGORITHM_PER_PARSER: - print("Replacing {} with value {} from env var DD_DEDUPLICATION_ALGORITHM_PER_PARSER".format(key, value)) + logger.info("Replacing {} with value {} (previously set to {}) from env var DD_DEDUPLICATION_ALGORITHM_PER_PARSER".format(key, value, DEDUPLICATION_ALGORITHM_PER_PARSER[key])) + DEDUPLICATION_ALGORITHM_PER_PARSER[key] = value + if key not in DEDUPLICATION_ALGORITHM_PER_PARSER: + logger.info("Adding {} with value {} from env var DD_DEDUPLICATION_ALGORITHM_PER_PARSER".format(key, value)) DEDUPLICATION_ALGORITHM_PER_PARSER[key] = value DUPE_DELETE_MAX_PER_RUN = env('DD_DUPE_DELETE_MAX_PER_RUN') diff --git a/dojo/templates/dojo/view_finding.html b/dojo/templates/dojo/view_finding.html index 8feb62214a..3cb576cd62 100755 --- a/dojo/templates/dojo/view_finding.html +++ b/dojo/templates/dojo/view_finding.html @@ -271,6 +271,9 @@

CWE Vulnerability Id Found by + {% if finding.vuln_id_from_tool %} + Vuln ID from tool + {% endif %} {% endblock header_head %} @@ -422,6 +425,9 @@

{{ scanner }} {% endfor %} {% endwith %} + {% if finding.vuln_id_from_tool %} + {{ finding.vuln_id_from_tool }} + {% endif %} {% endblock header_body %} diff --git a/dojo/templates/dojo/view_risk_acceptance.html b/dojo/templates/dojo/view_risk_acceptance.html index 5022493f6d..dfba51cd07 100644 --- a/dojo/templates/dojo/view_risk_acceptance.html +++ b/dojo/templates/dojo/view_risk_acceptance.html @@ -44,7 +44,7 @@

{% include "dojo/form_fields.html" with form=risk_acceptance_form %}
- +
@@ -222,8 +222,10 @@

Accept Additional Findings

{% include "dojo/paging_snippet.html" with page=add_findings prefix="apage" %}
- diff --git a/dojo/templates/notifications/alert/scan_added_empty.tpl b/dojo/templates/notifications/alert/scan_added_empty.tpl new file mode 120000 index 0000000000..4efb291e00 --- /dev/null +++ b/dojo/templates/notifications/alert/scan_added_empty.tpl @@ -0,0 +1 @@ +scan_added.tpl \ No newline at end of file diff --git a/dojo/templates/notifications/mail/scan_added_empty.tpl b/dojo/templates/notifications/mail/scan_added_empty.tpl new file mode 120000 index 0000000000..4efb291e00 --- /dev/null +++ b/dojo/templates/notifications/mail/scan_added_empty.tpl @@ -0,0 +1 @@ +scan_added.tpl \ No newline at end of file diff --git a/dojo/templates/notifications/msteams/scan_added_empty.tpl b/dojo/templates/notifications/msteams/scan_added_empty.tpl new file mode 120000 index 0000000000..4efb291e00 --- /dev/null +++ b/dojo/templates/notifications/msteams/scan_added_empty.tpl @@ -0,0 +1 @@ +scan_added.tpl \ No newline at end of file diff --git a/dojo/templates/notifications/slack/scan_added_empty.tpl b/dojo/templates/notifications/slack/scan_added_empty.tpl new file mode 120000 index 0000000000..4efb291e00 --- /dev/null +++ b/dojo/templates/notifications/slack/scan_added_empty.tpl @@ -0,0 +1 @@ +scan_added.tpl \ No newline at end of file diff --git a/dojo/tools/bundler_audit/parser.py b/dojo/tools/bundler_audit/parser.py index 0ab00ac583..39b836711b 100644 --- a/dojo/tools/bundler_audit/parser.py +++ b/dojo/tools/bundler_audit/parser.py @@ -18,6 +18,9 @@ def get_description_for_scan_types(self, scan_type): def get_findings(self, filename, test): lines = filename.read() + if isinstance(lines, bytes): + lines = lines.decode("utf-8") # passes in unittests, but would fail in production + dupes = dict() find_date = datetime.now() warnings = lines.split("\n\n") diff --git a/dojo/tools/harbor_vulnerability/parser.py b/dojo/tools/harbor_vulnerability/parser.py index 2d802fd53b..7f5d2b8898 100644 --- a/dojo/tools/harbor_vulnerability/parser.py +++ b/dojo/tools/harbor_vulnerability/parser.py @@ -27,14 +27,18 @@ def get_findings(self, filename, test): # When doing dictionary, we can detect duplications dupes = dict() + try: + vulnerability = data["vulnerabilities"] # json output of https://pypi.org/project/harborapi/ + except (KeyError): + pass # To be compatible with update in version try: vulnerability = data[next(iter(data.keys()))]["vulnerabilities"] - except (KeyError, StopIteration): - return list() + except (KeyError, StopIteration, TypeError): + pass # Early exit if empty - if vulnerability is None: + if 'vulnerability' not in locals() or vulnerability is None: return list() for item in vulnerability: diff --git a/dojo/tools/nuclei/parser.py b/dojo/tools/nuclei/parser.py index b5838c3238..76ed959eac 100644 --- a/dojo/tools/nuclei/parser.py +++ b/dojo/tools/nuclei/parser.py @@ -27,6 +27,8 @@ def get_description_for_scan_types(self, scan_type): def get_findings(self, filename, test): filecontent = filename.read() + if isinstance(filecontent, bytes): + filecontent = filecontent.decode("utf-8") data = [] if filecontent == "" or len(filecontent) == 0: return [] diff --git a/dojo/tools/sonarqube/parser.py b/dojo/tools/sonarqube/parser.py index d05c70d040..b8026fc453 100644 --- a/dojo/tools/sonarqube/parser.py +++ b/dojo/tools/sonarqube/parser.py @@ -56,21 +56,22 @@ def get_items(self, tree, test, mode): rulesDic = dict() for rule in rules_table: rule_properties = list(rule.iter("td")) - rule_name = list(rule_properties[0].iter("a"))[0].text + rule_name = list(rule_properties[0].iter("a"))[0].text.strip() rule_details = list(rule_properties[1].iter("details"))[0] rulesDic[rule_name] = rule_details for vuln in vulnerabilities_table: vuln_properties = list(vuln.iter("td")) - vuln_rule_name = list(vuln_properties[0].iter("a"))[0].text + rule_key = list(vuln_properties[0].iter("a"))[0].text + vuln_rule_name = rule_key and rule_key.strip() vuln_severity = self.convert_sonar_severity( - vuln_properties[1].text + vuln_properties[1].text and vuln_properties[1].text.strip() ) - vuln_file_path = vuln_properties[2].text - vuln_line = vuln_properties[3].text - vuln_title = vuln_properties[4].text - vuln_mitigation = vuln_properties[5].text - vuln_key = vuln_properties[6].text + vuln_file_path = vuln_properties[2].text and vuln_properties[2].text.strip() + vuln_line = vuln_properties[3].text and vuln_properties[3].text.strip() + vuln_title = vuln_properties[4].text and vuln_properties[4].text.strip() + vuln_mitigation = vuln_properties[5].text and vuln_properties[5].text.strip() + vuln_key = vuln_properties[6].text and vuln_properties[6].text.strip() if vuln_title is None or vuln_mitigation is None: raise ValueError( "Parser ValueError: can't find a title or a mitigation for vulnerability of name " diff --git a/dojo/utils.py b/dojo/utils.py index 5a759a53b4..f74ce8f33d 100644 --- a/dojo/utils.py +++ b/dojo/utils.py @@ -2398,17 +2398,57 @@ def sum_by_severity_level(metrics): def get_open_findings_burndown(product): - findings = Finding.objects.filter(test__engagement__product=product) + findings = Finding.objects.filter(test__engagement__product=product, duplicate=False) f_list = list(findings) curr_date = datetime.combine(datetime.now(), datetime.min.time()) start_date = curr_date - timedelta(days=90) - critical_count = len(list(findings.filter(date__lt=start_date).filter(severity='Critical'))) - high_count = len(list(findings.filter(date__lt=start_date).filter(severity='High'))) - medium_count = len(list(findings.filter(date__lt=start_date).filter(severity='Medium'))) - low_count = len(list(findings.filter(date__lt=start_date).filter(severity='Low'))) - info_count = len(list(findings.filter(date__lt=start_date).filter(severity='Info'))) + critical_count = 0 + high_count = 0 + medium_count = 0 + low_count = 0 + info_count = 0 + + # count all findings older than 90 days that are still active OR will be mitigated/risk-accepted in the next 90 days + for f in list(findings.filter(date__lt=start_date)): + if f.active: + if f.severity == 'Critical': + critical_count += 1 + if f.severity == 'High': + high_count += 1 + if f.severity == 'Medium': + medium_count += 1 + if f.severity == 'Low': + low_count += 1 + if f.severity == 'Info': + info_count += 1 + elif f.is_mitigated: + f_mitigated_date = f.mitigated.timestamp() + if f_mitigated_date >= start_date.timestamp(): + if f.severity == 'Critical': + critical_count += 1 + if f.severity == 'High': + high_count += 1 + if f.severity == 'Medium': + medium_count += 1 + if f.severity == 'Low': + low_count += 1 + if f.severity == 'Info': + info_count += 1 + elif f.risk_accepted: + f_risk_accepted_date = f.risk_acceptance.created.timestamp() + if f_risk_accepted_date >= start_date.timestamp(): + if f.severity == 'Critical': + critical_count += 1 + if f.severity == 'High': + high_count += 1 + if f.severity == 'Medium': + medium_count += 1 + if f.severity == 'Low': + low_count += 1 + if f.severity == 'Info': + info_count += 1 running_min, running_max = float('inf'), float('-inf') past_90_days = { @@ -2419,6 +2459,7 @@ def get_open_findings_burndown(product): 'Info': [] } + # count the number of open findings for the 90-day window for i in range(90, -1, -1): start = (curr_date - timedelta(days=i)) @@ -2426,6 +2467,7 @@ def get_open_findings_burndown(product): d_end = (start + timedelta(days=1)).timestamp() for f in f_list: + # If a finding was opened on this day we add it to the counter of that day f_open_date = datetime.combine(f.date, datetime.min.time()).timestamp() if f_open_date >= d_start and f_open_date < d_end: if f.severity == 'Critical': @@ -2439,6 +2481,7 @@ def get_open_findings_burndown(product): if f.severity == 'Info': info_count += 1 + # If a finding was mitigated on this day we subtract it if f.is_mitigated: f_mitigated_date = f.mitigated.timestamp() if f_mitigated_date >= d_start and f_mitigated_date < d_end: @@ -2453,6 +2496,21 @@ def get_open_findings_burndown(product): if f.severity == 'Info': info_count -= 1 + # If a finding was risk accepted on this day we subtract it + elif f.risk_accepted: + f_risk_accepted_date = f.risk_acceptance.created.timestamp() + if f_risk_accepted_date >= d_start and f_risk_accepted_date < d_end: + if f.severity == 'Critical': + critical_count -= 1 + if f.severity == 'High': + high_count -= 1 + if f.severity == 'Medium': + medium_count -= 1 + if f.severity == 'Low': + low_count -= 1 + if f.severity == 'Info': + info_count -= 1 + f_day = [critical_count, high_count, medium_count, low_count, info_count] if min(f_day) < running_min: running_min = min(f_day) diff --git a/helm/defectdojo/Chart.yaml b/helm/defectdojo/Chart.yaml index 29c8e9385d..0d1b3b8bea 100644 --- a/helm/defectdojo/Chart.yaml +++ b/helm/defectdojo/Chart.yaml @@ -2,7 +2,7 @@ apiVersion: v2 appVersion: "2.29.0-dev" description: A Helm chart for Kubernetes to install DefectDojo name: defectdojo -version: 1.6.96-dev +version: 1.6.98-dev icon: https://www.defectdojo.org/img/favicon.ico maintainers: - name: madchap diff --git a/requirements.txt b/requirements.txt index d914608aa4..06044afc4c 100644 --- a/requirements.txt +++ b/requirements.txt @@ -2,14 +2,14 @@ asteval==0.9.31 bleach==6.1.0 bleach[css] -celery==5.3.5 +celery==5.3.6 coverage==7.3.2 defusedxml==0.7.1 django_celery_results==2.5.1 django-auditlog==2.3.0 django-dbbackup==4.0.2 django-environ==0.11.2 -django-filter==23.3 +django-filter==23.4 django-imagekit==5.0.0 # This library is very outdated, but is a pillar of DefectDojo # django-multiselectfield==0.1.12 @@ -28,7 +28,7 @@ Django==4.1.13 djangorestframework==3.14.0 gunicorn==21.2.0 html2text==2020.1.16 -humanize==4.8.0 +humanize==4.9.0 jira==3.5.2 PyGithub==1.58.2 lxml==4.9.3 @@ -43,7 +43,7 @@ python-dateutil==2.8.2 pytz==2023.3.post1 redis==5.0.1 requests==2.31.0 -sqlalchemy==2.0.22 # Required by Celery broker transport +sqlalchemy==2.0.23 # Required by Celery broker transport supervisor==4.2.5 urllib3==1.26.18 uWSGI==2.0.23 @@ -78,7 +78,7 @@ django-ratelimit==4.1.0 argon2-cffi==23.1.0 blackduck==1.1.0 pycurl==7.45.2 # Required for Celery Broker AWS (SQS) support -boto3==1.29.1 # Required for Celery Broker AWS (SQS) support +boto3==1.29.7 # Required for Celery Broker AWS (SQS) support netaddr==0.8.0 vulners==2.1.1 fontawesomefree==6.4.2 diff --git a/unittests/scans/harbor_vulnerability/harborapipip.json b/unittests/scans/harbor_vulnerability/harborapipip.json new file mode 100644 index 0000000000..356d7ecf03 --- /dev/null +++ b/unittests/scans/harbor_vulnerability/harborapipip.json @@ -0,0 +1,86 @@ +{ + "generated_at": "2023-11-16T00:14:12.726598+00:00", + "artifact": null, + "scanner": { + "name": "Trivy", + "vendor": "Aqua Security", + "version": "v0.44.0" + }, + "severity": "High", + "vulnerabilities": [ + { + "id": "CVE-1999-123", + "package": "libs", + "version": "1.2.3.4.5.6", + "fix_version": "", + "severity": "Medium", + "description": "out-of-bounds write to the ram", + "links": [ + "https://avd.aquasec.com/nvd/cve-1999-123" + ], + "preferred_cvss": { + "score_v3": 9.8, + "score_v2": null, + "vector_v3": "", + "vector_v2": "" + }, + "cwe_ids": [ + "CWE-787" + ], + "vendor_attributes": { + "CVSS": { + "nvd": { + "V2Score": 7.5, + "V2Vector": "AV:N/AC:L/Au:N/C:P/I:P/A:P", + "V3Score": 9.8, + "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" + }, + "redhat": { + "V3Score": 4, + "V3Vector": "CVSS:3.0/AV:L/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L" + } + } + }, + "artifact_digests": [ + "sha256:1829318312389123819231839" + ] + }, + { + "id": "CVE-1999-1234", + "package": "asdf", + "version": "1.2.3.4.5", + "fix_version": "", + "severity": "High", + "description": "Lorem ipsum.", + "links": [ + "https://avd.aquasec.com/nvd/cve-1999-1234" + ], + "preferred_cvss": { + "score_v3": 7.5, + "score_v2": null, + "vector_v3": "", + "vector_v2": "" + }, + "cwe_ids": [ + "CWE-190" + ], + "vendor_attributes": { + "CVSS": { + "nvd": { + "V2Score": 5, + "V2Vector": "AV:N/AC:L/Au:N/C:N/I:N/A:P", + "V3Score": 7.5, + "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + }, + "redhat": { + "V3Score": 6.2, + "V3Vector": "CVSS:3.1/AV:L/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + }, + "artifact_digests": [ + "sha256:3db2f7b39ef243df9640a3844c95e5cd403447a0dcf8cb4f1cbb5e034971b29b" + ] + } + ] +} \ No newline at end of file diff --git a/unittests/scans/nmap_scanme.json b/unittests/scans/nmap_scanme.json deleted file mode 100644 index ca05d80a82..0000000000 --- a/unittests/scans/nmap_scanme.json +++ /dev/null @@ -1,190 +0,0 @@ -{ - "scan": { - "algorithm_version": 2, - "end_time": "Tue, 04 May 2021 09:07:02 GMT", - "grade": "F", - "hidden": false, - "likelihood_indicator": "MEDIUM", - "response_headers": { - "Accept-Ranges": "bytes", - "Connection": "Keep-Alive", - "Content-Type": "text/html; charset=utf-8", - "Date": "Tue, 04 May 2021 09:07:01 GMT", - "Keep-Alive": "timeout=5, max=100", - "Server": "Apache/2.4.6 (CentOS)", - "Strict-Transport-Security": "max-age=31536000; preload", - "Transfer-Encoding": "chunked" - }, - "scan_id": 18995270, - "score": 0, - "start_time": "Tue, 04 May 2021 09:07:00 GMT", - "state": "FINISHED", - "status_code": 200, - "tests_failed": 7, - "tests_passed": 5, - "tests_quantity": 12 - }, - "tests": { - "content-security-policy": { - "expectation": "csp-implemented-with-no-unsafe", - "name": "content-security-policy", - "output": { - "data": null, - "http": false, - "meta": false, - "policy": null - }, - "pass": false, - "result": "csp-not-implemented", - "score_description": "Content Security Policy (CSP) header not implemented", - "score_modifier": -25 - }, - "contribute": { - "expectation": "contribute-json-only-required-on-mozilla-properties", - "name": "contribute", - "output": { - "data": null - }, - "pass": true, - "result": "contribute-json-only-required-on-mozilla-properties", - "score_description": "Contribute.json isn't required on websites that don't belong to Mozilla", - "score_modifier": 0 - }, - "cookies": { - "expectation": "cookies-secure-with-httponly-sessions", - "name": "cookies", - "output": { - "data": null, - "sameSite": null - }, - "pass": true, - "result": "cookies-not-found", - "score_description": "No cookies detected", - "score_modifier": 0 - }, - "cross-origin-resource-sharing": { - "expectation": "cross-origin-resource-sharing-not-implemented", - "name": "cross-origin-resource-sharing", - "output": { - "data": { - "acao": null, - "clientaccesspolicy": null, - "crossdomain": null - } - }, - "pass": true, - "result": "cross-origin-resource-sharing-not-implemented", - "score_description": "Content is not visible via cross-origin resource sharing (CORS) files or headers", - "score_modifier": 0 - }, - "public-key-pinning": { - "expectation": "hpkp-not-implemented", - "name": "public-key-pinning", - "output": { - "data": null, - "includeSubDomains": false, - "max-age": null, - "numPins": null, - "preloaded": false - }, - "pass": true, - "result": "hpkp-invalid-cert", - "score_description": "HTTP Public Key Pinning (HPKP) header cannot be set, as site contains an invalid certificate chain", - "score_modifier": 0 - }, - "redirection": { - "expectation": "redirection-to-https", - "name": "redirection", - "output": { - "destination": "https://nmap.org/", - "redirects": true, - "route": [ - "http://nmap-scanme.nmap.org/", - "https://nmap.org/" - ], - "status_code": 301 - }, - "pass": false, - "result": "redirection-off-host-from-http", - "score_description": "Initial redirection from HTTP to HTTPS is to a different host, preventing HSTS", - "score_modifier": -5 - }, - "referrer-policy": { - "expectation": "referrer-policy-private", - "name": "referrer-policy", - "output": { - "data": null, - "http": false, - "meta": false - }, - "pass": true, - "result": "referrer-policy-not-implemented", - "score_description": "Referrer-Policy header not implemented", - "score_modifier": 0 - }, - "strict-transport-security": { - "expectation": "hsts-implemented-max-age-at-least-six-months", - "name": "strict-transport-security", - "output": { - "data": null, - "includeSubDomains": false, - "max-age": null, - "preload": false, - "preloaded": false - }, - "pass": false, - "result": "hsts-invalid-cert", - "score_description": "HTTP Strict Transport Security (HSTS) header cannot be set, as site contains an invalid certificate chain", - "score_modifier": -20 - }, - "subresource-integrity": { - "expectation": "sri-implemented-and-external-scripts-loaded-securely", - "name": "subresource-integrity", - "output": { - "data": { - "//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js": { - "crossorigin": null, - "integrity": null - } - } - }, - "pass": false, - "result": "sri-not-implemented-and-external-scripts-not-loaded-securely", - "score_description": "Subresource Integrity (SRI) not implemented, and external scripts are loaded over HTTP or use protocol-relative URLs via src=\"//...\"", - "score_modifier": -50 - }, - "x-content-type-options": { - "expectation": "x-content-type-options-nosniff", - "name": "x-content-type-options", - "output": { - "data": null - }, - "pass": false, - "result": "x-content-type-options-not-implemented", - "score_description": "X-Content-Type-Options header not implemented", - "score_modifier": -5 - }, - "x-frame-options": { - "expectation": "x-frame-options-sameorigin-or-deny", - "name": "x-frame-options", - "output": { - "data": null - }, - "pass": false, - "result": "x-frame-options-not-implemented", - "score_description": "X-Frame-Options (XFO) header not implemented", - "score_modifier": -20 - }, - "x-xss-protection": { - "expectation": "x-xss-protection-1-mode-block", - "name": "x-xss-protection", - "output": { - "data": null - }, - "pass": false, - "result": "x-xss-protection-not-implemented", - "score_description": "X-XSS-Protection header not implemented", - "score_modifier": -10 - } - } -} diff --git a/unittests/scans/sonarqube/sonar-table-in-table-with-whitespace.html b/unittests/scans/sonarqube/sonar-table-in-table-with-whitespace.html new file mode 100644 index 0000000000..c8554b35ec --- /dev/null +++ b/unittests/scans/sonarqube/sonar-table-in-table-with-whitespace.html @@ -0,0 +1,598 @@ + + + + + + SonarQube Vulnerability Report + + + + +
+ + +

SonarQube Vulnerability Report

+
+
Report Generated On
+
Fri Aug 02 2019
+
Project Name
+
java tomcat
+
Application
+
tomcat
+
Release
+
1.0.0
+
Delta analysis
+
No
+
+

Summary of the Detected Vulnerabilities +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
SeverityNumber of Issues
BLOCKER + 0 +
CRITICAL + 2 +
MAJOR + 0 +
MINOR + 5 +
+ + + + +
+ +
+ +

Detail of the Detected Vulnerabilities +

+ + + + + + + + + + + + + + + + + + + + + + + + + + +
RuleSeverityComponentLineDescriptionMessageStatus
+ squid:S2975 + + BLOCKER + + java/org/apache/catalina/util/URLEncoder.java + + 190 + + "clone" should not be overridden + + Remove this "clone" implementation; use a copy constructor or copy factory instead. + TO_REVIEW
+

Known Security Rules

+ + + + + + + + + + + + + + + + + + + + + + + + + +
RuleDescription
squid:S864 + +
+

The rules of operator precedence are complicated and can lead to errors. For this reason, + parentheses should be used for clarification in complex + statements. However, this does not mean that parentheses should be gratuitously added around + every operation.

+

This rule raises issues when && and || are used in + combination, when assignment and equality or relational + operators are used in together in a condition, and for other operator combinations according + to the following table:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+, -, *, /, %<<, >>, >>>&^|
+, -, *, /, %xxxx
<<, >>, >>>xxxx
&xxxx
^xxxx
|xxxx
+

Noncompliant Code Example

+
+x = a + b - c;
+x = a + 1 << b;  // Noncompliant
+
+if ( a > b || c < d || a == d) {...}
+if ( a > b && c < d || a == b) {...}  // Noncompliant
+if (a = f(b,c) == 1) { ... } // Noncompliant; == evaluated first
+
+

Compliant Solution

+
+x = a + b - c;
+x = (a + 1) << b;
+
+if ( a > b || c < d || a == d) {...}
+if ( (a > b && c < d) || a == b) {...}
+if ( (a = f(b,c)) == 1) { ... }
+
+

See

+
    +
  • MISRA C:2004, 12.1 - Limited dependence should be placed on C's operator precedence + rules in expressions +
  • +
  • MISRA C:2004, 12.2 - The value of an expression shall be the same under any order of + evaluation that the standard permits. +
  • +
  • MISRA C:2004, 12.5 - The operands of a logical && or || shall be + primary-expressions. +
  • +
  • MISRA C++:2008, 5-0-1 - The value of an expression shall be the same under any order of + evaluation that the standard permits. +
  • +
  • MISRA C++:2008, 5-0-2 - Limited dependence should be placed on C++ operator precedence + rules in expressions +
  • +
  • MISRA C++:2008, 5-2-1 - Each operand of a logical && or || shall be a + postfix-expression. +
  • +
  • MISRA C:2012, 12.1 - The precedence of operators within expressions should be made + explicit +
  • +
  • CERT, EXP00-C. - Use + parentheses for precedence of operation +
  • +
  • CERT, EXP53-J. - Use + parentheses for precedence of operation +
  • +
  • MITRE, CWE-783 - Operator + Precedence Logic Error +
  • +
+
+
squid:S2115 + +
+

Failure to password-protect a database is so careless or naive as to be almost negligent. + Databases should always be password protected, but the + use of a database connection with an empty password is a clear indication of a database that + is not protected.

+

This rule flags database connections with empty passwords.

+

Noncompliant Code Example

+
+Connection conn = DriverManager.getConnection("jdbc:derby:memory:myDB;create=true", "AppLogin", "");
+Connection conn2 = DriverManager.getConnection("jdbc:derby:memory:myDB;create=true?user=user&password=");
+
+

Compliant Solution

+
+DriverManager.getConnection("jdbc:derby:memory:myDB;create=true?user=user&password=password");
+
+DriverManager.getConnection("jdbc:mysql://address=(host=myhost1)(port=1111)(key1=value1)(user=sandy)(password=secret),address=(host=myhost2)(port=2222)(key2=value2)(user=sandy)(password=secret)/db");
+
+DriverManager.getConnection("jdbc:mysql://sandy:secret@[myhost1:1111,myhost2:2222]/db");
+
+String url = "jdbc:postgresql://localhost/test";
+Properties props = new Properties();
+props.setProperty("user", "fred");
+props.setProperty("password", "secret");
+DriverManager.getConnection(url, props);
+
+

See

+ +
+
squid:S1148 + +
+

Throwable.printStackTrace(...) prints a Throwable and its stack + trace to some stream. By default that stream + System.Err, which could inadvertently expose sensitive information.

+

Loggers should be used instead to print Throwables, as they have many + advantages:

+
    +
  • Users are able to easily retrieve the logs.
  • +
  • The format of log messages is uniform and allow users to browse the logs easily.
  • +
+

This rule raises an issue when printStackTrace is used without arguments, i.e. + when the stack trace is printed to the default + stream.

+

Noncompliant Code Example

+
+try {
+  /* ... */
+} catch(Exception e) {
+  e.printStackTrace();        // Noncompliant
+}
+
+

Compliant Solution

+
+try {
+  /* ... */
+} catch(Exception e) {
+  LOGGER.log("context", e);
+}
+
+

See

+ +
+
squid:S2975 + +
+

Many consider clone and Cloneable broken in Java, largely because the rules for overriding clone are tricky +and difficult to get right, according to Joshua Bloch:

+
+ Object's clone method is very tricky. It's based on field copies, and it's "extra-linguistic." It creates an object without calling a constructor. + There are no guarantees that it preserves the invariants established by the constructors. There have been lots of bugs over the years, both in and + outside Sun, stemming from the fact that if you just call super.clone repeatedly up the chain until you have cloned an object, you have a shallow + copy of the object. The clone generally shares state with the object being cloned. If that state is mutable, you don't have two independent objects. + If you modify one, the other changes as well. And all of a sudden, you get random behavior. +
+

A copy constructor or copy factory should be used instead.

+

This rule raises an issue when clone is overridden, whether or not Cloneable is implemented.

+

Noncompliant Code Example

+
+public class MyClass {
+  // ...
+
+  public Object clone() { // Noncompliant
+    //...
+  }
+}
+
+

Compliant Solution

+
+public class MyClass {
+  // ...
+
+  MyClass (MyClass source) {
+    //...
+  }
+}
+
+

See

+ +

See Also

+
    +
  • S2157 - "Cloneables" should implement + "clone" +
  • +
  • S1182 - Classes that override "clone" + should be "Cloneable" and call "super.clone()" +
  • +
+
+
+
+ + + + \ No newline at end of file diff --git a/unittests/tools/test_harbor_vulnerability_parser.py b/unittests/tools/test_harbor_vulnerability_parser.py index 433b8a193a..5f1048e1e4 100644 --- a/unittests/tools/test_harbor_vulnerability_parser.py +++ b/unittests/tools/test_harbor_vulnerability_parser.py @@ -54,3 +54,14 @@ def test_parse_file_with_multiple_vuln_has_multiple_trivy_findings(self): finding = findings[0] self.assertEqual(finding.severity, 'High') self.assertEqual(finding.cwe, '125') + + # Sample with harborapi pip + def test_parse_file_with_multiple_vuln_has_harborapi_pip_package(self): + testfile = open("unittests/scans/harbor_vulnerability/harborapipip.json") + parser = HarborVulnerabilityParser() + findings = parser.get_findings(testfile, Test()) + self.assertEqual(2, len(findings)) + + finding = findings[0] + self.assertEqual(finding.severity, 'Medium') + self.assertEqual(finding.cwe, '787') diff --git a/unittests/tools/test_sonarqube_parser.py b/unittests/tools/test_sonarqube_parser.py index 023f34d1c3..b688afa73e 100644 --- a/unittests/tools/test_sonarqube_parser.py +++ b/unittests/tools/test_sonarqube_parser.py @@ -408,3 +408,86 @@ def test_detailed_parse_file_with_vuln_issue_3725(self): findings = parser.get_findings(my_file_handle, test) # specific verifications self.assertEqual(322, len(findings)) + + def test_detailed_parse_file_table_has_whitespace(self): + """ + from version 3.1.1: sonarqube-report has new template with some change. + see: https://github.com/soprasteria/sonar-report/commit/7dab559e7ecf9ed319345e9262a8b160bd3af94f + Data table will have some whitespaces, parser should strip it before compare or use these properties. + """ + my_file_handle, product, engagement, test = self.init( + get_unit_tests_path() + "/scans/sonarqube/sonar-table-in-table-with-whitespace.html" + ) + parser = SonarQubeParser() + parser.set_mode('detailed') + findings = parser.get_findings(my_file_handle, test) + self.assertEqual(1, len(findings)) + + # check content + item = findings[0] + self.assertEqual(str, type(findings[0].title)) + self.assertEqual('"clone" should not be overridden', item.title) + self.assertEqual(int, type(item.cwe)) + self.assertEqual(0, item.cwe) + self.assertEqual(bool, type(item.active)) + self.assertEqual(True, item.active) + self.assertEqual(bool, type(item.verified)) + self.assertEqual(False, item.verified) + self.assertEqual(str, type(item.description)) + self.assertMultiLineEqual( + "Many consider clone and Cloneable broken in Java, largely because the rules for overriding clone are tricky\n" + "and difficult to get right, according to Joshua Bloch:\n" + "\n" + " Object's clone method is very tricky. It's based on field copies, and it's \"extra-linguistic.\" It creates an object without calling a constructor.\n" + " There are no guarantees that it preserves the invariants established by the constructors. There have been lots of bugs over the years, both in and\n" + " outside Sun, stemming from the fact that if you just call super.clone repeatedly up the chain until you have cloned an object, you have a shallow\n" + " copy of the object. The clone generally shares state with the object being cloned. If that state is mutable, you don't have two independent objects.\n" + " If you modify one, the other changes as well. And all of a sudden, you get random behavior.\n" + "\n" + "A copy constructor or copy factory should be used instead.\n" + "This rule raises an issue when clone is overridden, whether or not Cloneable is implemented.\n" + "**Noncompliant Code Example**\n" + "\n" + "public class MyClass {\n" + " // ...\n" + "\n" + " public Object clone() { // Noncompliant\n" + " //...\n" + " }\n" + "}\n" + "\n" + "**Compliant Solution**\n" + "\n" + "public class MyClass {\n" + " // ...\n" + "\n" + " MyClass (MyClass source) {\n" + " //...\n" + " }\n" + "}", + item.description, + ) + self.assertEqual(str, type(item.severity)) + self.assertEqual("Critical", item.severity) + self.assertEqual(str, type(item.mitigation)) + self.assertEqual( + 'Remove this "clone" implementation; use a copy constructor or copy factory instead.', + item.mitigation, + ) + self.assertEqual(str, type(item.references)) + self.assertMultiLineEqual( + "squid:S2975\n" "Copy Constructor versus Cloning\n" "S2157\n" "S1182", + item.references, + ) + self.assertEqual(str, type(item.file_path)) + self.assertEqual( + "java/org/apache/catalina/util/URLEncoder.java", item.file_path + ) + self.assertEqual(str, type(item.line)) + self.assertEqual("190", item.line) + self.assertEqual(str, type(item.unique_id_from_tool)) + self.assertEqual("AWK40IMu-pl6AHs22MnV", item.unique_id_from_tool) + self.assertEqual(bool, type(item.static_finding)) + self.assertEqual(True, item.static_finding) + self.assertEqual(bool, type(item.dynamic_finding)) + self.assertEqual(False, item.dynamic_finding)