Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add snapshot reports #153

Merged
merged 2 commits into from
Jan 17, 2025
Merged

feat: add snapshot reports #153

merged 2 commits into from
Jan 17, 2025

Conversation

camshaft
Copy link
Member

@camshaft camshaft commented Jan 16, 2025

Description of changes:

This change adds snapshot report functionality.

Before this change, codebases were left with the binary option of enforcing full coverage of requirements or not with the --ci flag. This isn't ideal, since it assumes that all of the requirements are covered already. As such, we need something to preserve the expected state of the requirement coverage in time and then assert it matches what we want it to in CI.

Codebases wanting to use this functionality would first enable the snapshot reports in their config:

[report.snapshot]
enable = true

Next, they run duvet report. This will write a snapshot file to .duvet/snapshot.txt. They should check this in to version control to ensure it stays in sync with the codebase.

Finally in CI, running duvet report --ci will ensure the current state of the requirement coverage matches the snapshot file. If not, an error is returned and the job fails:

$ duvet report --ci
EXIT: Some(1)
Extracting requirements
Extracted requirements from 1 specifications
Scanning sources
Scanned 1 sources
Parsing annotations
Parsed 1 annotations
Loading specifications
Loaded 1 specifications
Mapping sections
Mapped 1 sections
Matching references
Matched 1 references
Sorting references
Sorted 1 references
Writing .duvet/reports/report.html
Wrote .duvet/reports/report.html
Writing .duvet/snapshot.txt
Differences detected in .duvet/snapshot.txt:
@@ -1 +1,3 @@
SPECIFICATION: [Section](my-spec.md)
+ SECTION: [Section](#section)
+ TEXT[implementation]: here is a spec
× .duvet/snapshot.txt
╰─▶ Report snapshot does not match with CI mode enabled.

The snapshot format is a very simple text format that works well with tools like git diff. It consists of the following structure:

SPECIFICATION: [Spec Title](path/to/spec)
  SECTION: [Section Name](#section-name)
    TEXT[implementation,test]: Text goes here
    TEXT[implementation]: Other text goes here

  SECTION: [Other Section Name](#other-section-name)
    TEXT[!SHOULD,todo]: Not implemented

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@camshaft camshaft force-pushed the camshaft/snapshot-report branch from 5e0b2b9 to 24aa318 Compare January 16, 2025 01:13
@camshaft camshaft marked this pull request as ready for review January 16, 2025 01:15
@camshaft camshaft requested a review from a team as a code owner January 16, 2025 01:15
@camshaft camshaft merged commit e689f3a into main Jan 17, 2025
13 checks passed
@camshaft camshaft deleted the camshaft/snapshot-report branch January 17, 2025 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants