Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend code coverage to be more meaningful #278

Merged
merged 1 commit into from
Sep 24, 2024

Conversation

mikhailshilkov
Copy link
Member

@mikhailshilkov mikhailshilkov commented Sep 24, 2024

I noticed two issues with existing code coverage stats:

  1. Only in-module data are collected, e.g. tests in the github.com/pulumi/pulumi-go-provider/tests module don't output coverage data for github.com/pulumi/pulumi-go-provider
  2. The examples and tests folders have coverage data for its resource implementation code.

This PR changes the arguments to collect data for all submodules of github.com/pulumi/pulumi-go-provider but to exclude examples and tests/grpc in codecov itself.

Copy link

codecov bot commented Sep 24, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.08%. Comparing base (ade261c) to head (5bc7401).

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #278       +/-   ##
===========================================
+ Coverage   38.32%   71.08%   +32.75%     
===========================================
  Files          45       33       -12     
  Lines        4717     4098      -619     
===========================================
+ Hits         1808     2913     +1105     
+ Misses       2745      921     -1824     
- Partials      164      264      +100     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

codecov.yml Outdated
@@ -6,3 +6,7 @@ coverage:
patch:
default:
informational: true
ignore:
# Examples implementatio and test code should not count against coverage.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why shouldn't tests/grpc count against coverage? Everything in a testing dir should be part of a test, and thus count against coverage.

Suggested change
# Examples implementatio and test code should not count against coverage.
# Examples implementation and test code should not count against coverage.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I understand it, it's a part of tests, not an object being tested. Do we want to see the percentage of lines covered in that folder?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's useful as a sanity check on the tests. If there are tests that have low coverage percentages, its a good indicator that something has gone wrong in testing.

I don't feel strongly either way, but I do think that code coverage on the tests themselves is helpful.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't feel strong either - I'll return it to err on the side of more data

@mikhailshilkov mikhailshilkov force-pushed the mikhailshilkov/codecov-exp branch from 5bc7401 to 0b7446e Compare September 24, 2024 17:54
@mikhailshilkov mikhailshilkov merged commit 9378ea6 into main Sep 24, 2024
13 checks passed
@mikhailshilkov mikhailshilkov deleted the mikhailshilkov/codecov-exp branch September 24, 2024 18:00
@pulumi-bot
Copy link

This PR has been shipped in release v0.23.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants