Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(nodejs): add license parser to pnpm analyser #7036

Merged
merged 19 commits into from
Jul 3, 2024

Conversation

oscarbc96
Copy link
Contributor

@oscarbc96 oscarbc96 commented Jun 27, 2024

Description

pnpm-lock.yaml files do not contain dependency license information.
This PR parses */node_modules/<package_name>/package.json files alongside pnpm-lock.json and identify licenses.

Related Issues

Checklist

  • I've read the guidelines for contributing to this repository.
  • I've followed the conventions in the PR title.
  • I've added tests that prove my fix is effective or that my feature works.
  • I've updated the documentation with the relevant information (if needed).
  • I've added usage information (if the PR introduces new options)
  • I've included a "before" and "after" example to the description (if the PR is a user interface change).

@CLAassistant
Copy link

CLAassistant commented Jun 27, 2024

CLA assistant check
All committers have signed the CLA.

@oscarbc96 oscarbc96 changed the title feat(nodejs) add license parser to pnpm analyser feat(nodejs): add license parser to pnpm analyser Jun 27, 2024
@oscarbc96 oscarbc96 marked this pull request as ready for review June 27, 2024 14:55
@knqyf263
Copy link
Collaborator

You can reference this page to pass tests. Please feel free to ask if you need assistance.

@knqyf263
Copy link
Collaborator

knqyf263 commented Jul 1, 2024

Thanks for your contribution. LGTM.
One comment: I want as few test files as possible so we can easily maintain tests in the future. Can you please provide a minimum file structure that meets the case you want to test?

@oscarbc96
Copy link
Contributor Author

Thanks for your feedback! The tests are already consolidated into a minimal structure. The happy path test requires only two dependencies: ms, which has no additional dependencies, and @babel/parser, which has several indirect dependencies. This allows us to comprehensively test that everything is working correctly.

Due to the nature of these dependencies, it might appear that we have several test data files. However, to keep things simple, everything except the package.json file has been removed. This ensures that we maintain the necessary coverage while adhering to the minimal file structure.

@knqyf263
Copy link
Collaborator

knqyf263 commented Jul 1, 2024

I still feel like @babel/parser has too many dependencies. We don't need to use an actual package for unit tests. You can add it to integration tests.

@oscarbc96
Copy link
Contributor Author

oscarbc96 commented Jul 1, 2024

You're right. After giving it a second thought, I removed @babel/parser since testing the detection of indirect dependencies should be handled by the pnpm parser. The pnpm parser already has tests for these dependencies.

@knqyf263
Copy link
Collaborator

knqyf263 commented Jul 1, 2024

@DmitriyLewen Could you also take a look?

Copy link
Contributor

@DmitriyLewen DmitriyLewen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @oscarbc96
Thanks for your work!

Add comments. Take a look please.

docs/docs/coverage/language/nodejs.md Outdated Show resolved Hide resolved
logger *log.Logger
packageJsonParser *packagejson.Parser
lockParser language.Parser
comparer npm.Comparer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all package.json files don't use version range.
We don't need comparer to match version from package.json and pnpm-lock.yaml files.

packageJsonParser *packagejson.Parser
lockParser language.Parser
comparer npm.Comparer
license *license.License
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar case.
package.json file contain license name.
We don't need to use licenceClassifier.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need this link?

We skip this link in Required function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, it's not necessary actually.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove it please.

Copy link
Contributor

@DmitriyLewen DmitriyLewen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

cc. @knqyf263

Copy link
Collaborator

@knqyf263 knqyf263 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should update an integration test similar to npm (adding node_modules so licenses will be detected).

The test case is here.

{
name: "pnpm",
args: args{
scanner: types.VulnerabilityScanner,
input: "testdata/fixtures/repo/pnpm",
},
golden: "testdata/pnpm.json.golden",
},

You can update the golden file with mage test:updateGolden.

@oscarbc96 oscarbc96 requested a review from knqyf263 July 2, 2024 08:04
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to create a new gold file.

You need to update testdata/pnpm.json.golden file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I'm doing this right; the testdata/pnpm.json.golden file is not updating. 🤔
I'm running mage test:updateGolden. @DmitriyLewen Can you help me figure out what might be wrong?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, i missed that list-all-pkgs flag is not used for pnpm testcase.
I added flag and updated golden file in 602b4d9

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Copy link
Collaborator

@knqyf263 knqyf263 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your great contribution!

@knqyf263 knqyf263 added this pull request to the merge queue Jul 3, 2024
Merged via the queue into aquasecurity:main with commit 03ac93d Jul 3, 2024
17 checks passed
@aqua-bot aqua-bot mentioned this pull request Jul 3, 2024
skahn007gl pushed a commit to skahn007gl/trivy that referenced this pull request Jul 23, 2024
Co-authored-by: DmitriyLewen <dmitriy.lewen@smartforce.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pnpm license support
4 participants