Skip to content

Commit

Permalink
go: don't download pipeline modules when caching build modules (#29224)
Browse files Browse the repository at this point in the history
Various different CI jobs need Go modules in order to build or test
Vault. To speed this up in CI we cache them in Github Actions.
The caching requires downloading all modules first in order to upload
them to the actions cache, which is performed by calling the
`go-mod-download` Make target. This target will iterate over the
directory tree and download Go modules in all directories that include
a `go.mod` file.

There are two small problems with this approach that we resolved with
this PR:
* Our `go-mod-download` target would download modules for all
  `go.mod`'s present in the directory tree, regardless of whether or not
  they are required to build or test Vault. Only downloading those
  required results in slightly smaller caches.
* `tools/pipeline` is intentionally a separate Go module so as to not
  require its modules in order to build Vault, however, our
  `go-mod-download` downloading all modules requires the workflow
  environment to include auth credentials for internal modules. If a
  community contributed PRs modifies a `go.mod`, which in turn requires
  a new cache, the PR will always fail because it cannot download
  modules that require secrets.

Now we avoid installing our `tools/pipeline` modules when generating our
module cache which should allow community contributed PRs to execute
build and Go tests, while skipping enos workflows which already required
secrets and were thus skipped.

Signed-off-by: Ryan Cragun <me@ryan.ec>
  • Loading branch information
ryancragun authored Dec 19, 2024
1 parent 32ba53f commit f7ab5ca
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion scripts/go-helper.sh
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ mod_download() {
pushd "$(dirname "$mod")" > /dev/null || (echo "failed to push into module dir" && exit 1)
GOOS=linux GOARCH=amd64 GOPRIVATE=github.com/hashicorp go mod download -x
popd > /dev/null || (echo "failed to pop out of module dir" && exit 1)
done < <(find . -type f -name go.mod -print0)
done < <(find . -type f -name go.mod -not -path "./tools/pipeline/*" -print0 )
}

# Tidy all the go.mod's defined in the project.
Expand Down

0 comments on commit f7ab5ca

Please sign in to comment.