From f7ab5ca73d2fdf86278c68979f8b93f19410b305 Mon Sep 17 00:00:00 2001 From: Ryan Cragun Date: Thu, 19 Dec 2024 09:17:51 -0700 Subject: [PATCH] go: don't download pipeline modules when caching build modules (#29224) Various different CI jobs need Go modules in order to build or test Vault. To speed this up in CI we cache them in Github Actions. The caching requires downloading all modules first in order to upload them to the actions cache, which is performed by calling the `go-mod-download` Make target. This target will iterate over the directory tree and download Go modules in all directories that include a `go.mod` file. There are two small problems with this approach that we resolved with this PR: * Our `go-mod-download` target would download modules for all `go.mod`'s present in the directory tree, regardless of whether or not they are required to build or test Vault. Only downloading those required results in slightly smaller caches. * `tools/pipeline` is intentionally a separate Go module so as to not require its modules in order to build Vault, however, our `go-mod-download` downloading all modules requires the workflow environment to include auth credentials for internal modules. If a community contributed PRs modifies a `go.mod`, which in turn requires a new cache, the PR will always fail because it cannot download modules that require secrets. Now we avoid installing our `tools/pipeline` modules when generating our module cache which should allow community contributed PRs to execute build and Go tests, while skipping enos workflows which already required secrets and were thus skipped. Signed-off-by: Ryan Cragun --- scripts/go-helper.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/go-helper.sh b/scripts/go-helper.sh index 9d7c2a9014c5..5efaa61e427a 100755 --- a/scripts/go-helper.sh +++ b/scripts/go-helper.sh @@ -81,7 +81,7 @@ mod_download() { pushd "$(dirname "$mod")" > /dev/null || (echo "failed to push into module dir" && exit 1) GOOS=linux GOARCH=amd64 GOPRIVATE=github.com/hashicorp go mod download -x popd > /dev/null || (echo "failed to pop out of module dir" && exit 1) - done < <(find . -type f -name go.mod -print0) + done < <(find . -type f -name go.mod -not -path "./tools/pipeline/*" -print0 ) } # Tidy all the go.mod's defined in the project.