Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache rust dependencies in CI at lint stage #916

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
2 changes: 1 addition & 1 deletion .github/workflows/docker-hub-latest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/docker-hub-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Get release tag
run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
- name: Set up QEMU
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/docs-latest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ jobs:
deploy:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Use Node.js
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version-file: .node-version

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/docs-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ jobs:
deploy:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Use Node.js
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version-file: .node-version

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/helm-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
lint-helm:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Lint Helm
uses: WyriHaximus/github-action-helm3@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/helm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
fetch-depth: 0

Expand Down
87 changes: 43 additions & 44 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,87 +20,84 @@ concurrency:

jobs:
lint-node:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- name: Use Node.js
uses: actions/setup-node@v3
with:
node-version-file: .node-version
- run: yarn --ignore-scripts
- run: yarn lint:js
- uses: actions/checkout@v4
- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version-file: .node-version
- run: yarn --ignore-scripts
- run: yarn lint:js

lint-rust:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- uses: actions-rs/toolchain@v1
with:
toolchain: stable
profile: minimal
components: rustfmt
- run: cargo fmt --all -- --check
- run: cargo clippy -- -Dwarnings
- uses: actions/checkout@v4
- uses: Swatinem/rust-cache@v2
with:
shared-key: rust-cache
- run: cargo fmt --all -- --check
- run: cargo clippy -- -Dwarnings

config:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
needs:
# for cache population
- lint-rust
env:
HOOKSHOT_BUILD_PROFILE: " " # This is equivalent to dev
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Use Node.js
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version-file: .node-version
- uses: Swatinem/rust-cache@v2
with:
shared-key: rust-cache
- run: yarn # Need to build scripts to get rust bindings
- run: yarn --silent ts-node src/config/Defaults.ts --config | diff config.sample.yml -

metrics-docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Use Node.js
uses: actions/setup-node@v3
with:
node-version-file: .node-version
- run: yarn # Need to build scripts to get rust bindings
- run: yarn --silent ts-node ./scripts/build-metrics-docs.ts | diff docs/metrics.md -

test:
# Test on LTS-1
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
env:
HOOKSHOT_BUILD_PROFILE: " "
needs:
- lint-node
- lint-rust
strategy:
matrix:
node_version: [20, 21]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node_version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node_version }}
- uses: actions-rs/toolchain@v1
with:
toolchain: stable
profile: minimal
- uses: Swatinem/rust-cache@v2
with:
shared-key: rust-cache
- run: yarn
- run: yarn test:cover

build-homerunner:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
outputs:
homerunnersha: ${{ steps.gitsha.outputs.sha }}
steps:
- name: Checkout matrix-org/complement
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
repository: matrix-org/complement
- name: Get complement git sha
id: gitsha
run: echo sha=`git rev-parse --short HEAD` >> "$GITHUB_OUTPUT"
- name: Cache homerunner
id: cached
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: homerunner
key: ${{ runner.os }}-homerunner-${{ steps.gitsha.outputs.sha }}
Expand All @@ -120,28 +117,30 @@ jobs:
go build ./cmd/homerunner

integration-test:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
timeout-minutes: 30
needs:
- test
- build-homerunner
env:
HOOKSHOT_BUILD_PROFILE: " "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this (and the other instance) intentionally a whitespace to trick the default in build-app.sh? If yes, I'd note this in a comment here (or just assign --debug?)

steps:
- name: Install Complement Dependencies
run: |
sudo apt-get update && sudo apt-get install -y libolm3
- name: Load cached homerunner bin
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: homerunner
key: ${{ runner.os }}-homerunner-${{ needs.build-synapse.outputs.homerunnersha }}
fail-on-cache-miss: true # Shouldn't happen, we build this in the needs step.
- name: Checkout matrix-hookshot
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
path: matrix-hookshot
# Setup node & run tests
- name: Use Node.js
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version-file: matrix-hookshot/.node-version
- uses: Swatinem/rust-cache@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/newsfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
changelog:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with: # Needed for comparison
fetch-depth: 0
- uses: actions/setup-python@v1
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ jobs:
draft-release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Get changelog
id: extract-changelog
env:
Expand Down
1 change: 1 addition & 0 deletions changelog.d/916.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Cache more dependencies in CI to speed up runs.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"scripts": {
"build:web": "vite build",
"build:app": "tsc --project tsconfig.json",
"build:app:rs": "napi build --dts ../src/libRs.d.ts --release ./lib",
"build:app:rs": "napi build --dts ../src/libRs.d.ts $HOOKSHOT_BUILD_PROFILE ./lib",
"build:app:fix-defs": "ts-node scripts/definitions-fixer.ts src/libRs.d.ts",
"build:docs": "ts-node scripts/build-metrics-docs.ts > docs/metrics.md && mdbook build",
"dev:web": "vite dev",
Expand Down
4 changes: 3 additions & 1 deletion scripts/build-app.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
#!/bin/bash
#!/usr/bin/env bash

# exit when any command fails
set -e

export HOOKSHOT_BUILD_PROFILE="${HOOKSHOT_BUILD_PROFILE:---release}"

echo "Building Rust layer"
yarn run build:app:rs
echo "Running rust-typescript definitions fix"
Expand Down
Loading