Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PR時に走るworkflowの数を減らす #382

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/workflows/actionlint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: shellcheck
on:
push:
branches:
- main
pull_request:
paths:
- .github/workflows/**
- .github/actions/**/*

jobs:
actionlint:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
# ShellCheckとPyflakesをインストールしておくと、shell: bashとshell: pythonのコードを検査してくれるようになる
#
# 参考:
# - https://github.com/rhysd/actionlint/blob/main/docs/checks.md#shellcheck-integration-for-run
# - https://github.com/rhysd/actionlint/blob/main/docs/checks.md#pyflakes-integration-for-run
- name: Update ShellCheck
run: |
sudo apt-get update
sudo apt-get install -y shellcheck
- name: Set up Python 3.8
uses: actions/setup-python@v4
with:
python-version: "3.8"
- name: Install Pyflakes
run: pip install 'pyflakes>3,<4'
- name: actionlint
run: |
bash <(curl https://raw.githubusercontent.com/rhysd/actionlint/main/scripts/download-actionlint.bash)
./actionlint -color
12 changes: 9 additions & 3 deletions .github/workflows/build_and_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,16 @@ on:
types:
- published
pull_request:
paths:
- build_util/**
Comment on lines +16 to +17
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

参考: #345

Suggested change
paths:
- build_util/**
paths:
- .github/workflows/build_and_deploy.yml
- build_util/**

- Cargo.*
- crates/voicevox_core**
- model/**
- rust-toolchain
- scripts/downloads/*
push:
branches:
- "*"
- "**/*"
branches:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
branches:
branches:

- main
env:
# releaseタグ名か、workflow_dispatchでのバージョン名か、DEBUGが入る
VERSION: ${{ github.event.release.tag_name || github.event.inputs.version || 'DEBUG' }}
Expand Down
56 changes: 56 additions & 0 deletions .github/workflows/build_cpp_example.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Build C++ example
on:
push:
branches:
- main
pull_request:
paths:
- crates/voicevox_core**
Comment on lines +7 to +8
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
paths:
- crates/voicevox_core**
paths:
- .github/workflows/build_cpp_example.yml
- crates/voicevox_core**

- '!crates/voicevox_core_python_api/**'
- model/**
- example/cpp/unix/**

jobs:
build-unix-cpp-example:
strategy:
fail-fast: false
matrix:
include:
- os: macos-latest
artifact_name: osx-x64-cpu-cpp-shared
- os: ubuntu-latest
artifact_name: linux-x64-cpu-cpp-shared
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- name: Set up Rust
uses: ./.github/actions/rust-toolchain-from-file
- name: Install cargo-binstall
uses: taiki-e/install-action@cargo-binstall
- name: Install cbindgen
uses: ./.github/actions/cargo-binstall-cbindgen
- name: build voicevox_core_c_api
run: cargo build -p voicevox_core_c_api -vv
- name: voicevox_core.hを生成
run: cbindgen --crate voicevox_core_c_api -o ./example/cpp/unix/voicevox_core/voicevox_core.h
- name: 必要なfileをunix用exampleのディレクトリに移動させる
run: |
mkdir -p example/cpp/unix/voicevox_core/
cp -v target/debug/libvoicevox_core.{so,dylib} example/cpp/unix/voicevox_core/ || true
cp -v target/debug/build/onnxruntime-sys-*/out/onnxruntime_*/onnxruntime-*/lib/libonnxruntime.so.* example/cpp/unix/voicevox_core/ || true
cp -v target/debug/build/onnxruntime-sys-*/out/onnxruntime_*/onnxruntime-*/lib/libonnxruntime.*.dylib example/cpp/unix/voicevox_core/ || true

- if: startsWith(matrix.os, 'mac')
uses: jwlawson/actions-setup-cmake@v1.13
- name: Install build dependencies
if: startsWith(matrix.os, 'ubuntu')
shell: bash
run: |
sudo apt-get update
sudo apt-get install -y cmake
- name: Build
shell: bash
run: |
cd example/cpp/unix
cmake -S . -B build
cmake --build build
6 changes: 6 additions & 0 deletions .github/workflows/cargo-deny.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
name: cargo-deny
on:
push:
branches:
- main
pull_request:
paths:
- '**/Cargo.*'
- deny.toml
Comment on lines +7 to +9
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
paths:
- '**/Cargo.*'
- deny.toml
paths:
- '**/Cargo.*'
- .github/workflows/cargo-deny.yml
- deny.toml

- rust-toolchain
jobs:
cargo-deny:
runs-on: ubuntu-20.04
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/generate_document.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ on:
branches:
- main
pull_request:
paths:
- 'Cargo.*'
- crates/voicevox_core**
Comment on lines +8 to +9
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- 'Cargo.*'
- crates/voicevox_core**
- 'Cargo.*'
- .github/workflows/generate_document.yml
- crates/voicevox_core**

- docs/**
- rust-toolchain
jobs:
generate_api_document:
runs-on: ubuntu-latest
Expand Down
20 changes: 20 additions & 0 deletions .github/workflows/shellcheck.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: shellcheck
on:
push:
branches:
- main
pull_request:
paths:
- '**.sh'
- '**.bash'
jobs:
shellcheck:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- name: Update ShellCheck
run: |
sudo apt-get update
sudo apt-get install -y shellcheck
- name: ShellCheck
run: git ls-files | grep -E '\.(ba)?sh' | xargs shellcheck
167 changes: 27 additions & 140 deletions .github/workflows/test.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

build_cpp_example.ymlに分離したbuild-unix-cpp-exampleがまだ残っています。

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

指摘ありがとうございます!

昨日の時点で気づいてはいて、ローカルでは直しています。あとで他の変更と一緒にまとめてpushするつもりでした

Original file line number Diff line number Diff line change
@@ -1,73 +1,15 @@
name: test workflow
on:
push:
branches:
- "*"
- "**/*"
branches:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
branches:
branches:

- main
pull_request:
paths:
- crates/**
- Cargo.*
- rust-toolchain
jobs:
shellcheck:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- name: Update ShellCheck
run: |
sudo apt-get update
sudo apt-get install -y shellcheck
- name: ShellCheck
run: git ls-files | grep -E '\.(ba)?sh' | xargs shellcheck

actionlint:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
# ShellCheckとPyflakesをインストールしておくと、shell: bashとshell: pythonのコードを検査してくれるようになる
#
# 参考:
# - https://github.com/rhysd/actionlint/blob/main/docs/checks.md#shellcheck-integration-for-run
# - https://github.com/rhysd/actionlint/blob/main/docs/checks.md#pyflakes-integration-for-run
- name: Update ShellCheck
run: |
sudo apt-get update
sudo apt-get install -y shellcheck
- name: Set up Python 3.8
uses: actions/setup-python@v4
with:
python-version: "3.8"
- name: Install Pyflakes
run: pip install 'pyflakes>3,<4'
- name: actionlint
run: |
bash <(curl https://raw.githubusercontent.com/rhysd/actionlint/main/scripts/download-actionlint.bash)
./actionlint -color

validate-cargo-lock:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- name: Set up Rust
uses: ./.github/actions/rust-toolchain-from-file
- name: Validate Cargo.lock
run: cargo metadata --locked --format-version 1 > /dev/null

rust-lint:
runs-on: windows-latest
steps:
- uses: actions/checkout@v3
- name: Set up Rust
uses: ./.github/actions/rust-toolchain-from-file
with:
components: clippy,rustfmt
- name: Set up Python 3.8
uses: actions/setup-python@v4
with:
python-version: "3.8"
- uses: Swatinem/rust-cache@v2
- run: cargo clippy -vv --all-features --features onnxruntime/disable-sys-build-script --tests -- -D clippy::all -D warnings --no-deps
- run: cargo clippy -vv --all-features --features onnxruntime/disable-sys-build-script -- -D clippy::all -D warnings --no-deps
- run: cargo fmt -- --check

rust-test:
test-lint-build:
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -95,22 +37,16 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: "3.8"
- name: Set up Rust
- name: Set up Rust (test only)
if: matrix.os != 'windows-2022'
uses: ./.github/actions/rust-toolchain-from-file
- uses: Swatinem/rust-cache@v2
with:
# cargoのキャッシュが原因でテストが失敗する場合はバージョン部分をカウントアップすること
key: "v2-cargo-test-cache-${{ matrix.features }}-${{ matrix.os }}"
- name: Run cargo test
shell: bash
run: cargo test -vv --features ,${{ matrix.features }}

xtask-generate-c-header:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Rust
- name: Set up Rust (test and lint)
if: matrix.os == 'windows-2022'
uses: ./.github/actions/rust-toolchain-from-file
with:
components: clippy,rustfmt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

matrixに"components"を含む形でもよいのでは?
判定するときは${{ contains(matrix.components, 'clippy') }}とかにして。

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

確かに、とりあえず移植していたので思いつきませんでした...!
ちょっとやってみますね...!

- name: Validate Cargo.lock
run: cargo metadata --locked --format-version 1 > /dev/null
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

別workflowに分離してもよさそうな気がします。cargo-fmtも。

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ジョブの数が多いと並行ジョブ数上限に引っかかって、かえってCIにかかる時間が増えそうだったので、まとめられそうなものをまとめてみましたが、分ける方がメリットが大きそうでしょうか...?
いろんな視点からの意見が知りたいです:pray:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ちゃんと追えてないのですが、Actions起動のオーバーヘッドもあるので、対象のpath範囲が同じならまとめちゃった方が早いのかなと思いました!

- name: Install cargo-binstall
uses: taiki-e/install-action@cargo-binstall
- name: Install cbindgen
Expand All @@ -121,68 +57,19 @@ jobs:
run: cargo xtask generate-c-header -o ./voicevox_core_2.h
- name: Assert these header files are same
run: diff -u --color=always ./voicevox_core_{1,2}.h

build-unix-cpp-example:
strategy:
fail-fast: false
matrix:
include:
- os: macos-latest
artifact_name: osx-x64-cpu-cpp-shared
- os: ubuntu-latest
artifact_name: linux-x64-cpu-cpp-shared
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- name: Set up Rust
uses: ./.github/actions/rust-toolchain-from-file
- name: Install cargo-binstall
uses: taiki-e/install-action@cargo-binstall
- name: Install cbindgen
uses: ./.github/actions/cargo-binstall-cbindgen
- name: build voicevox_core_c_api
run: cargo build -p voicevox_core_c_api -vv
- name: voicevox_core.hを生成
run: cbindgen --crate voicevox_core_c_api -o ./example/cpp/unix/voicevox_core/voicevox_core.h
- name: 必要なfileをunix用exampleのディレクトリに移動させる
run: |
mkdir -p example/cpp/unix/voicevox_core/
cp -v target/debug/libvoicevox_core.{so,dylib} example/cpp/unix/voicevox_core/ || true
cp -v target/debug/build/onnxruntime-sys-*/out/onnxruntime_*/onnxruntime-*/lib/libonnxruntime.so.* example/cpp/unix/voicevox_core/ || true
cp -v target/debug/build/onnxruntime-sys-*/out/onnxruntime_*/onnxruntime-*/lib/libonnxruntime.*.dylib example/cpp/unix/voicevox_core/ || true

- if: startsWith(matrix.os, 'mac')
uses: jwlawson/actions-setup-cmake@v1.13
- name: Install build dependencies
if: startsWith(matrix.os, 'ubuntu')
shell: bash
run: |
sudo apt-get update
sudo apt-get install -y cmake
- name: Build
shell: bash
run: |
cd example/cpp/unix
cmake -S . -B build
cmake --build build

build-python-api:
strategy:
fail-fast: false
matrix:
include:
- os: windows-latest
- os: macos-latest
- os: ubuntu-latest
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.8
uses: actions/setup-python@v4
- uses: Swatinem/rust-cache@v2
with:
python-version: "3.8"
- name: Set up Rust
uses: ./.github/actions/rust-toolchain-from-file
# cargoのキャッシュが原因でテストが失敗する場合はバージョン部分をカウントアップすること
key: "v2-cargo-test-cache-${{ matrix.features }}-${{ matrix.os }}"
- name: Run cargo test
shell: bash
run: cargo test -vv --features ,${{ matrix.features }}
- run: cargo clippy -vv --all-features --features onnxruntime/disable-sys-build-script --tests -- -D clippy::all -D warnings --no-deps
if: matrix.os == 'windows-2022'
- run: cargo clippy -vv --all-features --features onnxruntime/disable-sys-build-script -- -D clippy::all -D warnings --no-deps
if: matrix.os == 'windows-2022'
- run: cargo fmt -- --check
if: matrix.os == 'windows-2022'
- name: venv作成
uses: ./.github/actions/create-venv
- shell: bash
Expand Down