ci: workflows for local runs and GH actions #18
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| name: GitHub CI/CD Workflow | ||
|
|
||
| on: [push, pull_request] | ||
|
|
||
| jobs: | ||
| build: | ||
|
|
||
| runs-on: ubuntu-latest | ||
| container: | ||
| image: ubuntu:24.04 | ||
| env: | ||
| OPENSSL_DIR: /usr/include/openssl | ||
| OPENSSL_LIB_DIR: /usr/lib/x86_64-linux-gnu | ||
| OPENSSL_INCLUDE_DIR: /usr/include/openssl | ||
|
|
||
| steps: | ||
| - name: Install dependencies | ||
| run: | | ||
| apt-get update | ||
| apt-get install -y curl git build-essential libssl-dev | ||
|
|
||
| - uses: actions/checkout@v4 # checkout the repository | ||
| - name: Install Rust | ||
| run: | | ||
| curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y | ||
| echo "$HOME/.cargo/bin" >> $GITHUB_PATH | ||
|
|
||
| - name: Set up JDK 11 # required for build-models.sh | ||
| uses: actions/setup-java@v3 | ||
| with: | ||
| distribution: 'adopt' | ||
| java-version: '11' | ||
|
|
||
| - name: Install Go | ||
| uses: actions/setup-go@v4 | ||
| with: | ||
| go-version: '^1.22' | ||
| - name: Install Funnel | ||
| run: | | ||
| if [ -d "funnel" ]; then rm -Rf funnel; fi | ||
| git clone https://github.com/ohsu-comp-bio/funnel.git | ||
| cd funnel && make && make install && cd .. | ||
| - uses: actions/checkout@v4 # checkout the repository | ||
| - name: Build models | ||
| run: | | ||
| bash ./build-models.sh | ||
| - name: Build | ||
| run: | | ||
| cargo build --verbose | ||
| - name: Run tests | ||
| run: | | ||
| bash ./run-tests.sh | ||
| - name: Lint | ||
| run: | | ||
| cargo clippy -- -D warnings | ||
| - name: Format | ||
| run: | | ||
| cargo fmt -- ./lib/src/serviceinfo/models/*.rs # workaround to fix autogenerated code formatting | ||
| cargo fmt -- ./lib/src/tes/models/*.rs | ||
| cargo fmt -- --check |
|
There was a problem hiding this comment. Minus the caching (which would also be useful when not triggered locally, no?), this is essentially the same workflow as the previous one. Couldn't you just add It is a considerable maintenance burden to keep multiple copies of (virtually) the same code in sync - sooner or later they will diverge! Also, please document (e.g., in There was a problem hiding this comment. I've just seen that triggering the workflow locally is documented in #29, so I guess you can ignore that. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| name: Local CI/CD Workflow | ||
|
|
||
| on: workflow_dispatch | ||
|
|
||
| jobs: | ||
| build: | ||
|
|
||
| runs-on: ubuntu-latest | ||
| container: | ||
| image: ubuntu:24.04 | ||
| env: | ||
| OPENSSL_DIR: /usr/include/openssl | ||
| OPENSSL_LIB_DIR: /usr/lib/x86_64-linux-gnu | ||
| OPENSSL_INCLUDE_DIR: /usr/include/openssl | ||
|
|
||
| steps: | ||
|
|
||
| - name: Cache Rust dependencies | ||
| uses: actions/cache@v3 | ||
| with: | ||
| path: ~/.cargo | ||
| key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} | ||
| restore-keys: ${{ runner.os }}-cargo- | ||
|
|
||
| - name: Cache Rust build output | ||
| uses: actions/cache@v3 | ||
| with: | ||
| path: target | ||
| key: ${{ runner.os }}-build-${{ hashFiles('**/Cargo.lock') }} | ||
| restore-keys: ${{ runner.os }}-build- | ||
|
|
||
| - name: Cache Maven dependencies | ||
| uses: actions/cache@v3 | ||
| with: | ||
| path: ~/.m2/repository | ||
| key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} | ||
| restore-keys: ${{ runner.os }}-maven- | ||
|
|
||
| - name: Cache Go modules | ||
| uses: actions/cache@v3 | ||
| with: | ||
| path: ~/.cache/go-build | ||
| key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }} | ||
| restore-keys: ${{ runner.os }}-go- | ||
|
|
||
| - name: Cache Funnel dependencies | ||
| uses: actions/cache@v3 | ||
| with: | ||
| path: ~/funnel/build | ||
| key: ${{ runner.os }}-funnel-${{ hashFiles('**/funnel/*') }} | ||
| restore-keys: ${{ runner.os }}-funnel- | ||
|
|
||
| - uses: actions/checkout@v4 # checkout the repository | ||
| - name: Install Rust | ||
| run: | | ||
| curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y | ||
| echo "$HOME/.cargo/bin" >> $GITHUB_PATH | ||
|
|
||
| - name: Set up JDK 11 # required for build-models.sh | ||
| uses: actions/setup-java@v3 | ||
| with: | ||
| distribution: 'adopt' | ||
| java-version: '11' | ||
|
|
||
| - name: Install Go | ||
| uses: actions/setup-go@v4 | ||
| with: | ||
| go-version: '^1.22' | ||
| - name: Install Funnel | ||
| run: | | ||
| if [ -d "funnel" ]; then rm -Rf funnel; fi | ||
| git clone https://github.com/ohsu-comp-bio/funnel.git | ||
| cd funnel && make && make install && cd .. | ||
| - uses: actions/checkout@v4 # checkout the repository | ||
| - name: Build models | ||
| run: | | ||
| . $HOME/.cargo/env | ||
| bash ./build-models.sh | ||
| - name: Build | ||
| run: | | ||
| . $HOME/.cargo/env | ||
| cargo build --verbose | ||
| - name: Run tests | ||
| run: | | ||
| . $HOME/.cargo/env | ||
| bash ./run-tests.sh | ||
| - name: Lint | ||
| run: | | ||
| . $HOME/.cargo/env | ||
| cargo clippy -- -D warnings | ||
| - name: Format | ||
| run: | | ||
| . $HOME/.cargo/env | ||
| # rustup install nightly – fails for some reason | ||
aaravm marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # rustup default nightly | ||
| cargo fmt -- ./lib/src/serviceinfo/models/*.rs # workaround to fix autogenerated code formatting | ||
| cargo fmt -- ./lib/src/tes/models/*.rs | ||
| cargo fmt -- --check # --config-path ./rustfmt.toml | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be useful to define separate jobs for running tests, linting and formatting checks. That way they can run in parallel, and it is easier to see at a glance where a problem arose. See here for an example.
Of course, for each job you only need to install what is really required for each job (e.g., JDK and Funnel will likely not be needed for linting and formatting).
Btw, this workflow also highlights the issues that may quickly arise when writing your unit/integration tests in a way that they depend on other services. Not only running the tests, but also setting them up (e.g. installing Rust, Go, JDK - and the list will quickly get longer the more services you require for testing) will take increasingly more time.
I think it is important to consider and implement a scalable testing strategy early, e.g.: