From ac746b97882f36c268c46261c82f953f08897415 Mon Sep 17 00:00:00 2001 From: Alfredo Gutierrez Date: Fri, 2 Aug 2024 10:18:33 -0600 Subject: [PATCH 1/6] including a release doc with the diagram flow Signed-off-by: Alfredo Gutierrez --- docs/release.md | 112 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) create mode 100644 docs/release.md diff --git a/docs/release.md b/docs/release.md new file mode 100644 index 00000000..cf484f39 --- /dev/null +++ b/docs/release.md @@ -0,0 +1,112 @@ +## Release Process Documentation + +### Overview + +This document outlines the release process for our project, detailing the steps involved in creating a new release, tagging, and publishing artifacts. + +### Kickstart Release Process + +1. **Milestone and Label Check**: + - Ensure all Pull Requests (PRs) have a milestone set and appropriate labels. This is important for generating accurate release notes. + - There is a CI check to ensure that all PRs have a milestone and labels set. If not, the PR will not be merged. + - Make sure all PRs and Issues related to the related milestone are already closed, if not closed them before proceeding or move them to the next milestone. + +### Release Automation Workflow + +The release process is automated using a GitHub Actions workflow (`release-automation.yaml`). The workflow must be triggered manually with a SemVer version input, which specifies the version to be released. The input can be: +- `x.n.0-rc1` for release candidates +- `x.n.0` for general availability (GA) versions +- `x.n.p` for patch versions, including hot-fixes + +#### Steps: + +1. **Create or Update Release Branch**: + - Create a new release branch `release/0.n` if it doesn't exist. + - If the release branch was created, means it's a new release, create a PR on the main branch to increase the snapshot version to the next version. + - Bump the version to the input provided in the release branch. + +2. **Tagging**: + - After bumping the version on the release branch, tag the version as `vX.n.p` (where `X.n.p` is the input). + - Push the tag, triggering the `Publish Release Workflow`. + +3. **Release Notes and Milestone**: + - Create release notes for the new Tag. + - Close the milestone associated with the release. + - Upload distribution artifacts to the release. + +### Publish Release Workflow + +The `Publish Release Workflow` (`publish-release.yaml`) is triggered by the Tag pushed in the previous step. This workflow performs the following tasks: + +1. **Docker Image Creation and Publishing**: + - Create a Docker image with the new release version and publish it to the GitHub Container Registry (`ghcr.io`). + +2. **Helm Chart Publishing**: + - Publish a new Helm chart with the same version to the custom GitHub Pages of the repo in the `Charts` folder. + +### Mermaid Diagram + +```mermaid +graph TD + subgraph Manual Process + A[Start Release Process] --> B{Milestone and Labels Set?} + B -->|Yes| C[Trigger release-automation.yaml] + B -->|No| D[Review Milestone and Labels] + D --> B + end + + subgraph Release Automation Workflow + C -->|Input: SemVer to release| E{Release Branch Exists?} + E -->|Yes| F[Release Branch: Bump Version to Input] + E -->|No| G[Create Release Branch] + G --> H[Main: Create PR to increase Snapshot Version] + H --> F + + F --> I[Tag Version as vX.X.X] + I --> J[Push Tag] + J --> K[Trigger publish-release.yaml] + J --> N[Create Release Notes] + N --> O[Close Milestone] + O --> P[End Release Process] + end + + subgraph Publish Release Workflow + K --> L[Create and Publish Docker Image] + L --> M[Publish Helm Chart to gh-pages] + + end +``` + +### Release Meta Process + +The meta process typically follows these steps: + +1. **Initial Release Candidate (RC)**: + - Trigger the Release Automation Workflow with an `rc` version (e.g., `x.n.0-rcY`). + - Perform integration and performance testing on this version. + +2. **General Availability (GA)**: + - Once testing is successful, trigger the Release Automation Workflow again for the same version but as a GA version (e.g., `x.n.0`). + +3. **Patch Versions**: + - For any patch versions, changes will be merged (cherry-picked from main) into the release branch. + - Start a new release automation process for the patch version (e.g., `x.n.p`). + +```mermaid +graph TD + A1[Trigger Release Automation Workflow for x.n.0-rcX] --> B1[RC Version Released] + B1 --> B2[Perform Integration and Performance Testing] + B2 --> B3{Testing Successful?} + B3 --> |No| R[Make changes and cherry-pick] --> A1 + B3-->|Yes| C1[Trigger Release Automation Workflow for x.n.0] + C1 --> D1[GA Version Released] + D1 --> E1[Merge Patches from Main to Release Branch] + E1 --> F1[Trigger Release Automation Workflow for x.n.p] + F1 --> G1[Patch Version Released] + +``` + +## Conclusion + +This release process ensures that all steps are automated, from checking PR milestones and labels to creating branches, tagging releases, and publishing Docker images and Helm charts. By following this documentation, you can streamline the release workflow and maintain consistency across all releases. + From 690efcd7422b12f384c26ef8dc277382e3f38cb5 Mon Sep 17 00:00:00 2001 From: Alfredo Gutierrez Date: Fri, 2 Aug 2024 12:16:49 -0600 Subject: [PATCH 2/6] improving doc and adding a base release automation GHA WF, copied from mirror-node, so changes are still necessary Signed-off-by: Alfredo Gutierrez --- .github/workflows/release-automation.yaml | 202 ++++++++++++++++++++++ docs/release.md | 5 +- 2 files changed, 205 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/release-automation.yaml diff --git a/.github/workflows/release-automation.yaml b/.github/workflows/release-automation.yaml new file mode 100644 index 00000000..abfb26dc --- /dev/null +++ b/.github/workflows/release-automation.yaml @@ -0,0 +1,202 @@ +name: Release Automation + +on: + workflow_dispatch: + inputs: + version: + description: "Version (semver)" + required: true + +permissions: + contents: write + issues: write + +defaults: + run: + shell: bash + +env: + LC_ALL: C.UTF-8 + +jobs: + release: + name: Release + runs-on: [self-hosted, Linux, large, ephemeral] + env: + RELEASE_NOTES_FILENAME: release_notes + outputs: + create_pr: ${{ env.CREATE_PR }} + next_version_snapshot: ${{ env.NEXT_VERSION_SNAPSHOT }} + pr_title: ${{ env.PR_TITLE }} + release_branch: ${{ env.RELEASE_BRANCH }} + + steps: + - name: Harden Runner + uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + with: + egress-policy: audit + + - name: Parse Version + id: version_parser + uses: step-security/semver-utils@c392dfe4e25826fe9fe6aa4c872c9c216b1982cb # v3.1.0 + with: + lenient: false + version: ${{ github.event.inputs.version }} + + - name: Set Release Environment Variables + run: | + PREMINOR_VERSION=${{ steps.version_parser.outputs.inc-preminor }} + NEXT_VERSION_SNAPSHOT=${PREMINOR_VERSION//-0/-SNAPSHOT} + RELEASE_BRANCH="release/${{ steps.version_parser.outputs.major }}.${{ steps.version_parser.outputs.minor }}" + [[ -z "${{ steps.version_parser.outputs.prerelease }}" ]] && \ + VERSION=${{ steps.version_parser.outputs.release }} || \ + VERSION="${{ steps.version_parser.outputs.release }}-${{ steps.version_parser.outputs.prerelease }}" + RELEASE_TAG="v${VERSION}" + + cat >> $GITHUB_ENV <> $GITHUB_ENV + echo "PR_TITLE=Bump versions for v$NEXT_VERSION_SNAPSHOT" >> $GITHUB_ENV + else + git checkout ${RELEASE_BRANCH} + fi + + - name: Gradle Release + run: ./gradlew release -Pversion=${{ env.VERSION }} + + - name: Close the Milestone + if: ${{ steps.version_parser.outputs.prerelease == '' }} + id: milestone + uses: Akkjon/close-milestone@88d3cb00ca452cb8d33f18fc0a1e22a730306b61 # v2.1.0 + with: + milestone_name: ${{ steps.version_parser.outputs.release }} + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + - name: Create Release Notes + if: ${{ steps.milestone.outputs.milestone_id != '' }} + uses: Decathlon/release-notes-generator-action@98423db7024696a339f3988ac8a2b051c5860741 # v3.1.6 + env: + FILENAME: ${{ env.RELEASE_NOTES_FILENAME }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + MILESTONE_NUMBER: ${{ steps.milestone.outputs.milestone_id }} + + - name: Commit and Tag + uses: stefanzweifel/git-auto-commit-action@8621497c8c39c72f3e2a999a26b4ca1b5058a842 # v5.0.1 + with: + commit_author: ${{ steps.gpg_importer.outputs.name }} <${{ steps.gpg_importer.outputs.email }}> + commit_message: Bump versions for ${{ env.RELEASE_TAG }} + commit_options: "--no-verify --signoff" + commit_user_name: ${{ steps.gpg_importer.outputs.name }} + commit_user_email: ${{ steps.gpg_importer.outputs.email }} + tagging_message: ${{ env.RELEASE_TAG }} + + - name: Create Github Release + uses: ncipollo/release-action@2c591bcc8ecdcd2db72b97d6147f871fcd833ba5 # v1.14.0 + with: + bodyFile: ${{ env.RELEASE_NOTES_FILENAME }}.md + commit: ${{ env.RELEASE_BRANCH }} + draft: ${{ steps.version_parser.outputs.prerelease == '' }} + name: ${{ env.RELEASE_TAG }} + omitBody: ${{ steps.milestone.outputs.milestone_id == '' }} + prerelease: ${{ steps.version_parser.outputs.prerelease != '' }} + tag: ${{ env.RELEASE_TAG }} + token: ${{ secrets.GITHUB_TOKEN }} + + create_pr: + name: Create PR + runs-on: [self-hosted, Linux, large, ephemeral] + needs: release + if: ${{ needs.release.outputs.create_pr == 'true' }} + env: + NEXT_VERSION_SNAPSHOT: ${{ needs.release.outputs.next_version_snapshot }} + RELEASE_BRANCH: ${{ needs.release.outputs.release_branch }} + + steps: + - name: Harden Runner + uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + with: + egress-policy: audit + + - name: Checkout Repository + uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 + with: + ref: main + + - name: Import GPG Key + id: gpg_importer + uses: crazy-max/ghaction-import-gpg@01dd5d3ca463c7f10f7f4f7b4f177225ac661ee4 # v6.1.0 + with: + git_commit_gpgsign: true + git_tag_gpgsign: true + git_user_signingkey: true + gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }} + passphrase: ${{ secrets.GPG_PASSPHRASE }} + + - name: Install JDK + uses: actions/setup-java@99b8673ff64fbf99d8d325f52d9a5bdedb8483e9 # v4.2.1 + with: + distribution: "temurin" + java-version: 21 + + - name: Setup Gradle + uses: gradle/actions/setup-gradle@d9c87d481d55275bb5441eef3fe0e46805f9ef70 # v3.5.0 + + - name: Reset main to release branch + run: | + git fetch origin $RELEASE_BRANCH:$RELEASE_BRANCH + git reset --hard $RELEASE_BRANCH + + - name: Gradle Release for Next Minor Snapshot + run: ./gradlew release -Pversion=${{ env.NEXT_VERSION_SNAPSHOT }} + + - name: Create Pull Request + uses: peter-evans/create-pull-request@c5a7806660adbe173f04e3e038b0ccdcd758773c # v6.1.0 + with: + author: ${{ steps.gpg_importer.outputs.name }} <${{ steps.gpg_importer.outputs.email }}> + body: Bump versions for v${{ env.NEXT_VERSION_SNAPSHOT }} + branch: create-pull-request/${{ env.RELEASE_BRANCH }} + commit-message: Bump versions for v${{ env.NEXT_VERSION_SNAPSHOT }} + committer: ${{ steps.gpg_importer.outputs.name }} <${{ steps.gpg_importer.outputs.email }}> + delete-branch: true + signoff: true + title: ${{ needs.release.outputs.pr_title }} + token: ${{ secrets.HEDERA_BOT_TOKEN }} diff --git a/docs/release.md b/docs/release.md index cf484f39..d9acd81a 100644 --- a/docs/release.md +++ b/docs/release.md @@ -22,8 +22,8 @@ The release process is automated using a GitHub Actions workflow (`release-autom 1. **Create or Update Release Branch**: - Create a new release branch `release/0.n` if it doesn't exist. - - If the release branch was created, means it's a new release, create a PR on the main branch to increase the snapshot version to the next version. - - Bump the version to the input provided in the release branch. + - If the release branch was created, means it's a new release, create a PR on the main branch to increase the snapshot version to the next version. + - Bump the version to the input provided in the release branch. 2. **Tagging**: - After bumping the version on the release branch, tag the version as `vX.n.p` (where `X.n.p` is the input). @@ -33,6 +33,7 @@ The release process is automated using a GitHub Actions workflow (`release-autom - Create release notes for the new Tag. - Close the milestone associated with the release. - Upload distribution artifacts to the release. + - Publish the artifacts to Sonatype Nexus / Maven Central. ### Publish Release Workflow From d26839174d89f454d9b8bc66e7cfa282f4a3ea6b Mon Sep 17 00:00:00 2001 From: Alfredo Gutierrez Date: Fri, 2 Aug 2024 12:27:13 -0600 Subject: [PATCH 3/6] improve docs Signed-off-by: Alfredo Gutierrez --- docs/release.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/release.md b/docs/release.md index d9acd81a..55504a08 100644 --- a/docs/release.md +++ b/docs/release.md @@ -9,7 +9,13 @@ This document outlines the release process for our project, detailing the steps 1. **Milestone and Label Check**: - Ensure all Pull Requests (PRs) have a milestone set and appropriate labels. This is important for generating accurate release notes. - There is a CI check to ensure that all PRs have a milestone and labels set. If not, the PR will not be merged. - - Make sure all PRs and Issues related to the related milestone are already closed, if not closed them before proceeding or move them to the next milestone. + - Make sure all PRs and Issues related to the related milestone are already closed, if not closed them before proceeding or move them to the next milestone. +2. **Trigger Release Automation Workflow**: + - Manually trigger the `release-automation.yaml` workflow with the desired version to be released. + - The workflow will create a new release branch, bump the version, and tag the release. + - The release branch will be used for any patch versions or hot-fixes. + - If the release branch already exists, the workflow should be triggered using the branch release, and the version should be bumped accordingly. + - for example: if `release/0.1` exists, and you want to release `0.1.1`, trigger the workflow with `0.1.1` as the semver input and select `release/0.1` on "Use workflow from" dropdown. ### Release Automation Workflow From 663b282b58969dae6f57a7ccd2ef26a8ed8d91e5 Mon Sep 17 00:00:00 2001 From: Alfredo Gutierrez Date: Mon, 5 Aug 2024 12:52:37 -0600 Subject: [PATCH 4/6] Adding gradle task to bumpVersion on gradle.properties and Chart.yaml Also improvements to the WF to be able to bump the version. Signed-off-by: Alfredo Gutierrez --- .github/workflows/release-automation.yaml | 10 +++++--- charts/hedera-block-node/Chart.yaml | 2 +- server/build.gradle.kts | 29 +++++++++++++++++++++++ 3 files changed, 37 insertions(+), 4 deletions(-) diff --git a/.github/workflows/release-automation.yaml b/.github/workflows/release-automation.yaml index abfb26dc..89e8a6c0 100644 --- a/.github/workflows/release-automation.yaml +++ b/.github/workflows/release-automation.yaml @@ -99,8 +99,12 @@ jobs: git checkout ${RELEASE_BRANCH} fi - - name: Gradle Release - run: ./gradlew release -Pversion=${{ env.VERSION }} + # task is currently failing due to needing credentials.username for PublishToMavenRepository task + #- name: Gradle Release + # run: ./gradlew release -Pversion=${{ env.VERSION }} + + - name: Bump Version + run: ./gradlew bumpVersion -Pversion=${{ env.VERSION }} - name: Close the Milestone if: ${{ steps.version_parser.outputs.prerelease == '' }} @@ -186,7 +190,7 @@ jobs: git reset --hard $RELEASE_BRANCH - name: Gradle Release for Next Minor Snapshot - run: ./gradlew release -Pversion=${{ env.NEXT_VERSION_SNAPSHOT }} + run: ./gradlew bumpVersion -Pversion=${{ env.NEXT_VERSION_SNAPSHOT }} - name: Create Pull Request uses: peter-evans/create-pull-request@c5a7806660adbe173f04e3e038b0ccdcd758773c # v6.1.0 diff --git a/charts/hedera-block-node/Chart.yaml b/charts/hedera-block-node/Chart.yaml index 19c33f98..f2e2b33a 100644 --- a/charts/hedera-block-node/Chart.yaml +++ b/charts/hedera-block-node/Chart.yaml @@ -1,5 +1,5 @@ apiVersion: v2 -appVersion: "0.1.0-SNAPSHOT" +appVersion: 0.1.0-SNAPSHOT description: A Helm chart for Hedera Block Node home: https://github.com/hashgraph/hedera-block-node keywords: diff --git a/server/build.gradle.kts b/server/build.gradle.kts index 75b890ef..615686a5 100644 --- a/server/build.gradle.kts +++ b/server/build.gradle.kts @@ -38,6 +38,35 @@ testModuleInfo { requiresStatic("com.github.spotbugs.annotations") } +// Release related tasks + +fun replaceVersion(files: String, match: String) { + ant.withGroovyBuilder { + "replaceregexp"( + "match" to match, + "replace" to project.version, + "flags" to "gm" + ) { + "fileset"( + "dir" to rootProject.projectDir, + "includes" to files, + "excludes" to "**/node_modules/" + ) + } + } +} + +tasks.register("bumpVersion") { + description = "Bump versions of the project" + group = "release" + + replaceVersion("charts/**/Chart.yaml", "(?<=^(appVersion|version): ).+") + replaceVersion("gradle.properties", "(?<=^version=).+") +} + + +// Docker related tasks + var updateDockerEnv = tasks.register("updateDockerEnv") { description = From 1b4d9679d92aa1c0e2c7f751691243cc32a8a8ab Mon Sep 17 00:00:00 2001 From: Alfredo Gutierrez Date: Wed, 7 Aug 2024 18:08:53 -0600 Subject: [PATCH 5/6] style fixes Signed-off-by: Alfredo Gutierrez --- .github/workflows/release-automation.yaml | 15 +++++++++++++++ server/build.gradle.kts | 7 +------ 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/.github/workflows/release-automation.yaml b/.github/workflows/release-automation.yaml index 89e8a6c0..54db05f8 100644 --- a/.github/workflows/release-automation.yaml +++ b/.github/workflows/release-automation.yaml @@ -1,3 +1,18 @@ +## +# Copyright (C) 2024 Hedera Hashgraph, LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +## name: Release Automation on: diff --git a/server/build.gradle.kts b/server/build.gradle.kts index 615686a5..83719732 100644 --- a/server/build.gradle.kts +++ b/server/build.gradle.kts @@ -42,11 +42,7 @@ testModuleInfo { fun replaceVersion(files: String, match: String) { ant.withGroovyBuilder { - "replaceregexp"( - "match" to match, - "replace" to project.version, - "flags" to "gm" - ) { + "replaceregexp"("match" to match, "replace" to project.version, "flags" to "gm") { "fileset"( "dir" to rootProject.projectDir, "includes" to files, @@ -64,7 +60,6 @@ tasks.register("bumpVersion") { replaceVersion("gradle.properties", "(?<=^version=).+") } - // Docker related tasks var updateDockerEnv = From 224bbf5540ab96ae2df59ae1fbff88ff0239dd26 Mon Sep 17 00:00:00 2001 From: Alfredo Gutierrez Date: Sun, 1 Sep 2024 08:10:37 -0600 Subject: [PATCH 6/6] Fixing runners, to use new ones Signed-off-by: Alfredo Gutierrez --- .github/workflows/release-automation.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/release-automation.yaml b/.github/workflows/release-automation.yaml index 54db05f8..7e71b4d8 100644 --- a/.github/workflows/release-automation.yaml +++ b/.github/workflows/release-automation.yaml @@ -36,7 +36,7 @@ env: jobs: release: name: Release - runs-on: [self-hosted, Linux, large, ephemeral] + runs-on: block-node-linux-medium env: RELEASE_NOTES_FILENAME: release_notes outputs: @@ -162,7 +162,7 @@ jobs: create_pr: name: Create PR - runs-on: [self-hosted, Linux, large, ephemeral] + runs-on: block-node-linux-medium needs: release if: ${{ needs.release.outputs.create_pr == 'true' }} env: