release 0.54.0 (#3667)

* chore: create release 0.54.0 * chore: Update pureconfig from 0.17.6 to 0.17.7 (#3666) Co-authored-by: RenkuBot <[email protected]> * chore: Update circe from 0.14.7 to 0.14.8 (#3683) Co-authored-by: Eike Kettner <[email protected]> * chore: Update scalafmt from 3.8.1 to 3.8.2 (#3681) Co-authored-by: Eike Kettner <[email protected]> * chore(deps): bump the gh-actions group with 2 updates (#3685) Bumps the gh-actions group with 2 updates: [actions/checkout](https://github.com/actions/checkout) and [Azure/docker-login](https://github.com/azure/docker-login). Updates `actions/checkout` from 4.1.2 to 4.1.7 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4.1.2...v4.1.7) Updates `Azure/docker-login` from 1 to 2 - [Release notes](https://github.com/azure/docker-login/releases) - [Commits](Azure/docker-login@v1...v2) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-patch dependency-group: gh-actions - dependency-name: Azure/docker-login dependency-type: direct:production update-type: version-update:semver-major dependency-group: gh-actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ralf Grubenmann <[email protected]> * feat: support for rotating secrets storage keys (#3653) * fix: properly encode generated secret service key (#3695) * feat: test storage credentials and prompt on v2 session start (#3693) Co-authored-by: Chandrasekhar Ramakrishnan <[email protected]> * Update CHANGELOG.rst Co-authored-by: Flora Thiebaut <[email protected]> * chore: Update CHANGELOG.rst wording * chore: Update scalatest from 3.2.18 to 3.2.19 (#3691) Co-authored-by: RenkuBot <[email protected]> Co-authored-by: Ralf Grubenmann <[email protected]> Co-authored-by: Rok Roškar <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: RenkuBot <[email protected]> Co-authored-by: Eike Kettner <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ralf Grubenmann <[email protected]> Co-authored-by: Lorenzo Cavazzi <[email protected]> Co-authored-by: Chandrasekhar Ramakrishnan <[email protected]> Co-authored-by: Flora Thiebaut <[email protected]> Co-authored-by: Rok Roškar <[email protected]>
SwissDataScienceCenter · Jun 26, 2024 · 9e5b368 · 9e5b368
1 parent 3850db7
commit 9e5b368
Show file tree

Hide file tree

Showing 17 changed files with 219 additions and 101 deletions.
diff --git a/.github/workflows/check-acceptance-test-code.yml b/.github/workflows/check-acceptance-test-code.yml
@@ -8,7 +8,7 @@ jobs:
     name: Scala dependencies and code check
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
       - name: run test compile
         run: |
           cd acceptance-tests

diff --git a/.github/workflows/check-acceptance-test-fmt.yml b/.github/workflows/check-acceptance-test-fmt.yml
@@ -8,7 +8,7 @@ jobs:
     name: Scala formatting check
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
       - name: run scalafmt
         run: |
           cd acceptance-tests

diff --git a/.github/workflows/create-release-branch.yml b/.github/workflows/create-release-branch.yml
@@ -16,7 +16,7 @@ jobs:
   create-release-pr:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
         with:
           fetch-depth: 0
           token: "${{ secrets.RENKUBOT_GITHUB_TOKEN }}"

diff --git a/.github/workflows/generate-values-script.yaml b/.github/workflows/generate-values-script.yaml
@@ -15,7 +15,7 @@ jobs:
         - os: macos-11
         - os: ubuntu-20.04
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
         with:
           fetch-depth: 0
       - uses: actions/setup-python@v5
@@ -49,10 +49,10 @@ jobs:
     runs-on: ubuntu-20.04
     needs: [test-script]
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
         with:
           fetch-depth: 0
-      - uses: Azure/docker-login@v1
+      - uses: Azure/docker-login@v2
         with:
           username: ${{ secrets.RENKU_DOCKER_USERNAME }}
           password: ${{ secrets.RENKU_DOCKER_PASSWORD }}

diff --git a/.github/workflows/publish-helm-chart.yml b/.github/workflows/publish-helm-chart.yml
@@ -9,7 +9,7 @@ jobs:
   publish-chart:
     runs-on: ubuntu-20.04
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
         with:
           fetch-depth: 0
       - name: Set version

diff --git a/.github/workflows/publish-master-merges.yaml b/.github/workflows/publish-master-merges.yaml
@@ -14,7 +14,7 @@ jobs:
   publish-chart:
     runs-on: ubuntu-20.04
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
         with:
           fetch-depth: 0
       - uses: azure/setup-helm@v4

diff --git a/.github/workflows/pull-request-test.yml b/.github/workflows/pull-request-test.yml
@@ -18,7 +18,7 @@ jobs:
     if: github.event.action != 'closed'
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
       - uses: actions/setup-java@v4
         with:
           distribution: "temurin"
@@ -60,7 +60,7 @@ jobs:
       test-enabled: ${{ steps.deploy-comment.outputs.test-enabled}}
       extra-values: ${{ steps.deploy-comment.outputs.extra-values}}
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
       - id: deploy-comment
         uses: SwissDataScienceCenter/renku-actions/[email protected]
         with:
@@ -75,7 +75,7 @@ jobs:
       name: ci-renku-${{ github.event.number }}
       url: https://ci-renku-${{ github.event.number }}.dev.renku.ch
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
       - name: renku build and deploy
         if: needs.check-deploy.outputs.pr-contains-string == 'true'
         uses: SwissDataScienceCenter/renku-actions/[email protected]

diff --git a/.github/workflows/renku-dev-test.yaml b/.github/workflows/renku-dev-test.yaml
@@ -22,7 +22,7 @@ jobs:
       github.event.client_payload.message == 'Helm test succeeded' }}
     runs-on: ubuntu-20.04
     steps:
-      - uses: actions/[email protected].2
+      - uses: actions/[email protected].7
       - uses: cypress-io/github-action@v6
         id: cypress
         env:

diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -1,5 +1,32 @@
 .. _changelog:
 
+0.54.0
+------
+
+User-Facing Changes
+~~~~~~~~~~~~~~~~~~~
+
+**✨ Improvements**
+
+- Test the cloud storage connection before persisting the configuration (`#3194 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3194>`_)
+- Prompt for cloud storage credentials on v2 session start (`#3203 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3203>`_)
+- Indicate repository permissions in Renku 2.0 (`#3136 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3136>`_)
+
+Internal Changes
+~~~~~~~~~~~~~~~~
+
+**🌟 New Features**
+
+- **Secrets**: Allow rotating the private key for secrets storage
+
+Individual Components
+~~~~~~~~~~~~~~~~~~~~~
+
+- `renku-data-services 0.15.0 <https://github.com/SwissDataScienceCenter/renku-data-services/releases/tag/v0.15.0>`__
+- `renku-notebooks 1.25.2 <https://github.com/SwissDataScienceCenter/renku-notebooks/releases/tag/1.25.2>`_
+- `renku-ui 3.29.0 <https://github.com/SwissDataScienceCenter/renku-ui/releases/tag/3.29.0>`_
+
+
 0.53.1
 ------
 

diff --git a/acceptance-tests/.scalafmt.conf b/acceptance-tests/.scalafmt.conf
@@ -1,4 +1,4 @@
-version = "3.8.1"
+version = "3.8.2"
 
 runner.dialect = "scala213"
 

diff --git a/acceptance-tests/build.sbt b/acceptance-tests/build.sbt
@@ -28,10 +28,10 @@ enablePlugins(AutomateHeaderPlugin)
 publish / skip := true
 publishTo := Some(Resolver.file("Unused transient repository", file("target/unusedrepo")))
 
-val circeVersion = "0.14.7"
+val circeVersion = "0.14.8"
 
 libraryDependencies += "ch.qos.logback"          % "logback-classic"               % "1.5.6"
-libraryDependencies += "com.github.pureconfig"  %% "pureconfig"                    % "0.17.6"     % Test
+libraryDependencies += "com.github.pureconfig"  %% "pureconfig"                    % "0.17.7"     % Test
 libraryDependencies += "eu.timepit"             %% "refined"                       % "0.11.2"     % Test
 libraryDependencies += "io.circe"               %% "circe-core"                    % circeVersion % Test
 libraryDependencies += "io.circe"               %% "circe-literal"                 % circeVersion % Test
@@ -40,7 +40,7 @@ libraryDependencies += "io.circe"               %% "circe-optics"
 libraryDependencies += "org.http4s"             %% "http4s-blaze-client"           % "0.23.16"    % Test
 libraryDependencies += "org.http4s"             %% "http4s-circe"                  % "0.23.27"    % Test
 libraryDependencies += "org.scalacheck"         %% "scalacheck"                    % "1.18.0"     % Test
-libraryDependencies += "org.scalatest"          %% "scalatest"                     % "3.2.18"     % Test
+libraryDependencies += "org.scalatest"          %% "scalatest"                     % "3.2.19"     % Test
 libraryDependencies += "org.scalatestplus"      %% "selenium-4-1"                  % "3.2.12.1"   % Test
 libraryDependencies += "org.seleniumhq.selenium" % "selenium-http-jdk-client"      % "4.13.0"     % Test
 libraryDependencies += "org.seleniumhq.selenium" % "selenium-java"                 % "4.18.1"     % Test

diff --git a/cypress-tests/cypress/e2e/useSession.cy.ts b/cypress-tests/cypress/e2e/useSession.cy.ts
@@ -265,7 +265,8 @@ describe("Basic public project functionality", () => {
         cy.get("#endpoint")
           .should("have.value", "")
           .type("http://s3.amazonaws.com");
-        cy.getDataCy("cloud-storage-edit-next-button")
+        cy.getDataCy("test-cloud-storage-button").should("be.visible").click();
+        cy.getDataCy("add-cloud-storage-continue-button")
           .should("be.visible")
           .click();
 
@@ -282,7 +283,7 @@ describe("Basic public project functionality", () => {
           .click();
 
         cy.getDataCy("cloud-storage-edit-body").contains(
-          "storage data_s3 has been succesfully added"
+          "storage data_s3 has been successfully added"
         );
         cy.getDataCy("cloud-storage-edit-close-button")
           .should("be.visible")

diff --git a/helm-chart/renku/templates/secrets-storage/deployment.yaml b/helm-chart/renku/templates/secrets-storage/deployment.yaml
@@ -24,6 +24,7 @@ spec:
         release: {{ .Release.Name }}
       {{- with .Values.secretsStorage.podAnnotations }}
       annotations:
+        checksum/privateKey: {{ .Values.global.platformConfig | sha256sum }}
         {{- toYaml . | nindent 8 }}
       {{- end }}
     spec:
@@ -67,6 +68,8 @@ spec:
                   key: dataServiceKeycloakClientSecret
             - name: SECRETS_SERVICE_PRIVATE_KEY_PATH
               value: /secrets/privateKey/privateKey
+            - name: PREVIOUS_SECRETS_SERVICE_PRIVATE_KEY_PATH
+              value: /secrets/privateKey/previousPrivateKey
             {{- include "certificates.env.python" $ | nindent 12 }}
           livenessProbe:
             httpGet:
@@ -99,9 +102,6 @@ spec:
         - name: secret-service-private-key
           secret:
             secretName: {{ template "renku.fullname" . }}-secret-service-private-key
-            items:
-            - key: privateKey
-              path: privateKey
         {{- include "certificates.volumes" . | nindent 8 }}
     {{- with .Values.secretsStorage.nodeSelector }}
       nodeSelector:

diff --git a/helm-chart/renku/templates/setup-job-platform-init.yaml b/helm-chart/renku/templates/setup-job-platform-init.yaml
@@ -60,6 +60,7 @@ rules:
       - list
       - patch
       - create
+      - delete
 ---
 apiVersion: v1
 kind: ServiceAccount

diff --git a/helm-chart/renku/values.yaml b/helm-chart/renku/values.yaml
@@ -7,8 +7,8 @@
 global:
   ## YAML string that contains all application level Renku configuration options.
   platformConfig: |
-    {}
-  # secretServicePrivateKey: ... RSA Private Key in PEM format
+  # secretServicePrivateKey: ... RSA Private Key in PKCS8 PEM format (`ssh-keygen -m PKCS8 -t rsa -b 4096`)
+  # secretServicePreviousPrivateKey: ... Previous Private key in PEM format, only set this when rotating keys
   # dataServiceEncryptionKey: 32 byte random string
   gitlab:
     ## Name of the postgres database to be used by Gitlab
@@ -672,7 +672,7 @@ ui:
     replicaCount: 1
     image:
       repository: renku/renku-ui
-      tag: "3.28.1"
+      tag: "3.29.0"
       pullPolicy: IfNotPresent
       ## Optionally specify an array of imagePullSecrets.
       ## Secrets must be manually created in the namespace.
@@ -861,7 +861,7 @@ ui:
     keepCookies: []
     image:
       repository: renku/renku-ui-server
-      tag: "3.28.1"
+      tag: "3.29.0"
       pullPolicy: IfNotPresent
     imagePullSecrets: []
     nameOverride: ""
@@ -1001,7 +1001,7 @@ notebooks:
     targetCPUUtilizationPercentage: 50
   image:
     repository: renku/renku-notebooks
-    tag: "1.25.1"
+    tag: "1.25.2"
     pullPolicy: IfNotPresent
     ## Optionally specify an array of imagePullSecrets.
     ## Secrets must be manually created in the namespace.
@@ -1119,15 +1119,15 @@ notebooks:
   gitRpcServer:
     image:
       name: renku/git-rpc-server
-      tag: "1.25.1"
+      tag: "1.25.2"
   gitHttpsProxy:
     image:
       name: renku/git-https-proxy
-      tag: "1.25.1"
+      tag: "1.25.2"
   gitClone:
     image:
       name: renku/git-clone
-      tag: "1.25.1"
+      tag: "1.25.2"
   service:
     type: ClusterIP
     port: 80
@@ -1180,12 +1180,12 @@ notebooks:
     sessionTypes: ["registered"]
     image:
       repository: renku/renku-notebooks-tests
-      tag: "1.25.1"
+      tag: "1.25.2"
       pullPolicy: IfNotPresent
   k8sWatcher:
     image:
       repository: renku/k8s-watcher
-      tag: "1.25.1"
+      tag: "1.25.2"
       pullPolicy: IfNotPresent
     resources: {}
     replicaCount: 1
@@ -1197,12 +1197,12 @@ notebooks:
   secretsMount:
     image:
       repository: renku/secrets-mount
-      tag: "1.25.1"
+      tag: "1.25.2"
   ssh:
     enabled: false
     image:
       repository: renku/ssh-jump-host
-      tag: "1.25.1"
+      tag: "1.25.2"
       pullPolicy: IfNotPresent
     resources: {}
     replicaCount: 1
@@ -1603,14 +1603,14 @@ platformInit:
 dataService:
   image:
     repository: renku/renku-data-service
-    tag: "0.14.1"
+    tag: "0.15.0"
     pullPolicy: IfNotPresent
   backgroundJobs:
     events:
       resources: {}
     image:
       repository: renku/data-service-background-jobs
-      tag: "0.14.1"
+      tag: "0.15.0"
       pullPolicy: IfNotPresent
     total:
       resources: {}
@@ -1663,7 +1663,7 @@ authz:
 secretsStorage:
   image:
     repository: renku/secrets-storage
-    tag: "0.14.1"
+    tag: "0.15.0"
     pullPolicy: IfNotPresent
   service:
     type: ClusterIP

diff --git a/helm-chart/values.yaml.changelog.md b/helm-chart/values.yaml.changelog.md
@@ -5,6 +5,17 @@ For changes that require manual steps other than changing values, please check o
 Please follow this convention when adding a new row
 * `<type: NEW|EDIT|DELETE> - *<resource name>*: <details>`
 
+## Upgrading to Renku 0.54.0
+
+* NEW ``global.platformConfig``: The YAML string can now contain a new key, `secretServicePreviousPrivateKey` which allows for rotating the secret-storage private key.
+  To rotate keys, set this to the previous `secretServicePrivateKey` value and set a new key for `secretServicePrivateKey`. Secrets-storage will then rotate all secrets
+  once its started. You can monitor the progress of the rotation in prometheus using the `secrets_rotation_count` (for total secrets rotated so far) and `secrets_rotation_state`
+  (either `running`, `finished` or `errored`) for the overall state of the rotation. Please make sure to unset `secretServicePreviousPrivateKey` once rotation is finished
+  as a matter of best practice.
+
+  NOTE: Make sure that you do not redeploy or rollback the Renku Helm chart while a key rotation is underway. Even if the 
+  deployment is broken it is best to wait for the key rotation to finish before attempting another deployment or a rollback.
+
 ## Upgrading to Renku 0.53.0
 
 The `data-service` configuration has been updated to support trusting reverse proxies.
@@ -18,6 +29,14 @@ The `data-service` configuration has been updated to support trusting reverse pr
 * NEW ``dataService.backgroundJobs.events.resources`` to set the resources for the users short period synchronization job
 * NEW ``dataService.backgroundJobs.total.resources`` to set the resources for the users long period synchronization job
 
+## Upgrading to Renku 0.52.0
+
+* NEW ``global.platformConfig`` a YAML string that contains the secret keys used by renku-data-services and secrets storage. In the future we plan to also consolidate other
+  platform specific configuration here. The YAML string should contain the following keys:
+    - `secretServicePrivateKey`: An RSA private key, generated by `ssh-keygen -m PKCS8 -t rsa -b 4096` without a password. You can leave this empty to have one automatically generated
+      but we recommend setting it manually.
+    - `dataServiceEncryptionKey`: A 32 byte random string, used for encryption at rest.
+
 ## Upgrading to Renku 0.51.0
 
 * NEW ``ui.client.sessionClassEmailUs`` to customize the content of the Email Us button on the Session class option.