Remove 1.23 dashboards and job #29230

ramrodo · 2023-04-05T22:30:33Z

Since v1.23 Kubernetes is out of support now, we should remove the 1.23 dashboards from testgrid and its Job

This PR is the following one of #29172

closes #29242

/sig release
/area release-eng

ramrodo · 2023-04-05T22:31:05Z

cc: @cici37 @gracenng @kubernetes/release-engineering

saschagrunert · 2023-04-06T07:03:49Z

/retest

cici37 · 2023-04-10T16:51:40Z

Other release jobs and testgrid settings should not be touched if the goal is to remove 1.23..

ramrodo · 2023-04-10T21:52:37Z

Other release jobs and testgrid settings should not be touched if the goal is to remove 1.23..

@cici37, I think we should update the generated.yaml file since the stable4 should be 1.24 now (instead of 1.23). Otherwise, there is an error at Prow:

could not apply prowjob annotations:
couldn't find dashboard "sig-release-1.23-blocking" for job "ci-kubernetes-e2e-gce-cos-k8sstable4-alphafeatures"
FAIL	k8s.io/test-infra/config/tests/testgrids	2.881s

And due to this file is auto-generated and should not be edited manually, I execute:

make -C releng prepare-release-branch

to update it. This command also adds changes to the config/jobs/kubernetes/sig-release/release-branch-jobs/1.24.yaml file.

BenTheElder · 2023-04-17T20:45:03Z

We should be updating the generated file, especially as the input spec changed earlier in this PR and merely running the generator would produce different results.

1.23 should be dropped from all of these jobs.

BenTheElder · 2023-04-17T20:57:48Z

releng/test_config.yaml

@@ -251,30 +251,30 @@ k8sVersions:
    - --extract-ci-bucket=k8s-release-dev
    version: master
  beta:
+    args:
+    - --extract=ci/latest-1.28


hmmm I don't think we should have 1.28 yet?

The marker already exists in fact (https://dl.k8s.io/ci/latest-1.28.txt) and points to the latest commit in k/k at the time of writing this comment.

cici37 · 2023-04-24T19:58:53Z

Seems like the auto generated yaml still binds to the wrong version: https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/generated/generated.yaml
generate_tests.py seems the one to make it right.

And also, we should not have 1.28 as Ben mentioned above. I forgot how we use beta before, seems like we were using it for the current released version til the next branch is cut in the past.

BenTheElder · 2023-04-24T20:00:11Z

Can we start with a more scoped PR to the other 1.23 jobs aside from the dashboard and the generated jobs?
And then track down who owns this job generator and get those resolved?

cici37 · 2023-04-27T22:13:31Z

Seems like the auto generated yaml still binds to the wrong version: https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/generated/generated.yaml
generate_tests.py seems the one to make it right.

The test will not pass if we didn't get generated jobs right with 1.23 removal. I agree we should start with minimized pr to remove 1.23 but I guess the above suggestion is still needed.

xmudrii · 2023-04-27T22:26:08Z

And also, we should not have 1.28 as Ben mentioned above. I forgot how we use beta before, seems like we were using it for the current released version til the next branch is cut in the past.

We discussed it on Slack a while ago in some thread and this is wrong. The released version should switch to stable1 as soon as .0 is out, but we were forgetting to do so in the past. That said, proposed changes are correct.

I'm wondering why is ramrodo@3940234 reverted? Was that file manually changed so it got reverted?

cici37 · 2023-04-27T22:42:14Z

A separate question, I tried to run

go run ./releng/config-forker   --job-config $(pwd)/config/jobs   --version 1.27   --go-version 1.20.2   --output $(pwd)/config/jobs/kubernetes/sig-release/release-branch-jobs/1.27.yaml

which gives me different changes than make -C releng prepare-release-branch. Which one is the source of truth? Are they going through different code path? @xmudrii

xmudrii · 2023-04-27T22:44:05Z

@cici37 config-forker is only half of the job, there's also config-rotator that takes care of existing jobs (IIRC). The make command unifies both forker and rotator, so that's the reason why results are different.

ramrodo · 2023-04-27T22:48:15Z

I'm wondering why is ramrodo@3940234 reverted? Was that file manually changed so it got reverted?

I reverted that commit since the script added the 1.28 reference. So, my initial thought was that something was wrong.

xmudrii · 2023-04-27T23:00:08Z

@ramrodo Can you try reverting that revert commit? I think that we'll get tests passing, from there we can figure out what to do with 1.28.

This reverts commit 3940234.

k8s-ci-robot · 2023-04-27T23:05:57Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ramrodo
Once this PR has been reviewed and has the lgtm label, please assign claudiubelu, justaugustus for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

xmudrii · 2023-04-27T23:24:01Z

config/jobs/kubernetes/generated/generated.yaml

@@ -224,7 +224,7 @@ periodics:
  cluster: k8s-infra-prow-build
  annotations:
    testgrid-tab-name: gce-cos-k8sbeta-slow
-    testgrid-dashboards: sig-release-1.27-informing
+    testgrid-dashboards: sig-release-1.28-informing
    testgrid-num-failures-to-alert: '6'
 - tags:


@ramrodo Can you try commenting all jobs above this one and then pushing that?

I'm aware this is autogenerated file, but let's give it a try and iterate from there.

I'm wondering if adding the # characters made this test fail at converting to JSON or it is validating the integrity of the file:
https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/test-infra/29230/pull-test-infra-prow-checkconfig/1651730089395621888

#29230 (comment)

xmudrii · 2023-04-27T23:32:40Z

config/jobs/kubernetes/generated/generated.yaml

-      - --env=ENABLE_CACHE_MUTATION_DETECTOR=true
-      env:
-      image: gcr.io/k8s-staging-test-infra/kubekins-e2e:v20230406-23cb1879e3-master
-      resources: &id001


@ramrodo You need to find the first uncommented reference of id001, then place it with L70-L76.

@xmudrii I'll do it with id002 also

k8s-ci-robot · 2023-04-27T23:52:54Z

@ramrodo: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-test-infra-unit-test	`32cc222`	link	true	`/test pull-test-infra-unit-test`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

xmudrii · 2023-04-28T14:17:39Z

I did a write up on this issue in #29387, please check it and comment there on opinions and next steps.

ramrodo · 2023-04-28T16:11:56Z

Closing in favor of #29387

k8s-ci-robot requested review from palnabarun and Verolop April 5, 2023 22:30

k8s-ci-robot added area/config Issues or PRs related to code in /config area/jobs area/testgrid sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Apr 5, 2023

k8s-ci-robot added sig/windows Categorizes an issue or PR as relevant to SIG Windows. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Apr 6, 2023

ramrodo added 5 commits April 9, 2023 15:41

Remove 1.23 dashboards

b46bf33

Update README to replace 1.23 dashboards adding the 1.27 ones

0c719b8

Remove 1.23 Windows jobs

e20161d

releng: Auto generated files

653421e

Delete 1.28 job auto generated

50a59f2

ramrodo force-pushed the remove-1.23-jobs branch from b55d615 to 50a59f2 Compare April 9, 2023 21:46

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 9, 2023

Delete 1.23 job

a9db93b

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 10, 2023

Revert changes made to the generated.yaml file

3940234

BenTheElder mentioned this pull request Apr 17, 2023

[WIP] Remove 1.23 dashboards and job #29321

Closed

BenTheElder reviewed Apr 17, 2023

View reviewed changes

ramrodo mentioned this pull request Apr 27, 2023

Remove 1.23 windows jobs #29379

Closed

Revert "Revert changes made to the generated.yaml file"

a6a1168

This reverts commit 3940234.

xmudrii reviewed Apr 27, 2023

View reviewed changes

Comment on all the jobs referencing 1.28

110b8ac

xmudrii reviewed Apr 27, 2023

View reviewed changes

ramrodo added 2 commits April 27, 2023 17:43

Move reference of &id001

76d7f42

Move reference of &id002

32cc222

xmudrii mentioned this pull request Apr 28, 2023

Remove Kubernetes 1.23 jobs #29387

Merged

ramrodo closed this Apr 28, 2023

ramrodo deleted the remove-1.23-jobs branch April 28, 2023 16:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove 1.23 dashboards and job #29230

Remove 1.23 dashboards and job #29230

ramrodo commented Apr 5, 2023 •

edited

Loading

ramrodo commented Apr 5, 2023

saschagrunert commented Apr 6, 2023

cici37 commented Apr 10, 2023

ramrodo commented Apr 10, 2023 •

edited

Loading

BenTheElder commented Apr 17, 2023

BenTheElder Apr 17, 2023

xmudrii Apr 27, 2023

cici37 commented Apr 24, 2023

BenTheElder commented Apr 24, 2023

cici37 commented Apr 27, 2023

xmudrii commented Apr 27, 2023

cici37 commented Apr 27, 2023

xmudrii commented Apr 27, 2023

ramrodo commented Apr 27, 2023

xmudrii commented Apr 27, 2023

k8s-ci-robot commented Apr 27, 2023

xmudrii Apr 27, 2023

xmudrii Apr 27, 2023

ramrodo Apr 27, 2023

ramrodo Apr 27, 2023

xmudrii Apr 27, 2023

xmudrii Apr 27, 2023

ramrodo Apr 27, 2023

k8s-ci-robot commented Apr 27, 2023

xmudrii commented Apr 28, 2023

ramrodo commented Apr 28, 2023

Remove 1.23 dashboards and job #29230

Remove 1.23 dashboards and job #29230

Conversation

ramrodo commented Apr 5, 2023 • edited Loading

ramrodo commented Apr 5, 2023

saschagrunert commented Apr 6, 2023

cici37 commented Apr 10, 2023

ramrodo commented Apr 10, 2023 • edited Loading

BenTheElder commented Apr 17, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cici37 commented Apr 24, 2023

BenTheElder commented Apr 24, 2023

cici37 commented Apr 27, 2023

xmudrii commented Apr 27, 2023

cici37 commented Apr 27, 2023

xmudrii commented Apr 27, 2023

ramrodo commented Apr 27, 2023

xmudrii commented Apr 27, 2023

k8s-ci-robot commented Apr 27, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented Apr 27, 2023

xmudrii commented Apr 28, 2023

ramrodo commented Apr 28, 2023

ramrodo commented Apr 5, 2023 •

edited

Loading

ramrodo commented Apr 10, 2023 •

edited

Loading