Improved multi-numa alignment in Topology Manager

[klueska]

Enhancement Description

• One-line enhancement description (can be used as a release note):
  An enhancement to the TopologyManager that considers relative distances between NUMA nodes when calculating "best hints" that include more than one NUMA node.

• Kubernetes Enhancement Proposal:
  https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/3545-improved-multi-numa-alignment/README.md

• Discussion Link:
  Private slack messages

• Primary contact (assignee):
  @PiotrProkop

• Responsible SIGs:
  /sig node

• Enhancement target (which target equals to which milestone):

  • Alpha release target: 1.26
  • Beta release target: 1.28
  • Stable release target: 1.30

• Alpha

  • KEP (k/enhancements) update PR(s):
    KEP-3545: Improved multi-numa alignment in Topology Manager #3549
  • Code (k/k) update PR(s):
    node: topologymanager: Improved multi-numa alignment in Topology Manager kubernetes#112914
  • Docs (k/website) update PR(s):
    topologymanager: document topology manager policy options website#37668

• Beta

  • KEP (k/enhancements) update PR(s):
    Promote Improved multi-numa alignment in Topology Manager to beta #4079
  • Code (k/k) update PR(s):
    topologymanager: Promote support for improved multi-numa alignment in Topology Manager to beta kubernetes#118816
  • Docs (k/website) update(s):
    topologymanager: document topology manager policy options promotion to beta  website#41718

[klueska]

/cc @fromanirh
/cc @swatisehgal

[ffromani]

thanks for the heads up, I'll review shortly

[derekwaynecarr]

/label lead-opted-in

[Atharva-Shinde]

Hey @klueska 👋, 1.26 Enhancements team here!

Just checking in as we approach Enhancements Freeze on 18:00 PDT on Thursday 6th October 2022.

This enhancement is targeting for stage alpha for 1.26

Here's where this enhancement currently stands:

• KEP file using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable
• KEP has an updated detailed test plan section filled out
• KEP has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would just need to:
-Get the PR #3549 merged before Enhancements Freeze to make this enhancement eligible for 1.26 release.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well.
Thank you :)

[Atharva-Shinde]

/milestone v1.26
/label tracked/yes

[k8s-ci-robot]

@Atharva-Shinde: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

In response to this:

/milestone v1.26
/label tracked/yes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Atharva-Shinde]

/stage alpha

[Atharva-Shinde]

Hello @klueska 👋, just a quick check-in again, as we approach the 1.26 Enhancements freeze.

Please plan to get the action items mentioned in my comment above done before Enhancements freeze on 18:00 PDT on Thursday 6th October 2022 i.e tomorrow

For note, the current status of the enhancement is marked at-risk :)

[rhockenbury]

We have this marked as tracked for v1.26 with #3549 merged.

[Atharva-Shinde]

Hi @klueska 👋,

Checking in as we approach 1.26 code freeze at 17:00 PDT on Tuesday 8th November 2022.

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes). If there are any other PRs beyond node: topologymanager: Improved multi-numa alignment in Topology Manager kubernetes#112914, please include them in the Issue Description too.
• All PRs are fully merged by the code freeze deadline.

As always, we are here to help should questions come up.
Thanks :)

[Atharva-Shinde]

Hey @klueska 👋, just a quick check-in again before 1.26 code freeze at 17:00 PDT Tuesday 8th November 2022 i.e tomorrow.
Looks like we would at least need to get the code PR/s: kubernetes/kubernetes#112914 (any more PRs?) merged before the code-freeze :)

[krol3]

Hello @klueska 👋, 1.26 Release Docs Lead here. This enhancement is marked as ‘Needs Docs’ for 1.26 release.

Please follow the steps detailed in the documentation to open a PR against dev-1.26 branch in the k/website repo. This PR can be just a placeholder at this time, and must be created by November 9. Also, take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Any doubt, reach us! Thank you!

[rhockenbury]

With kubernetes/kubernetes#112914 merged, we have this marked as tracked for code freeze.

[krol3]

Hello @klueska 👋 please take a look at Documenting for a release - PR Ready for Review to get your PR ready for review before deadline Tuesday 15th November 2022. Thank you!

[krol3]

Hi @klueska ! thank you for your doc PR here

[marosset]

/remove-label lead-opted-in
/remove-label tracked/yes
/label tracked/no
/milestone clear

[vsoch]

hey folks! My team is interested in this KEP - it looks like all the alpha/beta check boxes are purple (merged) and the release contender was 1.28? Was this released with 1.28 and I missed it https://kubernetes.io/blog/2023/08/15/kubernetes-v1-28-release/ or is it still TBA (and possibly not documented) because the issue here is still open? Thanks! And apologies for my naivete about this process.

[PiotrProkop]

@vsoch This is available as alpha option since 1.26 and as beta in 1.28. Check current documentation here: https://kubernetes.io/docs/tasks/administer-cluster/topology-manager/#topology-manager-policy-options

[salehsedghpour]

Hello 👋 1.30 Enhancements Lead here,

I'm closing milestone 1.28 now,
If you wish to progress this enhancement in v1.30, please follow the instructions here to opt in the enhancement and make sure the lead-opted-in label is set so it can get added to the tracking board and finally add /milestone v1.30. Thanks!

/milestone clear

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[PiotrProkop]

/remove-lifecycle rotten
still discussing how to approach e2e for GA

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[kannon92]

@PiotrProkop what are your plans for this KEP for 1.32?

[PiotrProkop]

@kannon92 I think we should try to graduate it to stable in 1.32 😄 I'll talk with @klueska how we can move forward.

[kannon92]

@PiotrProkop That is nice to hear.

[kannon92]

/remove-lifecycle rotten

[kannon92]

/remove-lifecycle stale

[klueska]

Happy to help move this forward to stable in 1.32

[haircommander]

/milestone v1.32
/label lead-opted-in

[kannon92]

Looking at our e2e tests around Multi-NUMA alignment, it seems that we don't have the right hardware to actually test this features. If you look at TopologyManager tests, they are being skipped because the tests do not have multi numa alignment.

• https://testgrid.k8s.io/sig-node-cri-o#ci-crio-cgroupv2-node-e2e-resource-managers
• https://testgrid.k8s.io/sig-node-containerd#node-kubelet-containerd-resource-managers

I posted this concern on #sigs-k8s-infra and it seems that we should be able to request the nodes if we know what kind of one to ask for.

Maybe we should get these jobs passing and not skipping for GA so we can make sure that these features don't regress.

[PiotrProkop]

@ffromani created an issue to track hardware requirements for sig-node e2e tests here:
kubernetes/k8s.io#7339

[impact-maker]

Hello @haircommander @PiotrProkop 👋, v1.32 Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 11th October 2024 / 19:00 PDT Thursday 10th October 2024.

This enhancement is targeting for stage stable for v1.32 (correct me, if otherwise).

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v1.32.
• KEP readme has up-to-date graduation criteria.
• KEP has submitted a production readiness review request for approval and has a reviewer assigned.
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday 3rd October 2024 so that the PRR team has enough time to review your KEP.

For this KEP, we would need to update the following:

• KEP has submitted a production readiness review request for approval and has a reviewer assigned.
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday 3rd October 2024 so that the PRR team has enough time to review your KEP.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!