Releases: karmada-io/karmada
karmada v1.4.1 release
Changes since v1.4.0
Bug Fixes
karmadactl
: Fixedkarmada-agent
installed by theregister
command can not delete works due to lack of permission issue. (#2904, @lonelyCZ)karmadactl
: Fixed the default ValidatingWebhookConfiguration forresourceinterpreterwebhook
not working issue. (#2924, @qingwave)karmadactl
: Fixed the default ValidatingWebhookConfiguration forresourceinterpretercustomizations
not working issue. (#2927, @chaunceyjiang)karmadactl
: Fixed the error of resources whose name contains colons failing to be created when usingkarmadactl apply
. (#2931, @Poor12)karmada-controller-manager
/karmada-agent
: Fixed misjudgment of deployment and statefuleset health status. (#2944, @Fish-pro)karmada-controller-manager
/karmada-agent
: Fixed failed to sync work status issue due to the informer being accidentally shut down. (#2937, @Poor12)karmada-scheduler
: Fixed a corner case that re-schedule be skipped in case of the cluster becomes not fit. (#2955, @jwcesign)
Others
karmada v1.2.5 release
Changes since v1.2.4
Bug Fixes
karmada-controller-manager
/karmada-agent
: Fixed failed to sync work status issue due to the informer being accidentally shut down. (#2939, @Poor12)karmadactl
: Fixed the error of resources whose name contains colons failing to be created when usingkarmadactl promote
. (#2933, @Poor12)karmadactl
: Fixed the default ValidatingWebhookConfiguration forresourceinterpreterwebhook
not working issue. (#2922, @qingwave)karmada-webhook
: Fixed failed to set resource selector default namespace if the relevant OverridePolicy and PropagationPolicy with namespace unset issue. (#2873, @carlory )
karmada v1.4.0 release
What's New
Declarative Resource Interpreter
The Interpreter Framework is designed for interpreting the structure of arbitrary resource types. It consists of built-in
and customized
interpreters, this release introduced another brand-new customized interpreter.
With the newly introduced declarative
interpreter, users can quickly customize resource interpreters for both Kubernetes resources and CRD resources by the rules declared in the ResourceInterpreterCustomization
API specification. Compared with the interpreter customized by the webhook, it gets the rules from the declarative specifications instead of requiring an additional webhook component.
The new command named interpret
in the karmadactl
could be used to test the rules before applying them to the system.
Some examples are provided to help users better understand how this interpreter can be used.
(Feature contributor: @jameszhangyukun @ikaven1024 @chaunceyjiang @XiShanYongYe-Chang @RainbowMango)
PropagationPolicy/ClusterPropagationPolicy priority
Users are now able to declare the priorities for both PropagationPolicy
and ClusterPropagationPolicy
. A policy will be applied for the matched resource templates if there are no other policies with higher priority at the point the resource template be processed.
The system administrator can use priorities to manage and control policies. Refer to Configure PropagationPolicy priority for more details.
(Feature contributor: @Garrybest @jwcesign)
Instrumentation improvement
This release enhanced observability significantly through metrics and events.
The metrics can be queried by the endpoint(/metrics
) of each component using an HTTP scrap, and they are served in Prometheus format. Events are reported to the relevant resource objects respectively.
Refer to events and metrics for more details.
(Feature contributor: @Poor12)
Other Notable Changes
API Changes
- Introduces priority to PropagationPolicy. (#2758, @RainbowMango)
- Introduced
LabelsOverrider
andAnnotationsOverrider
for overriding labels and annotations.(#2584, @chaunceyjiang) - Evoluted PropagateDeps FeatureGate to Beta and enabled it by default. (#2875, @XiShanYongYe-Chang)
- Evoluted Failover/GracefulEviction FeatureGate to Beta and enabled it by default. (#2876, @jwcesign)
- Evoluted CustomizedClusterResourceModeling FeatureGate to Beta and enabled it by default. (#2877, @Poor12)
Bug Fixes
karmada-search
: Fixed concurrent map writes panic while listing objects via proxy. (#2483, @ikaven1024)karmada-search
: Fixed the issue that the returned ResourceVersion by proxy was not stable. (#2746, @cmicat)karmada-controller-manager
/karmada-agent
: Fixed the issue that pod information cannot be collected when building resource summary. (#2489, @Poor12)karmada-controller-manager
: Used cluster secret ref namespace in unified-auth-controller when generating a ClusterRoleBinding. (#2516, @XiShanYongYe-Chang)karmada-controller-manager
: Fixed the failure of creating resources whose name contains colons. (#2549, @XiShanYongYe-Chang)karmada-controller-manager
: Fixed the panic when cluster ImpersonatorSecretRef is nil. (#2675, @stingshen)karmada-controller-manager
: Fixed serviceaccount continual regeneration by service account controller. (#2578, @Poor12)karmada-controller-manager
: Disabled the preemption matching of pp/cpp priority. (#2734, @XiShanYongYe-Chang)karmada-controller-manager
: Fixed the issue that clusterOverridePolicy and overridePolicy with nil resource selector could not work. (#2771, @wuyingjun-lucky)karmada-controlle-managerr
: Ignored resources that do not match with a policy before applying the policy. (#2786, @XiShanYongYe-Chang)karmada-agent
: Fixed the issue thatServiceExport
controller cannot reportendpointSlices
(due to the missingcreate
permission). (#2515, @lonelyCZ)karmadactl
: Fixedinit
cannot honor IPv6 addresses when generating the kubeconfig file. (#2450, @duanmengkk)karmadactl
: Fixed--karmada-data
directory was not initialized ininit
command. (#2548, @jwcesign)karmadactl
: Fixed the issue thatinit
commands print incorrect register command. (#2707, @Songjoy)karmadactl
: Fixed the issue that the namespace already exists. (#2505, @cleverhu)karmada-webhook
: Fixed the failure to set resource selector default namespace when the relevant OverridePolicy and PropagationPolicy are not configured with namespaces. (#2858, @carlory)
Security
Security
: Added limitReader toio.ReadAll
which could limit the memory request and avoid DoS attacks. (#2765, @Poor12)
Features & Enhancements
karmadactl
: Improved karmada init help output. (#2342, @my-git9)karmadactl
: karmadactl prohibits input extra arguments forinit
command. (#2497, @helen-frank)karmadactl
/chart
: Theinit
no longer creates redundant ServiceAccounts for components exceptkarmada-agent
. (#2523, @carlory)karmadactl
: Fixed the issue that options ofdeinit
cannot be shown. (#2540, @helen-frank)karmadactl/chart
: If Karmada is installed by using karmadactl or a Helm chart with default configurations, the image tag of karmada-kube-controller-manager/karmada-apiserver will be kube-controller-manager:v1.25.2/kube-apiserver:v1.25.2. (#2539, @jwcesign)karmadactl
: Introduced--karmada-apiserver-advertise-address
flag to specify Karmada APIserver's address to theinit
sub-command. (#2550, @wuyingjun-lucky)karmadactl
: Introduced --enable-cert-rotation option to register commands. (#2596, @lonelyCZ)karmadactl
: Added dryrun for uncordon. (#2760, @helen-frank)karmadactl
: Added validation on cluster existence for karmadactl get. (#2787, @helen-frank)karmadactl
: Added liveness probe into the kube-controller-manager component. (#2817, @carlory)karmadactl
: Added--image-registry
flags forinit
. (#2655, @helen-frank)karmadactl
: Added interpreter command for resource interpretation customizations. (#2750, @ikaven1024)karmadactl
: Added execute mod for the interpret command. (#2824, @ikaven1024)karmada-search
: Objects returned by proxy will haveresource.karmada.io/cached-from-cluster
annotation to indicate which member cluster it comes from. (#2469, @ikaven1024)karmada-search
: Users can get the real resource request metrics while using the proxy. (#2481, @ikaven1024)karmada-search
: Users now can use--disable-search
and--disable-proxy
options to disable search and proxy features (both enabled by default). (#2650, @ikaven1024)karmada-controller-manager
: Added implicit priority for PropagationPolicy. (#2267, @Garrybest)karmada-controller-manager
: Introduced resource labelnamespace.karmada.io/skip-auto-propagation: "true"
for namespaces that should be skipped from auto propagation. (#2696, @jwcesign)karmada-controller-manager
: Allowed users to update the.spec.resourceSelectors
field ofPropagationPolicy/ClusterPropagationPolicy
. (#2562, @XiShanYongYe-Chang)- karmada-controller-manager`: Introduced priority to PropagationPolicy. (#2767, @jwcesign)
karmada-scheduler-estimator
: Leveraged scheduler cache to estimate replicas. (#2704, @Garrybest)- `karmada-controller...
karmada v1.3.2 release
Changes since v1.3.1
Bug Fixes
karmada-controller-manager
: Fixed the panic when cluster ImpersonatorSecretRef is nil. (#2690, @stingshen)karmada-controller-manager
: Disable the preemption matching of pp/cpp priority. (#2738, @XiShanYongYe-Chang)karmada-controller-manager
: Fix clusterOverridePolicy and overridePolicy with nil resource selector could not work. (#2788, @wuyingjun-lucky)karmada-controller-manager
: ignore resource that does not match the policy before applying the policy. (#2813, @XiShanYongYe-Chang)HelmChart
: Fixed liveness probe misconfiguration which caused kube-controller-manager to alwaysCrashLoopBackup
. (#2836, @calvin0327)karmadactl
: Fixedinit
commands print incorrect register command issue. (#2744, @Songjoy)karmada-search
: Fixed returned ResourceVersion by proxy not stable issue. (#2816, @cmicat)
Others
Security
: Add limitReader toio.ReadAll
which could limit the memory request and avoid DoS attacks. (#2774, @Poor12)- Karmada(v1.3) is now built with Go1.18.8. (#2852, @RainbowMango)
karmada v1.2.4 release
Changes since v1.2.3
Bug Fixes
karmada-controller-manager
: Fixed the panic when cluster ImpersonatorSecretRef is nil. (#2691, @stingshen)karmada-controller-manager
: Disable the preemption matching of pp/cpp priority. (#2739, @XiShanYongYe-Chang)karmada-controller-manager
: Fix clusterOverridePolicy and overridePolicy with nil resource selector could not work. (#2798, @wuyingjun-lucky)karmada-controller-manager
: ignore resource that does not match the policy before applying the policy. (#2814, @XiShanYongYe-Chang)Security
: Add limitReader toio.ReadAll
which could limit the memory request and avoid DoS attacks. (#2775, @Poor12)- Karmada(v1.2) is now built with Go1.17.13. (#2853, @RainbowMango)
karmada v1.1.6 release
Changes since v1.1.5
Bug Fixes
karmada-controller-manager
: Fixed the panic when cluster ImpersonatorSecretRef is nil. (#2692, @stingshen)karmada-controller-manager
: Disable the preemption matching of pp/cpp priority. (#2740, @XiShanYongYe-Chang)karmada-controller-manager
: Fix ClusterOverridePolicy and OverridePolicy with nil resource selector could not work. (#2800, @wuyingjun-lucky)karmada-controller-manager
: ignore resource that does not match the policy before applying policy. (#2815, @XiShanYongYe-Chang)Security
: Add limitReader toio.ReadAll
which could limit the memory request and avoid DoS attacks. (#2776, @Poor12)- Karmada(v1.1) is now built with Go1.17.13. (#2854, @RainbowMango)
karmada v1.3.1 release
Changes since v1.3.0
Bug Fixes
karmada-controller-manager
/karmada-agent
: Fixed pod information can not be collected issue when building resource summary. (#2500, @Poor12)karmada-search
: fix concurrent map writes panic while list via proxy; And add cache source annotation to returned objects. (#2514, @ikaven1024)karmada-agent
: FixedServiceExport
controller can not reportendpointSlices
issue(due to misscreate
permission). (#2517, @lonelyCZ)karmadactl
: Fixedinit
can not honor IPV6 address issue when generating kubeconfig file. (#2520, @duanmengkk)karmada-controller-manager
: use cluster secret ref namespace in unified-auth-controller when generate ClusterRoleBinding. (#2530, @XiShanYongYe-Chang)karmadactl
: Fixed--karmada-data
directory not initialization issue ininit
command. (#2563, @jwcesign)karmadactl
: Fixed options ofdeinit
can not be shown issue. (#2567, @helen-frank)
karmada v1.2.3 release
Changes since v1.2.2
Bug Fixes
karmada-controller-manager
/karmada-agent
: Fixed a resource status can not be collected issue in case of Resource Interpreter returns an error. (#2429, @zhuwint)karmada-sechduler
: Fixed a panic issue whenreplicaDivisionPreference
isWeighted
andWeightPreference
is nil. (#2453, @XiShanYongYe-Chang)karmadactl
: Fixedinit
can not honor IPV6 address issue when generating kubeconfig file. (#2529, @duanmengkk)karmada-controller-manager
: use cluster secret ref namespace in unified-auth-controller when generate ClusterRoleBinding. (#2531, @XiShanYongYe-Chang)karmadactl
: Fixed options ofdeinit
can not be shown issue. (#2566, @helen-frank)
karmada v1.1.5 release
Changes since v1.1.4
Bug Fixes
Helm Chart
: Fixedkarmada-apiserver
always pending in case of upgrade issue. (#2436, @Poor12)karmadactl
: Fixedinit
can not honor IPV6 address issue when generating kubeconfig file. (#2535, @duanmengkk)karmada-controller-manager
: use cluster secret ref namespace in unified-auth-controller when generate ClusterRoleBinding.(#2538, @XiShanYongYe-Chang )
karmada v1.3.0 release
What's New
Taint-based eviction in graceful way
We introduced a new controller named taint manager
which aims to evict workloads from faulty clusters after a grace period.
Then the scheduler would select new best-fit clusters for the workloads. In addition, if the feature GracefulEviction
is enabled,
the eviction will be very smooth, that is, the removal of evicted workloads will be delayed until the workloads are available on
new clusters or reach the maximum grace period. For more details please refer to Failover Overview.
(Feature contributor: @Garrybest, @XiShanYongYe-Chang)
Global proxy for resources across multi-clusters
We introduced a new proxy feature to karmada-search
that allows users to access resources in multiple clusters in a way just like accessing resources in a single cluster. No matter whether the resources are managed by Karmada, by leveraging the proxy, users
can manipulate the resources from the Karmada control plane.
For more details please refer to Global Resource Proxy.
(Feature contributor: @ikaven1024, @XiShanYongYe-Chang)
Cluster resource modeling
To provide a more accurate scheduling basis for the scheduler, we introduced a way to model the cluster's available resources.
The cluster status controller will model the resources as per the customized resource models, which is more accurate than the general resource summary. For more details please refer to Cluster Resource Modeling.
(Feature contributor: @halfrost, @Poor12)
Bootstrap token-based cluster registration
Now for clusters in Pull
mode, we provide a way for them to register with the Karmada control plane. By leveraging the commands token
and register
in kubectl
, the registration process including deploying the karmada-agent
can be completed very easily. For more details please refer to Register cluster with Pull mode.
(Feature contributor: @lonelyCZ )
Significant improvement in system scalability
We improved the system scalability, such as:
- Enable pprof(#2008)
- Introduce
cachedRESTMapper
(#2187) - Adopt the transform function to reduce memory usage(#2383)
With these improvements, Karmada can easily manage hundreds of huge clusters. The detailed test report will be released soon.
Other Notable Changes
API changes
- The
Cluster
API is added optional fieldID
to uniquely identify the cluster. (@RainbowMango, #2180) - The
Cluster
API is added optional fieldProxyHeader
to specify the HTTP header required by the proxy server. (@mrlihanbo, #1874) - The
Cluster
API is added optional field namedResourceModels
to specify resource modeling. (@halfrost, #2386) - The
Work
andResourceBinding
/ClusterResourceBinding
APIs are added fieldhealth
to represent the state of workload. (@XiShanYongYe-Chang, #2351)
Bug Fixes
karmadactl
: Fixed issue that Kubernetes v1.24 cannot be joined. (@zgfh, #1972)karmadactl
: Fixed a panic issue when retrieving resources from an unknown cluster(karmadactl get xxx --cluster=not-exist
). (@my-git9, #2171)karmadactl
: Fixed failed promoting if a resource with another kind using the same name has been promoted before. (@wuyingjun-lucky, #1824)karmada-search
: Fixed panic when the resource annotation is nil. (@XiShanYongYe-Chang, #1921)karmada-search
: Fixed paniccomparing uncomparable type cache.ResourceEventHandlerFuncs
. (@liys87x, #1951)karmada-search
: Fixed failed query on a single namespace (@luoMonkeyKing, #2227)karmada-controller-manager
: Fixed thatJob
status might be incorrectly marked asCompleted
. (@Garrybest, #1987)karmada-controller-manager
: Fixed returning err when the interpreter webhook returns nil patch and nil patchType. (@CharlesQQ, #2161)karmada-controller-manager
: Fixed that Argo CD cannot assess Deployment health status. (@xuqianjins, #2241)karmada-controller-manager
: Fixed that Argo CD cannot assess StatefulSet/DaemonSet health status. (@RainbowMango, #2252)karmada-controller-manager
/karmada-agent
: Fixed an resource status can not be collected issue in case of Resource Interpreter returns an error. (@XiShanYongYe-Chang, #2428)karmada-sechduler
: Fixed a panic issue whenreplicaDivisionPreference
isWeighted
andWeightPreference
is nil. (@XiShanYongYe-Chang, #2451)
Features & Enhancements
karmadactl
: Added--force
flag todeinit
to skip confirmation. (@zgfh, #2016)karmadactl
: The flag-c
of sub-commandpromote
now has been changed to uppercase-C
. (@Fish-pro, #2140)- ``karmadactl
: Introduced
--cluster-zone` and `--cluster-region` flags to `join` command to specify the zone and region of joining cluster. (@chaunceyjiang, #2048) karmadactl
: Introduced--namespace
flag toexec
command to specify the workload namespace. (@carlory, #2092)karmadactl
: Allowed reading namespaces from the context field of karmada config forget
command. (@carlory, #2148)karmadactl
: Introducedapply
subcommand to apply a configuration to a resource by file name or stdin. (@carlory, #2000)karmadactl
: Introduced--namespace
flag todescribe
command to specify the namespace the workload belongs to. (@TheStylite, #2153)karmadactl
: Introduced--cluster
flag forapply
command to allow users to select one or many member clusters to propagate resources. (@carlory, #2192)karmadactl
: Introduced options subcmd to list global command-line options. (@lonelyCZ, #2283)karmadactl
: Introduced thetoken
command to manage bootstrap tokens. (@lonelyCZ, #2399)karmadactl
: Introduced theregister
command for joining PULL mode cluster. (@lonelyCZ, #2388)karmada-scheduler
: Introduced--enable-empty-workload-propagation
flag to enable propagating empty workloads. (@CharlesQQ, #1720)karmada-scheduler
: Allowed extended plugins in an out-of-tree mode. (@kerthcet, #1663)karmada-scheduler
: Introduced--disable-scheduler-estimator-in-pull-mode
flag to disable scheduler-estimator for clusters in pull mode. (@prodanlabs, #2064)karmada-scheduler
: Introduced--plugins
flag to enable or disable scheduler plugins. (@chaunceyjiang, #2135)karmada-scheduler
: Now the scheduler starts to re-schedule in case of cluster state changes. (@chaunceyjiang, #2301)karmada-search
: The search API supports searching for resources according to labels. (@XiShanYongYe-Chang, #1917)karmada-search
: The annotationcluster.karmada.io/name
which is used to represent the source of cache now has been changed toresource.karmada.io/cached-from-cluster
. (@calvin0327, #1960)karmada-search
: Fixed panic issue when dumping error info. (@AllenZMC, #2231)karmada-controller-manager/karmada-agent
: Cluster state controller now able to collect partial API list in the case of discovery failure. (@duanmengkk, #1968)karmada-controller-manager/karmada-agent
: Introduced--cluster-success-threshold
flag to specify cluster success threshold. Default to 30s. (@dddddai, #1884)karmada-controller-manager/karmada-agent
: Added CronJob support to the default resource interpreter framework. (@chaunceyjiang, #2060)karmada-controller-manager
/karmada-agent
: Introduced--leader-elect-lease-duration
,--leader-elect-renew-deadline
and--leader-elect-retry-period
flags to specify leader election behaviors. (@CharlesQQ, #2056)karmada-controller-manager
/karmada-agent
: Fixed panic issue when dumping error inf...