Knative Serving release v0.17.0

Meta

initialScale annotation to control the initial deployment size

There is a new annotation that can be used to control the number of pods that are initially deployed when new Revisions are rolled out.

net-contour and net-kourier have moved to Beta

In addition to net-istio, we now have three networking layers that we classify as (at least) Beta.

Kubernetes minimum version has NOT changed

It remains 1.16, but we may bump to 1.17 in the coming release depending on its availability for qualification testing (per our release principles).

Autoscaling

• Launched the initial scale with possibility of starting with 0 (thanks @taragu) [#8613, #8846]
• Launched new KPA statuses, which permit significant simplification of the state machine in revision and KPA itself:
  • Initial scale reached (thanks @markusthoemmes & @taragu)
  • SKS ready (thanks @vagababov)
• Concurrency & stat reporting rewrite in Activator (thanks @markusthoemmes) [#8787, #8796 ]
• Configurable idle conns/conns per host (thanks @vagababov & @julz) [#8810, #9027]
• Optimize pod counting in KPA (3 passes over pods to 1) (thanks @vagababov) [#8759, #8762]
• Tricky optimization of returned lambda in activator saving 16b allocations per every request in activator (thanks @julz) (#8851)
• Lots of new benchmarks thanks [@julz & @markusthoemmes]
• Various cleanups, test stability, code optimizations, etc [thanks @julz, @markusthoemmes, @vagababov, @skonto]

Core API

• Leader Election enabled by default (thanks @mattmoor)
  • By default control plane components now enable leader election, which can be disabled (for now) with --disable-ha.
• New feature flags are now available - see config-features for details
  • Enable affinity, nodeSelector and tolerations. #8645 (thanks @emaildanwilson)
  • Enable additional container & pod security context attributes. #9060 (thanks @dprotaso)
• Adopt a two-lane work queue for our controllers to prevent starvation during global re-syncs knative/pkg#1512 (thanks @vagababov)
• Add config knob "max-value," which allows for setting a cluster-wide value for the max scale of any revision that doesn't have the "autoscaling.knative.dev/maxScale" annotation. #8951 (thanks @arturenault)
• Adds a 60 second timeout for image digest resolution to guard against slow registries #8724 (thanks @julz)
• Implemented new garbage collector that allows for either time-based or min/max count bounds for automatic deletion of old revisions. #8621 (thanks @whaught)
  • To enable this a new v2 Labeler populates RoutingState and RoutingStateModified annotations on Revisions
• PodSpec DryRun also validates unparented (service-less) Configurations. #8828 (thanks @whaught)
• Users can specify the size of the initial deployment with both cluster-wide flag initial-scale, and annotation "autoscaling.internal.knative.dev/initialScale". Cluster-wide flag allow-zero-initial-scale controls whether the cluster-wide and revision initial scale can be zero. #8846, (thanks @taragu)
• When enabled, the ResponsiveGC feature flag disables lastPinned annotation timestamp refreshes #8757 (thanks @whaught)
• Added a workaround so Knative will work on AKS 1.17+ knative/pkg#1592 (thanks @n3wscott)
• Webhooks now drain for longer when shutting down knative/pkg#1517 (thanks @mattmoor)

Networking

• Net-contour is moved to Beta stage knative/docs#2737 (thanks @mattmoor)
• Net-kourier is moved to Beta stage knative/docs#2738 (thanks @mattmoor)
• The default Kingress timeout is increased to 48 hours to prevent gRPC stream timeout #8965 (thanks @tcnghia)
• Code in knative/serving/pkg/network is completely moved to knative/networking repo (thanks @tcnghia)
• Placeholder service's labels and annotations are propagated from Route. #8798 (thanks @nak3)
• When auto TLS is enabled, now net-istio controller generates Istio TLS Gateway per Kingress instead of reconciling the knative-ingress-gateway Gateway knative-extensions/net-istio#170 (thanks @ZhiminXiang)
• Kingress (net-istio) introduces RewriteHost feature knative-extensions/net-istio#174 (thanks @julz)
• Kingress prober improvement for net-istio: probing a single host instead of every host to improve the throughput of the prober queue knative-extensions/net-istio#190 (thanks @JRBANCEL)

Knative Serving release v0.14.3

Meta

Monitoring Bundle is deprecated

We have made the decision to deprecate the bundled monitoring tools that have remained unchanged since 2018 due to a lack of community interest. We will stop releasing them in a coming release and will instead focus on documenting how to integrate with existing monitoring systems using OpenTelemetry.

V1 is now our storage version

We have included a new migration Job to migrate existing resources. See the serving-storage-version-migration.yaml release artifact.

Several new net-* repos!

• Our Istio integration has moved out of Serving and into knative/net-istio.
• Kourier has moved to knative/net-kourier.
• We have a new knative/net-http01 project for implementing auto-TLS.

We have NOT bumped our minimum Kubernetes dependency (still 1.15)

We were unable to bump our minimum Kubernetes dependency to 1.16 this release as planned due to its lack of availability in GKE (on which we have a hard dependency for CI/CD). The principle behind our choice of minimum upstream version remains the same, and users should expect future releases to attempt to “catch up”.

Autoscaling

• Disable metric scraping in situations where the activator is always in path for increased efficiency #7431 (thanks @dsimansk)
• Added a metric for measuring metric scraping overhead #7232 (thanks @rmoe)
• The “Metric” resource now surfaces potential errors in its status #7525 (thanks @markusthoemmes)
• Activator tracks revision public service endpoints to assign downstream pods #7208 (thanks @vagababov)
• Documented the internal autoscaling systems #7126 (thanks @markusthoemmes)
  Cleanups and improvements (logging, metrics, config map, unit and e2e tests, etcs); many PRs (thanks @julz, @mgencur, @vagababov, @markusthoemmes)

Fixed various bugs

• Fixed races where a revision briefly scales below minScale only to immediately scale up again #7110, #7214 (thanks @tanzeeb)
• Fixed a bug where a revision would never become ready if minScale was set > 1 #7514 (thanks @markusthoemmes)
• Fixed a bug where request counts have been reported off by 1 on scale-from-0 #7109 (thanks @vagababov)
• Fixed potential panics around timeout handling in the queue-proxy #7138, #7146 (thanks @JRBANCEL)
• Fixed a rare race condition, where the activator would fail to schedule new , requests even though there is capacity in the system #7360 (thanks @markusthoemmes)

Core API

V1 is now our storage version #7204, #7499 (thanks @dprotaso)

After installing 0.14, a new migration Job must be run to migrate pre-existing resources, and remove v1alpha1 as a stored version from our CRDs.

Support for resolving AWS ECR images #7244 (thanks @mattmoor)

Fixes a long-standing issue where our tag resolutions does not work properly for AWS ECR.

Assorted Cleanups:

• Leader election config map cleaned up, defaulting is implemented, example verified as default values (thanks @vagababov)

Networking

Introducing knative/net-istio repository (thanks @mattmoor, @nghia, @tshafer):

Istio KIngress reconciler is now separated into its own repository knative/net-istio, enabling more focused testing on presubmits. In the future, Istio integration bugs should be filed to this new repository

Introducing knative/net-http01 repository (thanks @mattmoor):

knative/net-http01 is a simple standalone ACME HTTP01 solver for the Knative Certificate abstraction.

Introducing knative/net-kourier repository (thanks @dortiz, @jmprussi):

A new home for Kourier - a lightweight Envoy-based Knative Ingress reconciler previously hosted at https://github.com/3scale/kourier.

Support Istio canonical service and revision #6832 (thanks @tshafer):

Adding Istio canonical service labels (istio/istio#20943) to Knative objects for better integration with Istio UX.

Use /healthz for probe path for easier whitelisting #5918 (thanks itsmurugappa, shreejad)

We changed our probe path from /_internal/knative/activator/probe to /healthz and made that consistent across all probe receivers in Knative Serving.

Best effort Istio probing #6962 (thanks JRBANCEL)

Any scenario where probing would fail forever with the current implementation is now treated as a successful probing, to allow failing-open in cases where users use a 3-legged-oauth setup that would cause probing to fail indefinitely.

Generated VirtualService contains wrong gateways field knative-extensions/net-istio#44 (thanks @yanniszark)

Previously, we sometimes referred to unused Gateways in a VirtualService. That caused issues with Istio validation logic if those unused Gateways were non-existent. Unused Gateways are no longer referred from VirtualServices.

Assorted cleanups:

• Remove usages of deprecated field VirtualService.WebsocketUpgrade knative-extensions/net-istio#53 (thanks @nak3)
• Networking ConfigMap cleaned up, example verified as defaults and Go templates are cached, rather than parsed on every invocation #7403, #7408, #7395 (thanks @vagababov)

Knative Serving release v0.16.0

Meta

Minimum Kubernetes version supported is 1.16

• The previous release documented a minimum version of 1.16, where this release actually raises the hard limit as we have begun to take advantage of 1.16 features (namely CRD v1 API).

PodAutoscaler custom metrics API is dropped

• The autoscaler no longer implements the custom metrics API contract and we also no longer ship the APIService necessary to enable generic metric clients (like the HPA) to fetch those metrics from the autoscaler. Revisions can no longer be scaled using concurrency and/or request-per-second metrics when using the HPA.
• Multiple shoutouts via mailing lists and the community meeting yielded no usage of that feature.

We no longer release a serving.yaml manifest

• We stopped documenting this manifest some time ago, and it has long been the concatenation of several of the other manifests.

Post-Install Jobs

• The new serving-post-install-jobs.yaml is expected to be used with kubectl create - the jobs are idempotent

Autoscaling

• Improved load-balancing behavior for revisions that have the activator in their networking path #8226 #8263 (thanks @vagababov)
• Dropped support for HPA-scaling based on concurrency/RPS metrics #8318 (thanks @markusthoemmes)
• Stop renewing panic mode if it’s not necessary** #8125 (thanks @vagababov)**
• Added validation of autoscaler classes under the “knative.dev” domain #8224 (thanks @yanweiguo)
• Optimized the protocol between the Activator and the Autoscaler #8266 (thanks @julz)
• Made autoscaler calculations consistent between CPU architectures #8341 (thanks @mundaym)
• Keep connections alive during scrapes, if possible #8367 (thanks @julz)
• Scale non-routable revisions down quicker #8389 (thanks @vagababov)
• Remove endpoints informer from autoscaler reducing reducing our API server load for watches, memory usage and GC (thanks @vagababov)

Core API

• Support for multiple-containers is now alpha (many PRs, thanks @savitaashture, @skonto)
  • You can now use multiple-containers in the pod spec of a Knative Service when you set “multi-container” to “enabled” in our config-features ConfigMap.
• Support for disabling “service links” #8439, #8498, #8499 (thanks @dprotaso, @mattmoor, @vagababov)
  • This let’s through a field of the pod spec that was added in K8s 1.13 to disable a feature of the Kubernetes runtime environment called “service links”. This early service discovery feature injects 8 environment variables into the Pod’s containers for each Kubernetes Service in the same namespace, which leads to serious problems when many services are deployed. We have started to socialize a change to the default runtime behavior here, which would take effect in 0.19: #8563.
• Support for using the downwards API in environment variables #8126 (thanks @JRBANCEL)
  • This let’s users start to use fieldRef in their environment variable spec to project information like namespace into their containers.
  • This feature must be explicitly enabled by setting “kubernetes.podspec-fieldref” to “enabled” in our config-features ConfigMap.
• We have extended our leader election support to apply to Knative webhooks (many PRs, thanks @mattmoor, @yanweiguo)
  • This is the first part of this feature track.
• Ongoing improvements to our generated controller infrastructure (many PRs, thanks @whaught)
  • In reconcilers for “Knative-shaped” resources, the generated controller logic takes on a number of additional “best-practice” responsibilities (now by default!), including management of “observed generation”
• Reduce the idle queue-proxy CPU usage 10x #8148 (thanks @mattmoor)
  • The default exec probe frequency on minScale revisions led to a high CPU usage by the queue proxy due to the frequency. We reduced the frequency from 1s to 10s to reduce this overhead by roughly 10x.
• Users are now warned if they change the “_example” block in ConfigMaps seemingly by accident #8123 (thanks @markusthoemmes)
• Ingress conformance tests have been moved out of test files so they can be consumed downstream #8150 (thanks @dprotaso)
• Operators can now set queue proxy resource requests/limits in the config-deployment.yaml config map #8195 (thanks @julz)

Networking

• Fix Unknown cert status issue when cluster-local visibility is set #8043 (thanks @nak3)
• Support tag header based routing (thanks @igsong, @tanzeeb, @tcnghia, @ZhiminXiang)
• Split networking related resources from knative/serving repo into knative/networking repo (thanks @tcnghia)
• Reduce the high CPU usage of idle queue-proxy #8147 #8149 (thanks @mattmoor, @vagababov)
• Increase the QPS limit of networking probing #8054 (thanks @JRBANCEL)
• Fix the issue that namespace-level auto TLS feature does not work with web browser because of HTTP connection reuse #7495 (thanks @ZhiminXiang)
• Drop istio-injection=enabled label in knative-serving namespace from serving-core.yaml. #8482 (thanks @nak3)
• Add a documentation about how to use Istio Authorization with Knative (thanks @nak3)

Knative Serving release v0.15.2

Meta

go mod migration

Knative is now completely migrated to Golang modules.

Serving release artifact deprecations

serving.yaml and serving-cert-manager.yaml will be shipped for the last time in this release. They have been broken out into separate artifacts. Please refer to the current installation docs for guidance on how to install Knative Serving and its optional components.

Minimum supported Kubernetes version bumped to 1.16

As per the Kubernetes minimum version principle - our current minimum supported Kubernetes version is now 1.16.

Autoscaling

Activator Subsetting (thanks @vagababov)

We compute a subset of Activator pods for each revision in a consistent manner, rather than assigning all. This noticeably improves load balancing for smaller revisions with small container concurrency values.

• Improved pod scraping latency by directly scraping pods if available #7804 (thanks @vagababov)
• Autoscaling Documentation (thanks @markusthoemmes)
• Last pod retention period #7931 (thanks @vagababov)
• Unify Activator and QueueProxy stats reporting libraries and report more precise concurrency values from Activator #7775 (thanks @makusthoemmes)
• Add a global setting which prohibits setting container concurrency to 0 #7932 (thanks @julz)
• Progress deadline is now a configurable parameter #7649 (thanks @vagababov)
• Burst capacity is calculated over the panic window now (thanks @vagababov)
• General code cleanup, test stabilization, etc thanks (@julz, @markusthoemmes, @vagababov, @nak3)

Core API

• Our Revision shape has slightly changed to support multiple containers in the future #7373 (thanks @savitaashture)
  • Revision.Status.ImageDigest is deprecated and the digest will appear in Revision.Status.ContainerStatus.
• Enable K8s dry-run as an experimental feature to provide faster feedback when your template won't create a valid Pod #3425 (thanks @whaught)
  • These are currently opt-in via the current annotation (may change)
    • features.knative.dev/podspec-dryrun: enabled
    • features.knative.dev/podspec-dryrun: strict
  • Strict mode will return failures if dry-run is not supported. This happens when webhooks have side-effects.
• Webhook infrastructure now supports receiving a callback when a deletion occurs pkg/#1219 (thanks @whaught)
• Some lingering and deprecated v1alpha1 properties have been removed from our go types
  • Revision's concurrencyModel #7893 (thanks @Vagabov)
  • Revision's buildName and buildRef #7896 (thanks @Vagabov)
• Reduced some churn reconciling deleted objects when they were tracking dependent resources #7679 (thanks @markusthoemmes)
• genreconciler now allows developers to override the controller’s name pkg/#1137 (thanks @shashwathi @andrew-su)

Networking

• Remove /var/log symlink logic from the queue proxy #7882 (thanks @dprotaso)
  • /var/log log capture now supports containers that aren't named user-container.
• Add support for labels in DomainTemplate #7647 (thanks @duglin)
  • This allows users to create custom URLs via the template and to choose custom domains in the config-domain configMap via labels.
• net-certmanager repository setup and code migration (thanks @ZhiminXiang)
  • Cert-manager related resources for AutoTLS are generated and released from the net-certmanager repository now.
• KIngress no longer uses retries #7842 (thanks @tcnghia)
• Operation name for activator's proxy span and queue-proxy's span are renamed to {activator,queue}_proxy #7934 (thanks @nak3)
• Ingress conformance test for visibility and path #7666 (thanks @andrew-su)
• Better timeouts for the ingress prober #7702 (thanks @JRBANCEL)
• For ingress prober, use default http.Transport and context with timeout for better timeouts #7702 (thanks @JRBANCEL)
• Use "go mod" within net-istio, net-contour, net-certmanager, net-http01 (thanks @andrew-su, @mattmoor, @tcnghia, @ZhiminXiang)
• Propagate status from KCert to Route #7163 (thanks @nak3)

Knative Serving release v0.14.2

Meta

Monitoring Bundle is deprecated

We have made the decision to deprecate the bundled monitoring tools that have remained unchanged since 2018 due to a lack of community interest. We will stop releasing them in a coming release and will instead focus on documenting how to integrate with existing monitoring systems using OpenTelemetry.

V1 is now our storage version

We have included a new migration Job to migrate existing resources. See the serving-storage-version-migration.yaml release artifact.

Several new net-* repos!

• Our Istio integration has moved out of Serving and into knative/net-istio.
• Kourier has moved to knative/net-kourier.
• We have a new knative/net-http01 project for implementing auto-TLS.

We have NOT bumped our minimum Kubernetes dependency (still 1.15)

We were unable to bump our minimum Kubernetes dependency to 1.16 this release as planned due to its lack of availability in GKE (on which we have a hard dependency for CI/CD). The principle behind our choice of minimum upstream version remains the same, and users should expect future releases to attempt to “catch up”.

Autoscaling

• Disable metric scraping in situations where the activator is always in path for increased efficiency #7431 (thanks @dsimansk)
• Added a metric for measuring metric scraping overhead #7232 (thanks @rmoe)
• The “Metric” resource now surfaces potential errors in its status #7525 (thanks @markusthoemmes)
• Activator tracks revision public service endpoints to assign downstream pods #7208 (thanks @vagababov)
• Documented the internal autoscaling systems #7126 (thanks @markusthoemmes)
  Cleanups and improvements (logging, metrics, config map, unit and e2e tests, etcs); many PRs (thanks @julz, @mgencur, @vagababov, @markusthoemmes)

Fixed various bugs

• Fixed races where a revision briefly scales below minScale only to immediately scale up again #7110, #7214 (thanks @tanzeeb)
• Fixed a bug where a revision would never become ready if minScale was set > 1 #7514 (thanks @markusthoemmes)
• Fixed a bug where request counts have been reported off by 1 on scale-from-0 #7109 (thanks @vagababov)
• Fixed potential panics around timeout handling in the queue-proxy #7138, #7146 (thanks @JRBANCEL)
• Fixed a rare race condition, where the activator would fail to schedule new , requests even though there is capacity in the system #7360 (thanks @markusthoemmes)

Core API

V1 is now our storage version #7204, #7499 (thanks @dprotaso)

After installing 0.14, a new migration Job must be run to migrate pre-existing resources, and remove v1alpha1 as a stored version from our CRDs.

Support for resolving AWS ECR images #7244 (thanks @mattmoor)

Fixes a long-standing issue where our tag resolutions does not work properly for AWS ECR.

Assorted Cleanups:

• Leader election config map cleaned up, defaulting is implemented, example verified as default values (thanks @vagababov)

Networking

Introducing knative/net-istio repository (thanks @mattmoor, @nghia, @tshafer):

Istio KIngress reconciler is now separated into its own repository knative/net-istio, enabling more focused testing on presubmits. In the future, Istio integration bugs should be filed to this new repository

Introducing knative/net-http01 repository (thanks @mattmoor):

knative/net-http01 is a simple standalone ACME HTTP01 solver for the Knative Certificate abstraction.

Introducing knative/net-kourier repository (thanks @dortiz, @jmprussi):

A new home for Kourier - a lightweight Envoy-based Knative Ingress reconciler previously hosted at https://github.com/3scale/kourier.

Support Istio canonical service and revision #6832 (thanks @tshafer):

Adding Istio canonical service labels (istio/istio#20943) to Knative objects for better integration with Istio UX.

Use /healthz for probe path for easier whitelisting #5918 (thanks itsmurugappa, shreejad)

We changed our probe path from /_internal/knative/activator/probe to /healthz and made that consistent across all probe receivers in Knative Serving.

Best effort Istio probing #6962 (thanks JRBANCEL)

Any scenario where probing would fail forever with the current implementation is now treated as a successful probing, to allow failing-open in cases where users use a 3-legged-oauth setup that would cause probing to fail indefinitely.

Generated VirtualService contains wrong gateways field knative-extensions/net-istio#44 (thanks @yanniszark)

Previously, we sometimes referred to unused Gateways in a VirtualService. That caused issues with Istio validation logic if those unused Gateways were non-existent. Unused Gateways are no longer referred from VirtualServices.

Assorted cleanups:

• Remove usages of deprecated field VirtualService.WebsocketUpgrade knative-extensions/net-istio#53 (thanks @nak3)
• Networking ConfigMap cleaned up, example verified as defaults and Go templates are cached, rather than parsed on every invocation #7403, #7408, #7395 (thanks @vagababov)

Knative Serving release v0.15.1

Meta

go mod migration

Knative is now completely migrated to Golang modules.

Serving release artifact deprecations

serving.yaml and serving-cert-manager.yaml will be shipped for the last time in this release. They have been broken out into separate artifacts. Please refer to the current installation docs for guidance on how to install Knative Serving and its optional components.

Minimum supported Kubernetes version bumped to 1.16

As per the Kubernetes minimum version principle - our current minimum supported Kubernetes version is now 1.16.

Autoscaling

Activator Subsetting (thanks @vagababov)

We compute a subset of Activator pods for each revision in a consistent manner, rather than assigning all. This noticeably improves load balancing for smaller revisions with small container concurrency values.

• Improved pod scraping latency by directly scraping pods if available #7804 (thanks @vagababov)
• Autoscaling Documentation (thanks @markusthoemmes)
• Last pod retention period #7931 (thanks @vagababov)
• Unify Activator and QueueProxy stats reporting libraries and report more precise concurrency values from Activator #7775 (thanks @makusthoemmes)
• Add a global setting which prohibits setting container concurrency to 0 #7932 (thanks @julz)
• Progress deadline is now a configurable parameter #7649 (thanks @vagababov)
• Burst capacity is calculated over the panic window now (thanks @vagababov)
• General code cleanup, test stabilization, etc thanks (@julz, @markusthoemmes, @vagababov, @nak3)

Core API

• Our Revision shape has slightly changed to support multiple containers in the future #7373 (thanks @savitaashture)
  • Revision.Status.ImageDigest is deprecated and the digest will appear in Revision.Status.ContainerStatus.
• Enable K8s dry-run as an experimental feature to provide faster feedback when your template won't create a valid Pod #3425 (thanks @whaught)
  • These are currently opt-in via the current annotation (may change)
    • features.knative.dev/podspec-dryrun: enabled
    • features.knative.dev/podspec-dryrun: strict
  • Strict mode will return failures if dry-run is not supported. This happens when webhooks have side-effects.
• Webhook infrastructure now supports receiving a callback when a deletion occurs pkg/#1219 (thanks @whaught)
• Some lingering and deprecated v1alpha1 properties have been removed from our go types
  • Revision's concurrencyModel #7893 (thanks @Vagabov)
  • Revision's buildName and buildRef #7896 (thanks @Vagabov)
• Reduced some churn reconciling deleted objects when they were tracking dependent resources #7679 (thanks @markusthoemmes)
• genreconciler now allows developers to override the controller’s name pkg/#1137 (thanks @shashwathi @andrew-su)

Networking

• Remove /var/log symlink logic from the queue proxy #7882 (thanks @dprotaso)
  • /var/log log capture now supports containers that aren't named user-container.
• Add support for labels in DomainTemplate #7647 (thanks @duglin)
  • This allows users to create custom URLs via the template and to choose custom domains in the config-domain configMap via labels.
• net-certmanager repository setup and code migration (thanks @ZhiminXiang)
  • Cert-manager related resources for AutoTLS are generated and released from the net-certmanager repository now.
• KIngress no longer uses retries #7842 (thanks @tcnghia)
• Operation name for activator's proxy span and queue-proxy's span are renamed to {activator,queue}_proxy #7934 (thanks @nak3)
• Ingress conformance test for visibility and path #7666 (thanks @andrew-su)
• Better timeouts for the ingress prober #7702 (thanks @JRBANCEL)
• For ingress prober, use default http.Transport and context with timeout for better timeouts #7702 (thanks @JRBANCEL)
• Use "go mod" within net-istio, net-contour, net-certmanager, net-http01 (thanks @andrew-su, @mattmoor, @tcnghia, @ZhiminXiang)
• Propagate status from KCert to Route #7163 (thanks @nak3)

Knative Serving release v0.14.1

Meta

Monitoring Bundle is deprecated

We have made the decision to deprecate the bundled monitoring tools that have remained unchanged since 2018 due to a lack of community interest. We will stop releasing them in a coming release and will instead focus on documenting how to integrate with existing monitoring systems using OpenTelemetry.

V1 is now our storage version

We have included a new migration Job to migrate existing resources. See the serving-storage-version-migration.yaml release artifact.

Several new net-* repos!

• Our Istio integration has moved out of Serving and into knative/net-istio.
• Kourier has moved to knative/net-kourier.
• We have a new knative/net-http01 project for implementing auto-TLS.

We have NOT bumped our minimum Kubernetes dependency (still 1.15)

We were unable to bump our minimum Kubernetes dependency to 1.16 this release as planned due to its lack of availability in GKE (on which we have a hard dependency for CI/CD). The principle behind our choice of minimum upstream version remains the same, and users should expect future releases to attempt to “catch up”.

Autoscaling

• Disable metric scraping in situations where the activator is always in path for increased efficiency #7431 (thanks @dsimansk)
• Added a metric for measuring metric scraping overhead #7232 (thanks @rmoe)
• The “Metric” resource now surfaces potential errors in its status #7525 (thanks @markusthoemmes)
• Activator tracks revision public service endpoints to assign downstream pods #7208 (thanks @vagababov)
• Documented the internal autoscaling systems #7126 (thanks @markusthoemmes)
  Cleanups and improvements (logging, metrics, config map, unit and e2e tests, etcs); many PRs (thanks @julz, @mgencur, @vagababov, @markusthoemmes)

Fixed various bugs

• Fixed races where a revision briefly scales below minScale only to immediately scale up again #7110, #7214 (thanks @tanzeeb)
• Fixed a bug where a revision would never become ready if minScale was set > 1 #7514 (thanks @markusthoemmes)
• Fixed a bug where request counts have been reported off by 1 on scale-from-0 #7109 (thanks @vagababov)
• Fixed potential panics around timeout handling in the queue-proxy #7138, #7146 (thanks @JRBANCEL)
• Fixed a rare race condition, where the activator would fail to schedule new , requests even though there is capacity in the system #7360 (thanks @markusthoemmes)

Core API

V1 is now our storage version #7204, #7499 (thanks @dprotaso)

After installing 0.14, a new migration Job must be run to migrate pre-existing resources, and remove v1alpha1 as a stored version from our CRDs.

Support for resolving AWS ECR images #7244 (thanks @mattmoor)

Fixes a long-standing issue where our tag resolutions does not work properly for AWS ECR.

Assorted Cleanups:

• Leader election config map cleaned up, defaulting is implemented, example verified as default values (thanks @vagababov)

Networking

Introducing knative/net-istio repository (thanks @mattmoor, @nghia, @tshafer):

Istio KIngress reconciler is now separated into its own repository knative/net-istio, enabling more focused testing on presubmits. In the future, Istio integration bugs should be filed to this new repository

Introducing knative/net-http01 repository (thanks @mattmoor):

knative/net-http01 is a simple standalone ACME HTTP01 solver for the Knative Certificate abstraction.

Introducing knative/net-kourier repository (thanks @dortiz, @jmprussi):

A new home for Kourier - a lightweight Envoy-based Knative Ingress reconciler previously hosted at https://github.com/3scale/kourier.

Support Istio canonical service and revision #6832 (thanks @tshafer):

Adding Istio canonical service labels (istio/istio#20943) to Knative objects for better integration with Istio UX.

Use /healthz for probe path for easier whitelisting #5918 (thanks itsmurugappa, shreejad)

We changed our probe path from /_internal/knative/activator/probe to /healthz and made that consistent across all probe receivers in Knative Serving.

Best effort Istio probing #6962 (thanks JRBANCEL)

Any scenario where probing would fail forever with the current implementation is now treated as a successful probing, to allow failing-open in cases where users use a 3-legged-oauth setup that would cause probing to fail indefinitely.

Generated VirtualService contains wrong gateways field knative-extensions/net-istio#44 (thanks @yanniszark)

Previously, we sometimes referred to unused Gateways in a VirtualService. That caused issues with Istio validation logic if those unused Gateways were non-existent. Unused Gateways are no longer referred from VirtualServices.

Assorted cleanups:

• Remove usages of deprecated field VirtualService.WebsocketUpgrade knative-extensions/net-istio#53 (thanks @nak3)
• Networking ConfigMap cleaned up, example verified as defaults and Go templates are cached, rather than parsed on every invocation #7403, #7408, #7395 (thanks @vagababov)

Knative Serving release v0.15.0

Meta

go mod migration

Knative is now completely migrated to Golang modules.

Serving release artifact deprecations

serving.yaml and serving-cert-manager.yaml will be shipped for the last time in this release. They have been broken out into separate artifacts. Please refer to the current installation docs for guidance on how to install Knative Serving and its optional components.

Minimum supported Kubernetes version bumped to 1.16

As per the Kubernetes minimum version principle - our current minimum supported Kubernetes version is now 1.16.

Autoscaling

Activator Subsetting (thanks @vagababov)

We compute a subset of Activator pods for each revision in a consistent manner, rather than assigning all. This noticeably improves load balancing for smaller revisions with small container concurrency values.

• Improved pod scraping latency by directly scraping pods if available #7804 (thanks @vagababov)
• Autoscaling Documentation (thanks @markusthoemmes)
• Last pod retention period #7931 (thanks @vagababov)
• Unify Activator and QueueProxy stats reporting libraries and report more precise concurrency values from Activator #7775 (thanks @makusthoemmes)
• Add a global setting which prohibits setting container concurrency to 0 #7932 (thanks @julz)
• Progress deadline is now a configurable parameter #7649 (thanks @vagababov)
• Burst capacity is calculated over the panic window now (thanks @vagababov)
• General code cleanup, test stabilization, etc thanks (@julz, @markusthoemmes, @vagababov, @nak3)

Core API

• Our Revision shape has slightly changed to support multiple containers in the future #7373 (thanks @savitaashture)
  • Revision.Status.ImageDigest is deprecated and the digest will appear in Revision.Status.ContainerStatus.
• Enable K8s dry-run as an experimental feature to provide faster feedback when your template won't create a valid Pod #3425 (thanks @whaught)
  • These are currently opt-in via the current annotation (may change)
    • features.knative.dev/podspec-dryrun: enabled
    • features.knative.dev/podspec-dryrun: strict
  • Strict mode will return failures if dry-run is not supported. This happens when webhooks have side-effects.
• Webhook infrastructure now supports receiving a callback when a deletion occurs pkg/#1219 (thanks @whaught)
• Some lingering and deprecated v1alpha1 properties have been removed from our go types
  • Revision's concurrencyModel #7893 (thanks @Vagabov)
  • Revision's buildName and buildRef #7896 (thanks @Vagabov)
• Reduced some churn reconciling deleted objects when they were tracking dependent resources #7679 (thanks @markusthoemmes)
• genreconciler now allows developers to override the controller’s name pkg/#1137 (thanks @shashwathi @andrew-su)

Networking

• Remove /var/log symlink logic from the queue proxy #7882 (thanks @dprotaso)
  • /var/log log capture now supports containers that aren't named user-container.
• Add support for labels in DomainTemplate #7647 (thanks @duglin)
  • This allows users to create custom URLs via the template and to choose custom domains in the config-domain configMap via labels.
• net-certmanager repository setup and code migration (thanks @ZhiminXiang)
  • Cert-manager related resources for AutoTLS are generated and released from the net-certmanager repository now.
• KIngress no longer uses retries #7842 (thanks @tcnghia)
• Operation name for activator's proxy span and queue-proxy's span are renamed to {activator,queue}_proxy #7934 (thanks @nak3)
• Ingress conformance test for visibility and path #7666 (thanks @andrew-su)
• Better timeouts for the ingress prober #7702 (thanks @JRBANCEL)
• For ingress prober, use default http.Transport and context with timeout for better timeouts #7702 (thanks @JRBANCEL)
• Use "go mod" within net-istio, net-contour, net-certmanager, net-http01 (thanks @andrew-su, @mattmoor, @tcnghia, @ZhiminXiang)
• Propagate status from KCert to Route #7163 (thanks @nak3)

Knative Serving release v0.13.3

Meta

Minimum Kubernetes version remains 1.15

This is NOT a change from 0.12, however, with the adoption of Conversion webhooks this is no longer something that may be overridden without consequence.
The target minimum version for our next (0.14) release will be Kubernetes 1.16.

Deprecation of the Alpha and Beta Serving APIs

The v1 APIs are now available in every supported version of Knative, and our controllers are now consuming v1 themselves.
We will continue to ship the deprecated APIs for 9 months (6 releases), so these will be removed in the 0.19 release.

We now rely on CRD Conversion webhooks

We take advantage of this long-awaited Beta+ feature in 1.15+ to manage converting between v1alpha1, v1beta1, and v1 types.

Autoscaling

• Probe and forward traffic to non-ready pods (#6695 thanks @markusthoemmes)
• gRPC e2e autoscaling test (#6778, thanks @tanzeeb, @shashwathi)
• We no longer restrict min target at 1, permitting correct target utilization with CC=1 (#6951 thanks @vagababov)
• Ignore young pods from metric computation (#6649, #6626, thanks @vagababov)
• Metric Reporting Refactoring
  • Activator #6804, #6843 (thanks @markusthoemmes)
  • Autoscaler (#6774,#6707, #6712), QP #6852 (thanks @vagababov)
• Cleanups and improvements (logging, metrics, unit and e2e tests, etcs); many PRs (thanks @jpeach, @MIBc, @taragu, @markusthoemmes, @vagababov)

Core API

• We’ve adopted generated reconcilers to help minimize the boilerplate in our controllers (thanks @n3wscott, @mattmoor, @shashwathi #6993 #6973 #6969 #6952)
• We’re removed the serving stats reporter that was reporting some nonsensical metrics (thanks @mattmoor #6939)
• Webhook certificates now rotate (thanks @mpetason knative/pkg#1101)
• The validating admission webhook will now apply the correct defaults (thanks @itsmurugappan #6938)
• We’ve started our journey of actually deprecating the v1alpha1 APIs for the resources Service, Configuration, Revision and Route.
  • The controllers for these resources use the v1 APIs (thanks @dprotaso, @mattmoor #6933 #6949 #6950 #6957 #6958 #6959 #6960)
  • Thanks for the conversion webhook framework @dprotaso (knative/pkg#993)
  • Note: v1alpha1 will remain the storage version until we provide guidance on how to migrate the storage version to v1 - see #6726
  • Our current plan is remove v1alpha1 and v1beta1 APIs in 0.19
  • HPA auto scaling using Revision metrics (concurrency & requests per second) now use v1 APIs (thanks @dprotaso #6957).
    • Consuming revision metrics for the resource version v1alpha1 is deprecated and will be removed in the next release (0.14)

Networking

• Deprecate the istio.sidecar.includeOutboundIPRanges in config-network #6597 (thanks @nak3)
• Avoid unconditionally reconciling the Gateways on deletion #6934 (thanks @ZhiminXiang)
• Remove "internal" in class name of Certificates #6887 (thanks @ZhiminXiang)
• Wrong revision is picked up for traffic target marked as "latest" #6876 (thanks @taragu)
• Fix Ready -> NotReady- > Ready flip flops in Ingress Prober #6648 (thanks @JRBANCEL)
• Clean up orphaned VirtualService when migrating from Istio KIngress to other KIngress #6570 (thanks @nak3)
• Avoid specifying IngressTLS before Certificate reports Ready. #6870 (thanks @mattmoor)
• queue-proxy to returns 504 on connect timeouts #6859 (thanks @vagababov)
• KIngress to disallow ServiceNamespace that differs from its own #6868 (thanks @MIBc)
• Fix name collision when having two Route with name ${route} and ${route}-mesh #6362 (thanks @sreddy)
• Route reconciler to separate cluster local Ingress rules and external domain rules to avoid ClusterLocal special-casing in KIngress implementation #6727 (thanks @tcnghia, @andrew-su)
• Correctly set the network prober User-Agent #6644 (thanks @jpeach)
• gRPC, AutoTLS, and KIngress testing (thanks @ZhiminXiang, @tanzeeb, @sreddy, @rmoe, andrew-su@)

Knative Serving release v0.11.2

Meta

Load-balancing improvements with low containerConcurrency

At low containerConcurrency’s we now perform significantly better due to improvements in the application-specific load-balancing performed by the Activator component.

Kourier networking support

We have a new option for handling the ingress capabilities used by knative/serving. Kourier is the first Knative-native ingress implementation, which reconciles the Knative networking CRDs to program Envoy directly.

Autoscaling

Locally perfect loadbalancing and endpoint subsetting improvements (thanks @vagababov)

These are further improvements to the loadbalancing enhancements over the last releases. Given a stable activator count, loadbalancing of a revision with the activator on the path is now locally ideal. The graph.

Reduced the needed Kubernetes Services per Revision from 3 to 2 #5900 (thanks @markusthoemmes)

The third service used to be used for metric scraping exclusively. This is now done via the private service as well. Metric services are no longer created and actively removed in existing deployments.

Allow applications with a livenessProbe to properly scale down #5986 (thanks @nak3)

The queue-proxy wrongly counted requests sent via livenessProbes as actual requests, causing the revision to never shut down. These requests are now properly ignored.

Target annotation values can now exceed configured defaults #5975 (thanks @markusthoemmes)

This fixes a bug in the logic to determine the actual target of the autoscaler which capped the user-defined target value to the configured default value.

Report desired/actual scale in PodAutoscalers for the HPA as well (thanks @vagababov)

The values for desired and actual scale are now plumbed through from the HPA into the PodAutoscaler’s status.

Assorted code readability, optimizations and clean ups (thanks @vagababov, @markusthoemmes, @mgencur)

Core API

Improved error messages for image tag resolving #5920 (thanks @markusthoemmes)

Previous error messages did not indicate that the image pull failure occurred during digest resolution, and did not provide further details as to why the digest resolution failed. This change aides users in debugging problems in container registry permissions.

Enabled imagePullSecrets in PodSpec #5917 (thanks @taragu)

Users may now specify imagePullSecrets directly without attaching them to their Kubernetes ServiceAccount.

Add permissions for caching.internal.knative.dev to edit and view cluster roles #5992 (thanks @nak3)

Knative provides aggregated ClusterRoles that users can use to manage their Knative resources. These roles previously did not include the caching resource. This change adds the caching resource to both the edit and view roles.

Split apart defaulting and validation webhooks #5947 (thanks @mattmoor)

This fixes a problem where our validation wasn’t necessarily applied to the final object because it runs at the same time as defaulting, which might be before additional mutating webhooks. By separating things out we ensure that the validation occurs on the final object to be committed to etcd.

Configuration and Service now labeled with duck.knative.dev/podspecable #6121 (thanks @mattmoor)

This enables tools that reflect over the Kubernetes type system to reason about the podspec portion of these Knative resources.

Bug Fixes:

• Fix bug where latestRevision routes can point to wrong revision #5319 (thanks @taragu)
• Fix issue where config-defaults were not getting applied #5892 (thanks @taragu)
• Fix validation issue for lastModifier when using multiple service accounts #6072 (thanks @savitaashture)
• Fix problem with Configuration reporting Ready early #6096 (thanks @taragu)
• Validation added for name and generateName fields in RevisionTemplate #5110 (thanks @savitaashture)

Test Improvements:

• File access checks in conformance tests #5102 (thanks @shashwathi)
• Improve route visibility test #5831 (thanks @andrew-su)
• Added ability to run e2e tests with https #5157 (thanks @taragu)
• More reliable WaitForService function #5956 (thanks @mgencur)
• Added new upgrade test to catch broken defaulting behavior #6165 (thanks @dgerd)
• Fix digest validation in conformance to account for image templating #6106 (thanks @markusthoemmes)

Networking

Compatibility with Istio 1.4 #6058 (thanks @nak3)

Istio 1.4 introduced a breaking restriction to the length of regular expressions allowed in VirtualServices. We switched to using prefixes to be compatible with Istio 1.4.

Integration with istio/client-go #5969 pkg#208 pkg#831 (thanks @skaslev)

Knative now uses istio/client-go instead of its own version of Istio API client. This addressed a long pain-point of maintaining a manually-translated API client to a changing API.

3scale/kourier integration #5983 (thanks @bbrowning @davidor @jmprusi)

Kourier is a light-weight Ingress for Knative (a deployment of Kourier consists of an Envoy proxy and a control plane for it). In v0.11 we add Kourier as an option to run Knative e2e integration tests.

Better LoadBalancerReady condition when VirtualService failed to be reconciled #6048 (thanks @nak3)

Previously when VirtualService failed to be reconciled the LoadBalancerReady Condition isn’t updated. We fix this to surface reason and message from the failing VirtualService.

Post-ClusterIngress migration cleanups (thanks @markusthoemmes)

Clean up port names of Knative components to follow Istio convention #5070 (thanks @iamejboy)

Bug fix #5734: Do not permit cluster local kservices on the cluster ingress #6174 (thanks @vagababov)

Fix a bug where cluster-local ksvcs are erroneously exposed to the public ingress.

Monitoring

Add default request metrics backend in observability config #6022 (thanks @drpmma)

This change makes the default backend Prometheus and makes it consistent with the default example value in config-observability.yaml

Fix missing required selector for node-exporter #5934 (thanks @lionelvillard)