Ta update configs to enable mtls #3015

ItielOlenick · 2024-06-06T05:42:16Z

Description: When CertManager and secrets RBAC permissions are granted, mTLS will be used between the target allocator and the collector so that the latter can retrieve authentication secrets for endpoints that require them.

Link to Tracking Issue(s):

Second PR towards a solution for #1669

Testing: Unit tests added. E2E tests added. Tested in-cluster locally.

Documentation: Added documentation

Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.1 to 1.10.0. - [Release notes](https://github.com/gin-gonic/gin/releases) - [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md) - [Commits](gin-gonic/gin@v1.9.1...v1.10.0) --- updated-dependencies: - dependency-name: github.com/gin-gonic/gin dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…elemetry#2951) Bumps the prometheus group with 1 update: [github.com/prometheus/prometheus](https://github.com/prometheus/prometheus). Updates `github.com/prometheus/prometheus` from 0.51.2 to 0.52.0 - [Release notes](https://github.com/prometheus/prometheus/releases) - [Changelog](https://github.com/prometheus/prometheus/blob/main/CHANGELOG.md) - [Commits](prometheus/prometheus@v0.51.2...v0.52.0) --- updated-dependencies: - dependency-name: github.com/prometheus/prometheus dependency-type: direct:production update-type: version-update:semver-minor dependency-group: prometheus ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* enable readiness Probe for otel operator Signed-off-by: Janario Oliveira <[email protected]> * generate CRD and controller changes Signed-off-by: Janario Oliveira <[email protected]> * Adjusted code to be similar to Liveness logic Signed-off-by: Janario Oliveira <[email protected]> * Generated manifests Signed-off-by: Janario Oliveira <[email protected]> * Add changelog Signed-off-by: Janario Oliveira <[email protected]> * Fix lint Signed-off-by: Janario Oliveira <[email protected]> * Removed readinessProbe from alpha CRD Signed-off-by: Janario Oliveira <[email protected]> * Generated manifests Signed-off-by: Janario Oliveira <[email protected]> * Fix lint Signed-off-by: Janario Oliveira <[email protected]> * Centralized probe validation Signed-off-by: Janario Oliveira <[email protected]> --------- Signed-off-by: Janario Oliveira <[email protected]> Co-authored-by: hesam.hamdarsi <[email protected]>

Bumps [github.com/docker/docker](https://github.com/docker/docker) from 26.0.1+incompatible to 26.0.2+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v26.0.1...v26.0.2) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Added new Log Enconder Config Signed-off-by: Yuri Sa <[email protected]> * Added new Log Enconder Config Signed-off-by: Yuri Sa <[email protected]> * Added new Log Enconder Config Signed-off-by: Yuri Sa <[email protected]> * Added new Log Enconder Config Signed-off-by: Yuri Sa <[email protected]> * Added new Log Enconder Config Signed-off-by: Yuri Sa <[email protected]> * Added new Log Enconder Config Signed-off-by: Yuri Sa <[email protected]> * Added new Debug doc Signed-off-by: Yuri Sa <[email protected]> --------- Signed-off-by: Yuri Sa <[email protected]>

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Fix Signed-off-by: Pavol Loffay <[email protected]> * Fix Signed-off-by: Pavol Loffay <[email protected]> * Fix Signed-off-by: Pavol Loffay <[email protected]> * Fix Signed-off-by: Pavol Loffay <[email protected]> * Add test Signed-off-by: Pavol Loffay <[email protected]> --------- Signed-off-by: Pavol Loffay <[email protected]>

Signed-off-by: Yuri Sa <[email protected]>

…ility check (open-telemetry#2964) * Verify ServiceMonitor and PodMonitor are installed in prom cr availability check * Added changelog

…try#2968) Bumps [kyverno/action-install-chainsaw](https://github.com/kyverno/action-install-chainsaw) from 0.2.0 to 0.2.1. - [Release notes](https://github.com/kyverno/action-install-chainsaw/releases) - [Commits](kyverno/action-install-chainsaw@v0.2.0...v0.2.1) --- updated-dependencies: - dependency-name: kyverno/action-install-chainsaw dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Create a separate Service Monitor when the Prometheus exporter is present Signed-off-by: Israel Blancas <[email protected]> * Improve changelog Signed-off-by: Israel Blancas <[email protected]> * Fix prometheus-cr E2E test Signed-off-by: Israel Blancas <[email protected]> * Remove unused target Signed-off-by: Israel Blancas <[email protected]> * Add docstring Signed-off-by: Israel Blancas <[email protected]> * Fix typo Signed-off-by: Israel Blancas <[email protected]> * Change the label name Signed-off-by: Israel Blancas <[email protected]> * Change changelog description Signed-off-by: Israel Blancas <[email protected]> * Recover removed labels Signed-off-by: Israel Blancas <[email protected]> * Add missing labels Signed-off-by: Israel Blancas <[email protected]> * Remove wrong labels Signed-off-by: Israel Blancas <[email protected]> --------- Signed-off-by: Israel Blancas <[email protected]>

* Prepare release 0.100.0 Signed-off-by: Vineeth Pothulapati <[email protected]> * update the chlog * update the chlog with open-telemetry#2877 merge --------- Signed-off-by: Vineeth Pothulapati <[email protected]>

* Refactor consistent-hashing strategy * Refactor per-node strategy * Refactor least-weighted strategy * Minor allocation strategy refactor * Add some common allocation strategy tests * Fix collector and target reassignment * Minor allocator fixes * Add changelog entry * Fix an incorrect comment

* add back webhook port * chlog

…2967)

Signed-off-by: Pavol Loffay <[email protected]>

* Support for kubernetes 1.30 version * Update makefile

…or, target allocator, opamp bridge (open-telemetry#2933) * set things * fix kustomize shim * restore, better chlog

…y#2984)

Bumps alpine from 3.19 to 3.20. --- updated-dependencies: - dependency-name: alpine dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…emetry#2991) Bumps alpine from 3.19 to 3.20. --- updated-dependencies: - dependency-name: alpine dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [github.com/go-logr/logr](https://github.com/go-logr/logr) from 1.4.1 to 1.4.2. - [Release notes](https://github.com/go-logr/logr/releases) - [Changelog](https://github.com/go-logr/logr/blob/master/CHANGELOG.md) - [Commits](go-logr/logr@v1.4.1...v1.4.2) --- updated-dependencies: - dependency-name: github.com/go-logr/logr dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…try#2989) Bumps [kyverno/action-install-chainsaw](https://github.com/kyverno/action-install-chainsaw) from 0.2.1 to 0.2.2. - [Release notes](https://github.com/kyverno/action-install-chainsaw/releases) - [Commits](kyverno/action-install-chainsaw@v0.2.1...v0.2.2) --- updated-dependencies: - dependency-name: kyverno/action-install-chainsaw dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps the otel group with 5 updates: | Package | From | To | | --- | --- | --- | | [go.opentelemetry.io/otel](https://github.com/open-telemetry/opentelemetry-go) | `1.26.0` | `1.27.0` | | [go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp](https://github.com/open-telemetry/opentelemetry-go) | `1.26.0` | `1.27.0` | | [go.opentelemetry.io/otel/metric](https://github.com/open-telemetry/opentelemetry-go) | `1.26.0` | `1.27.0` | | [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go) | `1.26.0` | `1.27.0` | | [go.opentelemetry.io/otel/sdk/metric](https://github.com/open-telemetry/opentelemetry-go) | `1.26.0` | `1.27.0` | Updates `go.opentelemetry.io/otel` from 1.26.0 to 1.27.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.26.0...v1.27.0) Updates `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp` from 1.26.0 to 1.27.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.26.0...v1.27.0) Updates `go.opentelemetry.io/otel/metric` from 1.26.0 to 1.27.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.26.0...v1.27.0) Updates `go.opentelemetry.io/otel/sdk` from 1.26.0 to 1.27.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.26.0...v1.27.0) Updates `go.opentelemetry.io/otel/sdk/metric` from 1.26.0 to 1.27.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.26.0...v1.27.0) --- updated-dependencies: - dependency-name: go.opentelemetry.io/otel dependency-type: direct:production update-type: version-update:semver-minor dependency-group: otel - dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp dependency-type: direct:production update-type: version-update:semver-minor dependency-group: otel - dependency-name: go.opentelemetry.io/otel/metric dependency-type: direct:production update-type: version-update:semver-minor dependency-group: otel - dependency-name: go.opentelemetry.io/otel/sdk dependency-type: direct:production update-type: version-update:semver-minor dependency-group: otel - dependency-name: go.opentelemetry.io/otel/sdk/metric dependency-type: direct:production update-type: version-update:semver-minor dependency-group: otel ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: Jacob Aronoff <[email protected]>

tests/e2e-ta-collector-mtls/ta-collector-mtls/00-install.yaml

swiatekm

I think this looks good overall. I commented on some minor issues and missing (fairly trivial) unit tests, but otherwise I think we're almost good to go.

@ItielOlenick we still need to wait for the prometheus receiver fix in 0.108.0, right? Are you planning on adding a test that verifies it works?

internal/config/main.go

internal/manifests/targetallocator/certificate.go

internal/manifests/targetallocator/issuer.go

internal/manifests/collector/volume.go

internal/manifests/targetallocator/configmap.go

internal/manifests/targetallocator/service.go

internal/manifests/targetallocator/targetallocator.go

internal/manifests/targetallocator/volume.go

pkg/constants/env.go

tests/e2e-ta-collector-mtls/ta-collector-mtls/chainsaw-test.yaml

ItielOlenick · 2024-09-02T13:56:38Z

I think this looks good overall. I commented on some minor issues and missing (fairly trivial) unit tests, but otherwise I think we're almost good to go.

@ItielOlenick we still need to wait for the prometheus receiver fix in 0.108.0, right? Are you planning on adding a test that verifies it works?

The fix is already present in 0.108.0, and I've added it to the e2e test's image override until operator is bumped to that version - so we are already testing that in the current e2e tests for this branch.

swiatekm

👍 Thank you for all the work you've put into this change (and the related ones elsewhere)! I've left some questions about config handling in the Target Allocator, but I'm fine merging this PR as-is.

Do note that we have to wait until 0.108.0 is released before actually merging, as we depend on that collector version.

cmd/otel-allocator/config/config.go

cmd/otel-allocator/config/flags.go

ItielOlenick · 2024-09-04T16:39:01Z

👍 Thank you for all the work you've put into this change (and the related ones elsewhere)! I've left some questions about config handling in the Target Allocator, but I'm fine merging this PR as-is.

Do note that we have to wait until 0.108.0 is released before actually merging, as we depend on that collector version.

My pleasure! Quite the journey.
BTW Since this change is feature gated to work from collector v0.180.0 I believe it can already be merged.

swiatekm · 2024-09-05T11:05:47Z

👍 Thank you for all the work you've put into this change (and the related ones elsewhere)! I've left some questions about config handling in the Target Allocator, but I'm fine merging this PR as-is.
Do note that we have to wait until 0.108.0 is released before actually merging, as we depend on that collector version.

My pleasure! Quite the journey. BTW Since this change is feature gated to work from collector v0.180.0 I believe it can already be merged.

We're going to be releasing 0.108.0 very soon, and merging this afterwards will also let us avoid having to explicitly set the collector image in E2E tests, so I'd prefer to wait. I'd also like to give @jaronoff97 and @pavolloffay a bit of time to review and raise any concerns.

swiatekm

We should be good to go after you resolve conflicts and update collector versions. @jaronoff97 @pavolloffay can you also review? I'd like to get this in before 0.109.0.

tests/e2e-ta-collector-mtls/ta-collector-mtls/00-install.yaml

pkg/featuregate/featuregate.go

Removed collector image override for e2e test

Co-authored-by: Mikołaj Świątek <[email protected]>

jaronoff97

Looking REALLY GOOD @ItielOlenick Thank you so much for your work here. A few minor things, once resolved I think we should be g2g :D

go.mod

jaronoff97 · 2024-09-10T14:51:06Z

internal/manifests/collector/container.go

@@ -91,6 +93,14 @@ func Container(cfg config.Config, logger logr.Logger, otelcol v1beta1.OpenTeleme
 			})
 	}

+	if cfg.CertManagerAvailability() == certmanager.Available && featuregate.EnableTargetAllocatorMTLS.IsEnabled() {


thought (take it or leave it) we could move this featuregate check to the CertManagerAvailability call/package so we would only update there, idk how necessary though so feel free to ignore :)

jaronoff97 · 2024-09-10T14:52:31Z

internal/manifests/targetallocator/adapters/config_to_prom_config.go

@@ -257,10 +259,31 @@ func AddHTTPSDConfigToPromConfig(prometheus map[interface{}]interface{}, taServi
 	return prometheus, nil
 }

+func WithTLSConfig(caFile, certFile, keyFile, taServiceName string) TAOption {


this is fine for now, but I do want to simplify this in the future with a similar pattern to #3206

internal/manifests/targetallocator/adapters/config_to_prom_config_test.go

jaronoff97 · 2024-09-10T14:54:48Z

internal/manifests/targetallocator/certificate.go

+		Spec: cmv1.CertificateSpec{
+			DNSNames: []string{
+				naming.TAService(params.TargetAllocator.Name),
+				fmt.Sprintf("%s.%s.svc", naming.TAService(params.TargetAllocator.Name), params.TargetAllocator.Namespace),


(for a follow up) should we allow for adding DNS names here? (cc @swiatekm im thinking about the #3248 case here)

I half-feel like users who want to proxy connections to the API Server should just set GOPROXY and be done with it, but I'm not sure how this is usually solved.

I thought goproxy was only for modules? Maybe i misunderstand what that is for though...

Sorry, I meant HTTP_PROXY and the like.

internal/manifests/targetallocator/targetallocator.go

internal/manifests/manifestutils/utils.go

ItielOlenick · 2024-09-10T21:18:15Z

Looking REALLY GOOD @ItielOlenick Thank you so much for your work here. A few minor things, once resolved I think we should be g2g :D

Sure thing! Currently on vacation, will get on it once I get back.

ItielOlenick and others added 30 commits May 13, 2024 21:44

Initial commit

69b4c63

Added Cert Manager CRDs & RBAC validation and management

684afbd

Added relevant resources and started adding tests

50d568a

[chore] move VineethReddy02 to emeritus (open-telemetry#2957)

dfc7882

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

Fixed non-expected warnings on TA webhook. (open-telemetry#2962)

7fafce9

Signed-off-by: Yuri Sa <[email protected]>

Verify ServiceMonitor and PodMonitor are installed in prom cr availab…

53c5046

…ility check (open-telemetry#2964) * Verify ServiceMonitor and PodMonitor are installed in prom cr availability check * Added changelog

Prepare release 0.100.0 (open-telemetry#2960)

314d7c1

* Prepare release 0.100.0 Signed-off-by: Vineeth Pothulapati <[email protected]> * update the chlog * update the chlog with open-telemetry#2877 merge --------- Signed-off-by: Vineeth Pothulapati <[email protected]>

Bring back webhook port (open-telemetry#2973)

d1126a2

* add back webhook port * chlog

patch 0.100.1 (open-telemetry#2974)

435b900

Update the OpenTelemetry Java agent version to 2.4.0 (open-telemetry#…

2a9db5f

…2967)

simplify deletion logic (open-telemetry#2971)

eaa4e63

Update maintainers in the operator hub PR (open-telemetry#2977)

ac93c7e

Signed-off-by: Pavol Loffay <[email protected]>

Support for kubernetes 1.30 version (open-telemetry#2975)

1a731c9

* Support for kubernetes 1.30 version * Update makefile

[chore] Move TargetAllocator CRD to v1alpha1 (open-telemetry#2918)

cfc1dc3

[featuregate] Automatically set GOMEMLIMIT and GOMAXPROCS for collect…

cb7aaa8

…or, target allocator, opamp bridge (open-telemetry#2933) * set things * fix kustomize shim * restore, better chlog

Fix querying OpenShift user workload monitoring stack. (open-telemetr…

5326614

…y#2984)

ItielOlenick and others added 2 commits August 31, 2024 20:57

Update pkg/featuregate/featuregate.go

4093554

Co-authored-by: Jacob Aronoff <[email protected]>

Added https, serviceMonitor and tls resources assertions to e2e tests

6f7f817

swiatekm reviewed Sep 2, 2024

View reviewed changes

tests/e2e-ta-collector-mtls/ta-collector-mtls/00-install.yaml Outdated Show resolved Hide resolved

Using namespaced names for ClusterRoles

80cd205

swiatekm reviewed Sep 2, 2024

View reviewed changes

ItielOlenick added 4 commits September 2, 2024 19:42

Cleanup

06b1dfe

Added CertManager resources unit tests

e073de6

Added unit tests and e2e assertions

7e412ed

Added missing assertion call

17ff564

ItielOlenick requested a review from swiatekm September 3, 2024 13:34

swiatekm approved these changes Sep 4, 2024

View reviewed changes

cmd/otel-allocator/config/config.go Show resolved Hide resolved

cmd/otel-allocator/config/flags.go Show resolved Hide resolved

swiatekm approved these changes Sep 10, 2024

View reviewed changes

tests/e2e-ta-collector-mtls/ta-collector-mtls/00-install.yaml Outdated Show resolved Hide resolved

pkg/featuregate/featuregate.go Outdated Show resolved Hide resolved

ItielOlenick and others added 3 commits September 10, 2024 09:51

Update 00-install.yaml

7a8e8dd

Removed collector image override for e2e test

Merge branch 'main' into TA-update-configs-to-enable-mtls

e39f354

Update pkg/featuregate/featuregate.go

e8f7ae2

Co-authored-by: Mikołaj Świątek <[email protected]>

jaronoff97 reviewed Sep 10, 2024

View reviewed changes

Merge branch 'main' into TA-update-configs-to-enable-mtls

6a328d7

ItielOlenick requested review from a team as code owners September 25, 2024 11:28

ItielOlenick added 3 commits September 25, 2024 15:00

Minor fixes

27b887c

Fixed tests referencing logging exporter

f14821f

Moved mTLS file naming consts

9f16d57

jaronoff97 approved these changes Sep 25, 2024

View reviewed changes

swiatekm self-requested a review September 25, 2024 19:54

swiatekm approved these changes Sep 25, 2024

View reviewed changes

swiatekm requested a review from pavolloffay September 25, 2024 19:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ta update configs to enable mtls #3015

Ta update configs to enable mtls #3015

ItielOlenick commented Jun 6, 2024 •

edited

Loading

swiatekm left a comment

ItielOlenick commented Sep 2, 2024

swiatekm left a comment

ItielOlenick commented Sep 4, 2024 •

edited

Loading

swiatekm commented Sep 5, 2024

swiatekm left a comment

jaronoff97 left a comment

jaronoff97 Sep 10, 2024

jaronoff97 Sep 10, 2024

jaronoff97 Sep 10, 2024

swiatekm Sep 10, 2024

jaronoff97 Sep 10, 2024

swiatekm Sep 11, 2024

ItielOlenick commented Sep 10, 2024

Ta update configs to enable mtls #3015

Are you sure you want to change the base?

Ta update configs to enable mtls #3015

Conversation

ItielOlenick commented Jun 6, 2024 • edited Loading

swiatekm left a comment

Choose a reason for hiding this comment

ItielOlenick commented Sep 2, 2024

swiatekm left a comment

Choose a reason for hiding this comment

ItielOlenick commented Sep 4, 2024 • edited Loading

swiatekm commented Sep 5, 2024

swiatekm left a comment

Choose a reason for hiding this comment

jaronoff97 left a comment

Choose a reason for hiding this comment

jaronoff97 Sep 10, 2024

Choose a reason for hiding this comment

jaronoff97 Sep 10, 2024

Choose a reason for hiding this comment

jaronoff97 Sep 10, 2024

Choose a reason for hiding this comment

swiatekm Sep 10, 2024

Choose a reason for hiding this comment

jaronoff97 Sep 10, 2024

Choose a reason for hiding this comment

swiatekm Sep 11, 2024

Choose a reason for hiding this comment

ItielOlenick commented Sep 10, 2024

ItielOlenick commented Jun 6, 2024 •

edited

Loading

ItielOlenick commented Sep 4, 2024 •

edited

Loading