chore: Upgrade OpenTelemetry Collector to 0.104.0-sumo-1 (#3829)

* chore: update OpenTelemetry Collector to 0.104.0-sumo-1 * Remove deprecated compress_encoding to use compression * Change the bind address for the otel collector and the health check ext to the pod IP * Regenerate test files after changes to bind IP address configuration across all pods * Add otel collector labels to metrics collection
SumoLogic · Aug 20, 2024 · a10a4c7 · a10a4c7
1 parent ef923c9
commit a10a4c7
Show file tree

Hide file tree

Showing 100 changed files with 427 additions and 198 deletions.
diff --git a/.changelog/3829.changed.txt b/.changelog/3829.changed.txt
@@ -0,0 +1 @@
+chore: update OpenTelemetry Collector to 0.104.0-sumo-1
diff --git a/deploy/helm/sumologic/README.md b/deploy/helm/sumologic/README.md
@@ -169,7 +169,7 @@ The following table lists the configurable parameters of the Sumo Logic chart an
 | `sumologic.noProxy`                                                                   | List of comma separated hostnames which should be excluded from the proxy                                                                                                                                                                                                                                  | `kubernetes.default.svc`                                                                                                                                                             |
 | `sumologic.pullSecrets`                                                               | Optional list of secrets that will be used for pulling images for Sumo Logic's deployments and statefulsets.                                                                                                                                                                                               | `Nil`                                                                                                                                                                                |
 | `sumologic.otelcolImage.repository`                                                   | Default image repository for OpenTelemetry Collector. This can be overridden for specific components.                                                                                                                                                                                                      | `public.ecr.aws/sumologic/sumologic-otel-collector`                                                                                                                                  |
-| `sumologic.otelcolImage.tag`                                                          | Default image tag for OpenTelemetry Collector. This can be overridden for specific components.                                                                                                                                                                                                             | `0.102.1-sumo-0`                                                                                                                                                                     |
+| `sumologic.otelcolImage.tag`                                                          | Default image tag for OpenTelemetry Collector. This can be overridden for specific components.                                                                                                                                                                                                             | `0.104.0-sumo-1`                                                                                                                                                                     |
 | `sumologic.otelcolImage.addFipsSuffix`                                                | Add a `-fips` suffix to all image tags. See [help.sumologic.com/docs/send-data/kubernetes/security-best-practices](https://help.sumologic.com/docs/send-data/kubernetes/security-best-practices/) for more information.                                                                                    | `false`                                                                                                                                                                              |
 | `sumologic.podLabels`                                                                 | Additional labels for the pods.                                                                                                                                                                                                                                                                            | `{}`                                                                                                                                                                                 |
 | `sumologic.podAnnotations`                                                            | Additional annotations for the pods.                                                                                                                                                                                                                                                                       | `{}`                                                                                                                                                                                 |
@@ -322,7 +322,7 @@ The following table lists the configurable parameters of the Sumo Logic chart an
 | `opentelemetry-operator.manager.autoInstrumentationImage.nodejs.tag`                  | OpenTelemetry Operator NodeJS Instrumentation image tag.                                                                                                                                                                                                                                                   | `0.40.0`                                                                                                                                                                             |
 | `opentelemetry-operator.manager.image.repository`                                     | The default operator image repository for OpenTelemetry.                                                                                                                                                                                                                                                   | `public.ecr.aws/sumologic/opentelemetry-operator`                                                                                                                                    |
 | `opentelemetry-operator.manager.collectorImage.repository`                            | The default collector image repository for OpenTelemetryCollector CRDs.                                                                                                                                                                                                                                    | `public.ecr.aws/sumologic/sumologic-otel-collector`                                                                                                                                  |
-| `opentelemetry-operator.manager.collectorImage.tag`                                   | The default collector image tag for OpenTelemetryCollector CRDs.                                                                                                                                                                                                                                           | `0.102.1-sumo-0`                                                                                                                                                                     |
+| `opentelemetry-operator.manager.collectorImage.tag`                                   | The default collector image tag for OpenTelemetryCollector CRDs.                                                                                                                                                                                                                                           | `0.104.0-sumo-1`                                                                                                                                                                     |
 | `opentelemetry-operator.manager.resources.limits.cpu`                                 | Used to set limit CPU for OpenTelemetry-Operator Manager.                                                                                                                                                                                                                                                  | `250m`                                                                                                                                                                               |
 | `opentelemetry-operator.manager.resources.limits.memory`                              | Used to set limit Memory for OpenTelemetry-Operator Manager.                                                                                                                                                                                                                                               | `512Mi`                                                                                                                                                                              |
 | `opentelemetry-operator.manager.resources.requests.cpu`                               | Used to set requested CPU for OpenTelemetry-Operator Manager.                                                                                                                                                                                                                                              | `150m`                                                                                                                                                                               |

diff --git a/deploy/helm/sumologic/conf/events/otelcol/config.yaml b/deploy/helm/sumologic/conf/events/otelcol/config.yaml
@@ -29,7 +29,8 @@ extensions:
     directory: {{ .Values.sumologic.events.persistence.persistentVolume.path }}
     timeout: 10s
 {{- end }}
-  health_check: {}
+  health_check:
+    endpoint: ${env:MY_POD_IP}:13133
   pprof: {}
 
 processors:

diff --git a/deploy/helm/sumologic/conf/instrumentation/otelcol.instrumentation.conf.yaml b/deploy/helm/sumologic/conf/instrumentation/otelcol.instrumentation.conf.yaml
@@ -12,7 +12,7 @@ exporters:
     endpoint: ${SUMO_ENDPOINT_DEFAULT_METRICS_SOURCE}
     ## Compression encoding format, either empty string (""), gzip or deflate (default gzip).
     ## Empty string means no compression
-    compress_encoding: gzip
+    compression: gzip
     ## Max HTTP request body size in bytes before compression (if applied). By default 1_048_576 (1MB) is used.
     max_request_body_size: 1_048_576 # 1MB
     ## Format to use when sending logs to Sumo. (default json) (possible values: json, text)
@@ -62,27 +62,27 @@ receivers:
   jaeger:
     protocols:
       thrift_compact:
-        endpoint: "0.0.0.0:6831"
+        endpoint: "${env:MY_POD_IP}:6831"
       thrift_binary:
-        endpoint: "0.0.0.0:6832"
+        endpoint: "${env:MY_POD_IP}:6832"
       grpc:
-        endpoint: "0.0.0.0:14250"
+        endpoint: "${env:MY_POD_IP}:14250"
       thrift_http:
-        endpoint: "0.0.0.0:14268"
+        endpoint: "${env:MY_POD_IP}:14268"
   opencensus:
-    endpoint: "0.0.0.0:55678"
+    endpoint: "${env:MY_POD_IP}:55678"
   otlp:
     protocols:
       grpc:
-        endpoint: "0.0.0.0:4317"
+        endpoint: "${env:MY_POD_IP}:4317"
       http:
-        endpoint: "0.0.0.0:4318"
+        endpoint: "${env:MY_POD_IP}:4318"
   otlp/deprecated:
     protocols:
       http:
-        endpoint: "0.0.0.0:55681"
+        endpoint: "${env:MY_POD_IP}:55681"
   zipkin:
-    endpoint: "0.0.0.0:9411"
+    endpoint: "${env:MY_POD_IP}:9411"
 
 processors:
   ## Source processor adds Sumo Logic related metadata
@@ -173,7 +173,8 @@ processors:
     ## Time duration after which a batch will be sent regardless of size
     timeout: 5s
 extensions:
-  health_check: {}
+  health_check:
+    endpoint: ${env:MY_POD_IP}:13133
   memory_ballast:
     ## Memory Ballast size should be max 1/3 to 1/2 of memory.
     size_mib: 512

diff --git a/deploy/helm/sumologic/conf/instrumentation/traces.gateway.conf.yaml b/deploy/helm/sumologic/conf/instrumentation/traces.gateway.conf.yaml
@@ -24,9 +24,9 @@ receivers:
   otlp:
     protocols:
       grpc:
-        endpoint: "0.0.0.0:4317"
+        endpoint: "${env:MY_POD_IP}:4317"
       http:
-        endpoint: "0.0.0.0:4318"
+        endpoint: "${env:MY_POD_IP}:4318"
 processors:
   ## The memory_limiter processor is used to prevent out of memory situations on the collector.
   memory_limiter:
@@ -55,7 +55,8 @@ processors:
     ## Never more than this many spans are being sent in a batch
     # send_batch_max_size: 512
 extensions:
-  health_check: {}
+  health_check:
+    endpoint: ${env:MY_POD_IP}:13133
   memory_ballast:
     ## Memory Ballast size should be max 1/3 to 1/2 of memory.
     size_mib: 512

diff --git a/deploy/helm/sumologic/conf/instrumentation/traces.sampler.conf.yaml b/deploy/helm/sumologic/conf/instrumentation/traces.sampler.conf.yaml
@@ -33,9 +33,9 @@ receivers:
   otlp:
     protocols:
       grpc:
-        endpoint: "0.0.0.0:4317"
+        endpoint: "${env:MY_POD_IP}:4317"
       http:
-        endpoint: "0.0.0.0:4318"
+        endpoint: "${env:MY_POD_IP}:4318"
 processors:
   ## The memory_limiter processor is used to prevent out of memory situations on the collector.
   memory_limiter:
@@ -70,7 +70,8 @@ processors:
     timeout: 5s
 
 extensions:
-  health_check: {}
+  health_check:
+    endpoint: ${env:MY_POD_IP}:13133
 {{- if .Values.tracesSampler.persistence.enabled }}
   ## Configuration for File Storage extension
   file_storage:

diff --git a/deploy/helm/sumologic/conf/logs/collector/otelcloudwatch/config.yaml b/deploy/helm/sumologic/conf/logs/collector/otelcloudwatch/config.yaml
@@ -15,7 +15,8 @@ extensions:
     directory: /var/lib/storage/otc
     timeout: 10s
 {{ end }}
-  health_check: {}
+  health_check:
+    endpoint: ${env:MY_POD_IP}:13133
   pprof: {}
 
 processors:

diff --git a/deploy/helm/sumologic/conf/logs/collector/otelcol/config.yaml b/deploy/helm/sumologic/conf/logs/collector/otelcol/config.yaml
@@ -17,7 +17,8 @@ extensions:
       on_rebound: true
     directory: /var/lib/storage/otc
     timeout: 10s
-  health_check: {}
+  health_check:
+    endpoint: ${env:MY_POD_IP}:13133
   pprof: {}
 
 processors:

diff --git a/deploy/helm/sumologic/conf/logs/collector/otellogswindows/config.yaml b/deploy/helm/sumologic/conf/logs/collector/otellogswindows/config.yaml
@@ -17,7 +17,8 @@ extensions:
       on_rebound: true
     directory: /var/lib/storage/otc
     timeout: 10s
-  health_check: {}
+  health_check:
+    endpoint: ${env:MY_POD_IP}:13133
   pprof: {}
 
 processors:

diff --git a/deploy/helm/sumologic/conf/logs/otelcol/config.yaml b/deploy/helm/sumologic/conf/logs/otelcol/config.yaml
@@ -95,7 +95,8 @@ exporters:
 {{- end }}
 
 extensions:
-  health_check: {}
+  health_check:
+    endpoint: ${env:MY_POD_IP}:13133
 {{ if .Values.metadata.persistence.enabled }}
   ## Configuration for File Storage extension
   file_storage:
@@ -509,7 +510,7 @@ receivers:
   otlp:
     protocols:
       http:
-        endpoint: 0.0.0.0:4318
+        endpoint: ${env:MY_POD_IP}:4318
 {{ end }}
 
 service:

diff --git a/deploy/helm/sumologic/conf/metrics/collector/otelcol/config.yaml b/deploy/helm/sumologic/conf/metrics/collector/otelcol/config.yaml
@@ -15,7 +15,8 @@ exporters:
     disable_keep_alives: true
 
 extensions:
-  health_check: {}
+  health_check:
+    endpoint: ${env:MY_POD_IP}:13133
   pprof: {}
   file_storage:
     directory: /var/lib/storage/otc

diff --git a/deploy/helm/sumologic/conf/metrics/otelcol/config.yaml b/deploy/helm/sumologic/conf/metrics/otelcol/config.yaml
@@ -11,7 +11,8 @@ extensions:
     directory: /var/lib/storage/otc
     timeout: 10s
 {{ end }}
-  health_check: {}
+  health_check:
+    endpoint: ${env:MY_POD_IP}:13133
   pprof: {}
 
 processors:
@@ -40,7 +41,7 @@ receivers:
   otlp:
     protocols:
       http:
-        endpoint: 0.0.0.0:4318
+        endpoint: ${env:MY_POD_IP}:4318
 
 service:
   extensions:

diff --git a/deploy/helm/sumologic/templates/_helpers/_common.tpl b/deploy/helm/sumologic/templates/_helpers/_common.tpl
@@ -493,6 +493,14 @@ Example Usage:
 {{- end -}}
 {{- end -}}
 
+{{- define "pod-ip" -}}
+- name: MY_POD_IP
+  valueFrom:
+    fieldRef:
+      apiVersion: v1
+      fieldPath: status.podIP
+{{- end -}}
+
 {{/*
 Pod anti affinity "hard"
 

diff --git a/deploy/helm/sumologic/templates/events/otelcol/statefulset.yaml b/deploy/helm/sumologic/templates/events/otelcol/statefulset.yaml
@@ -120,6 +120,7 @@ spec:
 {{- $ctx := .Values -}}
 {{- include "kubernetes.sources.envs" (dict "Context" $ctx "Type" "events") | nindent 8 -}}
 {{- include "proxy-env-variables" . | nindent 8 -}}
+{{- include "pod-ip" . | nindent 8 -}}
         {{- if .Values.otelevents.statefulset.extraEnvVars }}
 {{ toYaml .Values.otelevents.statefulset.extraEnvVars | nindent 8 }}
         {{- end }}

diff --git a/deploy/helm/sumologic/templates/instrumentation/otelcol-instrumentation/statefulset.yaml b/deploy/helm/sumologic/templates/instrumentation/otelcol-instrumentation/statefulset.yaml
@@ -136,6 +136,7 @@ spec:
         env:
 {{- $ctx := .Values -}}
 {{- include "kubernetes.sources.env" (dict "Context" $ctx "Type" "metrics" "Name" "default" ) | nindent 8 -}}
+{{- include "pod-ip" . | nindent 8 -}}
 {{- if $otelcolInstrumentation.statefulset.extraEnvVars }}
 {{- toYaml $otelcolInstrumentation.statefulset.extraEnvVars | nindent 8 }}
 {{- end }}

diff --git a/deploy/helm/sumologic/templates/instrumentation/traces-gateway/deployment.yaml b/deploy/helm/sumologic/templates/instrumentation/traces-gateway/deployment.yaml
@@ -75,6 +75,7 @@ spec:
 {{- include "kubernetes.sources.envs" (dict "Context" $ctx "Type" "traces") | nindent 8 -}}
 {{- include "kubernetes.sources.env" (dict "Context" $ctx "Type" "metrics" "Name" "default" ) | nindent 8 }}
 {{- include "proxy-env-variables" . | nindent 8 -}}
+{{- include "pod-ip" . | nindent 8 -}}
 {{- if $tracesGateway.deployment.extraEnvVars }}
 {{- toYaml $tracesGateway.deployment.extraEnvVars | nindent 8 }}
 {{- end }}

diff --git a/deploy/helm/sumologic/templates/instrumentation/traces-sampler/deployment.yaml b/deploy/helm/sumologic/templates/instrumentation/traces-sampler/deployment.yaml
@@ -66,6 +66,7 @@ spec:
 {{- $ctx := .Values -}}
 {{- include "kubernetes.sources.envs" (dict "Context" $ctx "Type" "traces") | nindent 8 -}}
 {{- include "proxy-env-variables" . | nindent 8 -}}
+{{- include "pod-ip" . | nindent 8 -}}
 {{- if $tracesSampler.deployment.extraEnvVars }}
 {{- toYaml $tracesSampler.deployment.extraEnvVars | nindent 8 }}
 {{- end }}

diff --git a/deploy/helm/sumologic/templates/logs/collector/otelcol-cloudwatch/statefulset.yaml b/deploy/helm/sumologic/templates/logs/collector/otelcol-cloudwatch/statefulset.yaml
@@ -117,4 +117,5 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.namespace
+{{- include "pod-ip" . | nindent 8 -}}
 {{- end }}
diff --git a/deploy/helm/sumologic/templates/logs/collector/otelcol/daemonset.yaml b/deploy/helm/sumologic/templates/logs/collector/otelcol/daemonset.yaml
@@ -127,6 +127,7 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.namespace
+{{- include "pod-ip" . | nindent 8 -}}
 {{- if $daemonset.extraEnvVars }}
 {{ toYaml $daemonset.extraEnvVars | nindent 8 }}
 {{- end }}

diff --git a/deploy/helm/sumologic/templates/logs/collector/otellogswindows/daemonset.yaml b/deploy/helm/sumologic/templates/logs/collector/otellogswindows/daemonset.yaml
@@ -132,6 +132,7 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.namespace
+{{- include "pod-ip" . | nindent 8 -}}
 {{- if $daemonset.extraEnvVars }}
 {{ toYaml $daemonset.extraEnvVars | nindent 8 }}
 {{- end }}

diff --git a/deploy/helm/sumologic/templates/logs/otelcol/statefulset.yaml b/deploy/helm/sumologic/templates/logs/otelcol/statefulset.yaml
@@ -141,6 +141,7 @@ spec:
 {{- $ctx := .Values -}}
 {{- include "kubernetes.sources.envs" (dict "Context" $ctx "Type" "logs") | nindent 8 -}}
 {{- include "proxy-env-variables" . | nindent 8 -}}
+{{- include "pod-ip" . | nindent 8 -}}
 {{- if .Values.metadata.logs.statefulset.extraEnvVars }}
 {{- toYaml .Values.metadata.logs.statefulset.extraEnvVars | nindent 8 }}
 {{- end }}

diff --git a/deploy/helm/sumologic/templates/metrics/collector/otelcol/opentelemetrycollector.yaml b/deploy/helm/sumologic/templates/metrics/collector/otelcol/opentelemetrycollector.yaml
@@ -98,6 +98,11 @@ spec:
     valueFrom:
       fieldRef:
         fieldPath: metadata.namespace
+  - name: MY_POD_IP
+    valueFrom:
+      fieldRef:
+        apiVersion: v1
+        fieldPath: status.podIP
   podAnnotations:
     ## The operator adds this annotation by default, but we use our own ServiceMonitor
     prometheus.io/scrape: "false"

diff --git a/deploy/helm/sumologic/templates/metrics/otelcol/statefulset.yaml b/deploy/helm/sumologic/templates/metrics/otelcol/statefulset.yaml
@@ -137,6 +137,7 @@ spec:
 {{- $ctx := .Values -}}
 {{- include "kubernetes.sources.envs" (dict "Context" $ctx "Type" "metrics") | nindent 8 -}}
 {{- include "proxy-env-variables" . | nindent 8 -}}
+{{- include "pod-ip" . | nindent 8 -}}
         {{- if .Values.metadata.metrics.statefulset.extraEnvVars }}
 {{ toYaml .Values.metadata.metrics.statefulset.extraEnvVars | nindent 8 }}
         {{- end }}

diff --git a/deploy/helm/sumologic/values.yaml b/deploy/helm/sumologic/values.yaml
@@ -264,7 +264,7 @@ sumologic:
   ### Global configuration for OpenTelemetry Collector
   otelcolImage:
     repository: "public.ecr.aws/sumologic/sumologic-otel-collector"
-    tag: "0.102.1-sumo-0"
+    tag: "0.104.0-sumo-1"
 
     ## Add a -fips suffix to all image tags. With default tags, this results in FIPS-compliant otel images.
     ## See https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/fips.md for more information.
@@ -2487,7 +2487,7 @@ opentelemetry-operator:
       repository: public.ecr.aws/sumologic/opentelemetry-operator
     collectorImage:
       repository: "public.ecr.aws/sumologic/sumologic-otel-collector"
-      tag: "0.102.1-sumo-0"
+      tag: "0.104.0-sumo-1"
     autoInstrumentationImage:
       java:
         repository: public.ecr.aws/sumologic/autoinstrumentation-java

diff --git a/docs/README.md b/docs/README.md
@@ -107,7 +107,7 @@ The following table displays the currently used software versions for our Helm c
 
 | Name                                      | Version |
 | ----------------------------------------- | ------- |
-| OpenTelemetry Collector                   | 0.102.1 |
+| OpenTelemetry Collector                   | 0.104.0 |
 | OpenTelemetry Operator                    | 0.56.1  |
 | kube-prometheus-stack/Prometheus Operator | 40.5.0  |
 | Falco                                     | 3.8.7   |

diff --git a/tests/helm/testdata/goldenfile/events_otc/basic.output.yaml b/tests/helm/testdata/goldenfile/events_otc/basic.output.yaml
@@ -24,7 +24,8 @@ data:
       file_storage:
         directory: /var/lib/storage/events
         timeout: 10s
-      health_check: {}
+      health_check:
+        endpoint: ${env:MY_POD_IP}:13133
       pprof: {}
     processors:
       batch:

diff --git a/tests/helm/testdata/goldenfile/events_otc/options.output.yaml b/tests/helm/testdata/goldenfile/events_otc/options.output.yaml
@@ -20,7 +20,8 @@ data:
         sending_queue:
           enabled: true
     extensions:
-      health_check: {}
+      health_check:
+        endpoint: ${env:MY_POD_IP}:13133
       pprof: {}
     processors:
       batch:

diff --git a/tests/helm/testdata/goldenfile/events_otc/sumologic-mock.output.yaml b/tests/helm/testdata/goldenfile/events_otc/sumologic-mock.output.yaml
@@ -32,7 +32,8 @@ data:
       file_storage:
         directory: /var/lib/storage/events
         timeout: 10s
-      health_check: {}
+      health_check:
+        endpoint: ${env:MY_POD_IP}:13133
       pprof: {}
     processors:
       batch: