Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OTLP Zipkin Collector #5

Closed
wants to merge 34 commits into from
Closed
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
facbedc
WIP
marcingrzejszczak Jun 12, 2024
c5bad8b
WIP
marcingrzejszczak Jun 13, 2024
37ef965
WIP
marcingrzejszczak Jun 14, 2024
0f7ee5a
Passing build
marcingrzejszczak Jun 14, 2024
45c1d40
Collectors are working fine
marcingrzejszczak Jun 14, 2024
538da4f
HTTP collector tests passing
marcingrzejszczak Jun 21, 2024
a810184
Added OTel HTTP exporter test
marcingrzejszczak Jun 24, 2024
81e36ed
Polish
marcingrzejszczak Jun 24, 2024
d55db5f
Removed OTel GRPC module
marcingrzejszczak Jun 24, 2024
f0680f4
Removed OTel GRPC module
marcingrzejszczak Jun 24, 2024
b766e59
Removed OTel GRPC module
marcingrzejszczak Jun 24, 2024
3639e4d
WIP on ported tests from OTel
marcingrzejszczak Jun 24, 2024
7738906
Fixed tests
marcingrzejszczak Jun 25, 2024
c868600
Trying to make the tests pass
marcingrzejszczak Jun 25, 2024
02c49f1
Trying to make the tests pass
marcingrzejszczak Jun 25, 2024
437f9e9
SpanTranslator tests
marcingrzejszczak Jun 25, 2024
d9faf6a
Added moar tests
marcingrzejszczak Jun 25, 2024
d0d2fe2
Added Zipkin tests
marcingrzejszczak Jun 25, 2024
dbbe43c
Added links
marcingrzejszczak Jun 26, 2024
791175f
Removed tests module
marcingrzejszczak Jun 26, 2024
dc7b762
Fixed translation from OTel to Zipkin
marcingrzejszczak Jun 26, 2024
f30ade5
Removing GRPC
marcingrzejszczak Jun 26, 2024
cf9c6dc
Merged removal of GRPC branch
marcingrzejszczak Jun 26, 2024
d64ff9b
Left only Zipkin OTLP collector
marcingrzejszczak Jun 26, 2024
e5bc2a4
Version bumps
marcingrzejszczak Jun 27, 2024
afe15ae
Merge branch 'main' into receiver
marcingrzejszczak Jun 27, 2024
03ac1ea
Merge branch 'versionBumps' into receiver
marcingrzejszczak Jun 27, 2024
bcbfc36
Uses decorators instead of manually decoding payload
marcingrzejszczak Jun 27, 2024
265e373
Optimize imports
marcingrzejszczak Jun 27, 2024
c706abb
Simplified Collector logic
marcingrzejszczak Jun 27, 2024
579ef19
Improved byte decoding
marcingrzejszczak Jun 27, 2024
45b26b4
Merge branch 'main' into receiver
marcingrzejszczak Jun 28, 2024
d87046d
Changes following the review
marcingrzejszczak Jun 28, 2024
cb1387e
Polish
marcingrzejszczak Jun 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 14 additions & 16 deletions .github/workflows/create_release.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
# yamllint --format github .github/workflows/create_release.yml
---
name: create_release

# We create a release version on a trigger tag, regardless of if the commit is documentation-only.
#
# See https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet
on:
# We create a release version on a trigger tag, regardless of if the commit is
# documentation-only.
on: # yamllint disable-line rule:truthy
push:
tags: 'release-[0-9]+.[0-9]+.[0-9]+**' # Ex. release-1.2.3
tags: # e.g. release-1.2.3
- 'release-[0-9]+.[0-9]+.[0-9]+**'

jobs:
create_release:
Expand All @@ -16,30 +15,29 @@ jobs:
- name: Checkout Repository
uses: actions/checkout@v4
with:
# Prevent use of implicit GitHub Actions read-only token GITHUB_TOKEN. We don't deploy on
# the tag MAJOR.MINOR.PATCH event, but we still need to deploy the maven-release-plugin main commit.
# Prevent use of implicit GitHub Actions read-only GITHUB_TOKEN
# because maven-release-plugin pushes commits to master.
token: ${{ secrets.GH_TOKEN }}
fetch-depth: 1 # only need the HEAD commit as license check isn't run
- name: Setup java
uses: actions/setup-java@v4
with:
distribution: 'zulu' # zulu as it supports a wide version range
java-version: '11' # earliest LTS and last that can compile the 1.6 release profile.
java-version: '17' # earliest LTS supported by Spring Boot 3
- name: Cache local Maven repository
uses: actions/cache@v3
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-maven-
key: ${{ runner.os }}-jdk-17-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-jdk-17-maven-
- name: Create Release
env:
# GH_USER=<user that created GH_TOKEN>
GH_USER: ${{ secrets.GH_USER }}
# GH_TOKEN=<hex token value>
# - makes release commits and tags
# - needs repo:status, public_repo
# - referenced in .settings.xml
# * makes release commits and tags
# * needs repo:status, public_repo
# * referenced in .settings.xml
GH_TOKEN: ${{ secrets.GH_TOKEN }}
run: | # GITHUB_REF will be refs/tags/release-MAJOR.MINOR.PATCH
build-bin/git/login_git &&
build-bin/maven/maven_release $(echo ${GITHUB_REF} | cut -d/ -f 3)
build-bin/maven/maven_release $(echo ${GITHUB_REF} | cut -d/ -f 3)
55 changes: 31 additions & 24 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@
---
name: deploy

# We deploy on main and release versions, regardless of if the commit is
# We deploy on master and release versions, regardless of if the commit is
# documentation-only or not.
#
# See https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet
on:
on: # yamllint disable-line rule:truthy
push:
# Don't deploy tags as they conflict with [maven-release-plugin] prepare release MAJOR.MINOR.PATCH
tags: ''
branches: main
branches:
- main
# Don't deploy tags because the same commit for MAJOR.MINOR.PATCH is also
# on master: Redundant deployment of a release version will fail uploading.
tags-ignore:
- '*'

jobs:
deploy:
Expand All @@ -19,21 +20,20 @@ jobs:
- name: Checkout Repository
uses: actions/checkout@v4
with:
# Prevent use of implicit GitHub Actions read-only token GITHUB_TOKEN.
# We push Javadocs to the gh-pages branch on commit.
# Prevent use of implicit GitHub Actions read-only GITHUB_TOKEN
# because javadoc_to_gh_pages pushes commits to the gh-pages branch.
token: ${{ secrets.GH_TOKEN }}
fetch-depth: 0 # allow build-bin/idl_to_gh_pages to get the full history
- name: Setup java
uses: actions/setup-java@v4
with:
distribution: 'zulu' # zulu as it supports a wide version range
java-version: '11' # earliest LTS and last that can compile the 1.6 release profile.
java-version: '17' # earliest LTS supported by Spring Boot 3
- name: Cache local Maven repository
uses: actions/cache@v3
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-maven-
key: ${{ runner.os }}-jdk-17-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-jdk-17-maven-
# Don't attempt to cache Docker. Sensitive information can be stolen
# via forks, and login session ends up in ~/.docker. This is ok because
# we publish DOCKER_PARENT_IMAGE to ghcr.io, hence local to the runner.
Expand All @@ -42,23 +42,30 @@ jobs:
# GH_USER=<user that created GH_TOKEN>
GH_USER: ${{ secrets.GH_USER }}
# GH_TOKEN=<hex token value>
# - pushes Docker images to ghcr.io
# - create via https://github.com/settings/tokens
# - needs repo:status, public_repo, write:packages, delete:packages
# * pushes gh-pages during build-bin/javadoc_to_gh_pages
# * pushes Docker images to ghcr.io
# * create via https://github.com/settings/tokens
# * needs repo:status, public_repo, write:packages, delete:packages
GH_TOKEN: ${{ secrets.GH_TOKEN }}
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
# GPG_PASSPHRASE=<passphrase for GPG_SIGNING_KEY>
# - referenced in .settings.xml
# * referenced in .settings.xml
GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
# SONATYPE_USER=<sonatype account token>
# - deploys snapshots and releases to Sonatype
# - needs access to io.zipkin via https://issues.sonatype.org/browse/OSSRH-16669
# - generate via https://oss.sonatype.org/#profile;User%20Token
# - referenced in .settings.xml
# * deploys snapshots and releases to Sonatype
# * needs access to io.zipkin via OSSRH-16669
# * generate via https://oss.sonatype.org/#profile;User%20Token
# * referenced in .settings.xml
SONATYPE_USER: ${{ secrets.SONATYPE_USER }}
# SONATYPE_PASSWORD=<password to sonatype account token>
# - referenced in .settings.xml
# * referenced in .settings.xml
SONATYPE_PASSWORD: ${{ secrets.SONATYPE_PASSWORD }}
run: | # GITHUB_REF will be refs/heads/main or refs/tags/MAJOR.MINOR.PATCH
# DOCKERHUB_USER=<typically dockerzipkindeployer>
# * only push repos in openzipkin org to Docker Hub on release
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
# DOCKERHUB_TOKEN=<access token for DOCKERHUB_USER>
# * Access Token from here https://hub.docker.com/settings/security
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: | # GITHUB_REF = refs/heads/main or refs/tags/MAJOR.MINOR.PATCH
build-bin/configure_deploy &&
build-bin/deploy $(echo ${GITHUB_REF} | cut -d/ -f 3)
build-bin/deploy $(echo ${GITHUB_REF} | cut -d/ -f 3)
27 changes: 15 additions & 12 deletions .github/workflows/docker_push.yml
Original file line number Diff line number Diff line change
@@ -1,35 +1,38 @@
# yamllint --format github .github/workflows/docker_push.yml
---
name: docker_push

# We re-push docker images on a trigger tag, regardless of if the commit is documentation-only.
#
# See https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet
on:
# We re-push docker on a trigger tag, regardless of if the commit is
# documentation-only.
on: # yamllint disable-line rule:truthy
push:
tags: 'docker-[0-9]+.[0-9]+.[0-9]+**' # Ex. docker-1.2.3
tags: # e.g. docker-1.2.3
- 'docker-[0-9]+.[0-9]+.[0-9]+**'

jobs:
docker_push:
runs-on: ubuntu-22.04 # newest available distribution, aka jellyfish
steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
fetch-depth: 1 # only needed to get the sha label
# Don't attempt to cache Docker. Sensitive information can be stolen
# via forks, and login session ends up in ~/.docker. This is ok because
# we publish DOCKER_PARENT_IMAGE to ghcr.io, hence local to the runner.
- name: Docker Push
run: | # GITHUB_REF will be refs/tags/docker-MAJOR.MINOR.PATCH
run: | # GITHUB_REF = refs/tags/docker-MAJOR.MINOR.PATCH
build-bin/git/login_git &&
build-bin/docker/configure_docker_push &&
build-bin/docker_push $(echo ${GITHUB_REF} | cut -d/ -f 3)
env:
# GH_USER=<user that created GH_TOKEN>
GH_USER: ${{ secrets.GH_USER }}
# GH_TOKEN=<hex token value>
# - pushes Docker images to ghcr.io
# - create via https://github.com/settings/tokens
# - needs repo:status, public_repo, write:packages, delete:packages
# * pushes Docker images to ghcr.io
# * create via https://github.com/settings/tokens
# * needs repo:status, public_repo, write:packages, delete:packages
GH_TOKEN: ${{ secrets.GH_TOKEN }}
# DOCKERHUB_USER=<typically dockerzipkindeployer>
# * only push repos in openzipkin org to Docker Hub on release
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
# DOCKERHUB_TOKEN=<access token for DOCKERHUB_USER>
# * Access Token from here https://hub.docker.com/settings/security
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
66 changes: 24 additions & 42 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,59 +1,40 @@
# yamllint --format github .github/workflows/test.yml
---
name: test

# We don't test documentation-only commits.
on:
# We run tests on non-tagged pushes to main that aren't a commit made by the release plugin
push:
tags: ''
branches: main
paths-ignore: '**/*.md'
# We also run tests on pull requests targeted at the main branch.
pull_request:
branches: main
paths-ignore: '**/*.md'
on: # yamllint disable-line rule:truthy
push: # non-tagged pushes to main
branches:
- main
tags-ignore:
- '*'
paths-ignore:
- '**/*.md'
- './build-bin/*lint'
- ./build-bin/mlc_config.json
pull_request: # pull requests targeted at the main branch.
branches:
- main
paths-ignore:
- '**/*.md'
- './build-bin/*lint'
- ./build-bin/mlc_config.json

jobs:
test-javadoc:
name: Test JavaDoc Builds
runs-on: ubuntu-22.04 # newest available distribution, aka jellyfish
if: "!contains(github.event.head_commit.message, 'maven-release-plugin')"
steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
fetch-depth: 0 # full git history for license check
- name: Setup java
uses: actions/setup-java@v4
with:
distribution: 'zulu' # zulu as it supports a wide version range
java-version: '11' # earliest LTS and last that can compile the 1.6 release profile.
- name: Cache local Maven repository
uses: actions/cache@v3
with:
path: ~/.m2/repository
key: ${{ runner.os }}-jdk-11-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-jdk-11-maven-
- name: Build JavaDoc
run: ./mvnw clean javadoc:aggregate -Prelease

test:
name: test (JDK ${{ matrix.java_version }})
runs-on: ubuntu-22.04 # newest available distribution, aka jellyfish
runs-on: ubuntu-22.04 # newest available distribution, aka jellyfish
if: "!contains(github.event.head_commit.message, 'maven-release-plugin')"
strategy:
fail-fast: false # don't fail fast as sometimes failures are operating system specific
matrix: # use latest available versions and be consistent on all workflows!
fail-fast: false # don't fail fast as some failures are LTS specific
matrix: # match with maven-enforcer-plugin rules in pom.xml
include:
- java_version: 11 # Last that can compile zipkin core to 1.6 for zipkin-reporter
maven_args: -Prelease -Dgpg.skip -Dmaven.javadoc.skip=true
- java_version: 17 # earliest LTS supported by Spring Boot 3
maven_args: -Prelease -Dgpg.skip
- java_version: 21 # Most recent LTS
steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
fetch-depth: 0 # full git history for license check
- name: Setup java
uses: actions/setup-java@v4
with:
Expand All @@ -63,10 +44,11 @@ jobs:
uses: actions/cache@v3
with:
path: ~/.m2/repository
# yamllint disable-line rule:line-length
key: ${{ runner.os }}-jdk-${{ matrix.java_version }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-jdk-${{ matrix.java_version }}-maven-
# Don't attempt to cache Docker. Sensitive information can be stolen
# via forks, and login session ends up in ~/.docker. This is ok because
# we publish DOCKER_PARENT_IMAGE to ghcr.io, hence local to the runner.
- name: Test
run: build-bin/configure_test && build-bin/test
run: build-bin/configure_test && build-bin/test ${{ matrix.maven_args }}
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,17 @@ persists them to a configured collector component.
|------------------------------------|-----------------------------------------------------------------------------------------|
| [collector-http](./collector-http) | Implements the [OTLP/HTTP protocol](https://opentelemetry.io/docs/specs/otlp/#otlphttp) |

### Encoders

Encoding is library-specific, as some libraries use `zipkin2.Span` and others
`brave.handler.MutableSpan`. Both options are available to encode to the
OTLP format.

| Encoder | Description |
|-------------------------------------------|------------------------------------------------|
| [`OtelEncoder.V1`](./encoder-otel-zipkin) | zipkin-reporter `AsyncReporter<Span>` |
| [`OtelEncoder`](./encoder-otel-brave) | zipkin-reporter-brave `AsyncZipkinSpanHandler` |

## Server integration

If you cannot use our [Docker image](./docker/README.md), you can still integrate
Expand Down
22 changes: 18 additions & 4 deletions collector-http/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@
<artifactId>zipkin-collector</artifactId>
<version>${zipkin.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>zipkin-encoder-otel</artifactId>
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>${armeria.groupId}</groupId>
Expand All @@ -52,11 +57,20 @@
<version>${zipkin.version}</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>${armeria.groupId}</groupId>
<artifactId>armeria-junit5</artifactId>
<version>${armeria.version}</version>
<groupId>org.awaitility</groupId>
<artifactId>awaitility</artifactId>
<version>${awaitility.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-otlp</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
Expand Down
Loading
Loading