ArgumentError thrown from auto-instrumentation code when running OpenTelemetry in parallel
#3575
Comments
anandhasudhan
added
bug
|
@anandhasudhan Thanks for reporting this! Just want to let you know the team has seen this, and is starting to look at it. I'll tag in someone here to walk through this issue with you. |
|
@marcotc Thanks for looking into this. For additional context, I initialized 1_datadog.rb first and then 2_opentelemetry.rb. I tried changing the order to initialize 1_opentelemetry.rb first and then 2_datadog.rb. The opposite seems to happen. ie. open telemetry code ends up calling data dog code and throws the Argument Error Stack traceI came across this PR where it indicates that opentelemetry tracers have been built using the datadog auto-instrumentation library as a starting point and wonder if that could have any impact here. I've provided my Opentelemetry configuration below. OpenTelemetry configurationrequire 'opentelemetry/sdk'
require 'opentelemetry/instrumentation/all'
require 'opentelemetry-exporter-otlp'
OpenTelemetry::SDK.configure do |c|
c.service_name = 'basic-web'
c.use 'OpenTelemetry::Instrumentation::ActiveSupport'
c.use 'OpenTelemetry::Instrumentation::Rack'
c.use 'OpenTelemetry::Instrumentation::AwsSdk'
c.use 'OpenTelemetry::Instrumentation::HTTP'
c.use 'OpenTelemetry::Instrumentation::ConcurrentRuby'
c.use 'OpenTelemetry::Instrumentation::Dalli'
c.use 'OpenTelemetry::Instrumentation::Ethon'
c.use 'OpenTelemetry::Instrumentation::Faraday'
c.use 'OpenTelemetry::Instrumentation::HttpClient'
c.use 'OpenTelemetry::Instrumentation::Net::HTTP'
c.use 'OpenTelemetry::Instrumentation::Rake'
c.use 'OpenTelemetry::Instrumentation::Redis'
c.use 'OpenTelemetry::Instrumentation::RestClient'
c.use 'OpenTelemetry::Instrumentation::Sidekiq'
c.add_span_processor(
OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(
OpenTelemetry::Exporter::OTLP::Exporter.new(endpoint: "http://#{ENV['DD_AGENT_HOST']}:4318/v1/traces")
)
)
end |
|
Seeing this as well, following |
|
I'm taking a look into this issue next week. I'll report back. |
|
Ran into this as well. Subscribed. Thanks. |
|
Based on my investigation, I realized that it is not possible to have both data dog and opentelemetry auto-instrumentation code running in parallel. Please correct me if I'm wrong. I took a deep dive at the implementation of both datadog and opentelemetry auto-instrumentation code. Both overload the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Current behaviour
When running Datadog and OpenTelemetry in parallel, the auto-instrumentation code from datadog throws an
ArgumentErrorsince it ends up calling the opentelemetry library code. It appears that datadog instrumentation is clashing with the opentelemetry instrumentation. I briefly looked at the ddtrace code and it has this methodannotate_span_with_response!that takes 3 arguments and opentelemetry code has the same methodannotate_span_with_response!that takes 2 arguments.The
ArgumentErrorwe received was for this particular method where opentelemetry expected 2 arguments but received 3 (from the datadog instrumentation). Although, they have their own separate name spaces, I'm not sure why datadog called the opentelemetry library code.Expected behaviour
Datadog does not call the opentelemetry library code and continues instrumenting my application code without errors when opentelemetry is enabled.
Steps to reproduce
Run datadog and opentelemetry in parallel. The auto-instrumentation code from datadog ends up calling the opentelemetry library code resulting in
ArgumentError.Below is the complete stack trace for the
ArgumentErrorwhen running datadog and opentelemetry in parallel.How does
ddtracehelp you?Environment
Datadog.configure ...):The text was updated successfully, but these errors were encountered: