Consider to add OpenTelemetry support

[qiaoleiatms]

Summary

For a better monitoring on the running state of build service which leveraging buildpacks library, please consider add OpenTelemetry support on phases
Here's the golang version SDK of OpenTelemetry:
https://opentelemetry.io/docs/instrumentation/go/

---

Proposal

1. Need to know the success/fail for each phase.
2. For detect phase, need to know which BP pass the detection and which is not.
3. For build phase, besides the success/fail and also the time duration for each BP.
4. For export phase, besides the success/fail and also the time duration.
5. Telemetry should be disabled by default, and enabled by a "OPEN_TELEMTRY_ENABLED=true" (or whatever provided) flag
6. Telemetry endpoint should be available to be configured over another flag, such as OPEN_TELEMETRY_ENDPOINT

Context

We're running build service to enable multi-language build support in multi-tenant environment. Sometimes part of the users may encounter failures when a BP is not able to download the dependencies due to network connection issue. While sometime, the overall build is extreme slow.

For a better understanding on the service availability and detailed statistics of BP, please consider add OpenTelemetry support

[joshwlewis]

@qiaoleiatms thanks for bringing this up. I have been in talks about something like this with the buildpacks team myself. I presume you'd want OpenTelemetry tracing (rather than metrics or logs)?

[qiaoleiatms]

Hi @joshwlewis thanks for checking this. To me, the priority is:

1. metrics
2. tracing
3. logs

With the metrics, we're able to have a quick overview on the service availability

[natalieparellano]

@qiaoleiatms @joshwlewis thanks for starting this conversation. I think the ideal next step would be to start an RFC so we can formalize how this would be integrated within the lifecycle, pack, and buildpacks. Would either of you be interested in starting this and pushing it forward?

Edit: we put this on the agenda for tomorrow's Working Group meeting which is at 2p UTC this week.

[jjbustamante]

Some related work from Paketo folks

[qiaoleiatms]

@natalieparellano already submitted a PR for RFC