First will there be an official process to make appeals against the projects design decisions? I am worried that collectively open-telemetry is implementing the former projects designs under what I believe to be a false premise it can "always change things later". I feel such changes will be impossible once the library is in wide spread use.
I have a number of concerns with the actual implementation here but it will be a large effort for me to enumerate them. For now I would like to start with some higher level issues that would resolve many of my code related concerns.
Exporters should not exist in this project (fundamental issue with OpenTelemetry as a whole)
If I could choose a single issue to focus on this would be it, as I feel it creates most of the design issues I have to begin with. This is an issue with the greater OpenTelemetry project as a hole. As far as "openness" OpenTelemetry is a step backwards from OpenTracing. It's placing OpenTelemetry as a gatekeeper of implementations, in fact it defines no formal specifications at all which I reported an issue about open-telemetry/opentelemetry-specification#20 and discuss in this gist. Instead of regurgitating these points in detail here I would ask anyone interested to please read this conversation, to summarize:
OpenTracing - Each SDK provides an interface in opentracing- which trace providers may implement in their own repositories to ship traces, for example
For tracing to be useful you must have participation from many teams across your organization. It's not uncommon for a request to span many languages (java, C#, Go, Python, Nodejs, ...) at a medium sized company. Thus if you want to provide a new tracer and integrate well with an existing ecosystem you must commit the dev cycles to implement all of these languages. There really is no reason to continue on this path with the "next generation" convergence efforts of "OpenTelemetry" that will leave a footprint on the ecosystem for years.
OpenTelemetry - Each SDK provides no interface as a "standard" design principle (not that I've seen anything resembling a standard design principle as I've echo'd here, in gitter, etc). This project seems to provide an ABI only, which is a step backwards from the API of OpenTracing. In addition it seems there is a drive to have the default OpenTelemetry SDK's all talk to a central "OpenTelemetry" agent which will be bundled with exporters further diminishing "openness". Push for an agent after your SDK's reduces the per-language burden we see above with OpenTracing but inserts OpenTelemetry as the gatekeeper of tracing implementations. At the very worst the "maintainers / governing body / process" could be captured by commerical interests to impede competition. At best it slowly grows to be a more arduous and painful process as layers of "exporters" increase bloat, complexity and reduce maintainability until it's no longer worth the efforts.
MyProposal - Stop writing code. Collectively define a structural specification that will slowly produce the initial version for a wire format for carrying trace events. Separate trace backends from the clients at the protocol level. Now you can resume writing code.
Trace vendors implement 1 server for every language
Open telemetry can provide reference implementations of clients that ship to these trace servers, which is what this repository would become. But this leaves room for those who want to implement their own clients because the chosen balance of ergonomics, performance, was not right for them. As it stands now you have a monumental effort beyond writing a client for a well defined wire-format. If the maintainers of this project disagree with my stance on plugins, great! I don't care. I can write my own client. As it stands I won't have that option without also implementing the glue for the tracer which defeats the purpose of investing resources in these convergence efforts. I really would like to urge the project owners to pump the brakes a little and consider the amount of effort that could spared by taking a different approach. Or at least provide some strong technical reasons to march on with this paradigm that expand beyond "this is what we had".
Testing Frameworks
Unit tests are not just for testing results, they also lock in your public API. Using a large unit tests abstraction that accepts interfaces everywhere means that type changes are possible without breaking unit tests. It also imposes additional cognitive load for potential contributors who are unfamiliar with the chosen libraries. In short they only provide a tool to circumvent engineering rigor, relieving authors of both the responsibility to think about struct composition and the benefits of "testing" the said structures to begin with. I strongly believe the projects quality would suffer substantially by using them.
Plugins
I am not sure the design considerations behind the scenes that led to requiring a plugin for tracing via "stdout" but I very strongly object. Plugins should be entirely out of the question as an API requirement.
- Plugins may not be unloaded
- Plugins provide a potential source for API compatibility mismatches, as plugins:
- can't be locked / versioned by a go.mod file
- can't have large interfaces or impose compile time errors
- insert any argument against ABI as an API here
- Plugins can negatively impact the API from evolving as suddenly ABI compatibility is a consideration
- Plugins add additional deployment complexity and opportunity for runtime failures
- Plugins do not provide nor enforce any code signing by default, adding potential attack vectors
I could continue on, but I feel this is a decent start for a technical debate. The most important argument against plugins is that they may still be provided for proprietary tracing solutions with careful design, you define the Tracer
interface and write a Tracer
which is backed by a plugin
. Knowing this, I think it's easy to prove there is no real reason to circumvent an API to provide an ABI.
Dependencies
First, I feel this project should depend on only the standard library, maybe a couple packages from golang.org/x if well justified. I strongly believe this should be a non-negotiable requirement because it will force a healthier design than what the current tracing libraries of today have. As it stands now this library depends on a massive (257) amount of packages go.sum and this project is just getting started. I understand that part of this is because there is a fundamental problem with the design of OpenTelemetry as a whole providing implementations rather than specifications, but there is still plenty of opportunity to improve here.