Comments (6)
@tsloughter I wonder if there's two issues here that are related. One that I noticed was creating many entries in persistent_term
on boot, but devoid of crashes. I think what you're saying can excacerbate the issue ofc.
I wonder if persistent_term
in the end is a good fit here, without more understanding of opentelemetry it's hard for me to say, but I think regardless, it may be advantageous to consider not using persistent_term
. All the configuration I see in it right now may be fine, but only if it can all be stuffed into one big term and what's more never updated.
If that's not the case then we may want to consider using ets instead, the reads would be slower, but the trade off would be worth it. Specifically, we could make an ets table directly in a supervisor that is public, such that it has most of the same properties of persistent_term, but not effecting the systems use of persistent_term as a whole.
The only rub there would be if telemetry app crashes and/or the supervisor bounces.
from opentelemetry-erlang.
There should only be like 4 terms at boot. But then there is 1 term per-Tracer that gets used. A Tracer by default is per-Application. So 1 term per Application that has tracing code in it. So technically the day a project has 100s of instrumented Applications that may be a problem. But that'll be a good day :).
I'd need to see what you seeing to know if maybe its another bug that is creating more terms than expected. If so we can add a test to in the future make sure that on startup nothing new starts getting added.
from opentelemetry-erlang.
And term is used because tracing needs to have as close to 0 overhead as possible. OTP logger does the same.
from opentelemetry-erlang.
There should only be like 4 terms at boot. But then there is 1 term per-Tracer that gets used. A Tracer by default is per-Application. So 1 term per Application that has tracing code in it. So technically the day a project has 100s of instrumented Applications that may be a problem. But that'll be a good day :).
I'd need to see what you seeing to know if maybe its another bug that is creating more terms than expected. If so we can add a test to in the future make sure that on startup nothing new starts getting added.
I think you stated it perfectly, yet yeah, I might be seeing something abnormal too. In a nutshell I see keys for lots of modules, not just the application names.
As an example and locally in dev mode, I'll see {:opentelemetry, Credo.Check.Refactor.WithClauses}
, without debugging myself, but by looking at the code this is a result of opentelemetry:registry_application_tracer/1
, which is being called by opentelemetry_app
on boot (i.e., it will enumerate all loaded applications and call register_application_tracer/1
for all apps). This is configurable, so that's good.
I'm all for using persistent_term :) Just have to figure out how how not create so many terms.
from opentelemetry-erlang.
Oooh, you are on an old version aren't you?
from opentelemetry-erlang.
Oooh, you are on an old version aren't you?
Indeed. So, what I described, the second part, seems like a non-issue now.
Edit:
Ok, brought myself up to date. For posterity, otel no longer does what I described as a default, instead, if the sdk is enabled (config), loaded applications are inserted into persistent_term, but not an entry for each module. Adding modules is now done lazily in a with_span
. Now what you are saying about how it does this like logger makes sense 😄
I wonder if there's a better way to do this still, but it is off topic.
from opentelemetry-erlang.
Related Issues (20)
- Support for Metric Exemplars
- Add new add_link function
- Rename view_aggregation to stream HOT 1
- service.instance.id is sent as an integer, not as a string in protobuf messages HOT 1
- logging a keyword list is incorrectly detected as a string HOT 1
- Injecting and extracting traceparent for distributed tracing HOT 8
- Update exemplar random seleection
- Metric Filters
- Metric cardinality limits
- Metric sync gauge
- Metrics: Investigate using metrics key with bucket index for histograms
- Exponential histogram
- Use a sorted list for attributes in metrics HOT 1
- Wrong seen count in exemplars test HOT 1
- broken typespec for otel_span:start_opts HOT 1
- Histogram metric with cumulative temporality HOT 19
- Histogram transient test failure
- Flapping exemplars test failure
- Codecov always fail
- breaking changes in recent version HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opentelemetry-erlang.