Comments (5)
Hi, sorry for the late response. I saw the UnparentedContext
in your PR and this could definitely be an option for context management. But before going further to simply add that, I would like to know more about your exception on tracing under pubsub module.
Here are several options:
- Set context missing strategy to
LOR_ERROR
soSegmentNotFoundException
won't be thrown. This way you will lose tracing data that doesn't have a context. - Like what you did with the
UnparentedContext
. This way each unparented segment carries a different trace id so you won't be able to easily correlates tracing data sent from various thread. Maybe you are doing some extra work in addition to what you actually pasted in the PR comment? - We have an item on our backlog for supporting automatic context propagation from main thread to worker threads with
future.ThreadPoolExecutor
. The downside is that this only fits very limited use case and for some libraries likes3transfer
you have to go very deep to figure out what threading module it is using.
Please let me know in an ideal case how you expect your tracing data to be structured so I can provide further help.
from aws-xray-sdk-python.
late? that was really fast :)
- Ya that's an existing option, however we lose these traces like you stated
- That's what I'm trying now, we'll see how messy it gets :) Basically the google pubsub client library creates a bunch of background helpers which contact the network and I want to get some visibility into what they're doing. Unfortunately when they run I have no active "root" thread so they're completely isolated without any hierarchy. In my scenario I'm using the apscheduler, and while my callback is inactive these threads are firing off network calls.
- That would be nice but there are cases where the workers do work w/o the thread which owns the pool having an active segment (like my case).
I think it's worth discussing what the ideal solution would be, I have no firm proposal.
from aws-xray-sdk-python.
Hi,
Please let me know if solution2 works for you. In that case I would imagine your service graph is split into two parts. One represents the request flow starts from your main thread and the other one represents the background work. All background work subsegments will be aggregated to a single node with name like unparented segment
and you still see errors, outbound calls etc if applicable.
You can also do some more complicated logic on how to create the unparented segment so you categorize the workers into different logical nodes on the service graph to fit your need.
Please let me know your findings and I can see a general enough unparented context
being added to the SDK built-in context.
from aws-xray-sdk-python.
ya solution 2 seems to be working for me. Right now I have them all in one unparented bucket, but I could imagine some people may want them categorized by thread or perhaps pool (which could include a name for more context). However I decided not to do that because the threads may be one-offs.
from aws-xray-sdk-python.
I'm closing this issue now. Will put the UnparentedContext
as a backlog and prioritize it accordingly should more customers request this specific use case.
from aws-xray-sdk-python.
Related Issues (20)
- aws-xray-sdk 2.11.0 removes causes from exceptions HOT 1
- ERROR: cannot find the current segment/subsegment when segment is open and uploading file to s3. HOT 3
- Installing SDK without botocore and other transitive dependencies HOT 4
- Custom emitter based on boto3 creates an infinite loop in the SDK HOT 3
- Bug: nested subsegments don't work across threads HOT 4
- IndexError when using AWS X-Ray SDK with SQLAlchemy HOT 1
- EKSPlugin HOT 1
- Very rare ReferenceError HOT 2
- Support for psycopg3 HOT 2
- Link a Textract async operation with downstream process HOT 1
- Current tox versions do not like how testenv.passenv is set
- aws_xray_sdk.core.exceptions.exceptions.SegmentNameMissingException: Segment name is required. HOT 1
- Using psycopg2 connection_factory throws exceptions HOT 4
- Documentation for patch_all HOT 2
- Sampling configuration should discuss DefaultSampler vs. LocalSampler HOT 1
- Flask middleware errors when an earlier Flask extension throws an exception in a before_request method HOT 1
- Patched DB cursor and template may record outside of XRayMiddleware HOT 2
- Segment not propgated when subsegment metadata is sufficiently long HOT 4
- sqlalchemy_core patch errors for unencoded special characters in db url HOT 1
- Error when using AsyncContext in python 3.11.5 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aws-xray-sdk-python.