Comments (5)
That's a fair point that data for events does come in with nano second granularity for start and end timestamps.
AFaIK the main constraint is Chrome Trace format. We use these 'X' Complete events to represent the kernels and operators. They unfortunately specify the timing in resolution of microseconds only
There is an extra parameter dur to specify the tracing clock duration of complete events in microseconds. All other parameters are the same as in duration events.
And all timestamsp are in microseconds too
ts: The tracing clock timestamp of the event. The timestamps are provided at microsecond granularity.
https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview
from kineto.
That's a fair point that data for events does come in with nano second granularity for start and end timestamps.
AFaIK the main constraint is Chrome Trace format. We use these 'X' Complete events to represent the kernels and operators. They unfortunately specify the timing in resolution of microseconds only
There is an extra parameter dur to specify the tracing clock duration of complete events in microseconds. All other parameters are the same as in duration events.
And all timestamsp are in microseconds too
ts: The tracing clock timestamp of the event. The timestamps are provided at microsecond granularity.
https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview
Yes, ts and dur is in microseconds granularity. But they can be floating point type as far as i know.
So if there are no other constraints, I think it would be necessary to perform the conversion from nanoseconds(int64) to microseconds(double) only when saving to a Chrome trace file. Using floating-point types for 'ts' and 'dur' would prevent the loss of time in the order of nanoseconds or even hundreds of nanoseconds per event.
Additionally, it need to use relative timestamps for 'ts'.Due to the timestamps in nanoseconds require 19 digits. But double can typically represent around 15-16 significant digits of precision.
from kineto.
But they can be floating point type as far as i know.
Oh I didn't know that. We can try that actually.
Additionally, it need to use relative timestamps for 'ts'.Due to the timestamps in nanoseconds require 19 digits. But double can typically represent around 15-16 significant digits of precision.
How about just using a fixed point representation with 3 digits after the decimal point?
from kineto.
How about just using a fixed point representation with 3 digits after the decimal point?
Sorry. I don't fully understand that.Using what filed to save these 3 digits?
If the 'ts' in nanosecond is 1706495062881409718. It was like this before the modification.
{
"ph": "X", "cat": "kernel", "name": "kernel_name", "pid": 0, "tid": 1,
"ts": 1706495062881409, "dur": 36,
"args": {
...
}
}
What should it be like after the modification?
from kineto.
Resolved in this PR: pytorch/pytorch#123650
from kineto.
Related Issues (20)
- Question about how to run "make test" correctly? HOT 2
- CUPTI symbols are undefined after libkineto build HOT 1
- The trace json file in the "Input Dims" field of aten::conv2d only has input shape and kernel shape
- TB_Plugin_CI failing with AttributeError: module 'mpmath' has no attribute 'rational'
- [Plugin-Bug]The Operators of baseline-run and exp-run are showed in a misaligned order HOT 1
- How to add customized metadata with on demand profiling ? HOT 7
- [RFC] Support XPU Backend With PTI-sdk in Kineto HOT 3
- [Discussion] Which clock should we be using for timestamps? HOT 2
- GPU traces fail when using PyTorch lightning due to square braces in traceName HOT 2
- Support memory profiling feature from on-demand path
- Roctracer crashes when number of samples too high
- TypeError: bad operand type for unary -: 'NoneType' HOT 4
- [Synchronization events] Missing StreamWait event in cases
- KeyError: <torch_tb_profiler.profiler.node.OperatorNode object at 0x7f4a45dc3e80> HOT 1
- Module View cannot show device time HOT 5
- CUDA time difference between print function and Profiler TensorBoard
- 【Feature Request】Add Process Status Check Before Profiling to Handle Non-Running Training Tasks
- Upgrade to CUDA 12.4 is causing segfaults in 4 Range Profiler Tests
- [BUG] Number of communication kernels don't match between workers in run
- Train process is blocked when kineto is processing traceEvents HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kineto.