[RFC] Support XPU Backend With PTI-sdk in Kineto about kineto HOT 3 OPEN

Hi @xunsongh,

At Graphcore we are looking to make changes to Kineto to support an IPU profiling backend.

We want to add another activity - e.g. IPUActivityAPI but we don't want to compile time link this to Kineto/PyTorch. i.e. We would like to register IPUActivityAPI (or perhaps IPUActivityProfiler) at run-time with Kineto.
We had a few questions relating to this RFC:

1. We understand that this RFC is a plan to add an XPUActivityAPI within CuptiActivityProfiler - extending CuptiActivityProfiler to support the XPU with additional XPU specific members and paths. This will be an addition to the existing CUPTI and Roctracer code paths. On its own, this change will add XPU support, but does not provide a generic way to add new profiling backends as you described in #647. Is our understanding correct?
2. What was the main difficulty with #647 and was there any investigation into how this could be broken into smaller changes?
3. You state that "To my knowledge, Kineto is planning to separate CUPTI and Roctracer from the main profiler." Is there any more information about this to give regarding details and timeline? Will this introduce a system to register profiling backends?

Thanks in advance for taking the time to go over these questions and for any answers you can provide.

from kineto.

Hi @harryPorter828 ,
Thank you for your attention to this RFC and for taking the time to read and respond. However, after an online discussion in April, our XPU team has decided to follow the advice of the Kineto developers. We will adopt the form of a Kineto plugin, registering the XPU profiler as a relatively independent child profiler to the existing main profiler of Kineto. This approach avoids making changes to the main profiler of Kineto and does not involve directly pushing XPU-related integration code into the Kineto code repository. Instead, this code will be placed in PyTorch. Moreover, this plugin-based implementation is not directly related to what I previously proposed and what you mentioned in #647; they can even be considered two separate topics.

As developers on the XPU side, our primary goal is to enable PyTorch XPU users to utilize a profiler supported by Kineto without changing their usage habits, meaning with minimal changes to the front-end code. To achieve this, we are willing to first adapt to all existing designs and code logic of Kineto, reaching this goal in a way that is acceptable to all parties. In the future, we are open to participating in the structural reconstruction of Kineto, continuing to develop it into a more open, general, and user-friendly library. #647 was merely a conceptual idea towards this goal, and through extensive practice over this period, we have become aware of various issues with the original plan.

As for the current this RFC, it has been rejected by Meta's meeting, so we do not intend to discuss or explain it further. Once we complete and submit our design based on the Kineto plugin, I will close this RFC to avoid any misunderstandings.

Thank you.

from kineto.