Comments (3)
你好有没有关于此项目的微信交流群
您好,谢谢关注此项目。
抱歉6月4号下午7点左右才看到。
目前没有微信交流群(未来应该也不会有)。
对项目感兴趣的开发人员,可以提交issue report或者code PR.目前有两个重点方向:
1.通过开源社区协同的方式,完全实现ggml推理框架的Qualcomm backend,有如下几个问题需要解决:
- 量化数据的处理,只需要处理add,mul,mulmat。参考资料:explanation of k-quants, ggerganov/llama.cpp#1684
- NPU backend的完整实现(先略过GPU backend),目前NPU后端的性能太差
2.改进ggml推理框架的推理性能:现在的ggml backend subsystem有改进的空间,尤其不少backend实际上是单线程执行推理。
如果没有疑问,此issue将会被关闭。如您有需要,未来也可以再打开。谢谢。
from kantv.
大牛,你接下来准备做onnxruntime+qnn的后端支持?
from kantv.
大牛,你接下来准备做onnxruntime+qnn的后端支持?
1.😂 AI初学者
2.onnxruntime的QNN后端貌似已经有了:https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/core/providers/qnn 最近看了一眼,个人感觉封装的太多了,对个人而言没啥太大参考价值.我在做ggml的QNN后端时,最开始得到了高通一位工程师的指点,直接使用最原始的QNN API。后面基本是用逆向工程的方法做实验一步一步试验出来的,主要参考了如下资料:
(1) https://github.com/pytorch/executorch/tree/main/backends/qualcomm (provided by Qualcomm Technologies, Inc.)
(2) QNN samples (provided by Qualcomm Technologies, Inc.)
(3) /opt/qcom/aistack/qnn/2.20.0.240223/examples/Models/InceptionV3/model/Inception_v3.cpp which is
generated automatically by Qualcomm's dedicated tool and it contains more then 20,000 lines C++ code
from kantv.
Related Issues (20)
- Build succeed but not run on iOS HOT 1
- Build failed HOT 1
- how to complete the two-step authentication(2FA)? HOT 5
- PoC:clean-room implementation of real-time AI subtitle for English online-TV(OTT TV) HOT 20
- snapshots HOT 42
- integrate gstreamer to KanTV HOT 1
- Enhance English language support in UI HOT 2
- PoC:How to integrate proprietary/open-source codes to KanTV for personal/proprietary/commercial device-side/edge AI R&D activity HOT 1
- bug in UI HOT 2
- [whisper.cpp] sometimes some repeated sentences were shown in real-time English subtitle for English online-tv
- need to build the apk for "armeabi-v7a", there is error: no find libkantv-core.so HOT 3
- [llama.cpp] AI answer does not stop automatically when inference is launched on Android phone HOT 1
- PoC: Add Qualcomm mobile SoC native backend for GGML HOT 25
- How to use ggml_qnn.cpp and ggml_qnn.h in llamp.cpp? HOT 4
- Graphic: 2D graphic benchmark does not work properly on Xiaomi 14
- MNIST: inference result is not correct HOT 1
- How to implement a single whispercpp with qnn on Qualcomm SoC? HOT 2
- Kantv NPU
- 高通8Gen3设备上使用QNN的HTP加速效果不理想
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kantv.