The Pipeline example based on AXear-Pi (AX620A) , AXera-Pi Pro (AX650N) and AXera-Pi Zero (AX620Q) shows the software development skills of ISP, Image Processing, NPU, Encoding, and Display modules, which is helpful for users to develop their own multimedia applications.
License: BSD 3-Clause "New" or "Revised" License
正确的应该为：ax-pipeline/build$ tree install
error: ‘AX_VENC_RC_ATTR_T’ {aka ‘struct axVENC_RC_ATTR_T’} has no member named ‘stFrameRate’; did you mean ‘uFrameRate’?
303 | stVencChnAttr.stRcAttr.stFrameRate.fSrcFrameRate = (config.nSrcFrameRate); /* input frame rate /
| ^~~~~~~~~~~
| uFrameRate
/root/ax-pipeline/examples/common/common_pipeline/ax650/common_pipeline_venc.cpp:304:28 error: ‘AX_VENC_RC_ATTR_T’ {aka ‘struct axVENC_RC_ATTR_T’} has no member named ‘stFrameRate’; did you mean ‘uFrameRate’?
304 | stVencChnAttr.stRcAttr.stFrameRate.fDstFrameRate = (config.nDstFrameRate); / target frame rate */
| ^~~~~~~~~~~
| uFrameRate
这里
example中的案例，全部都将gModels句柄暴露出来了，是否可以考虑封装性更好，内部将其隐藏起来，无需上层调用者主动去持有该句柄
need ax-pipeline/examples/libaxdl/src/ax_joint_runner_ax630.cpp
官方编译之后得到下面的：
ax-pipeline/build$ tree install
install
├── bin
│ ├── config
│ │ ├── dinov2.json
│ │ ├── scrfd.json
│ │ ├── yolov5_seg.json
│ │ ├── yolov5s_650.json
│ │ ├── yolov5s_face.json
│ │ ├── yolov6.json
│ │ ├── yolov7.json
│ │ ├── yolov7_face.json
│ │ ├── yolov8_pose.json
│ │ └── yolox.json
│ ├── sample_demux_ivps_joint_hdmi_vo
│ ├── sample_demux_ivps_joint_rtsp
│ ├── sample_demux_ivps_joint_rtsp_hdmi_vo
│ ├── sample_multi_demux_ivps_joint_hdmi_vo
│ ├── sample_multi_demux_ivps_joint_multi_rtsp
│ └── sample_multi_demux_ivps_joint_multi_rtsp_hdmi_vo\
而我编译之后得到如下：
tree install
install
├── bin
│ ├── config
│ │ ├── clip.json
│ │ ├── crowdcount.json
│ │ ├── custom_model.json
│ │ ├── depth_anything.json
│ │ ├── dinov2_depth.json
│ │ ├── dinov2.json
│ │ ├── glpdepth.json
│ │ ├── owlvit.json
│ │ ├── ppyoloe.json
│ │ ├── scrfd.json
│ │ ├── scrfd_recognition.json
│ │ ├── yolo_nas.json
│ │ ├── yolov5_seg.json
│ │ ├── yolov5s_face.json
│ │ ├── yolov5s_face_recognition.json
│ │ ├── yolov5s.json
│ │ ├── yolov6.json
│ │ ├── yolov7_face.json
│ │ ├── yolov7.json
│ │ ├── yolov8.json
│ │ ├── yolov8_pose.json
│ │ ├── yolov8_seg.json
│ │ └── yolox.json
│ ├── sample_demux_ivps_npu_hdmi_vo
│ ├── sample_demux_ivps_npu_rtsp
│ ├── sample_demux_ivps_npu_rtsp_hdmi_vo
│ ├── sample_multi_demux_ivps_npu_hdmi_vo
│ ├── sample_multi_demux_ivps_npu_multi_rtsp
│ ├── sample_multi_demux_ivps_npu_multi_rtsp_hdmi_vo
│ ├── sample_vin_ivps_npu_hdmi_vo
│ └── sample_vin_ivps_npu_venc_rtsp
├── include
│ ├── ByteTrack
│ │ └── bytetrack.h
│ ├── libRtspServer
│ │ └── RtspServerWarpper.h
│ ├── Mp4Demuxer
│ │ └── Mp4Demuxer.h
│ └── rtspclisvr
│ ├── AC3RTPSource.h
│ ├── Base64.hh
│ ├── BasicHashTable.hh
│ ├── BitVector.hh
│ ├── ClientSocket.h
│ ├── DigestAuthentication.hh
│ ├── Event.h
│ ├── H264RTPSource.h
│ ├── H265RTPSource.h
│ ├── HashTable.hh
│ ├── JPEGRTPSource.h
│ ├── LiveServerMediaSession.h
│ ├── MediaSession.h
│ ├── MPEG4ESRTPSource.h
│ ├── MPEG4GenericRTPSource.h
│ ├── Mutex.h
│ ├── MyList.h
│ ├── MySemaphore.h
│ ├── MySock.h
│ ├── NetAddress.h
│ ├── NetCommon.h
│ ├── OnDemandServerMediaSession.h
│ ├── our_md5.h
│ ├── OutPacketBuffer.h
│ ├── rtcp_from_spec.h
│ ├── RTCP.h
│ ├── RTCPInstance.h
│ ├── RTPPacketBuffer.h
│ ├── RTPSource.h
│ ├── RTSPClient.h
│ ├── RTSPCommonEnv.h
│ ├── RTSPCommon.h
│ ├── RTSPServer.h
│ ├── ServerMediaSession.h
│ ├── SockCommon.h
│ ├── TaskScheduler.h
│ ├── Thread.h
│ └── util.h
└── lib
├── libaxdl.a
├── libByteTrack.a
├── libcommon_pipeline.a
├── libMp4Demuxer.a
├── librtspclisvr.a
└── libRtspServer.a
为什么不一样呢？
We tested MaixPy in a project, using the ax-pipeline built into a binary file named lib_sample_vin_ivps_joint_venc_rtsp_sipy.so
We need a feature to enable night vision or black-and-white mode at night using IR and stream it via RTSP.
Could you suggest additional features or an easier way to develop this functionality?
硬件环境
ax-pi软件版本
ax-pipeline commit c183e85353ce84577679723a8eeefc914bdba506执行脚本
./bin/sample_vin_ivps_joint_vo -m resource/model/yolov5s_sub_nv12_11.joint -p resource/config/yolov5s.json -c 2错误log
[N][ main][ 266]: sample begin
[N][ main][ 320]: eSysCase=2,eHdrMode=1
2560 1440
[AX_SYS_LOG] AX_SYS_Log2ConsoleThread_Start
AX_POOL_SetConfig success!
[COMMON_SYS_Init]-85: AX_POOL_Init success!
[INFO]: Virtual npu mode is 1_1
Tools version: 0.6.1.16
141d3e2
[I][ sample_run_joint_init][ 231]: NV12 MODEL
[N][ main][ 485]: load model resource/model/yolov5s_sub_nv12_11.joint success!
[COMMON_ISP_GetI2cDevNode]-448: get board_id = 2
[RegisterSns]-556: set sensor bus idx 0
gc4653_sensor_set_mode
[sample_isp][COMMON_CAM_Open][167] pFile
[COMMON_CAM_Open]-170: AX_ISP_LoadBinParams will user sensor.h
Cannot open /sys/class/gpio/gpio496/direction.
Cannot open /sys/class/gpio/gpio496/direction.
i2c_init: i2c device is /dev/i2c-0
line_period 26.000000us
ISP IFE INIT done.
ISP ITP INIT done.
[N][ main][ 512]: camera 0 is open
AX_IVPS_Init
[N][ StartOverlay][ 52]: AX_IVPS_RGN_AttachToFilter(Grp: 0, Filter: 0x11) successfully, hChnRgn=0.
[ParseVoPubAttr-103] dsi0 output
[ParseVoPubAttr-144] reso: 480x854-60
g_vo_log_target = 0, g_vo_log_level = 0, g_perf_trace_enable = 0
[VoInit-202] u32LayerNr = 1
[VoInit-226] layer0 u32Width = 480, u32Height = 854
[SAMPLE_VO_CREATE_POOL-167] u32BlkCnt = 3, u64BlkSize = 0x961e0, pPoolID = 7
[VoInit-237] layer pool id = 0x7
[SAMPLE_COMM_VO_StartDev-37] failed with 0x80038a80!
[SAMPLE_COMM_VO_StartVO-305] SAMPLE_COMM_VO_StartDev failed!
[VoInit-243] SAMPLE_COMM_VO_StartVO failed, s32Ret = 0x80038a80
check vo_dev->state != VO_STS_INIT && vo_dev->state != VO_STS_DISABLED failed
[VoInit-285] done, s32Ret = 0x80038a80
[E][ main][ 534]: VoInit failed,s32Ret:0x80038a80
ISP IFE DEINIT done.
ISP ITP DEINIT done.
[COMMON_CAM_Close]-269: COMMON_CAM_Close: pipe 0: exit.
[AX_SYS_LOG] Waiting thread(2990784992) to exit
[AX_SYS_LOG] AX_Log2ConsoleRoutine terminated!!!
exit[AX_SYS_LOG] join thread(2990784992) ret:0
[N][ main][ 576]: sample end
In file included from /root/Desktop/ax-pipeline/examples/common/common_pipeline/ax650/common_pipeline_vo.cpp:30:
/root/Desktop/ax-pipeline/build/../bsp/msp/out/../sample/common/common_vo.h:130:5: error: ‘AX_VO_MODE_E’ does not name a type; did you mean ‘AX_VO_MODE_MAX’?
130 | AX_VO_MODE_E enMode;
| ^~~~~~~~~~~~
| AX_VO_MODE_MAX
/root/Desktop/ax-pipeline/examples/common/common_pipeline/ax650/common_pipeline_vo.cpp: In function ‘AX_BOOL SetChnFrameRate(VO_CHN, VO_LAYER, VO_DEV, AX_U32)’:
/root/Desktop/ax-pipeline/examples/common/common_pipeline/ax650/common_pipeline_vo.cpp:296:56: error: could not convert ‘nFps’ from ‘AX_U32’ {aka ‘unsigned int’} to ‘AX_FRAME_RATE_CTRL_U’ {aka ‘AX_FRAME_RATE_CTRL_U’}
296 | AX_S32 ret = AX_VO_SetChnFrameRate(voLayer, voChn, nFps);
| ^~~~
| |
| AX_U32 {aka unsigned int}
In file included from /root/Desktop/ax-pipeline/examples/common/common_pipeline/ax650/common_pipeline_vo.cpp:22:
/root/Desktop/ax-pipeline/examples/common/common_pipeline/ax650/common_pipeline_vo.cpp: In function ‘int _create_vo_hdmi(pipeline_t*)’:
/root/Desktop/ax-pipeline/examples/common/common_pipeline/./../../utilities/sample_log.h:34:21: warning: format ‘%d’ expects argument of type ‘int’, but argument 11 has type ‘std::vector<axVO_CHN_ATTR_T>::size_type’ {aka ‘long unsigned int’} [-Wformat=]
34 | #define MACRO_GREEN "\033[1;30;32m"
| ^
/root/Desktop/ax-pipeline/examples/common/common_pipeline/./../../utilities/sample_log.h:57:12: note: in expansion of macro ‘MACRO_GREEN’
57 | printf(MACRO_GREEN "[I][%32s][%4d]: " fmt MACRO_END "\n", FUNCTION, LINE, ##VA_ARGS)
| ^~~~~~~~~~~
/root/Desktop/ax-pipeline/examples/common/common_pipeline/ax650/common_pipeline_vo.cpp:428:13: note: in expansion of macro ‘ALOGI’
428 | ALOGI("layer %d: [(%d, %d) %dx%d], dispatch mode %d, layer depth %d, ChnNr %d, part mode %d, tolerance %d",
| ^~~~~
/root/Desktop/ax-pipeline/examples/common/common_pipeline/ax650/common_pipeline_vo.cpp:428:89: note: format string is defined here
428 | ALOGI("layer %d: [(%d, %d) %dx%d], dispatch mode %d, layer depth %d, ChnNr %d, part mode %d, tolerance %d",
| ~^
| |
| int
| %ld
make[2]: *** [examples/common/common_pipeline/CMakeFiles/common_pipeline.dir/build.make:146: examples/common/common_pipeline/CMakeFiles/common_pipeline.dir/ax650/common_pipeline_vo.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:520: examples/common/common_pipeline/CMakeFiles/common_pipeline.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
运行./sample_demux_ivps_npu_hdmi_vo -f /root/boxvideos/08.mp4 -p ./config/yolov8.json报错
_ivps_get_frame_thread][ 58]: IvpsGrp 1: AX_IVPS_GetChnFrame failed!s32Ret:0x800d0120
不知是什么原因？
硬件环境
ax-pi软件版本
ax-pipeline commit c183e85353ce84577679723a8eeefc914bdba506执行脚本
./bin/sample_vin_joint -m /home/models/yolov5s.joint -p resource/config/yolov5s.json -c 0错误log
2688 1520
[AX_SYS_LOG] AX_SYS_Log2ConsoleThread_Start
AX_POOL_SetConfig success!
[COMMON_SYS_Init]-85: AX_POOL_Init success!
[INFO]: Virtual npu mode is 1_1
Tools version: d696ee2f
[I][ sample_run_joint_init][ 241]: BGR MODEL
[N][ main][ 635]: load model /home/models/yolov5s.joint success!
[COMMON_ISP_GetI2cDevNode]-448: get board_id = 2
[RegisterSns]-556: set sensor bus idx 0
[sample_isp][COMMON_CAM_Open][167] pFile
[COMMON_CAM_Open]-170: AX_ISP_LoadBinParams will user sensor.h
Cannot open /sys/class/gpio/gpio496/direction.
Cannot open /sys/class/gpio/gpio496/direction.
i2c_init: i2c device is /dev/i2c-0
i2c_read: Failed to read reg: Remote I/O error.!
i2c_read: Failed to read reg: Remote I/O error.!
i2c_read: Failed to read reg: Remote I/O error.!
[ os04a10_get_chipid][ 105][ERROR]os04a10_get_chipid: Failed to read sensor os04a10 id
[ sensor_i2c_init][ 152][ERROR]can't find os04a10 sensor id ret=6.
i2c_write: Failed to write reg: Remote I/O error.!
i2c_write: Failed to write reg: Remote I/O error.!
i2c_write: Failed to write reg: Remote I/O error.!
...
...
i2c_read: Failed to read reg: Remote I/O error.!
ISP IFE INIT done.
ISP ITP INIT done.
i2c_write: Failed to write reg: Remote I/O error.!
[N][ main][ 660]: camera 0 is open
[N][ IspRun][ 206]: cam 0 is running...
[E][ getYuv][ 237]: return error! gCams[i].nPipeId = 0, retval = 0x80010127
[E][ getYuv][ 237]: return error! gCams[i].nPipeId = 0, retval = 0x80010127
[E][ getYuv][ 237]: return error! gCams[i].nPipeId = 0, retval = 0x80010127
[E][ getYuv][ 237]: return error! gCams[i].nPipeId = 0, retval = 0x80010127
^Ci2c_write: Failed to write reg: Remote I/O error.!
ISP IFE DEINIT done.
ISP ITP DEINIT done.
[COMMON_CAM_Close]-269: COMMON_CAM_Close: pipe 0: exit.
[AX_SYS_LOG] Waiting thread(2974753248) to exit
[AX_SYS_LOG] AX_Log2ConsoleRoutine terminated!!!
exit[AX_SYS_LOG] join thread(2974753248) ret:0其他现象
使用板子上自带的`/home/libmaix/examples/axpi_classification_cam` 可以正常跑．可以提供一下hrnet模型转换的配置吗，在AX620A芯片
Can you help me with running samples with yolov3 tiny?
Hi,
I already have an AXera-Pi kit, it's a very interesting piece of hardware.
I'm working on a device base on AXera device, but I need to add a new image sensor, what should be the way? How can I access the Linux kernel source? Where should I be asking this question, I think this is not the right channel.
BR
ax-pipeline-main/examples/libaxdl/src/ax_common_api.cpp:9:10: fatal error: npu_cv_kit/ ax_npu_imgproc.h: No such file or directory
9 | #include "npu_cv_kit/ax_npu_imgproc.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
在build文件夹内make -j4 也报错
Hi,
while building pipeline for AX650 using cmake -DAXERA_TARGET_CHIP=AX650 -DBSP_MSP_DIR="/home/sagarkhimani/Downloads/2.Pulsar/ax-pipeline/ax650n_bsp_sdk/msp/out" -DOpenCV_DIR="/home/sagarkhimani/Downloads/2.Pulsar/ax-pipeline/3rdparty/libopencv-4.5.5-aarch64/lib/cmake/opencv4" -DSIPY_BUILD=OFF -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE="/home/sagarkhimani/Downloads/2.Pulsar/ax-pipeline/toolchains/aarch64-none-linux-gnu.toolchain.cmake" -DCMAKE_INSTALL_PREFIX=install .. and make install, i'm getting below error.
sagarkhimani@sagarkhimani:~/Downloads/2.Pulsar/ax-pipeline/build$ make install
[ 23%] Built target rtspclisvr
[ 46%] Built target RtspServer
[ 47%] Built target rtsp_h264_file
[ 48%] Built target rtsp_pusher
[ 50%] Built target rtsp_server
[ 54%] Built target ByteTrack
[ 55%] Built target Mp4Demuxer
[ 59%] Built target common_pipeline
[ 68%] Built target axdl
[ 69%] Linking CXX executable sample_demux_ivps_npu_rtsp
/home/sagarkhimani/Downloads/2.Pulsar/ax-pipeline/3rdparty/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/bin/ld: cannot find -ldrm
collect2: error: ld returned 1 exit status
make[2]: *** [examples/sample_demux_ivps_npu_rtsp/CMakeFiles/sample_demux_ivps_npu_rtsp.dir/build.make:167: examples/sample_demux_ivps_npu_rtsp/sample_demux_ivps_npu_rtsp] Error 1
make[1]: *** [CMakeFiles/Makefile2:577: examples/sample_demux_ivps_npu_rtsp/CMakeFiles/sample_demux_ivps_npu_rtsp.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
该怎么转换后的yolov5模型，调用摄像头，检测模型中的某个类别，在屏幕上只画这个类别的框，保存检测图片
目前我是这样做，能只输出人信息，但是屏幕会画其它框。
import time
from ax import pipeline
pipeline.load([
'libsample_vin_ivps_joint_vo_sipy.so',
'-p', '/home/config/yolov5s.json',
'-c', '2',
])
while pipeline.work():
time.sleep(0.001)
tmp = pipeline.result()
if tmp and tmp['nObjSize']:
for i in tmp['mObjects']:
if i['label'] == 0:
print(i)
else:
pass
本地编译报错，找不到 common_venc.h
解决方法：
需要按 交叉编译 的方式，把 ax650n_bsp_sdk 加进来。
然后重新 cmake
cmake -DBSP_MSP_DIR=/soc/ -DAXERA_TARGET_CHIP=AX650 -DBSP_MSP_DIR=$PWD/../ax650n_bsp_sdk/msp/out -DSIPY_BUILD=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=install ..
sample_vin_ivps_npu_venc_rtsp 编译完成之后无法正常运行 joint，camera初始化失败
sample_vin_ivps_npu_venc_rtsp 编译完成之后可以正常运行
复现方法：某些RTSP服务端处理DESCRIBE请求的回复中没有"Content-Base"项且SDP信息里也没有a=control:
文件：
ax-pipeline/third-party/RTSP/src/RTSPClient/RTSP/RTSPClient.cpp
中的函数:
char const* RTSPClient::sessionURL(MediaSession const& session) const
返回值 url (fControlPath)和 fBaseURL 可能同时为NULL,后续会导致程序崩
建议在函数:
char* RTSPClient::describeURL(const char url, Authenticator authenticator, bool allowKasennaProtocol)
解析后增加对 fBaseURL 判空，如果为空时可将参数url给fBaseURL
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.