linaro / uadk Goto Github PK
View Code? Open in Web Editor NEWUADK (User space Accelerator Development Kit), is a user space framework for using accelerators. Active branch is 'master'.
License: Apache License 2.0
UADK (User space Accelerator Development Kit), is a user space framework for using accelerators. Active branch is 'master'.
License: Apache License 2.0
User Space Accelerator Development Kit ====================================== UADK is a framework for user application to access hardware accelerator in a unified, secure, and efficient way. UADK is comprised of UACCE, libwd and many other algorithm libraries for different applications. UADK contains the following elements: wd.c UADK fundamental library which wraps the basic operations to the UACCE device. libwd is this library. wd_[alg].c UADK algorithm libraries. libwd_comp is for compression/decompression, libwd_crypto is for all encryption/decryption and hash algorithm. wd_utils.[ch] Some utility functions used by UADK and its drivers. drv/* Hardware drivers. It helps to fulfill the semantic of algorithm libraries for particular hardware. test/* Test applications to use UADK. include/* Head files for user APP and hardware drivers. docs/* Documentations. ====================================== The information of deploying UADK is in INSTALL file. The design document is in docs/wd_design.md file. The rules of library version are in docs/maintenance.md file.
GCC 12 emits this error:
wd.c: In function 'wd_is_isolate':
wd.c:193:21: error: the comparison will always evaluate as 'true' for the address of 'dev_root' will never be NULL [-Werror=address]
193 | if (!dev || !dev->dev_root)
| ^
In file included from wd.c:21:
./include/wd.h:131:14: note: 'dev_root' declared here
131 | char dev_root[PATH_STR_SIZE];
| ^~~~~~~~
dev_root being an initialized array, it cannot be NULL.
It should be fixed with this patch:
--- a/wd.c
+++ b/wd.c
@@ -190,7 +190,7 @@ int wd_is_isolate(struct uacce_dev *dev)
int value = 0;
int ret;
- if (!dev || !dev->dev_root)
+ if (!dev)
return -WD_EINVAL;
ret = access_attr(dev->dev_root, "isolate", F_OK);
1/test/hisi_sec_test_sgl/Makefile.am:9:test_hisi_sec_sgl_LDADD=../../../.libs/libwd.la $(with_openssl_dir)/libcrypto_wd.so
v1/test/hisi_sec_test_sgl/Makefile.am:11:test_hisi_sec_sgl_LDADD=../../../.libs/libwd.so $(with_openssl_dir)/libcrypto_wd.so
v1/test/hisi_hpre_test/Makefile.am:20:test_hisi_hpre_LDADD+=$(with_openssl_dir)/libcrypto_wd.so
v1/test/hisi_hpre_test/Makefile.am:21:hpre_test_tools_LDADD+=$(with_openssl_dir)/libcrypto_wd.so
v1/test/hisi_hpre_test/Makefile.am:22:test_hisi_hpre_times_LDADD+=$(with_openssl_dir)/libcrypto_wd.so
v1/test/hisi_sec_test/Makefile.am:9:test_hisi_sec_LDADD=../../../.libs/libwd.la $(with_openssl_dir)/libcrypto_wd.so
v1/test/hisi_sec_test/Makefile.am:11:test_hisi_sec_LDADD=../../../.libs/libwd.so $(with_openssl_dir)/libcrypto_wd.so
test/hisi_hpre_test/Makefile.am:17:test_hisi_hpre_LDADD+= $(with_openssl_dir)/libcrypto_wd.so
/bin/bash ../../libtool --tag=CC --mode=link aarch64-linux-gnu-gcc -Wall -fno-strict-aliasing -I../../include -pthread -g -O2 -Wl,-rpath,'/usr/local/lib' -o test_hisi_hpre test_hisi_hpre.o ../../.libs/libwd.a ../../.libs/libwd_crypto.a ../../.libs/libhisi_hpre.a -ldl
libtool: link: aarch64-linux-gnu-gcc -Wall -fno-strict-aliasing -I../../include -pthread -g -O2 -Wl,-rpath -Wl,/usr/local/lib -o test_hisi_hpre test_hisi_hpre.o ../../.libs/libwd.a ../../.libs/libwd_crypto.a ../../.libs/libhisi_hpre.a -ldl -pthread
/usr/bin/ld: ../../.libs/libhisi_hpre.a(hisi_hpre.o): in function ecc_sign_out_transfer': /home/buildslave/uadk/drv/hisi_hpre.c:1225: undefined reference to
wd_sm2_get_sign_out_params'
/usr/bin/ld: ../../.libs/libhisi_hpre.a(hisi_hpre.o): in function sm2_enc_out_transfer': /home/buildslave/uadk/drv/hisi_hpre.c:1279: undefined reference to
wd_sm2_get_enc_out_params'
/usr/bin/ld: ../../.libs/libhisi_hpre.a(hisi_hpre.o): in function sm2_kg_out_transfer': /home/buildslave/uadk/drv/hisi_hpre.c:1259: undefined reference to
wd_sm2_get_kg_out_params'
/usr/bin/ld: ../../.libs/libhisi_hpre.a(hisi_hpre.o): in function ecc_prepare_pubkey': /home/buildslave/uadk/drv/hisi_hpre.c:847: undefined reference to
wd_ecc_get_pubkey_params'
/usr/bin/ld: ../../.libs/libhisi_hpre.a(hisi_hpre.o): in function ecc_prepare_prikey': /home/buildslave/uadk/drv/hisi_hpre.c:784: undefined reference to
wd_ecc_get_prikey_params'
/usr/bin/ld: ../../.libs/libhisi_hpre.a(hisi_hpre.o): in function ecc_prepare_sign_in': /home/buildslave/uadk/drv/hisi_hpre.c:970: undefined reference to
wd_ecc_get_prikey_params'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:395: test_hisi_hpre] Error 1
make[3]: Leaving directory '/home/buildslave/uadk/test/hisi_hpre_test'
make[2]: *** [Makefile:352: all-recursive] Error 1
make[2]: Leaving directory '/home/buildslave/uadk/test'
make[1]: *** [Makefile:722: all-recursive] Error 1
make[1]: Leaving directory '/home/buildslave/uadk'
make: *** [Makefile:516: all] Error 2
UADK multi-threaded performance cannot be increased to the hardware speed limit
8 threads:
algname: length: perf: iops: CPU_rate:
aes-128-cbc 1024Bytes 1254809.0KB/s 1225.2Kops 785.40%
16 threads:
algname: length: perf: iops: CPU_rate:
aes-128-cbc 1024Bytes 1420551.0KB/s 1387.2Kops 1564.60%
32 threads:
algname: length: perf: iops: CPU_rate:
aes-128-cbc 1024Bytes 539262.2KB/s 526.6Kops 3138.20%
In contrast, in the same synchronization mode, warpdrive can reach the hardware speed limit by adding threads:
8 threads:
algname: length: perf: iops: CPU_rate:
aes-128-cbc 1024Bytes 2970516.8KB/s 2900.8Kops 799.20%
16 threads:
algname: length: perf: iops: CPU_rate:
aes-128-cbc 1024Bytes 5664155.0KB/s 5531.4Kops 1597.80%
32 threads:
algname: length: perf: iops: CPU_rate:
aes-128-cbc 1024Bytes 8293766.6KB/s 8099.2Kops 3194.80%
nginx can not support since cipher and digest can not re-entry.
prevented by
if (wd_cipher_setting.config.ctx_num) {
return -WD_EEXIST;
Nginx master process will init cipher and digest, doing negotiation
then fork work process, init cipher and digest, doing process, here failed.
How to setup nginx, see https://docs.qq.com/doc/DRVdaZ2tqTmdzc1hI
DVM: "Distributed Virtual Memory", an Arm interconnect
protocol. When sharing a process address space, TLB invalidations from the
CPU are broadcasted to the SMMU, so we don't have to send commands through
the SMMU queue to invalidate IOTLBs
guest: fail
estuary:/mnt$ ./test_sva_bind -s 0x1000 -b 0x1000 -k t
TLB test failed, broken invalidate! VA=0xffff9d3f9000-0xffff9d3fafff
host: succeed
sudo ./test_sva_bind -s 0x1000 -b 0x1000 -k t bad status (s=66, t=3, expected)
wd_sched_work: -14
TLB test success
SUCCESS
test zip using cmd:
zip_sva_perf -b 4096 -l 1 -v -m 0
host sva and guest vsva is enabled.
host: success
guest: fail, both host and guest report errors
detail steps:
insmod zip driver in host:
insmod uacce.ko
insmod hisi_qm.ko
insmod hisi_zip.ko uacce_mode=1 vfs_num=2
start vm, using 75:00.1:
echo 0000:75:00.1 > /sys/bus/pci/drivers/hisi_zip/unbind
echo vfio-pci > /sys/devices/pci0000:74/0000:74:00.0/0000:75:00.1/driver_override
echo 19e5 a251 > /sys/bus/pci/drivers/vfio-pci/new_id
./qemu-system-aarch64
-enable-kvm
-kernel ../Image
-smp 4
-m 4G
-machine virt,kernel_irqchip=on,gic-version=3,iommu=smmuv3
-device pcie-root-port,port=0x8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1
-net none
-device virtio-net-pci,netdev=net0,mac="52:54:11:12:34:55"
-netdev type=tap,id=net0,script=no,downscript=no,ifname=tap0
-device vfio-pci,host=75:00.1,bus=pci.1,addr=0x0,id=acc2
-initrd ../v688fs-guest
-cpu host
-nographic
-append "rdinit=init console=ttyAMA0 earlycon=pl011,0x9000000 ip=192.168.100.3"
insmod zip driver in guest:
insmod /uacce.ko
insmod /hisi_qm.ko
insmod /hisi_zip.ko uacce_mode=1
test zip using cmd:
zip_sva_perf -b 4096 -l 1 -v -m 0
[ 262.127989] hisi_zip 0000:75:00.0: qm_db_random_invalid [error status=0x1000] found
[ 262.127989] hisi_zip 0000:75:00.0: qm_db_random_invalid [error status=0x1000] found
guest zip driver(hisi_zip 0000:01:00.0 ) reports errors and ras error:
[ 23.974799] hisi_zip 0000:01:00.0: uacce release queue's qp_id = 0, BD sqe head = 0, tail = 0
NOTICE: RasIntBase = 0x140090000,ErrMapping = 0x29
NOTICE: [RasEriInterrupt]:[140L]NodeTYP0Status = 0x0
NOTICE: [RasEriInterrupt]:[156L]NodeTYP1Status = 0x1
NOTICE: [NimbusZipNodeType1]:[2254L]This is zip, Base = 0x141000000
NOTICE: [NimbusZipHandle]:[2188L]QM_PF_ABNORMAL_INT_SOURCE = 0x40
NOTICE: [NimbusZipHandle]:[2189L]ZIP_CORE_INT_SOURCE = 0x0
NOTICE: [PrintSecurityType]:[349L] SecurityType is RECOVERABLE!
NOTICE: [RasErrorDataPcieOemProcessor]:[1400L]BDF[0x75:0x0:0x0]
NOTICE: [HestGhesV2SetGenericErrorData]:[121L]
NOTICE: [HestGhesV2SetGenericErrorData]:[171L] Fill in HEST TABLE ,AckRegister=44010000
NOTICE: [ErrorBlockUpdateStatusStructure]:[38L] memset_s() run failed!
NOTICE: [HestNotifiedOS]:[18L]
NOTICE: [RasEriInterrupt]:[172L]NodeTYP2Status = 0x0
ERROR: [FiqInterruptHandler]:[115L] GicInterrupt1 : 0x1d3!!!
[ 1129.637064] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 0
[ 1129.637064] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 0
[ 1129.656343] {1}[Hardware Error]: event severity: recoverable
[ 1129.656343] {1}[Hardware Error]: event severity: recoverable
[ 1129.667667] {1}[Hardware Error]: Error 0, type: recoverable
[ 1129.667667] {1}[Hardware Error]: Error 0, type: recoverable
[ 1129.678991] {1}[Hardware Error]: section_type: PCIe error
[ 1129.678991] {1}[Hardware Error]: section_type: PCIe error
[ 1129.690139] {1}[Hardware Error]: version: 4.0
[ 1129.690139] {1}[Hardware Error]: version: 4.0
[ 1129.699198] {1}[Hardware Error]: command: 0x0006, status: 0x0010
[ 1129.699198] {1}[Hardware Error]: command: 0x0006, status: 0x0010
[ 1129.711566] {1}[Hardware Error]: device_id: 0000:75:00.0
[ 1129.711566] {1}[Hardware Error]: device_id: 0000:75:00.0
[ 1129.722540] {1}[Hardware Error]: slot: 0
[ 1129.722540] {1}[Hardware Error]: slot: 0
[ 1129.730728] {1}[Hardware Error]: secondary_bus: 0x00
[ 1129.730728] {1}[Hardware Error]: secondary_bus: 0x00
[ 1129.741006] {1}[Hardware Error]: vendor_id: 0x19e5, device_id: 0xa250
[ 1129.741006] {1}[Hardware Error]: vendor_id: 0x19e5, device_id: 0xa250
[ 1129.754244] {1}[Hardware Error]: class_code: 120000
[ 1129.754244] {1}[Hardware Error]: class_code: 120000
[ 30.004443] hisi_zip 0000:01:00.0: QM mailbox operation timeout!
[ 30.004875] hisi_zip 0000:01:00.0: Failed to dump sqc!
[ 30.005242] hisi_zip 0000:01:0[0.0: Failed to drain out data for stopping!
[ 30.005853] hisi_zip 0000:01:00.0: uacce release queue's qp_id = 1, BD sqe head = 0, tail = 0
[ 30.007466] hisi_zip 0000:01:00.0: QM mailbox is busy to start!
[ 30.007880] hisi_zip 0000:01:00.0: Failed to dump sqc!
[ 30.008236] hisi_zip 0000:01:00.0: Failed to drain out data for stopping!
[ 30.008715] hisi_zip 0000:01:00.0: uacce release queue's qp_id = 2, BD sqe head = 0, tail = 0
[ 30.010319] hisi_zip 0000:01:00.0: QM mailbox is busy to start!
[ 30.010733] hisi_zip 0000:01:00.0: Failed to dump sqc!
[ 30.011093] hisi_zip 0000:01:00.0: Failed to drain out data for stopping!
[ 30.011563] hisi_zip 0000:01:00.0: uacce release queue's qp_id = 3, BD sqe head = 0, tail = 0
[ 30.013155] hisi_zip 0000:01:00.0: QM mailbox is busy to start!
[ 30.013639] hisi_zip 0000:01:00.0: Failed to dump sqc!
[ 30.013998] hisi_zip 0000:01:00.0: Failed to drain out data for stopping!
Dear developers:
Thank you for your checking. In the method test_sec_cipher_async
, the lock &test_sec_mutex
maybe not be released correctly when the control flow goes to the else if
branch and go to out
, bypassing the unreleased lock.
uadk/test/hisi_sec_test/test_hisi_sec.c
Line 1055 in 41313cd
uadk/test/hisi_sec_test/test_hisi_sec.c
Line 1064 in 41313cd
static int test_sec_cipher_async(void *arg)
{
...;
pthread_mutex_lock(&test_sec_mutex);
...;
do {
try_do_again:
ret = wd_do_cipher_async(h_sess, req);
if (ret == -EBUSY) { // busy
usleep(100);
goto try_do_again;
} else if (ret) {
SEC_TST_PRT("test sec cipher send req is error!\n");
goto out; // bypass the unlock, leading to unreleased lock
}
cnt--;
} while (cnt);
pthread_mutex_unlock(&test_sec_mutex);
...;
out:
if (h_sess)
wd_cipher_free_sess(h_sess);
return ret;
}
Best,
$ openssl engine -t uadk
(uadk) uadk hardware engine support
[ available ]
281473677588864:error:25066067:DSO support routines:dlfcn_load:could not load the shared library:crypto/dso/dso_dlfcn.c:117:filename(libproviders.so): libproviders.so: cannot open shared object file: No such file or directory
281473677588864:error:25070067:DSO support routines:DSO_load:could not load the shared library:crypto/dso/dso_lib.c:162:
281473677588864:error:0E07506E:configuration file routines:module_load_dso:error loading dso:crypto/conf/conf_mod.c:221:module=providers, path=providers
281473677588864:error:0E076071:configuration file routines:module_run:unknown module name:crypto/conf/conf_mod.c:162:module=providers
Sanity_test.sh depends on /var/log/syslog
to run correctly.
However, on openEuler 22.04, there is no such a file. Somebody need to fix this.
wd_util.c
num = fmin(config_numa->async_poll_num, config_numa->async_ctx_num);
depend on -lm
core lib should not depend too much library
board: 201
kernel: https://github.com/Linaro/linux-kernel-uadk/tree/uacce-devel-5.11
make defconfig
make menuconfig
Device Drivers --->
-- IOMMU Hardware Support --->
[] Shared Virtual Addressing support for the ARM SMMUv3
Misc devices --->
<> Accelerator Framework for User Land
-- Cryptographic API --->
[] Hardware crypto devices --->
<> Support for HiSilicon SEC2 crypto block cipher accelerator
<> Support for HiSilicon ZIP accelerator
<> Support for HISI HPRE accelerator
[*] Tracers --->
uadk: master version: update to 2.2.14
openssl-uadk: master: a96fbbf ras: work with nginx via both async and sync mode
https://github.com/Linaro/openssl-uadk
https://github.com/Linaro/openssl-uadk/blob/master/INSTALL.md
Test:
for((i=0; i<10; i++))
do
echo $i
openssl speed -elapsed -engine uadk -async_jobs 36 rsa2048
done
Hung since poll->recv can not get new data
Sometime report ras error first, then hung.
NOTICE: [RasEriInterrupt]:[141L]NodeTYP0Status = 0x0
NOTICE: [RasEriInterrupt]:[157L]NodeTYP1Status = 0x2
NOTICE: [NimbusHpreNodeType1]:[2405L]This is hpre, Base = 0x208000000
NOTICE: [NimbusHpreHandle]:[2348L] HpreHacIntSt = 0x33e7c8
NOTICE: [NimbusHpreHandle]:[2349L] HpreQmIntStatus = 0x22
NOTICE: [PrintSecurityType]:[389L] SecurityType is RECOVERABLE!
NOTICE: [RasErrorDataPcieOemProcessor]:[1748L]BDF[0x79:0x0:0x0]
NOTICE: [HestGhesV2SetGenericErrorData]:[188L] Fill in HEST TABLE ,AckRegister=44010000
NOTICE: [HestNotifiedOS]:[37L]
NOTICE: [RasEriInterrupt]:[173L]NodeTYP2Status = 0x0
[ 496.016498] {2}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 0
[ 496.024727] {2}[Hardware Error]: event severity: recoverable
[ 496.030361] {2}[Hardware Error]: Error 0, type: recoverable
[ 496.035994] {2}[Hardware Error]: section_type: PCIe error
[ 496.041539] {2}[Hardware Error]: version: 4.0
[ 496.046047] {2}[Hardware Error]: command: 0x0006, status: 0x0010
[ 496.052199] {2}[Hardware Error]: device_id: 0000:79:00.0
[ 496.057659] {2}[Hardware Error]: slot: 0
[ 496.061735] {2}[Hardware Error]: secondary_bus: 0x00
[ 496.066849] {2}[Hardware Error]: vendor_id: 0x19e5, device_id: 0xa258
[ 496.073431] {2}[Hardware Error]: class_code: 100000
root@ubuntu:/sys/kernel/debug/hisi_hpre/0000:79:00.0/qm# cat regs
QM_ECC_1BIT_CNT = 0x00000000
QM_ECC_MBIT_CNT = 0x00000000
QM_DFX_MB_CNT = 0x0000248e
QM_DFX_DB_CNT = 0x2ce1cada
QM_DFX_SQE_CNT = 0x08971cfc
QM_DFX_CQE_CNT = 0x08971cfc
QM_DFX_SEND_SQE_TO_ACC_CNT = 0x1670ef9d
QM_DFX_WB_SQE_FROM_ACC_CNT = 0x00000000
QM_DFX_ACC_FINISH_CNT = 0x1670ef9b
QM_DFX_CQE_ERR_CNT = 0x00000206
QM_DFX_FUNS_ACTIVE_ST = 0x00000002
QM_ECC_1BIT_INF = 0x00010045
QM_ECC_MBIT_INF = 0x000600c6
QM_DFX_ACC_RDY_VLD0 = 0x01000000
QM_DFX_ACC_RDY_VLD1 = 0x0000ff00
QM_DFX_AXI_RDY_VLD = 0x00001802
QM_DFX_FF_ST0 = 0x00000ff4
QM_DFX_FF_ST1 = 0x0208c000
QM_DFX_FF_ST2 = 0x01d73fff
QM_DFX_FF_ST3 = 0x00000000
QM_DFX_FF_ST4 = 0x0fffffff
QM_DFX_FF_ST5 = 0x00ff00ff
QM_DFX_FF_ST6 = 0x00078081
QM_IN_IDLE_ST = 0x00000000
uadk master can not reproduce since only support -t 2
test_hisi_hpre rsa-sgn --mode=crt --perf --trd_mode=async --seconds=10 -t 2
If -t 3, report "failed to send: retry exit!"
test aes-cbc, use sync mode. Treads == queues. packet size = 1K. The lengths of other packets are similar. maybe worse.
NO-SVA SVA
Threads = 1, 502.4 MB/s 555.7 MB/s
Threads = 2, 989.3 MB/s 1067.6 MB/s
Threads = 4, 1963.9 MB/s 1910.8 MB/s
Threads = 8, 13604.3 MB/s 3189.9 MB/s
Threads = 16, 6571.5 MB/s 1911.6 MB/s
Threads = 32, 9403.3 MB/s 966.9 MB/s
环境:
硬件:920
OS: openeuler + kernel 6.1.x
步骤:
host执行:
1.modprobe hisi_sec2 uacce_mode=1 pf_q_num=256 ctx_q_num=2 vfs_num=0
2.echo 63 >/sys/bus/pci/devices/0000:76:00.0/sriov_numvfs
3.echo 0000:76:01.0 > /sys/bus/pci/devices/0000:76:01.0/driver/unbind
4. echo vfio-pci > /sys/bus/pci/devices/0000:76:01.0/driver_override
5. echo 0000:76:01.0 > /sys/bus/pci/probe
启动虚拟机参数:
smp 1 -device vfio-pci,host=76:01.0
kill一次虚拟机再重启。
vm执行业务:
1.modprobe hisi_sec2 uacce_mode=1
2.while true; do kp_utools u_accv2 -a cipher -n encrypt-cbc-aes -k 16-16-1000000 -s 0 -e 1||exit;done
16-16-1000000:(key长16,iv长16,输入1000000 )
业务流程:CBC-AES算法(SEC其他算法也可),单进程单线程同步模式,硬件加密,再进行硬件环回解密。
https://github.com/Linaro/linux-kernel-uadk/tree/uacce-devel-5.12
make defconfig
make -j4
linaro@ubuntu:$ zip_sva_perf$ NotifiedOS]:[37L]
Couldn't open file /sys/kernel/debug/tracing/events/iommu/dev_fault/id
IOPF statistic unavailable
NOTICE: RasIntBase = 0x140090000,ErrMapping = 0x29
NOTICE: [RasEriInterrupt]:[141L]NodeTYP0Status = 0x0
NOTICE: [RasEriInterrupt]:[157L]NodeTYP1Status = 0x1
NOTICE: [NimbusZipNodeType1]:[2593L]This is zip, Basbad status(ctx_st=0x0, s=0x41, t=3)
Compress bz=512000 nb=1×10, speed=184.3 MB/s (±0.0% N=1) overall=161.7 MB/s (±0.0%)
e = 0x141000000
NOTICE: [NimbusZipHandle]:[2542L]QM_PF_ABNORMAL_INT_SOURCE = 0x0
NOTICE: [NimbusZipHandle]:[2543L]ZIP_CORE_INT_SOURCE = 0xc
NOTICE: [PrintSecurityType]:[389L] SecurityType is RECOVERABLE!
NOTICE: [RasErrorDataPcieOemProcessor]:[1748L]BDF[0x75:0x0:0x0]
NOTICE: [HestGhesV2SetGenericErrorData]:[188L] Fill in HEST TABLE ,AckRegister=44010000
NOTICE: [Hestlinaro@ubuntu:
NOTICE: [RasEriInterrupt]:[173L]NodeTYP2Status = 0x0
[ 849.220545] {7}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 0
[ 849.228776] {7}[Hardware Error]: event severity: recoverable
[ 849.234409] {7}[Hardware Error]: Error 0, type: recoverable
[ 849.240043] {7}[Hardware Error]: section_type: PCIe error
[ 849.245589] {7}[Hardware Error]: version: 4.0
[ 849.250097] {7}[Hardware Error]: command: 0x0006, status: 0x0010
[ 849.256248] {7}[Hardware Error]: device_id: 0000:75:00.0
[ 849.261707] {7}[Hardware Error]: slot: 0
[ 849.265784] {7}[Hardware Error]: secondary_bus: 0x00
[ 849.270897] {7}[Hardware Error]: vendor_id: 0x19e5, device_id: 0xa250
[ 849.277479] {7}[Hardware Error]: class_code: 120000
linaro@ubuntu:~$ test_hisi_sec
this is a hisi sec test.
Test sec Cipher parameter default, alg:ecb(aes), set_times:10,set_pktlen:16 bytes, set_keylen:128 bit.
hisi sec init Kunpeng920!
req src--------->:
\0x00\0x11\0x22\0x33\0x44\0x55\0x66\0x77
\0x88\0x99\0xaa\0xbb\0xcc\0xdd\0xee\0xff
cipher req key--------->:
SEC BD cipher fail! done=0x1, etype=0xd
SEC BD cipher fail! done=0x1, etype=0xd
SEC BD cipher fail! done=0x1, etype=0xd
SEC BD cipher fail! done=0x1, etype=0xd
SEC BD cipher fail! done=0x1, etype=0xd
SEC BD cipher fail! done=0x1, etype=0xd
SEC BD cipher fail! done=0x1, etype=0xd
SEC BD cipher fail! done=0x1, etype=0xd
SEC BD cipher fail! done=0x1, etype=0xd
SEC BD cipher fail! done=0x1, etype=0xd
time_used:275 us, send task num:10
Pro-5277, thread_id-5277, speed:36363.637 ops, Perf: 568 KB/s
Test cipher sync function: output dst-->
\0x00\0x00\0x00\0x00\0x00\0x00\0x00\0x00
\0x00\0x00\0x00\0x00\0x00\0x00\0x00\0x00
linaro@ubuntu:~$ test_hisi_hpre
test start run rsa-gen :
key_bits = 1024
trd_mode = sync
trd_num = 2
core_mask = [0x0][0x0]
msg_type = 0
msg_len = 0xffffffff
id_len = 0xffffffff
hash_type = 1
rand_type = 1
rand_len = 0xffffffff
data_from = 0
perf_test = 0
check = 0
cycles = 100
seconds = 0
HPRE do rsa fail!done=0x2, etype=0x4
Proc-5278, T-5279:hpre rsa-gen 0th status=0 fail!
HPRE do rsa fail!done=0x2, etype=0x4
Proc-5278, T-5280:hpre rsa-gen 0th status=0 fail!
<< test finish!
NOTICE: RasIntBase = 0x2010a0000,ErrMapping = 0x29
NOTICE: [RasEriInterrupt]:[141L]NodeTYP0Status = 0x0
NOTICE: [RasEriInterrupt]:[157L]NodeTYP1Status = 0x2
NOTICE: [NimbusHpreNodeType1]:[2405L]This is hpre, Base = 0x208000000
NOTICE: [NimbusHpreHandle]:[2348L] HpreHacIntSt = 0xe408
NOTICE: [NimbusHpreHandle]:[2349L] HpreQmIntStatus = 0x0
NOTICE: [PrintSecurityType]:[389L] SecurityType is RECOVERABLE!
NOTICE: [RasElinaro@ubuntu:~$ rrorDataPcieOemProcessor]:[1748L]BDF[0x79:0x0:0x0]
NOTICE: [HestGhesV2SetGenericErrorData]:[188L] Fill in HEST TABLE ,AckRegister=44010000
NOTICE: [HestNotifiedOS]:[37L]
NOTICE: [RasEriInterrupt]:[173L]NodeTYP2Status = 0x0
[ 1090.772613] {8}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 0
[ 1090.780842] {8}[Hardware Error]: event severity: recoverable
[ 1090.786474] {8}[Hardware Error]: Error 0, type: recoverable
[ 1090.792107] {8}[Hardware Error]: section_type: PCIe error
[ 1090.797653] {8}[Hardware Error]: version: 4.0
[ 1090.802161] {8}[Hardware Error]: command: 0x0006, status: 0x0010
[ 1090.808311] {8}[Hardware Error]: device_id: 0000:79:00.0
[ 1090.813771] {8}[Hardware Error]: slot: 0
[ 1090.817848] {8}[Hardware Error]: secondary_bus: 0x00
[ 1090.822961] {8}[Hardware Error]: vendor_id: 0x19e5, device_id: 0xa258
[ 1090.829543] {8}[Hardware Error]: class_code: 100000
if you try to build using make -jN (where N is >= 8), the build fails:
/usr/bin/ld: cannot find -lwd_crypto
UADK's scheduling for multiple queues will decrease performance after more than 16 queues.
In synchronous mode, 4threads,16 queues
algname: length: perf: iops: CPU_rate:
aes-128-cbc 1024Bytes 1026159.6KB/s 1002.0Kops 398.80%
In synchronous mode, 4threads, 24 queues
algname: length: perf: iops: CPU_rate:
aes-128-cbc 1024Bytes 1001670.6KB/s 978.0Kops 398.60%
In synchronous mode, 4threads, 32 queues
algname: length: perf: iops: CPU_rate:
aes-128-cbc 1024Bytes 821041.8KB/s 801.6Kops 386.40%
https://github.com/Linaro/linux-kernel-warpdrive/tree/uacce-devel-5.12
make defconfig
buildin, cmdline:
hisi_zip.uacce_mode=2 hisi_hpre.uacce_mode=2 hisi_sec2.uacce_mode=2
Or use module:
make menuconifg
-- Cryptographic API --->
[] Hardware crypto devices --->
Support for HiSilicon SEC2 crypto block cipher accelerator
Support for HiSilicon ZIP accelerator
Support for HISI HPRE accelerator
sudo insmod hisi_qm.ko
sudo insmod sec2/hisi_sec2.ko uacce_mode=2
sudo insmod hpre/hisi_hpre.ko uacce_mode=2
$ cat /sys/class/uacce/hisi_sec2-0/flags
128
--with-openssl_dir=~/work/openssl \
make clean
./conf.sh; make -j4
sudo make install
// Or kae.sh for local build
#!/bin/bash -e
export WORKSPACE=${PWD}
cd ${WORKSPACE}/uadk-master
./cleanup.sh; ./autogen.sh;
./configure
--prefix=${WORKSPACE}/uadk-shared-v2/usr/local
--includedir=${WORKSPACE}/uadk-shared-v2/usr/local/include/uadk
--disable-static
--enable-shared
make -j$(nproc)
make install
#sudo \
cd ${WORKSPACE}/KAE
make clean
./configure --openssl_path=${WORKSPACE}/uadk-shared-v2
LIBRARY_PATH=${WORKSPACE}/uadk-shared-v2/usr/local/lib C_INCLUDE_PATH=${WORKSPACE}/uadk-master/v1 make
make install
export LD_LIBRARY_PATH=${WORKSPACE}/uadk-shared-v2/usr/local/lib
openssl engine -c kae
SM4 CTR/XTS/CBC/ECB/OFB modes, and AES ECB/CTR/XTS/CBC modes are supported by wd v1, should also all be supported by uadk
//async
test_hisi_hpre rsa-sgn --mode=crt --perf --trd_mode=async -t 1 --key_bits=2048
//sync
test_hisi_hpre rsa-sgn --mode=crt --perf --trd_mode=sync --key_bits=2048
sync:
23:<< sync 2 thread rsa-gen crt mode 2048 key_bits at 20899.988 ops!
45:<< sync 2 thread rsa-gen crt mode 2048 key_bits at 18341.109 ops!
67:<< sync 2 thread rsa-gen crt mode 2048 key_bits at 21087.043 ops!
89:<< sync 2 thread rsa-gen crt mode 2048 key_bits at 21136.064 ops!
111:<< sync 2 thread rsa-gen crt mode 2048 key_bits at 20772.770 ops!
async:
22:<< async 1 thread rsa-gen crt mode 2048 key_bits at 3570.233 ops!
43:<< async 1 thread rsa-gen crt mode 2048 key_bits at 58572.301 ops!
64:<< async 1 thread rsa-gen crt mode 2048 key_bits at 156542.062 ops!
85:<< async 1 thread rsa-gen crt mode 2048 key_bits at 4922.516 ops!
106:<< async 1 thread rsa-gen crt mode 2048 key_bits at 27717.783 ops!
async+bind:
numactl --cpubind=0 --membind=0 test_hisi_hpre rsa-sgn --mode=crt --perf --trd_mode=async -t 1 --key_bits=2048
linaro@ubuntu:~/test$ grep -rn "async 1 thread rsa-gen crt" log.2
22:<< async 1 thread rsa-gen crt mode 2048 key_bits at 6391.080 ops!
43:<< async 1 thread rsa-gen crt mode 2048 key_bits at 10102.447 ops!
64:<< async 1 thread rsa-gen crt mode 2048 key_bits at 18771.873 ops!
85:<< async 1 thread rsa-gen crt mode 2048 key_bits at 11188.066 ops!
106:<< async 1 thread rsa-gen crt mode 2048 key_bits at 8320.621 ops!
127:<< async 1 thread rsa-gen crt mode 2048 key_bits at 19878.707 ops!
148:<< async 1 thread rsa-gen crt mode 2048 key_bits at 6042.568 ops!
169:<< async 1 thread rsa-gen crt mode 2048 key_bits at 7707.399 ops!
190:<< async 1 thread rsa-gen crt mode 2048 key_bits at 1772.593 ops!
211:<< async 1 thread rsa-gen crt mode 2048 key_bits at 5408.329 ops!
232:<< async 1 thread rsa-gen crt mode 2048 key_bits at 4921.700 ops!
253:<< async 1 thread rsa-gen crt mode 2048 key_bits at 19461.562 ops!
274:<< async 1 thread rsa-gen crt mode 2048 key_bits at 17051.154 ops!
295:<< async 1 thread rsa-gen crt mode 2048 key_bits at 17449.238 ops!
316:<< async 1 thread rsa-gen crt mode 2048 key_bits at 4229.448 ops!
337:<< async 1 thread rsa-gen crt mode 2048 key_bits at 17965.992 ops!
358:<< async 1 thread rsa-gen crt mode 2048 key_bits at 19314.020 ops!
379:<< async 1 thread rsa-gen crt mode 2048 key_bits at 18230.029 ops!
400:<< async 1 thread rsa-gen crt mode 2048 key_bits at 4987.757 ops!
421:<< async 1 thread rsa-gen crt mode 2048 key_bits at 17950.393 ops!
442:<< async 1 thread rsa-gen crt mode 2048 key_bits at 18874.283 ops!
463:<< async 1 thread rsa-gen crt mode 2048 key_bits at 3388.029 ops!
484:<< async 1 thread rsa-gen crt mode 2048 key_bits at 120603.016 ops!
505:<< async 1 thread rsa-gen crt mode 2048 key_bits at 16211.062 ops!
526:<< async 1 thread rsa-gen crt mode 2048 key_bits at 21487.039 ops!
547:<< async 1 thread rsa-gen crt mode 2048 key_bits at 58381.984 ops!
568:<< async 1 thread rsa-gen crt mode 2048 key_bits at 10157.367 ops!
589:<< async 1 thread rsa-gen crt mode 2048 key_bits at 15453.960 ops!
610:<< async 1 thread rsa-gen crt mode 2048 key_bits at 5289.558 ops!
631:<< async 1 thread rsa-gen crt mode 2048 key_bits at 20218.229 ops!
prepare:
dd if=/dev/urandom of=origin bs=1K count=512
zip_sva_perf --in origin --dest
test:
zip_sva_perf -d --in dest --out origin2
result:
Missing IN list file!
Fail to run hw_ifl_perf() (-22)!
I think the error is caused by the 'load_ilist'. But I can't understand the use of 'ilist'. So I need you help.
@hzhuang1
After numa source code make install, apt-get install libnuma-dev can not work again, both on x86 and ARM.
So numa has to install from source code
2,
$git clone https://github.com/numactl/numactl.git
$cd numactl
$./autogen.sh
./configure, make; sudo make install
Can remove library in /use/lib, or sudo make uninstall
On x86
apt-get remove libnuma-dev
apt-get install libnuma-dev
ls /usr/lib/x86_64-linux-gnu/libnuma.*
/usr/lib/x86_64-linux-gnu/libnuma.a /usr/lib/x86_64-linux-gnu/libnuma.so
Only has link libnuma.so, but no libnuma.so.1.0.0
$ ll /usr/lib/x86_64-linux-gnu/libnuma.so
lrwxrwxrwx 1 root root 16 10月 20 2017 /usr/lib/x86_64-linux-gnu/libnuma.so -> libnuma.so.1.0.0
$ ls /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0
ls: cannot access '/usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0': No such file or directory
On ARM
$ sudo apt-get remove libnuma-dev
$ sudo apt-get install libnuma-dev
$ numactl
numactl: error while loading shared libraries: libnuma.so.1: cannot open shared object file: No such file or directory
$ ll /usr/lib/aarch64-linux-gnu/libnuma*
-rw-r--r-- 1 root root 77642 Sep 8 2018 /usr/lib/aarch64-linux-gnu/libnuma.a
lrwxrwxrwx 1 root root 16 Sep 8 2018 /usr/lib/aarch64-linux-gnu/libnuma.so -> libnuma.so.1.0.0
Only link, no real library
https://github.com/Linaro/uadk/tree/nginx-fork
test_hisi_sec --digest 0 --optype 0 --pktlen 16 --keylen 16 --times 1 --sync --multi 1
SEC BD digest fail! done=0x1, etype=0x11
In the middle of sec_digest_sync_once, fork a process start sec_digest_sync_once again,
report error: SEC BD digest fail! done=0x1, etype=0x11
After appended rand case in "example.c", stream mode is failed in NOSVA scenario. It could be observed in "hzhuang1-sva-3.2" branch.
digest stream mode must be 64-byte aligned!
In guest system (openEuler), init2 fails since wd_get_usable_list failed to get usable devices(-19)!
Host system init2 works.
Guest:
cat /sys/class/uacce/hisi_sec2-0/device/numa_node
-1
uadk_tool test --m sec --digest 0 --sync --optype 0 --pktlen 16 --keylen 16 --times 1 --multi 1 --init 2
driver_type=3
HW
numa_id=-1 p=0x3cdbd9e0
p=(nil)
failed to get usable devices(-19)!
Host:
HW
numa_id=2 p=0xaaaac695a030
numa_id=0 p=0xaaaac695a440
https://github.com/Linaro/linux-kernel-warpdrive/tree/uacce-devel-5.12
make defconfig
make menuconifg
-- Cryptographic API --->
[] Hardware crypto devices --->
Support for HiSilicon SEC2 crypto block cipher accelerator
Support for HiSilicon ZIP accelerator
Support for HISI HPRE accelerator
sudo insmod hisi_qm.ko
sudo insmod hisi_sec2.ko uacce_mode=2
$ cat /sys/class/uacce/hisi_sec2-0/flags
128
./cleanup.sh;./autogen.sh;
./configure
--host aarch64-linux-gnu
--target aarch64-linux-gnu
--includedir=/usr/local/include/uadk
--with-openssl_dir=~/work/openssl
--enable-static --with-static_drv --disable-shared
make
$ ./v1/test/hisi_sec_test/test_hisi_sec
Segmentation fault (core dumped)
It could be observed in "master" branch.
Currently, we found the version number in the ChangeLog, however, it it not convenient to DFX. For example, we can easily obtain the version number in the interface.
commit b3db20a
Author: Haojian Zhuang [email protected]
Date: Wed Dec 30 20:38:35 2020 +0800
wd: replace mutex lock by spin lock in ctx structure
Is io page fault for intel x86 platform supported in this project?
When test multiple blocks in block mode, case in "example.c" fails. It could be observed in both "master" and "hzhuang1-sva-3.3" branch.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.