jackweiwang / android-facedetection-ultranet-mnn Goto Github PK

when I set sch_config.type = (MNNForwardType)MNN_FORWARD_OPENCL get error.

2020-08-06 17:20:00.360 5393-5393/? A/DEBUG: #00 pc 0001d458 /data/app/com.facesdk-g1tWUdusJJNcF4watyMQ7w==/lib/arm/libMNN.so (MNN::Interpreter::resizeTensor(MNN::Tensor*, std::__ndk1::vector<int, std::__ndk1::allocator> const&)+40) (BuildId: 3011315747c81ab99070c67313433009112c4630)
2020-08-06 17:20:00.361 5393-5393/? A/DEBUG: #1 pc 0000d13b /data/app/com.facesdk-g1tWUdusJJNcF4watyMQ7w==/lib/arm/libfacedetect.so (Inference_engine::infer_img(unsigned char*, int, int, int, int, int, Inference_engine_tensor&)+326) (BuildId: 8260643125673747a779f569c3c3b3a3c3fff245)
2020-08-06 17:20:00.361 5393-5393/? A/DEBUG: #2 pc 00007dc5 /data/app/com.facesdk-g1tWUdusJJNcF4watyMQ7w==/lib/arm/libfacedetect.so (UltraFace::detect(unsigned char*, int, int, int, std::__ndk1::vector<FaceInfo, std::__ndk1::allocator>&)+256) (BuildId: 8260643125673747a779f569c3c3b3a3c3fff245)
2020-08-06 17:20:00.361 5393-5393/? A/DEBUG: #3 pc 00005b33 /data/app/com.facesdk-g1tWUdusJJNcF4watyMQ7w==/lib/arm/libfacedetect.so (Java_com_facesdk_FaceSDKNative_FaceDetect+414) (BuildId: 8260643125673747a779f569c3c3b3a3c3fff245)

大佬 能不能修复一下。谢谢啦

Error while executing process C:\Users\Username\AppData\Local\Android\Sdk\cmake\3.10.2.4988404\bin\cmake.exe with arguments {--build D:\Projects\Face detection\Android-FaceDetection-UltraNet-MNN-master\app.externalNativeBuild\cmake\debug\arm64-v8a --target facedetect}
Unknown argument detection\Android-FaceDetection-UltraNet-MNN-master\app.externalNativeBuild\cmake\debug\arm64-v8a

The original detection result is the bbox position (x1, y1, x2, y2), located in line 99 of "Ultra_jni.cpp":

allfaceInfo[4*i+1] = face_info[i].x1;//left
allfaceInfo[4*i+2] = face_info[i].y1;//top
allfaceInfo[4*i+3] = face_info[i].x2;//right
allfaceInfo[4*i+4] = face_info[i].y2;//bottom

But I also want to get facial point detection results, such as eyes, nose, mouse position...
Can the model obtain the information?
If possible, how to modify the cpp file to obtain the information.

@jackweiwang 您好，想问下为啥RFB-320.mnn测不出人脸呢？？请教下如果我想用RFB-320.mnn而不是量化模型RFB-320-quant-ADMM-32.mnn来进行检测的话需要怎么修改呢？麻烦作者帮我解答下啦

CMakeLists.txt中36行find_library(log-lib log); 41行MNN_CL; 45行 z。分别是什么意思，你能解释一下吗？我刚接触这个不太懂。

1、当我在Windows使用AndroidStudio打开该项目是，会出现“java.lang.NullPointerException (no error message)”的报错
2、我尝试将project中app的build.gradle中的:

    externalNativeBuild {
        cmake {
            path "CMakeLists.txt"
        }
    }

注释掉后再Sync就能正常，但是build的apk在手机上运行是无法进入的，直接闪退
3、目前我还没有找到解决的办法，请问可能的解决办法有哪些？

I m trying to change this code to run with real-time video by using thread.
but It tends to slow down like half of its speed after several minutes.
so I ran this code as thread with static photo, but had same result.
any suggestion would be Thankful.
TIA !

Could anyone please give any instructions on how to run this application? I am new to android and i am trying to run this on my phone. But i am not able to do so. I am trying to solve all the errors that i am getting while trying to run this but i am struck at a point and getting the following error.

E/AndroidRuntime: FATAL EXCEPTION: main Process: com.facesdk, PID: 13680 java.lang.UnsatisfiedLinkError: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.facesdk-9GOwYJgoL4MH3zef9FGT6w==/base.apk"],nativeLibraryDirectories=[/data/app/com.facesdk-9GOwYJgoL4MH3zef9FGT6w==/lib/x86, /system/lib, /vendor/lib]]] couldn't find "libfacedetect.so"

    at java.lang.Runtime.loadLibrary0(Runtime.java:1011)
    at java.lang.System.loadLibrary(System.java:1657)
    at com.facesdk.FaceSDKNative.<clinit>(FaceSDKNative.java:14)
    at com.facesdk.MainActivity.<init>(MainActivity.java:41)
    at java.lang.Class.newInstance(Native Method)
    at android.app.Instrumentation.newActivity(Instrumentation.java:1174)
    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2669)
    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2856)
    at android.app.ActivityThread.-wrap11(Unknown Source:0)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1589)
    at android.os.Handler.dispatchMessage(Handler.java:106)
    at android.os.Looper.loop(Looper.java:164)
    at android.app.ActivityThread.main(ActivityThread.java:6494)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)`

** My app gradle file **

` apply plugin: 'com.android.application'
apply plugin: 'com.google.gms.google-services'

android {

compileSdkVersion 28
defaultConfig {
    applicationId "com.facesdk"
    minSdkVersion 16
    targetSdkVersion 28
    versionCode 1
    versionName "1.0"
    testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"

    externalNativeBuild {
        cmake {
            arguments "-DANDROID_STL=c++_shared", "-DANDROID_ARM_NEON=TRUE", "-DANDROID_PLATFORM=android-21",
                      "-DMNN_OPENCL=true",  "-DMNN_OPENGL=true"
            abiFilters  'arm64-v8a', 'armeabi-v7a'
        }
    }
}
buildTypes {
    debug {
        ndk {
            abiFilters "armeabi-v7a", "x86", "arm64-v8a", "x86_64"
            abiFilters.clear()
        }
    }
    splits {
        abi {
            enable true //enables the ABIs split mechanism
            reset() //reset the list of ABIs to be included to an empty string
            include 'arm64-v8a', 'armeabi-v7a', 'x86'
            universalApk true
        }
    }
    release {
        minifyEnabled false
        proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
    }
}

externalNativeBuild {
    cmake {
        path "CMakeLists.txt"
    }
}
ndkVersion '21.0.6113669'

}

dependencies {
// implementation fileTree(dir: 'libs', include: ['*.jar'])
// implementation 'com.android.support:appcompat-v7:28.0.0'
// implementation 'com.android.support.constraint:constraint-layout:1.1.3'
// testImplementation 'junit:junit:4.12'
// androidTestImplementation 'com.android.support.test:runner:1.0.2'
// androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.2'
// implementation 'com.android.support:design:28.0.0'
// implementation 'com.android.support:cardview-v7:28.0.0'

implementation fileTree(include: ['*.jar'], dir: 'libs')
androidTestImplementation('com.android.support.test.espresso:espresso-core:3.0.2', {
    exclude group: 'com.android.support', module: 'support-annotations'
})
implementation 'com.android.support:appcompat-v7:28.0.0'
implementation 'com.android.support:support-v4:28.0.0'
implementation 'com.google.firebase:firebase-analytics:17.4.3'
//implementation 'com.google.gms:google-services:4.2.0'
//implementation 'com.google.firebase:firebase-core:16.0.7'
testImplementation 'junit:junit:4.12'
implementation 'com.google.firebase:firebase-appindexing:10.2.1'

}
//apply plugin: 'com.google.gms.google-services' `

** My Project Gradle file **
` // Top-level build file where you can add configuration options common to all sub-projects/modules.

buildscript {
repositories {
jcenter()
maven {
url 'https://maven.google.com/'
name 'Google'
}
}
dependencies {
classpath 'com.android.tools.build:gradle:4.0.0'
classpath 'com.google.gms:google-services:4.3.3'
//classpath 'com.google.gms:google-services:4.2.0'

    // NOTE: Do not place your application dependencies here; they belong
    // in the individual module build.gradle files
}

}

allprojects {
repositories {
jcenter()
maven {
url 'https://maven.google.com/'
name 'Google'
}
}
}

allprojects {
repositories {
google()
jcenter()
}
}
task clean(type: Delete) {
delete rootProject.buildDir
} `

@jackweiwang could you please look into it. Thank You

do you encounter this problem? how to solve this? thank you

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x28
2021-04-01 18:20:15.426 5703-5703/? A/DEBUG: Cause: null pointer dereference
2021-04-01 18:20:15.426 5703-5703/? A/DEBUG: x0 000000000000003e x1 000000789b534921 x2 0000000000000001 x3 6552007070632e72
2021-04-01 18:20:15.426 5703-5703/? A/DEBUG: x4 000000789b534922 x5 b4000078c33d50f4 x6 000000000000000a x7 000000000000000a
2021-04-01 18:20:15.426 5703-5703/? A/DEBUG: x8 b4000078c3695390 x9 b4000078c36953a0 x10 0000000000004001 x11 0000000000000000
2021-04-01 18:20:15.426 5703-5703/? A/DEBUG: x12 0000000000000000 x13 0000000000000000 x14 00000079b1ec8be2 x15 0000201afe7fcb66
2021-04-01 18:20:15.426 5703-5703/? A/DEBUG: x16 00000079b1ec4930 x17 00000079b1eb97fc x18 00000078b222e000 x19 0000000000000000
2021-04-01 18:20:15.426 5703-5703/? A/DEBUG: x20 0000000000000000 x21 00000078b96b3fe8 x22 00000078b96b6000 x23 b4000079113e34b8
2021-04-01 18:20:15.426 5703-5703/? A/DEBUG: x24 0000007910e89238 x25 00000078b96b6000 x26 000000000000001f x27 0000000000000005
2021-04-01 18:20:15.426 5703-5703/? A/DEBUG: x28 00000078b96b4250 x29 00000078b96b4050
2021-04-01 18:20:15.426 5703-5703/? A/DEBUG: lr 000000789b45b69c sp 00000078b96b3e20 pc 000000789b45b6a0 pst 0000000060001000
2021-04-01 18:20:15.471 603-603/? E/SELinux: avc: denied { find } for pid=7538 uid=10206 name=tethering scontext=u:r:vendor_systemhelper_app:s0:c512,c768 tcontext=u:object_r:tethering_service:s0 tclass=service_manager permissive=0
2021-04-01 18:20:15.621 4916-6674/? E/MiuiFastConnectService: check adv data error
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: backtrace:
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #00 pc 00000000000356a0 /data/app/~~uqU_6wx4vThePgo6c-I9hg==/com.timehut.ailab-PjN-FEtLVJMiUKGBnDpimA==/lib/arm64/libMNN.so (MNN::Interpreter::resizeTensor(MNN::Tensor*, std::__ndk1::vector<int, std::__ndk1::allocator > const&)+52) (BuildId: 28bed86e363f251ace0f36b236592180a7046cfd)
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #1 pc 0000000000019108 /data/app/~~uqU_6wx4vThePgo6c-I9hg==/com.timehut.ailab-PjN-FEtLVJMiUKGBnDpimA==/lib/arm64/libfacedetect.so (Inference_engine::infer_img(unsigned char*, int, int, int, int, int, Inference_engine_tensor&)+240) (BuildId: c0ffa6de2baa09966a554d5df99764ed7bed9d95)
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #2 pc 000000000000f930 /data/app/~~uqU_6wx4vThePgo6c-I9hg==/com.timehut.ailab-PjN-FEtLVJMiUKGBnDpimA==/lib/arm64/libfacedetect.so (UltraFace::detect(unsigned char*, int, int, int, std::__ndk1::vector<FaceInfo, std::__ndk1::allocator >&)+192) (BuildId: c0ffa6de2baa09966a554d5df99764ed7bed9d95)
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #3 pc 000000000000b5dc /data/app/~~uqU_6wx4vThePgo6c-I9hg==/com.timehut.ailab-PjN-FEtLVJMiUKGBnDpimA==/lib/arm64/libfacedetect.so (Java_com_facesdk_FaceSDKNative_FaceDetect+412) (BuildId: c0ffa6de2baa09966a554d5df99764ed7bed9d95)
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #4 pc 000000000013ced4 /apex/com.android.art/lib64/libart.so (art_quick_generic_jni_trampoline+148) (BuildId: e8a61841e9d1ad33244bcfb65974940b)
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #5 pc 0000000000133564 /apex/com.android.art/lib64/libart.so (art_quick_invoke_stub+548) (BuildId: e8a61841e9d1ad33244bcfb65974940b)
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #6 pc 00000000001a8a78 /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+200) (BuildId: e8a61841e9d1ad33244bcfb65974940b)
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #7 pc 0000000000318f34 /apex/com.android.art/lib64/libart.so (art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread*, art::ArtMethod*, art::ShadowFrame*, unsigned short, art::JValue*)+376) (BuildId: e8a61841e9d1ad33244bcfb65974940b)
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #8 pc 000000000030f260 /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*)+996) (BuildId: e8a61841e9d1ad33244bcfb65974940b)
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #9 pc 000000000067d104 /apex/com.android.art/lib64/libart.so (MterpInvokeVirtual+848) (BuildId: e8a61841e9d1ad33244bcfb65974940b)
2021-04-01 18:20:15.665 5703-5703/? A/DEBUG: #10 pc 000000000012d814 /apex/com.android.art/lib64/libart.so (mterp_op_invoke_virtual+20) (BuildId: e8a61841e9d1ad33244bcfb65974940b)

用的小米10 Android 11

我从ocrlitemnn中把mnn模型拿过来，为啥crnn不出结果呢？
https://github.com/thomaszheng/OcrLiteMnn
模型和 libmnn.so 有强匹配关系吗？

When tried to integrate in the other sample app, getting this error.

Hi,

I successfully ran on android . Its detecting from picture.

Is it possible to open Camera and detect realtime ?

Best

使用RFB量化模型测试的时候检测框基本没有效果，我想问下您有试过量化模型吗

I did not successfully tried it on RK3399. Do you think if I need to re-compile libMNN on the new device?

Hi,
I made facesdk directory under /storage/emulated/0/ and copied RFB-320-quant-ADMM-32.mnn file into the "facesdk".

but MNN::Interpreter::createFromFile(file.c_str()) returns null and application is terminated with following error message.

--------- beginning of crash

A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x4e in tid 17530 (ample.testcmake)

what should i do to solve this problem?
could anyone give me advice?
thanks

Hey Is there any way for direct pass NV21 imageFormat from Camera API?
Because conversion from NV21 to RBGA taking much time in Real Time Face Detection.

这个代码里所给出的模型中，只有经过量化的模型是能够得到正确结果的，当我加入人脸识别算法的inference时，识别算法的模型同样需要量化，量化之后就可以得到正确结果。查看人脸检测的inference过程，发现图片数据经过网络之后得到的结果值全都为0.所以想问一下作者，代码中是有一些特别需要注意的地方吗？

Can you help me to solve this error?