Code Monkey home page Code Monkey logo

multimediaproject's Introduction

Introduction

This is an Android app that performs face recognition and identification through the smartphone's camera. The app uses an internal database containing the known identities, including sample photos. The app allows the user to check the database content, delete or add new identities.

Implementation overview

  • Face detection using Haar Cascade Classifier
  • Face features exctracted using VGG2
  • Classification performed with weighted kNN

OpenCV library was used for data processing.

For more details, see the presentation.

Cannot find the caffe model?

If you get errors similar to Can't open "/data/user/0/melonizippo.org.facerecognition/cache/caffe_model/resnet50_ft_caffe/resnet50_ft.caffemodel", then you must first extract the 7z multipart file called resnet50_ft.7z.*** in the path src/main/assets/caffe_model/resnet50_ft_caffe/.

multimediaproject's People

Contributors

enricomeloni avatar rzippo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

princeamoako

multimediaproject's Issues

App crashing when using LightenedCNN_A model

I want to have a light model (light in terms of size in MB) so I switched the model from resnet50 to LightenedCNN_A model. I had to change the forward() method called from extract() method of DNNExtractor class from

net.forward(outputBlobs, Parameters.DEEP_LAYER);
to
net.forward(outputBlobs);

But I am facing 2 problems with these changes-

  1. I am getting "No faces detected" although the white box around face is showing accurately. Thus, nothing is being saved under "unspecified".
  2. When I add an identity from gallery or capture from video and when I click on "Save Identity" button, the app crashes.

Here is the stacktrace for the crash-

E/cv::error(): OpenCV(3.4.1) Error: Assertion failed (srcMat.dims == 2 && srcMat.cols == weights.cols && dstMat.rows == srcMat.rows && dstMat.cols == weights.rows && srcMat.type() == weights.type() && weights.type() == dstMat.type() && srcMat.type() == 5 && (biasMat.empty() || (biasMat.type() == srcMat.type() && biasMat.isContinuous() && (int)biasMat.total() == dstMat.cols))) in static void cv::dnn::FullyConnectedLayerImpl::FullyConnected::run(const cv::Mat&, const cv::Mat&, const cv::Mat&, cv::Mat&, const cv::dnn::experimental_dnn_v4::ActivationLayer*, int), file /build/master_pack-android/opencv/modules/dnn/src/layers/fully_connected_layer.cpp, line 154 2019-08-14 18:10:39.017 24057-24174/melonizippo.org.facerecognition E/org.opencv.dnn: dnn::forward_13() caught cv::Exception: OpenCV(3.4.1) /build/master_pack-android/opencv/modules/dnn/src/layers/fully_connected_layer.cpp:154: error: (-215) srcMat.dims == 2 && srcMat.cols == weights.cols && dstMat.rows == srcMat.rows && dstMat.cols == weights.rows && srcMat.type() == weights.type() && weights.type() == dstMat.type() && srcMat.type() == 5 && (biasMat.empty() || (biasMat.type() == srcMat.type() && biasMat.isContinuous() && (int)biasMat.total() == dstMat.cols)) in function static void cv::dnn::FullyConnectedLayerImpl::FullyConnected::run(const cv::Mat&, const cv::Mat&, const cv::Mat&, cv::Mat&, const cv::dnn::experimental_dnn_v4::ActivationLayer*, int) 2019-08-14 18:10:39.019 24057-24174/melonizippo.org.facerecognition E/AndroidRuntime: FATAL EXCEPTION: Thread-4 Process: melonizippo.org.facerecognition, PID: 24057 CvException [org.opencv.core.CvException: cv::Exception: OpenCV(3.4.1) /build/master_pack-android/opencv/modules/dnn/src/layers/fully_connected_layer.cpp:154: error: (-215) srcMat.dims == 2 && srcMat.cols == weights.cols && dstMat.rows == srcMat.rows && dstMat.cols == weights.rows && srcMat.type() == weights.type() && weights.type() == dstMat.type() && srcMat.type() == 5 && (biasMat.empty() || (biasMat.type() == srcMat.type() && biasMat.isContinuous() && (int)biasMat.total() == dstMat.cols)) in function static void cv::dnn::FullyConnectedLayerImpl::FullyConnected::run(const cv::Mat&, const cv::Mat&, const cv::Mat&, cv::Mat&, const cv::dnn::experimental_dnn_v4::ActivationLayer*, int) ] at org.opencv.dnn.Net.forward_3(Native Method) at org.opencv.dnn.Net.forward(Net.java:302) at melonizippo.org.facerecognition.deep.DNNExtractor.extract(DNNExtractor.java:40) at melonizippo.org.facerecognition.AddIdentityActivity.addImage(AddIdentityActivity.java:626) at melonizippo.org.facerecognition.AddIdentityActivity.addImage(AddIdentityActivity.java:605) at melonizippo.org.facerecognition.AddIdentityActivity.addImage(AddIdentityActivity.java:598) at melonizippo.org.facerecognition.AddIdentityActivity.lambda$processImages$10$AddIdentityActivity(AddIdentityActivity.java:533) at melonizippo.org.facerecognition.AddIdentityActivity$$Lambda$8.run(Unknown Source:20) at java.lang.Thread.run(Thread.java:764)

I am missing something? Some change I am failing to do to make this model work for face detection and saving the identity?

Getting error- "Can't open "/data/user/0/melonizippo.org.facerecognition/cache/caffe_model/resnet50_ft_caffe/resnet50_ft.caffemodel""

Hi, I am running the project as it is from android studio and getting an exception about not finding 'resnet50_ft.caffemodel'

Here is the stacktrace-

E/cv::error(): OpenCV(3.4.1) Error: Unspecified error (FAILED: fs.is_open(). Can't open "/data/user/0/melonizippo.org.facerecognition/cache/caffe_model/resnet50_ft_caffe/resnet50_ft.caffemodel") in bool cv::dnn::ReadProtoFromBinaryFile(const char*, google::protobuf::Message*), file /build/master_pack-android/opencv/modules/dnn/src/caffe/caffe_io.cpp, line 1126
2019-08-08 10:41:11.911 5421-5421/? E/org.opencv.dnn: dnn::readNetFromCaffe_10() caught cv::Exception: OpenCV(3.4.1) /build/master_pack-android/opencv/modules/dnn/src/caffe/caffe_io.cpp:1126: error: (-2) FAILED: fs.is_open(). Can't open "/data/user/0/melonizippo.org.facerecognition/cache/caffe_model/resnet50_ft_caffe/resnet50_ft.caffemodel" in function bool cv::dnn::ReadProtoFromBinaryFile(const char*, google::protobuf::Message*)
2019-08-08 10:41:11.915 5421-5421/? E/AndroidRuntime: FATAL EXCEPTION: main
    Process: melonizippo.org.facerecognition, PID: 5421
    java.lang.RuntimeException: Unable to create application melonizippo.org.facerecognition.FaceRecognitionApp: CvException [org.opencv.core.CvException: cv::Exception: OpenCV(3.4.1) /build/master_pack-android/opencv/modules/dnn/src/caffe/caffe_io.cpp:1126: error: (-2) FAILED: fs.is_open(). Can't open "/data/user/0/melonizippo.org.facerecognition/cache/caffe_model/resnet50_ft_caffe/resnet50_ft.caffemodel" in function bool cv::dnn::ReadProtoFromBinaryFile(const char*, google::protobuf::Message*)
    ]
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6112)
        at android.app.ActivityThread.access$1200(ActivityThread.java:212)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1806)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:193)
        at android.app.ActivityThread.main(ActivityThread.java:6912)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:860)
     Caused by: CvException [org.opencv.core.CvException: cv::Exception: OpenCV(3.4.1) /build/master_pack-android/opencv/modules/dnn/src/caffe/caffe_io.cpp:1126: error: (-2) FAILED: fs.is_open(). Can't open "/data/user/0/melonizippo.org.facerecognition/cache/caffe_model/resnet50_ft_caffe/resnet50_ft.caffemodel" in function bool cv::dnn::ReadProtoFromBinaryFile(const char*, google::protobuf::Message*)
    ]
        at org.opencv.dnn.Dnn.readNetFromCaffe_0(Native Method)
        at org.opencv.dnn.Dnn.readNetFromCaffe(Dnn.java:107)
        at melonizippo.org.facerecognition.deep.DNNExtractor.<init>(DNNExtractor.java:26)
        at melonizippo.org.facerecognition.FaceRecognitionApp.initFaceRecognition(FaceRecognitionApp.java:94)
        at melonizippo.org.facerecognition.FaceRecognitionApp.onCreate(FaceRecognitionApp.java:70)
        at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1176)
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6107)
        	... 8 more

Do I have to train the model myself and generate the said file?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.