lzane / fingers-detection-using-opencv-and-python Goto Github PK

View Code? Open in Web Editor NEW

435.0 17.0 178.0 1.37 MB

A simple Fingers Detection (or Gesture Recognition) using OpenCV and Python with background substraction 简单手势识别

Home Page: https://www.lzane.com

License: MIT License

Python 100.00%

opencv python fingers-detection gesture-recognition gesture-detection

fingers-detection-using-opencv-and-python's Introduction

for people using python2 and opencv2, please check out the lzane:py2_opencv2 branch.

for people using opencv4, please change line 96 in the new.py to contours, hierarchy = cv2.findContours(thresh1, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) according to the opencv api change.

Environment

OS: MacOS El Capitan
Platform: Python 3
Librarys:
- OpenCV 3
- appscript

Demo Videos

Youtube: Click here
Youku: Click here

How to run it?

run it in python
press 'b' to capture the background model (Remember to move your hand out of the blue rectangle)
press 'r' to reset the backgroud model
press 'ESC' to exit

Process

Capture original image

Capture video from camera and pick up a frame.

Capture background model & Background subtraction

Use background subtraction method called Gaussian Mixture-based Background/Foreground Segmentation Algorithm to subtract background.

For more information about the method, check Zivkovic2004

Here I use the OpenCV's built-in function BackgroundSubtractorMOG2 to subtract background.

bgModel = cv2.BackgroundSubtractorMOG2(0, bgSubThreshold)

Build a background subtractor model

fgmask = bgModel.apply(frame)

Apply the model to a frame

res = cv2.bitwise_and(frame, frame, mask=fgmask)

Get the foreground(hand) image

Gaussian blur & Threshold

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

First convert the image to gray scale.

blur = cv2.GaussianBlur(gray, (blurValue, blurValue), 0)

By Gaussian blurring, we create smooth transition from one color to another and reduce the edge content.

ret, thresh = cv2.threshold(blur, threshold, 255, cv2.THRESH_BINARY)

We use thresholding to create binary images from grayscale images.

Contour & Hull & Convexity

We now need to find out the hand contour from the binary image we created before and detect fingers (or in other words, recognize gestures)

contours, hierarchy = cv2.findContours(thresh1, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

This function will find all the contours from the binary image. We need to get the biggest contours (our hand) based on their area since we can assume that our hand will be the biggest contour in this situation. (it's obvious)

After picking up our hand, we can create its hull and detect the defects by calling :

hull = cv2.convexHull(res)
defects = cv2.convexityDefects(res, hull)

Now we have the number of fingers. How to use this information? It's based on your imagination...

I add in a keyboard simulation package named appscript as interface to control Chrome's dinosaur game.

References & Tutorials

OpenCV documentation: http://docs.opencv.org/2.4.13/
Opencv python hand gesture recognition: http://creat-tabu.blogspot.com/2013/08/opencv-python-hand-gesture-recognition.html
Mahaveerverma's hand gesture recognition project: hand-gesture-recognition-opencv

fingers-detection-using-opencv-and-python's People

Contributors

Stargazers

Watchers

Forkers

ajrulez hoangnghiem205 ohalkhateeb ggyimah1031 benjamesbabala shenyang70s sourabhmisal hema91 gradient-zero mbencherif karima1212 rotron computer-vision-dhankar-rohit kaumil abioyeayo redragonx kaashmonee culms nikhilm19 shrikantnarayankar15 bthiru005c architsingh15 singulartnt narenvamsi comeonkai prav10194 suavelad subtleseeker splevine arnovandash dengkedu nilay27 nikomateos auserj fitrialif gfishliu zy84338719 shakram02 cupid725 dragonwarri0r orensbruli negativeone1 pujoseno dataphobe anh-ngo-saltovn shubham4u sunnykmarr loyasto balajinataraj gpsbird sujithkanna pandinosaurus nandavikas weltry akhilagp spandy12 kevinkm kansalraj ljhnick suryatmodulus shokosanma loganryanjing adityachamp tinawu-23 hacker0829 zzundark zhoukaii 1005281342 kilbee yanhuizen and9527 habhabhabs ayushkumar07 manassaj handsomeboy samratsukhi2 levrado kmvinayaka chakraroy williredale spectoter sangyy shivamsahu4u bengood362 homerj233 mfarismuzakki hareruya2008 baidu88vip si-lhouette rahulghangas softskillsunion peiyongxia abecedariann cc8848 1164513233 knight-x ohwei en-1999 neubb staceycy

fingers-detection-using-opencv-and-python's Issues

dinosaur appscript source?

Where is this dinosaur appscript control file??

An error occurred after pressing'b'.

line 96, in
_,contours, hierarchy = cv2.findContours(thresh1, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
ValueError: not enough values to unpack (expected 3, got 2)

cannot running as demo

when i press b to get background and n to start,it can running a little time but gradually my hand integrate into backgroud(black).

Error installing appscript(Windows)

Do you suggest any similar libraries like appscript. It is not not installing on windows successfully.

ValueError: not enough values to unpack (expected 3, got 2)

Background is not substracting

Hello, I tried to use your code (only the part of substracting the background) , but seems like nothing is happening. The image i give input, the code simply shows me the same image.

如果空格键之后，键盘会被占用啊

Display the number of fingers

How to display the count of fingers like 1 or 2 or 10

Deprecated code

This line

 bgModel = cv2.BackgroundSubtractorMOG2(0, bgSubThreshold)

should be

cv2.createBackgroundSubtractorMOG2(0, bgSubThreshold)

and

contours, hierarchy = cv2.findContours(thresh1, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

should be

x, contours, y = cv2.findContours(thresh1, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

threshold wll affect accuracy?

Cannot open the window of camera

Hi, when i was test the code you put, the light of the camera becomes green. However, the window is not open. i am new in this area. How could i fix this problem?

Small change to calculateFingers

How about moving the plotting of the circle outside the defand then using the new cavity_pts output to plot the points?

PS. What kind of license you have for your software? Can I use a modified version of this function in a non-commercial research purpose? How do you want yourself to be mentioned if you allow the usage?

def calculateFingers(max_c):
    """
    https://github.com/lzane/Fingers-Detection-using-OpenCV-and-Python/blob/master/new.py#L39-L60
    """
    hull = cv2.convexHull(max_c, returnPoints=False)
    if len(hull) > 3:
        try:
            defects = cv2.convexityDefects(max_c, hull)
        except Exception:
            defects = None
        
        counter = 0
        if defects is not None:
            cavity_pts = []
            for i in range(defects.shape[0]):  # calculate the angle
                s, e, f, d = defects[i][0]
                start = tuple(max_c[s][0])
                end = tuple(max_c[e][0])
                far = tuple(max_c[f][0])
                a = math.sqrt((end[0] - start[0]) ** 2 + (end[1] - start[1]) ** 2)
                b = math.sqrt((far[0] - start[0]) ** 2 + (far[1] - start[1]) ** 2)
                c = math.sqrt((end[0] - far[0]) ** 2 + (end[1] - far[1]) ** 2)
                angle = math.acos((b ** 2 + c ** 2 - a ** 2) / (2 * b * c))  # cosine theorem
                if angle <= math.pi / 2:  # angle less than 90 degree, treat as fingers
                    counter += 1
                    cavity_pts.append(far)
            return counter, cavity_pts
    return 0, []