ofshellohicy / wechat-miniprogram-webar Goto Github PK

1. Chinese README

2. Chinese Source Code Analysis

This is a WeChat Web AR Demo. On July 5, 2019, WeChat miniprogram supports AR. It has been added a new API named "CameraFrameListener".

CameraFrameListener API

We can create AR effects with the new API. This demo demonstrates a color tracker effect using "tracking.js" and "jsfeat" library.

The "tracking.js" brings computer vision algorithms and techniques into browser environment. We can do real-time color tracking, face detection and much more.

tracking.js

The "jsfeat" is a JavaScript computer vision library.

JSFeat

The Demo includes color tracker, face tracker, image tracker and object tracker. There are two modes for each item. It includes 'taking a photo' and 'access a camera'.

Use the Demo to scan people's face.

Expect the effect below.

Use "Take a Photo" mode.

When face recognition is not exact, the result is not good.

When face recognition is exact, the result is good.

a sample pattern picture is below.

Use the Demo to scan a rotate picture below.

Expected:

Use the Demo to scan a skew picture below.

Expected:

Use the Demo to scan a translate and scale picture below.

Expected:

Use the Demo to scan a perspective picture below.

Expected:

Use the Demo to scan the picture below.

Expect the effect below.

Use "Take a Photo" mode.

Use "Access a camera" mode.

There will be a rect aboves people's mouth. It is slow and not better than face tracker.

Face tracker is slow, but we can do some thing to improve speed. For example, we can blur image, grayscale image, sobel image, compress image and so on. After testing, we found that reducing image size is more proper. When image size is reduced, the parameters of tracker needs to be updated.

frame size of camera: the image size is smaller, the tracker's speed is faster.

const frameWidth = 150;

face tracker parameter：the "initialScale" is bigger, the face tracker's speed is faster.

const initialScale = 2;

color tracker parameter：the "minDimension" is bigger, the color tracker's speed is faster.

const minDimension = 4;

interval time: interval time should be greater than cost time. The unit is milliseconds.

const intervalTime = 350;

Image tracker and face tracker are very slow on iOS WeChat.