An Augmented Reality Android App, made using Unity (C#) and OpenCV, to implement virtual trials of ornamental rings.

The hand and fingers detection is based on skin color detection. The application is initialized by sampling color from the hand of the user with the help of seven square color trackers or ROIs.

The hand, the foreground, is then extracted from the background by thresholding with the sampled color profile. Each of the seven color in the profile produces a binary segmentation image. Then, all these seven binary images are added up to form a single binary image. This binary image is made smooth and denoised using image median blurring filters.

This procedure helps in finding the ring finger of the user and getting the appropriate information required to place the ring in the next stage.

But, this procedure produces some noisy convexity defects too. To filter out these false convexity defects, we set the following constraints:

1. The length or depth of the defect (L) > L_bb / 5.5
2. The angle between the lines going from the defect to the neighboring convex polygon vertices (α) < 95

Now, to finally place the ring on the user's finger, the coordinates of the two leftmost convexity defects are found.

The following 3 transformational components are then set and passed to Unity,

1. Position: The ring is placed on the midpoint of the points P1 and P2
2. Rotation: The ring is rotated by an angle equal to the angle made by the line P1P2 with the horizontal axis
3. Scale: The scale of the ring is set according to the length of the line segment P1P2

The Unity Game Engine enables us to add a third dimension to the 2D input of the camera. A Scene is set in Unity as follows:

The Scene consists of the following components:

1. Main Camera – Captures the scene and displays the final output
2. Plane - Acts as a screen to display the input camera feed
3. Directional Light – Illuminates the scene
4. Rings – 5 3D ring models (Imported into Unity)

The rings are referenced as Game Objects and are transformed (position, rotation and scale) according to the data received from the 1st phase. This finally places the ring model on the user's finger.

• Unity 2018.1.2f1
• OpenCV

This project is licensed under the MIT License - see the LICENSE file for details.