Finger Controller uses a hand-character model as a natural input for video games. In this model, the player's hand represents their character in the video game, where the index and middle finger serve as the character's legs, and the rest of the hand acts as a large upper-body. Finger Controller is implemented with a single RGB camera.

See Finger Controller in action with Super Mario Bros. in this video!

In (a), we see the hand-character model, where the character's legs are boxed in green, and the character's body is boxed in magenta. Cells (b), (c), and (d) show some actions including idling, walking, and jumping, respectively. Finger Controller currently supports these actions, including walking, and movement directions. This model is most suitable for 2D side-scrolling platformers such as Super Mario Bros. and Sonic the Hedgehog.

Finger Controller currently supports a flexible range of input sources (cameras). The user can choose from a built-in camera, an Android device's camera (over LAN), or a video (for testing purposes). Finger Controller is best used with an Android device's camera because of its flexible placement.

For more information about Finger Controller, please refer to the paper, or look right at the code. Happy gaming!