The setup was tested only for Windows 10. If you are using some other system you are on your own.
Supported browsers:
- Chrome (Chrome has a strict policy regarding voice output. To enable TTS feature click left mouse btn once the website is loaded)
- Edge
- Follow
SETUP LEAP MOTION
- Open Edge / Chrome (As of writting this doc (01-06-2021) Edge has way nicer TTS voice)
- Go to
https://kind-plant-0d14acb03.azurestaticapps.net/
- Accept permissions to use microphone
- (Chrome only - click left mouse button)
- Test by saying
start
Additional notes:
For better speach recognition use headphones
(v - voice command, g - gesture)
v: select <phrase> - select phrase in text, stop reading, switch to SELECTED context
v: stop - stop reading
v: start / repeat - jump to the beginning of the current sentence (or jump to the previous sentence if cursor is at the beginning of sentence)
v: restart - jump to the beginning of the text
v: undo - undo the last change
g: wave in - jump to the beginning of the current sentence (or jump to the previous sentence if cursor is at the beginning of sentence)
g: stop - stop reading
g: swipe_left - undo the last change
v: stop - unselect the prase, switch to DEFAULT context, resume reading
v: delete - delete selected phrase, switch to DEFAULT context, resume reading
v: replace - switch to REPLACE context
v: insert before / before - switch to INSERT_BEFORE context
v: insert after / after - switch to INSERT_AFTER context
g: fist_and_throw_away - delete selected phrase, switch to DEFAULT context, resume reading
g: hand_upward - switch to REPLACE context
g: index_finger_left - switch to INSERT_BEFORE context
g: index_finger_right - switch to INSERT_AFTER context
v: stop - unselect the prase, switch to DEFAULT context, resume reading
v: <phrase> - insert the phrase before the previously selected text, switch to DEFAULT context, resume reading
v: stop - unselect the prase, switch to DEFAULT context, resume reading
v: <phrase> - insert the phrase after the previously selected text, switch to DEFAULT context, resume reading
v: stop - unselect the prase, switch to DEFAULT context, resume reading
v: <phrase> - replace the previously selected text with the new prase, switch to DEFAULT context, resume reading
- Download Leap Motion Developer Kit - ORION 3.2.1 (it will require registartion)
https://developer.leapmotion.com/releases/leap-motion-orion-321-39frn-3b659
- Connect Leap Motion to your laptop
- Open Leap Motion Controller Panel
- Test your connection in
Visualizer
(right click Leap Motion Controller in taskbar >Visualizer...
) - Turn on
Allow Web Apps
(right click Leap Motion Controller in taskbar >Settings...
)
- Install node.js (
https://nodejs.org/en/download/
) - Install dependencies (
npm install
inside project's root folder) - Start (
npm start
) - In your browser (Chrome / Edge) open
http://localhost:3000/
Leap Motion API reference:
https://developer-archive.leapmotion.com/documentation/javascript/api/Leap.Hand.html
Contact:
Patryk Pomykalski
[email protected]