JuliusJS
A speech recognition library for the web
Try the live demo.
JuliusJS is an opinionated port of Julius to JavaScript.
It actively listens to the user to transcribe what they are saying through a callback.
// bootstrap JuliusJS
var julius = new Julius();
julius.onrecognition = function(sentence) {
console.log(sentence);
};
// say "Hello, world!"
// console logs: `> HELLO WORLD`Features:
- Real-time transcription
- Use the provided grammar, or write your own
- 100% JavaScript implementation
- All recognition is done in-browser through a
Worker - Familiar event-inspired API
- No external server calls
Quickstart
Using Express 4.0
- Grab the latest version with bower
bower install juliusjs --save
- Include
julius.jsin your html
<script src="julius.js"></script>
- Make the scripts available to the client through your server
var express = require('express'),
app = express();
app.use(express.static('path/to/dist'));- In your main script, bootstrap JuliusJS and register an event listener for recognition events
// bootstrap JuliusJS
var julius = new Julius();
// register listener
julius.onrecognition = function(sentence, score) {
// ...
console.log(sentence);
};- Your site now has real-time speech recognition baked in!
Configure your own recognition grammar
In order for JuliusJS to use it, your grammar must follow the Julius grammar specification. The site includes a tutorial on writing grammars.
By default, phonemes are defined in voxforge/hmmdefs, though you might find other sites more useful as reference.
- Building your own grammar requires the
mkdfa.plscript and associated binaries, distributed with Julius. - On Mac OS X
- Use
./bin/mkdfa.pl, included with this repo
- Use
- On other OS
- Run
emscripten.shto populatebinwith the necessary files
- Run
- Write a
yourGrammar.vocafile with words to be recognized
- The
.vocafile defines "word candidates" and their pronunciations.
- Write a
yourGrammar.grammarfile with phrases composed of those words
- The
.grammarfile defines "category-level syntax, i.e. allowed connection of words by their category name."
- Compile the grammar using
./bin/mkdfa.pl yourGrammar
- The
.vocaand.grammarmust be prefixed with the same name - This will generate
yourGrammar.dfaandyourGrammar.dict
- Give the new
.dfaand.dictfiles to theJuliusconstructor
// when bootstrapping JuliusJS
var julius = new Julius('path/to/dfa', 'path/to/dict');Advanced Use
Configuring the engine
The Julius constructor takes three arguments which can be used to tune the engine:
new Julius('path/to/dfa', 'path/to/dict', options)Both 'path/to/dfa' and 'path/to/dict' must be set to use a custom grammar
'path/to/dfa'
- path to a valid
.dfafile, generated as described above - if left
null, the default grammar will be used
'path/to/dict'
- path to a valid
.dictfile, generated as described above - if left
null, the default grammar will be used
options
options.verbose- iftrue, JuliusJS will log to the consoleoptions.stripSilence- iftrue, silence phonemes will not be included in callbackstrueby defaultoptions.transfer- iftrue, captured microphone input will be piped to your speakers- this is mostly useful for debugging
options.*- Julius supports a wide range of options. Most of these are made available here, by specifying the flag name as a key. For example:
options.zc = 30will lower the zero-crossing threshold to 30.
Some of these options will break JuliusJS, so use with caution. - A reference to available options can be found in the JuliusBook.
- Currently, the only supported hidden markov model is from voxforge. The
handhlistoptions are unsupported.
Examples
Voice Command
Coming soon...
Keyword Spotting (e.g., API integration)
Coming soon...
In the wild
If you use JuliusJS let me know, and I'll add your project to this list (or issue a pull request yourself).
- Coming soon...
Motivation
- Implement speech recognition in...
- 100% JavaScript - no external dependencies
- A familiar and easy-to-use context
- Follow standard eventing patterns (e.g.,
onrecognition)
- Follow standard eventing patterns (e.g.,
- As far as accessibility, allow...
- Out-of-the-box use
- Minimal barrier to use
- This means commited sample files (e.g. commited emscripted library)
- Minimal configuration
- Real-time (opinionated) use only
- Hide mfcc/wav/rawfile configurations
- Real-time (opinionated) use only
- Minimal barrier to use
- Useful examples (not so much motivation, as my motivating goals)
- Voice command
- Keyword spotting
Future goals
- Better sample recognition grammar (improves out-of-the-box usability)
- Examples
Developers
Contributions are welcome! See CONTRIBUTING.md for guidelines.
Build from source
You'll need emscripten, the LLVM to JS compiler, to build this from the C source. Once you have that, run ./emscript.sh. If you are missing other tools, the script will let you know.
As emscript.sh reloads and recompiles static libraries, ./reemscript.sh is available once you've already run emscript.sh. reemscript.sh will only recompile to JavaScript based on your latest changes. This can also be run with npm make.
Additionally, tests are set will be made to run using npm test.
In the meantime, a blank page with the JuliusJS library can be served using npm start.
Codemap
emscript.sh / reemscript.sh
These scripts will compile/recompile Julius C source to JavaScript, as well as copy all other necessary files, to the js folder.
emscript.sh will also compile binaries, which you can use to create recognition grammars or compile grammars to smaller binary files. These are copied to the bin folder.
src
This is where the source for Julius will go once emscript.sh is run. emscript.sh will replace certain files in src/julius4 with those in src/include in order to make src/emscripted, the files eventually compiled to JavaScript.
- src/include/julius/app.h - the main application header
- src/include/julius/main.c - the main application
- src/include/julius/recogloop.c - a wrapper around the recognition loop
- src/include/libjulius/src/adin_cut.c - interactions with a microphone
- src/include/libjulius/src/m_adin.c - initialization to Web Audio
- src/include/libjulius/src/recogmain.c - the main recognition loop
- src/include/libsent/configure[.in] - configuration to add Web Audio
- src/include/libsent/src/adin/adin_mic_webaudio.c - input on Web Audio
Files in bold were changed to replace a loop with eventing, to simulate multithreading in a Worker.
js
The home to the testing server run with npm start. Files are copied to this folder from dist with emscript.sh and reemscript.sh. If they are modified, they should be commited back to the dist folder.
dist
The home for committed copies of the compiled library, as well as the wrappers that make them work: julius.js and worker.js. dist/listener/converter.js is the file that actually pipes Web Audio to Julius (the compiled C program).
JuliusJS is a port of the "Large Vocabulary Continuous Speech Recognition Engine Julius" to JavaScript
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent