Code Monkey home page Code Monkey logo

keras-trigger-word's Introduction

Trigger word detection, aka. wake/hot word detection. Like Amazon's "Alexa" or Google Home's "OK, Google" to wake them up. Will it be cool to build one yourself and run it in Real-time?

In this post, I am going to show you exactly how to build a Keras model to do the same thing from scratch. No third party voice API or network connection required to make it functional.

Background information is shown in my blog post.

How to Run

Require Python 3.5+ and Jupyter notebook installed

Clone or download this repo

git clone https://github.com/Tony607/Keras-Trigger-Word

Install required libraries

pip3 install -r requirements.txt

Real-time demo

In the project directory start a command line, then run command

jupyter notebook

If you are only interested in playing with the pre-trained trigger word model with real-time demo. In the opened browser window choose

trigger_word_real_time_demo.ipynb

Optionally if you want to learn about data preparation and model training. Continue on with my write up. In the opened browser window choose this notebook.

Trigger word detection - v1.ipynb

Download the train/dev Data from the releases if you want to follow along the notebook, Data.zip. Extract XY_dev and XY_train folders to the root of the project directory.

Happy coding! Leave a comment if you have any question.

keras-trigger-word's People

Contributors

tony607 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

keras-trigger-word's Issues

reset_after error

Hi @Tony607 when I tried loading the tr_model.h5 saved in the models file, I got an error as shown below,
GRU(reset_after=False) is not compatible with GRU(reset_after=True)
image

License

Hi Tony,

Could you provide a license in this repo?

Kernel dying

Hi,
I'm trying to test your "trigger_word_real_time_demo" notebook.
At this part of the code:

stream = get_audio_input_stream(callback)

The kernel dies and gets restarted, I can't see any error not in the jupyter console, nor in the notebook itself.
I've also tried to export a .py and run it from my console but I only get "segmentation fault (core dumped)", in the same line.

I know that this information is not much, maybe you can help me troubleshot this problem.
Thanks for your help in advance.

Mattia

EDIT:
Investigating the problem, it seems that I'm missing a default_input_device, therefore when your program tries to access the device list, using "input_device_index=0", it goes into segmentation fault, since the list is empty.
I'm not the only one encountering this problem, there are numerous cases like mine on stack overflow.
https://stackoverflow.com/questions/42404408/pyaudio-cannot-find-any-output-devices, this is just one of many example I've found, unfortunately i still couldn't find a way to make it work.

How many samples required for custom keyword

Hi I have created the dataset for my own custom key word with 15 backgrounds and some 60 keys and 60 negatives.I have created 4000 samples with randomly overlaying the keys over the background.I ran some 500 epochs starting from scratch and the accuracy is0.77. Its not even recognizing my keyword.
So I tried to create huge dataset of about 40000 samples in ubuntu instance of 32GB RAM,It ran out of memory?
How many samples do I need ?
If large samples are required what file format should I use ?

How to generate X,Y dataset?

How do you create X,Y dataset for training, should we add arrays of voices and then save it as that or should we create an array file for train.wav?

got some confused result

figure_1
i retrain this project background number is 522(home+factort+street noise), 24 activate audio and 24negatigve audio .
the train log is as foloow:
522/522 [==============================] - 72s 138ms/step - loss: 0.0530 - acc: 0.9998
Epoch 19/20
522/522 [==============================] - 72s 138ms/step - loss: 0.0524 - acc: 0.9998
Epoch 20/20
522/522 [==============================] - 72s 138ms/step - loss: 0.0519 - acc: 0.9998
537/537 [==============================] - 17s 32ms/step
could pelase give me some guide or suggestion to train the model to a success trigger?
or could you give me some guide on making training data set?
@Tony607

Require actual large dataset

Hello sir,
From where can I get the actual large dataset of 4000 samples so that I can fit the model by myself only.

Device is detected in the system why not in the code?!

So I am using a dedicated USB mic for RPI. It is detected in the computer and all able to record audio in browser and in local software. But for some reason I get this:

/home/harshil/Downloads/kerass/try1/lib/python3.6/site-packages/keras/models.py:318: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
warnings.warn('Error in loading the saved optimizer '
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm_route.c:867:(find_matching_chmap) Found no matching channel map

And there are other usual tensorflow warnings but they are not the issue I guess. Please can someone tell me what can i do to overcome this???

.

.

Not able to change random range in cell 23

I tried changing the random range for my negatives and activates from 0, 4 to 0, 9 it worked fine but when I'm changing it to 0, 11 it just get stuck there (for 0, 9 it works very fast but for 0, 11 it just get stuck)

How can we create confusion matrix?

I am trying to make my custom model. And I want to generate a confusion matrix for that as accuracy is not a good measure for this data set.
I am trying it by following code.

from sklearn.metrics import confusion_matrix
cm=confusion_matrix(X_dev,Y_dev)

But getting following error:-

ValueError                                Traceback (most recent call last)
<ipython-input-10-0dcf84d46f09> in <module>
----> 1 cm=confusion_matrix(X_dev,Y_dev)

~/anaconda3/lib/python3.7/site-packages/sklearn/metrics/classification.py in confusion_matrix(y_true, y_pred, labels, sample_weight)
    251 
    252     """
--> 253     y_type, y_true, y_pred = _check_targets(y_true, y_pred)
    254     if y_type not in ("binary", "multiclass"):
    255         raise ValueError("%s is not supported" % y_type)

~/anaconda3/lib/python3.7/site-packages/sklearn/metrics/classification.py in _check_targets(y_true, y_pred)
     86     # No metrics support "multiclass-multioutput" format
     87     if (y_type not in ["binary", "multiclass", "multilabel-indicator"]):
---> 88         raise ValueError("{0} is not supported".format(y_type))
     89 
     90     if y_type in ["binary", "multiclass"]:

ValueError: unknown is not supported

Will someone help me for this.

loading model doesn't work.

Hi !
I would like to test the demo file.
but loading model doesn't work !

model = load_model('./models/tr_model.h5')

ValueError Traceback (most recent call last)
in
----> 1 model = load_model('./models/tr_model.h5')

~\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\saving\save.py in load_model(filepath, custom_objects, compile, options)
180 if (h5py is not None and (
181 isinstance(filepath, h5py.File) or h5py.is_hdf5(filepath))):
--> 182 return hdf5_format.load_model_from_hdf5(filepath, custom_objects, compile)
183
184 filepath = path_to_string(filepath)

~\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\saving\hdf5_format.py in load_model_from_hdf5(filepath, custom_objects, compile)
179
180 # set weights
--> 181 load_weights_from_hdf5_group(f['model_weights'], model.layers)
182
183 if compile:

~\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\saving\hdf5_format.py in load_weights_from_hdf5_group(f, layers)
706 str(len(weight_values)) + ' elements.')
707 weight_value_tuples += zip(symbolic_weights, weight_values)
--> 708 K.batch_set_value(weight_value_tuples)
709
710

~\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\util\dispatch.py in wrapper(*args, **kwargs)
199 """Call target, and fall back on dispatchers if there is a TypeError."""
200 try:
--> 201 return target(*args, **kwargs)
202 except (TypeError, ValueError):
203 # Note: convert_to_eager_tensor currently raises a ValueError, not a

~\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\backend.py in batch_set_value(tuples)
3574 if ops.executing_eagerly_outside_functions():
3575 for x, value in tuples:
-> 3576 x.assign(np.asarray(value, dtype=dtype(x)))
3577 else:
3578 with get_graph().as_default():

~\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\ops\resource_variable_ops.py in assign(self, value, use_locking, name, read_value)
856 with _handle_graph(self.handle):
857 value_tensor = ops.convert_to_tensor(value, dtype=self.dtype)
--> 858 self._shape.assert_is_compatible_with(value_tensor.shape)
859 assign_op = gen_resource_variable_ops.assign_variable_op(
860 self.handle, value_tensor, name=name)

~\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\framework\tensor_shape.py in assert_is_compatible_with(self, other)
1132 """
1133 if not self.is_compatible_with(other):
-> 1134 raise ValueError("Shapes %s and %s are incompatible" % (self, other))
1135
1136 def most_specific_compatible_shape(self, other):

ValueError: Shapes (2, 384) and (384,) are incompatible

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.