Code Monkey home page Code Monkey logo

stream-audio-fingerprint's Introduction

Adblock Radio (project archived)

Adblock Radio

A library to block ads on live radio streams and podcasts. Machine learning meets Shazam.

Engine of AdblockRadio.com. Demo standalone player available here.

Build status: CircleCI

Help the project grow: Donate using Liberapay

Overview

A technical discussion is available here.

Radio streams are downloaded in predictor.js with the module adblockradio/stream-tireless-baler. Podcasts are downloaded in predictor-file.js.

In both cases, audio is then decoded to single-channel, 22050 Hz PCM with ffmpeg.

Chunks of about one second of PCM audio are piped into two sub-modules:

  • a time-frequency analyser (predictor-ml/ml.js), that analyses spectral content with a neural network.
  • a fingerprint matcher (predictor-db/hotlist.js), that searches for exact occurrences of known ads, musics or jingles.

In post-processing.js, results are gathered for each audio segment and cleaned.

A Readable interface, Analyser, is exposed to the end user. It streams objects containing the audio itself and all analysis results.

On a regular laptop CPU and with the Python time-frequency analyser, computations run at 5-10X for files and at 10-20% usage for live stream.

Getting started

Installation

Mandatory prerequisites:

You need Node.js (>= v10.12.x, but < 11) and NPM. Download it here. Pro-tip: to manage several node versions on your platform, use NVM.

On Debian Stretch:

apt-get install -y git ssh tar gzip ca-certificates build-essential sqlite3 ffmpeg

Note: works on Jessie, but installing ffmpeg is a bit painful. See here and there.

Optional prerequisites:

For best performance (~2x speedup) you should choose to do part of the computations with Python. Additional prerequisites are the following: Python (tested with v2.7.9), Keras (tested with v2.0.8) and Tensorflow (tested with CPU v1.4.0 and GPU v1.3.0).

On Debian:

apt-get install python-dev portaudio19-dev
pip install python_speech_features h5py numpy scipy keras tensorflow zerorpc sounddevice psutil

Note: if you do not have pip follow these instructions to install it.

Then install this module:
git clone https://github.com/adblockradio/adblockradio.git
cd adblockradio
npm install

Testing

Validate your installation with the test suite:

npm test

Command-line demo

At startup and periodically during runtime, filter configuration files are automatically updated from adblockradio.com/models/:

  • a compatible machine-learning model (model.keras or model.json + group1-shard1of1), for the time-frequency analyser.
  • a fingerprint database (hotlist.sqlite), for the fingerprint matcher.

Live stream analysis

Run the demo on French RTL live radio stream:

node demo.js

Here is a sample output of the demo script, showing an ad detected:

{
	"gain": 74.63,
	"ml": {
		"class": "0-ads",
		"softmaxraw": [
			0.996,
			0.004,
			0
		],
		"softmax": [
			0.941,
			0.02,
			0.039
		],
		"slotsFuture": 4,
		"slotsPast": 5
	},
	"hotlist": {
		"class": "9-unsure",
		"file": null,
		"matches": 1,
		"total": 7
	},
	"class": "0-ads",
	"metadata": {
		"artist": "Laurent Ruquier",
		"title": "L'été des Grosses Têtes",
		"cover": "https://cdn-media.rtl.fr/cache/wQofzw9SfgHNHF1rqJA3lQ/60v73-2/online/image/2014/0807/7773631957_laurent-ruquier.jpg"
	},
	"streamInfo": {
		"url": "http://streaming.radio.rtl.fr/rtl-1-44-128",
		"favicon": "https://cdn-static.rtl.fr/versions/www/6.0.637/img/apple-touch-icon.png",
		"homepage": "http://www.rtl.fr/",
		"audioExt": "mp3"
	},
	"predictorStartTime": 1531150137583,
	"playTime": 1531150155250,
	"tBuffer": 15.98,
	"audio": ...
}

Podcast analysis

It is also possible to analyse radio recordings. Run the demo on a recording of French RTL radio, including ads, talk and music:

node demo-file.js

Gradual outputs are similar to those of live stream analysis. An additional post-processing specific to recordings hides the uncertainties in predictions and shows big chunks for each class, with time stamps in milliseconds, making it ready for slicing.

[
	{
		"class": "1-speech",
		"tStart": 0,
		"tEnd": 58500
	},
	{
		"class": "0-ads",
		"tStart": 58500,
		"tEnd": 125500
	},
	{
		"class": "1-speech",
		"tStart": 125500,
		"tEnd": 218000
	},
	{
		"class": "2-music",
		"tStart": 218000,
		"tEnd": 250500
	},
	{
		"class": "1-speech",
		"tStart": 250500,
		"tEnd": 472949
	}
]

Note that when analyzing audio files, you still need to provide the name of a radio stream, because the algorithm has to load acoustic parameters and DB of known samples. Analysis of podcasts not tied to a radio is not yet supported, but may possibly be in the future.

Documentation

Usage

Below is a simple usage example. More thorough usage examples are available in the tests:

  • file/podcast analysis: test/file.js
  • live stream analysis: test/online.js
  • record a live stream, analyse it later: test/offline.js
const { Analyser } = require("adblockradio");

const abr = new Analyser({
	country: "France",
	name: "RTL",
	config: {
		...
	}
});

abr.on("data", function(obj) {
	...
});
Property Description Default
country Country of the radio stream according to radio-browser.info None
name Name of the radio stream according to radio-browser.info None
file File to analyse (optional, analyse the live stream otherwise) None

Methods

Acoustic model and hotlist files are refreshed automatically on startup. If you plan to continuously run the algo for a long time, you can trigger manual updates. Note those methods are only available in live stream analysis mode.

Method Parameters Description
refreshPredictorMl None Manually refresh the ML model (live stream only)
refreshPredictorHotlist None Manually refresh the hotlist DB (live stream only)
refreshMetadata None Manually refresh the metadata scraper (live stream only)
stopDl None Stop Adblock Radio (live stream only)

Optional configuration

Properties marked with a * are meant to be used only with live radio stream analysis, not file analysis where they are ignored.

Scheduling

Property Description Default
predInterval Send stream status to listener every N seconds 1
saveDuration* If enabled, save audio file and metadata every N predInterval times 10
modelUpdatesInterval If enabled, update model files every N minutes 60

Switches

Property Description Periodicity Default
enablePredictorMl Perform machine learning inference predInterval true
JSPredictorMl Use tfjs instead of Python for ML inference (slower) false
enablePredictorHotlist Compute audio fingerprints and search them in a DB predInterval true
saveAudio* Save stream audio data in segments on hard drive saveDuration true
saveMetadata Save a JSON with predictions saveDuration true
fetchMetadata* Gather metadata from radio websites saveDuration true
modelUpdates Keep ML and hotlist files up to date modelUpdatesInterval true

Paths

Property Description Default
modelPath Directory where ML models and hotlist DBs are stored process.cwd() + '/model'
modelFile Path of ML file relative to modelPath country + '_' + name + '/model.keras'
hotlistFile Path of the hotlist DB relative to modelPath country + '_' + name + '/hotlist.sqlite'
saveAudioPath* Root folder where audio and metadata are saved process.cwd() + '/records'

Output

Readable streams constructed with Analyser emit objects with the following properties. Some properties are only available when doing live radio analysis. They are marked with a *. Other specific to file analysis are marked with **.

  • audio*: Buffer containing a chunk of original (compressed) audio data.

  • ml: null if not available, otherwise an object containing the results of the time-frequency analyser

    • softmaxraw: an array of three numbers representing the softmax between ads, speech and music.
    • softmax: same as softmaxraw, but smoothed in time with slotsFuture data points in the future and slotsPast data points in the past. Smoothing weights are defined by consts.MOV_AVG_WEIGHTS in post-processing.js.
    • class: either 0-ads, 1-speech, 2-music or 9-unsure. The classification according to softmax.
  • hotlist: null if not available, otherwise an object containing the results of the fingerprint matcher.

    • file: if class is not "9-unsure", the reference of the file recognized.
    • total: number of fingerprints computed for the given audio segment.
    • matches: number of matching fingerprints between the audio segment and the fingerprint database.
    • class: either 0-ads, 1-speech, 2-music, 3-jingles or 9-unsure if not enough matches have been found.
  • class: final prediction of the algorithm. Either 0-ads, 1-speech, 2-music, 3-jingles or 9-unsure.

  • metadata*: live metadata, fetched and parsed by the module adblockradio/webradio-metadata.

  • streamInfo*: static metadata about the stream. Contains stream url, favicon, bitrate in bytes / s, audio files extension audioExt (mp3 or aac) and homepage URL.

  • gain: a dB value representing the average volume of the stream. Useful if you wish to normalize the playback volume. Calculated by mlpredict.py.

  • tBuffer*: seconds of audio buffer. Calculated by adblockradio/stream-tireless-baler.

  • predictorStartTime*: timestamp of the algorithm startup. Useful to get the uptime.

  • playTime*: approximate timestamp of when the given audio is to be played. TODO check this.

  • tStart**: lower boundary of the time interval linked with the prediction (in milliseconds)

  • tEnd**: upper boundary of the time interval linked with the prediction (in milliseconds)

Supported radios

The list of supported radios is available here.

Note to developers

Integrations of this module are welcome. Suggestions are available here.

A standalone demo player for web browsers is available here.

License

See LICENSE file.

Your contribution to this project is welcome, but might be subject to a contributor's license agreement.

stream-audio-fingerprint's People

Contributors

dest4 avatar omgimalexis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

stream-audio-fingerprint's Issues

Error opening input files: Invalid data found when processing input

Description:
I encountered an error while running my Node.js application using version 18.14.0. The application throws the following error message:

`
Error opening input files: Invalid data found when processing input
node:events:491
throw er; // Unhandled 'error' event
^

Error: write EPIPE
at afterWriteDispatched (node:internal/stream_base_commons:160:15)
at writevGeneric (node:internal/stream_base_commons:143:3)
at Socket._writeGeneric (node:net:928:11)
at Socket._writev (node:net:937:8)
at doWrite (node:internal/streams/writable:409:12)
at clearBuffer (node:internal/streams/writable:564:5)
at onwrite (node:internal/streams/writable:464:7)
at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:106:10)

Emitted 'error' event on Socket instance at:
at Socket.onerror (node:internal/streams/readable:785:14)
at Socket.emit (node:events:513:28)
at emitErrorNT (node:internal/streams/destroy:151:8)
at emitErrorCloseNT (node:internal/streams/destroy:116:3)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
errno: -4047,
code: 'EPIPE',
syscall: 'write'
}

Node.js version: v18.14.0
`

plotting landmarks

Hello, I have tried to do some plotting turning DO_PLOT=true.
It works, but the plots seems to be stretched in some way
out-fft
out-thr
Any idea?

Thanks again for your help!

Could you explain dependencies that you are using?

I got the following error when i run
curl http://radiofg.impek.com/fg | node node_modules/stream-audio-fingerprint/codegen_demo.js

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: spawn ffmpeg ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:190:19)
    at onErrorNT (internal/child_process.js:362:16)
    at _combinedTickCallback (internal/process/next_tick.js:139:11)
    at process._tickCallback (internal/process/next_tick.js:181:9)
    at Function.Module.runMain (module.js:696:11)
    at startup (bootstrap_node.js:204:16)
    at bootstrap_node.js:625:3
100  5808    0  5808    0     0   7902      0

Possible ways of reducing complexity

  1. According to https://en.wikipedia.org/wiki/Just-noticeable_difference humans can only notice tones of 10 cents (1/10 of a semitone) or more, so is it possible to reduce tone differences to an integer to the nearest 10cents instead of a float?
  2. According to https://www.reddit.com/r/askscience/comments/5dpu0z/what_is_the_fastest_beats_per_minute_we_can_hear/ humans can only notice beats of 1500-1800BPM or 25-30BPS or slower, so is it possible to reduce the timed difference to an integer to the nearest 40ms (or 50ms, or 20ms/25ms for higher accuracy) instead of a float?
  3. According to https://en.wikipedia.org/wiki/List_of_chords most chords have less than 8 notes, so would adding f3~f6 make everything more accurate?

P.S. It would be great to have an awesome-audio-fingerprint that shows a list of projects that uses this repo

blast from the past!

Hi! You don't know me, I don't;t I know you, but I couldn't believe my eyes when randomly skimming reddit I saw your post. For reasons soon to be apparent it and it filled me with joy and happy memories)

Well more than a decade ago (I think 2001), me and some wet behind the ears friends had this crazy idea that had never occurred to anyone before -- through the constructor of certain transfer functions, we could APPROXIMATE a metric space of sound, where distance corresponded to "how similar are these two sounds" This metric, when combined with more classical ML tools like PCA then SVM then Q trees for search and neighborest matching (thoughthat was slow) allowed us to so what felt at the time completely unheard of. We could actually take a random CD from an investors car, play any point on it, and within seconds know exactly the song and location.

But what I was always more curious about is the function AXA not AXB. THe function showing varying levels of self similarity within the same clip i just KNEW would eventually be worth it and at the least, it's really cool!

So, in the final death throws of the company, I put together a quick demo that would take as input audio stations and give as output exact timestmps of every time every single commercial was played. I thouhght it was cool, but had no idea of anyone would want such a thing.

Anyway, I'm nto sure where I was going but thanks for a pleasant walk down memory land, =)

wav header is included when running the sample code.

in the demo js, I believe it processes the wav header as well as the data :(.
the ffmpeg options are:
'-i', 'pipe:0',
'-acodec', 'pcm_s16le',
'-ar', 22050,
'-ac', 1,
'-f', 'wav',
'-v', 'fatal',
'pipe:1'

these are better, providing only the raw samples:
'-i', 'pipe:0',
'-map', '0:a',
'-acodec', 'pcm_s16le',
'-ar', 22050,
'-ac', 1,
'-f', 'data',
'-v', 'fatal',
'pipe:1'

The fingerprints produced are different.
It is quite odd that the fingerprints are SO different just for a few added bytes at the start.... quite a few common fingerprints, but >75% are different, and not just offset by time.

How does matching work?

This might be a dumb question, if so I am sorry.

I looked through the code and the example and am now properly confused.

What I'm doing to test (albeit a bit awkwardly) is piping my microphone input using ffmpeg, to an example file with node, but I notice that the fingerprints very rarely match those that I generated based on the source file.

The way I thought it worked was:

  1. Create fingerprints for a source file (I used the example in the readme)
  2. Create fingerprints for the input (for example microphone) and match against against previously generated fingerprints.

Like I said, I rarely have any results. Do I need to make it more sensitive? If so, how would I do that? Or am I not supposed to use my microphone as input with this lib?

Thanks!

Update:

If it helps, here's the code I'm using (a mess because it's just me going nuts):

var decoder = require('child_process').spawn('ffmpeg', [
	'-i', 'pipe:0',
	'-acodec', 'pcm_s16le',
	'-ar', 22050,
	'-ac', 1,
	'-f', 'wav',
	'-v', 'fatal',
	'pipe:1'
], { stdio: ['pipe', 'pipe', process.stderr] });

process.stdin.pipe(decoder.stdin);

var Codegen = require("stream-audio-fingerprint");
var fingerprinter = new Codegen();
decoder.stdout.pipe(fingerprinter);

const buildDB = process.argv[2] === 'db';
const live = process.argv[2] === 'live';
const database = [];
const db = buildDB ? null : require('./db.json');

fingerprinter.on("data", function(data) {
  const suspects = [];

  for (var i=0; i < data.tcodes.length; i++) {
    if (buildDB) {
      database.push({ time: data.tcodes[i], fingerprint: data.hcodes[i] })
    } else if (live) {
      suspects.push(data.hcodes[i]);
    } else {
      database.push(data.hcodes[i]);
    }
  }

  if (live) {
    const res = db.filter(x => suspects.indexOf(x.fingerprint) > -1);

    console.log({ tada: res });
  }
}).on('end', () => {
  if (buildDB) {
    console.log(JSON.stringify(database));
  } else {
    const db = require('./db.json');
    const results = [];

    const res = db.filter(x => database.indexOf(x.fingerprint) > -1);

    console.log({ tada: res });
  }
});

The way I run it is:

  1. Store above file as index.js in a directory.
  2. Make sure there's a source .wav file in the same directory. Let's call it my_file.wav
  3. Make sure you have this file on an external device for playing. I have it on my phone (iPhone 11 pro)
  4. "Index" the audio file: cat my_file.wav | node index.js db > db.json
  5. Pipe your mic into the file: ffmpeg -f avfoundation -i ":0" -ar 22050 -acodec pcm_s16le -v fatal -f wav pipe:1 | node index.js live
  6. Start playing the source audio on the external device (again, I'm playing it on my phone).
  7. Observe some matches, very rarely (example: { tada: [ { time: 19433, fingerprint: 139300 } ] })

Get second from tcodes

image
Hi i want to convert tcodes to seconds, is there any way to convert the tcodes to the actual seconds on the music? (i have the original song fingerprint on the db, the hashes shown on the image is just a part of the original one)
Also, is the hash generator sensitive about distortion, noise and the other audio glitch? (i hope not)

Getting more than one fingerprint for each offset time?

Hi, this module is awesome! I am brand new to audio fingerprint, this module get me a good opportunity to learn how to code.

But I found that I got more than one fingerprint for each offset time, is it normal?

Sample Output:

time=34 fingerprint=238707
time=34 fingerprint=216179
time=34 fingerprint=347159
time=34 fingerprint=478295
time=43 fingerprint=601676
time=43 fingerprint=929303
time=43 fingerprint=1060439
time=45 fingerprint=132398
time=45 fingerprint=1049879
time=45 fingerprint=2360577

I am confused how to use these hashes to search in the database for matching hashes.

For example, if I have a record in database time=43 fingerprint=601676, does it mean the sample hit this db record?
or it requires a full matching time=43 fingerprint=601676, time=43 fingerprint=929303, time=43 fingerprint=1060439?

Consistent Hashes size among dataset

I would like to keep all the hcodes array hashes of the same size for every audio sample. Now assumed that we consider the same file chunk size for each audio sample, would be that possibile?

fingerprints stream ended, no output, no errors

starting
type awesome_music.mp3 | node codegen_demo.js
gives me just one console output:
"fingerprints stream ended"

no other errors or logs appeared. awesome_music.mp3 is ~4s long.

please help

Error opening input files: Invalid data found when processing input

I encountered an error while running my Node.js application using version 18.14.0. The application throws the following error message:

Error opening input files: Invalid data found when processing input
node:events:491
      throw er; // Unhandled 'error' event
      ^

Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:160:15)
    at writevGeneric (node:internal/stream_base_commons:143:3)
    at Socket._writeGeneric (node:net:928:11)
    at Socket._writev (node:net:937:8)
    at doWrite (node:internal/streams/writable:409:12)
    at clearBuffer (node:internal/streams/writable:564:5)
    at onwrite (node:internal/streams/writable:464:7)
    at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:106:10)

Emitted 'error' event on Socket instance at:
    at Socket.onerror (node:internal/streams/readable:785:14)
    at Socket.emit (node:events:513:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -4047,
  code: 'EPIPE',
  syscall: 'write'
}

Node.js version: v18.14.0

Interrupt fingerprinting process

image
Any chance to stop the fingerprinting process instead leave it untill it emit 'end' event?
Figure1.1: Audio recognized as A Music
Figure1.2: End fingerprinting process because the Music already fingerprinted
Thank you, Anyway Merry Christmas! 😃

Using shorter intervals?

Hi @dest4 ,

If I'm not mistaken, you use 12-second windows to generate fingerprints, thus it is not possible to use this to match parts of audio that are shorter than 12 seconds and that are spaced by less than 12 seconds from a longer stream. Is this correct? Would it be possible to do it my by changing some of the constants and which (e.d. to make it work with 3 or 4 seconds)?

To give a bit of context - I have a longer audio recording (a voice over recording session - several hours), from which a selection of takes have already been cut out into separate files and slightly processed (so volume and quality is not the same). I need to find at which positions (in seconds) did each of the files originate. As it is right now, for some of them it doesn't find a single match - presumably because they are too close together when in the large file.

Thanks!

events.js:183 throw er; // Unhandled 'error' event && bash: nodejs: command not found

Louisde-MBP:stream-audio-fingerprint louis$ sudo curl http://radiofg.impek.com/fg | nodejs codegen_demo.js
bash: nodejs: command not found
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 17376 0 17376 0 0 6608 0 --:--:-- 0:00:02 --:--:-- 6609
curl: (23) Failed writing body (0 != 1448)

I figured maybe just my problem so I also tried this below:

Louisde-MBP:stream-audio-fingerprint louis$ curl http://radiofg.impek.com/fg | node codegen_demo.js
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0events.js:183
throw er; // Unhandled 'error' event
^
Error: spawn ffmpeg ENOENT
at _errnoException (util.js:1024:11)
at Process.ChildProcess._handle.onexit (internal/child_process.js:190:19)
at onErrorNT (internal/child_process.js:372:16)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickCallback (internal/process/next_tick.js:180:9)
at Function.Module.runMain (module.js:678:11)
at startup (bootstrap_node.js:187:16)
at bootstrap_node.js:608:3
100 23168 0 23168 0 0 11617 0 --:--:-- 0:00:01 --:--:-- 11613
curl: (23) Failed writing body (0 != 8688)

Still failed

Input from local file

I'm trying to pipe the output of ffmpeg to the codegen passing the input file directly to ffmpeg like:

decode = function (uri) {
      var self = this;

      // defaults
      var loglevel = self.logger.isDebug() ? 'debug' : 'warning';
      const args = [
        '-y',
        '-loglevel', loglevel,
        '-acodec', 'pcm_s16le',
        '-ar', 22050,
        '-ac', 1,
        '-f', 'wav',
        '-v', 'fatal',
        '-i', uri, // <--- I'm using local input instead of 'pipe:0
        'pipe:1'
      ];
      const opts = {
        stdio: ['pipe', 'pipe', process.stderr]
      };
      var decoder = cp.spawn('ffmpeg', args, opts);
      return decoder;
    }//decode

and then

// get decoder instance
var decoder = this.decode(mediaPath);
var fingerprinter = new Codegen();
// pipe fingerprint
decoder.stdout.pipe(fingerprinter);
// attach handlers
fingerprinter.on('end', resolve)
fingerprinter.on('message', msg => self.logger.info(msg))
fingerprinter.on('error', reject)
fingerprinter.on('close', resolve)
fingerprinter.on("data", function (data) {
    for (var i = 0; i < data.tcodes.length; i++) {
        console.log("time=" + data.tcodes[i] + " fingerprint=" + data.hcodes[i]);
      }
});

but I don't get any result here. What it seems to me is that I get the end handler, but not the data handler.

Integrating With ExpressJS/MQ

Is there had chance to do this fingerprinting with ExpressJS or MQ so its run as standalone instead of executing cat music.mp3 | node src.js ? Just wonder how since this works only with stdin pipe

Oh! also, any chance to get more fingerprint (way more accurate)? Is this just play with the overlap?

It is not an issue, help request

Sorry to bother but any lead on how to match 2 fingerprints ?

Say I have generated some fingerprints and saved to a database, and now have a radio stream and want to match againts that database. Any lead on this ?

Thanks a lot for your work.

Any way to tune threshold?

Hello,
First of all thanks for such a nice tool.

I was curious if there is a threshold which can be tuned?
For example I have indexed 2 hours long audio file in json (I took an example from one of closed issues here) and cut out a 15 second clip from the same audio file which I am then comparing against this json file. It seems like some of the fingerprints match up throughout all the 2 hour audio file, but then repeats a lot of times exactly where it should. ( I have tuned the code a little bit so timecode is put as exact timestamp)

So for example if I compare the small 15 second clip against the whole 2 hours of audio, I get this -

{ time: '0:40', fingerprint: 920332 },
{ time: '0:56', fingerprint: 207915 },
{ time: '1:53', fingerprint: 2826250 },
{ time: '2:10', fingerprint: 4787722 },
{ time: '2:56', fingerprint: 2826250 },
{ time: '6:34', fingerprint: 4157 },
{ time: '6:34', fingerprint: 15632 },
{ time: '12:07', fingerprint: 14114 },
{ time: '12:07', fingerprint: 8759 },
{ time: '13:07', fingerprint: 47075 },
{ time: '13:07', fingerprint: 58295 },
{ time: '15:51', fingerprint: 1706252 },
{ time: '25:35', fingerprint: 920332 },
{ time: '26:03', fingerprint: 3934509 },
{ time: '26:04', fingerprint: 8759 },
{ time: '26:04', fingerprint: 14114 },
{ time: '27:48', fingerprint: 92813 },
{ time: '30:40', fingerprint: 599052 },
{ time: '32:24', fingerprint: 2439691 },
{ time: '33:51', fingerprint: 68644 },
{ time: '35:03', fingerprint: 211492 },
{ time: '36:32', fingerprint: 921098 },
{ time: '39:33', fingerprint: 8759 },
{ time: '39:33', fingerprint: 14114 },
{ time: '40:36', fingerprint: 1706252 },
{ time: '44:24', fingerprint: 4787722 },
{ time: '45:24', fingerprint: 2949928 },
{ time: '46:37', fingerprint: 1901062 },
{ time: '48:08', fingerprint: 4457230 },
{ time: '48:33', fingerprint: 1904898 },
{ time: '48:36', fingerprint: 197136 },
{ time: '49:23', fingerprint: 68644 },
{ time: '49:57', fingerprint: 4260376 },
{ time: '50:21', fingerprint: 270338 },
{ time: '50:28', fingerprint: 1185794 },
{ time: '50:28', fingerprint: 4260376 },
{ time: '59:07', fingerprint: 145932 },
{ time: '59:54', fingerprint: 1706252 },
{ time: '1:02:23', fingerprint: 599052 },
{ time: '1:05:09', fingerprint: 985892 },
{ time: '1:08:26', fingerprint: 2439691 },
{ time: '1:09:03', fingerprint: 42892 },
{ time: '1:09:03', fingerprint: 36007 },
{ time: '1:09:49', fingerprint: 145932 },
{ time: '1:12:17', fingerprint: 1704723 },
{ time: '1:12:24', fingerprint: 374162 },
{ time: '1:12:33', fingerprint: 393732 },
{ time: '1:12:55', fingerprint: 131587 },
---- HERE STARTING THE 15 sec clip -----
{ time: '1:13:00', fingerprint: 197152 },
{ time: '1:13:00', fingerprint: 278816 },
{ time: '1:13:00', fingerprint: 60099 },
{ time: '1:13:00', fingerprint: 50154 },
{ time: '1:13:00', fingerprint: 615489 },
{ time: '1:13:00', fingerprint: 736100 },
{ time: '1:13:01', fingerprint: 1114683 },
{ time: '1:13:01', fingerprint: 2687520 },
{ time: '1:13:01', fingerprint: 666951 },
{ time: '1:13:01', fingerprint: 3672578 },
{ time: '1:13:01', fingerprint: 4786747 },
{ time: '1:13:01', fingerprint: 921098 },
{ time: '1:13:02', fingerprint: 4100733 },
{ time: '1:13:03', fingerprint: 5243420 },
{ time: '1:13:04', fingerprint: 1250050 },
{ time: '1:13:04', fingerprint: 605971 },
{ time: '1:13:04', fingerprint: 1835539 },
{ time: '1:13:04', fingerprint: 2900031 },
{ time: '1:13:04', fingerprint: 3489811 },
{ time: '1:13:04', fingerprint: 266560 },
{ time: '1:13:04', fingerprint: 1904898 },
{ time: '1:13:04', fingerprint: 730129 },
{ time: '1:13:04', fingerprint: 992320 },
{ time: '1:13:04', fingerprint: 2630658 },
{ time: '1:13:05', fingerprint: 5139008 },
{ time: '1:13:05', fingerprint: 1974808 },
{ time: '1:13:06', fingerprint: 2838562 },
{ time: '1:13:06', fingerprint: 3774058 },
{ time: '1:13:06', fingerprint: 993872 },
{ time: '1:13:06', fingerprint: 3811874 },
{ time: '1:13:06', fingerprint: 2168618 },
{ time: '1:13:06', fingerprint: 3151696 },
{ time: '1:13:06', fingerprint: 1051671 },
{ time: '1:13:06', fingerprint: 3214378 },
{ time: '1:13:07', fingerprint: 590348 },
{ time: '1:13:07', fingerprint: 1638935 },
{ time: '1:13:07', fingerprint: 2556461 },
{ time: '1:13:08', fingerprint: 1580627 },
{ time: '1:13:08', fingerprint: 472094 },
{ time: '1:13:08', fingerprint: 2045011 },
{ time: '1:13:08', fingerprint: 1134644 },
{ time: '1:13:08', fingerprint: 1593374 },
{ time: '1:13:08', fingerprint: 268112 },
{ time: '1:13:08', fingerprint: 1382196 },
{ time: '1:13:08', fingerprint: 1840926 },
{ time: '1:13:10', fingerprint: 928576 },
{ time: '1:13:10', fingerprint: 1746845 },
{ time: '1:13:10', fingerprint: 2032171 },
{ time: '1:13:10', fingerprint: 1874599 },
{ time: '1:13:10', fingerprint: 226970 },
{ time: '1:13:10', fingerprint: 1144401 },
{ time: '1:13:11', fingerprint: 69146 },
{ time: '1:13:11', fingerprint: 2172969 },
{ time: '1:13:12', fingerprint: 3023912 },
{ time: '1:13:12', fingerprint: 211492 },
{ time: '1:13:12', fingerprint: 1575226 },
{ time: '1:13:12', fingerprint: 669961 },
{ time: '1:13:12', fingerprint: 1114681 },
{ time: '1:13:13', fingerprint: 75345 },
{ time: '1:13:13', fingerprint: 393766 },
{ time: '1:13:13', fingerprint: 2430722 },
{ time: '1:13:14', fingerprint: 2818583 },
---- HERE ENDING THE 15 sec clip -----
{ time: '1:14:45', fingerprint: 328196 },
{ time: '1:14:53', fingerprint: 268290 },
{ time: '1:15:14', fingerprint: 1901062 },
{ time: '1:16:43', fingerprint: 2630658 },
{ time: '1:18:06', fingerprint: 68644 },
{ time: '1:20:19', fingerprint: 3083304 },
{ time: '1:21:07', fingerprint: 204804 },
{ time: '1:21:07', fingerprint: 135200 },
{ time: '1:26:13', fingerprint: 4787722 },
{ time: '1:27:25', fingerprint: 96683 },
{ time: '1:27:41', fingerprint: 135200 },
{ time: '1:30:44', fingerprint: 1908750 },
{ time: '1:36:48', fingerprint: 204802 },
{ time: '1:40:22', fingerprint: 183452 },
{ time: '1:42:27', fingerprint: 938380 },
{ time: '1:45:04', fingerprint: 1517582 },
{ time: '1:55:11', fingerprint: 210434 },
{ time: '1:55:55', fingerprint: 918591 },
---- HERE STARTING THE 15 sec clip again which is expected -----
{ time: '1:56:21', fingerprint: 393732 },
{ time: '1:56:22', fingerprint: 2884100 },
{ time: '1:56:22', fingerprint: 666951 },
{ time: '1:56:22', fingerprint: 2501890 },
{ time: '1:56:22', fingerprint: 1408327 },
{ time: '1:56:22', fingerprint: 1182253 },
{ time: '1:56:22', fingerprint: 3672578 },
{ time: '1:56:23', fingerprint: 2826250 },
{ time: '1:56:23', fingerprint: 3899517 },
{ time: '1:56:23', fingerprint: 5275719 },
{ time: '1:56:23', fingerprint: 1969696 },
{ time: '1:56:23', fingerprint: 4787722 },
{ time: '1:56:25', fingerprint: 70439 },
{ time: '1:56:25', fingerprint: 1704723 },
{ time: '1:56:25', fingerprint: 1770279 },
{ time: '1:56:26', fingerprint: 2100994 },
{ time: '1:56:27', fingerprint: 2168618 },
{ time: '1:56:28', fingerprint: 1638935 },
{ time: '1:56:28', fingerprint: 2556461 },
{ time: '1:56:29', fingerprint: 4079149 },
{ time: '1:56:31', fingerprint: 1901059 },
{ time: '1:56:31', fingerprint: 938380 },
{ time: '1:56:32', fingerprint: 69146 },
{ time: '1:56:33', fingerprint: 131642 },
{ time: '1:56:33', fingerprint: 270338 },
{ time: '1:56:33', fingerprint: 401466 },
{ time: '1:56:35', fingerprint: 2818583 },
{ time: '1:56:35', fingerprint: 4935959 },
---- HERE ENDING THE 15 sec clip again which is expected -----
{ time: '1:56:48', fingerprint: 134147 },
{ time: '1:59:00', fingerprint: 131587 },
{ time: '1:59:01', fingerprint: 197136 },
{ time: '1:59:30', fingerprint: 657666 },
{ time: '1:59:42', fingerprint: 268290 },
{ time: '1:59:48', fingerprint: 4787722 },
{ time: '2:00:19', fingerprint: 1185828 },
{ time: '2:01:43', fingerprint: 920332 },
{ time: '2:02:39', fingerprint: 3214378 },
{ time: '2:05:22', fingerprint: 2826250 },
{ time: '2:06:55', fingerprint: 4157 },
{ time: '2:06:55', fingerprint: 15632 },
{ time: '2:07:38', fingerprint: 68644 },
{ time: '2:08:48', fingerprint: 1775628 }
]

So as you can see there is a lot of "noise" but I expect thats because the audio is containing voice which might be similar troughout the audio fingerprintwise. Or maybe I am doing something completely wrong?

not creat png

hi
thanks for publish this library
run in window command prompt with this command
type two.mp3 | node codegen_demo.js

but not creat out-fft.png OR out-thr.png OR out-raw.png
please help me
thanks

Comparing short audio files

Hi,
I'm interested in finding near-duplicate audio files. My dataset is about 3000 thousands short audio files, between 0.5 seconds to 5 seconds. Unlike Shazam, both the "target" audio (i.e. the songs in Shazam's case) and the user input are short, and both might contain noise.

Can this library help?
If so, are there any recommendations for tuning parameters?

N.B - if a file is matched to multiple other files, it's fine - I have a less efficient algorithm that can verify which match is correct. In other words, I can handle some amount of false positives, but I don't want false negatives.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.