Comments (13)
I found the problem!!!
I was using vCap.read()
called with requestAnimationFrame()
. By using vCap.asyncRead()
instead, I get sub-millisecond results. Yes! Just in time for the week-end.
Thanks a lot for your help and for your work on this library!
from opencv4nodejs.
Ok atleast then there is no problem with getData in nw.js, that's good to know. :D
As far as detectMultiScale
is concerned, going with default parameters usually does not provide the expected performance, especially with larger images. To achieve realtime performance I would highly suggest adjusting the parameters scaleFactor
to limit the step size between each image scaling operation and minSize
(minimum size of faces to expect) depending on your usecase.
For example running this with default parameters for a 1120x630 image on my machine takes me roughly 160ms on average. If I simply limit the search space however, e.g:
classifier.detectMultiScale(grayImg, { scaleFactor: 1.2, minSize: new cv.Size(100, 100) })
this brings it down to ~12ms, which I would consider realtime.
The delay you reported for vCap.read()
seems strange to me. I will investigate what's the problem here.
from opencv4nodejs.
Just to check if it’s not an issue with your webcam, did you try to open a video stream by giving an url as a parameter to cv.VideoCapture() ? I’ve tested this OK by running mjpg-streamer (https://github.com/jacksonliam/mjpg-streamer) on a Raspberry pi. It works fine.
from opencv4nodejs.
I tried with a local video file and I'm getting detection times below 1ms. Seeing that, I switched from my Macbook's onboard webcam to an external Logitech webcam and I'm getting better results (somewhere between 15-60ms).
What puzzles me is that, using the same webcam, I get 60FPS when I use FaceOSC for face detection...
from opencv4nodejs.
You should definitely expect to get real time performance with opencv. Looking at your code that you posted in another issue I am assuming you are drawing the result into a canvas?
Unfortunately with electron, apparently also with nw.js there is a problem with the mat.getData function (see this issue: justadudewhohacks/opencv-electron#2). Could you verify whether this is your bottleneck via simple console.time or similar?
I tried to figure out whats going on there and also opened an issue at electron-rebuild without success.
from opencv4nodejs.
Thanks for the quick reply. I am indeed drawing to canvas. However, this does not seem to be the bottleneck. According to console.time()
, these are the culprits:
const frame = vCap.read().bgrToGray();
±60ms
const faceResult = this.faceClassifier.detectMultiScale(frame);
±95ms
The whole detection loop takes about 155ms. This means that, besides those 2 lines, the rest is negligible.
If you want me to run more tests, I'll happily oblige.
from opencv4nodejs.
Thanks a lot for your help! By tweaking the settings for detectMultiScale()
, I'm now getting detection times below 1ms! However, vCap.read()
is still a huge bottleneck. For that single function, console.time()
reports times in the range of 50-100ms.
from opencv4nodejs.
Hm ok, great that atleast the detectMultiScale problem is fixed. Did you have better results for reading from a webcam with one of the other javascript libraries you used before?
from opencv4nodejs.
Did you have better results for reading from a webcam with one of the other javascript libraries you used before?
Yeah, that's the thing. For example, clmtracker gives me a much higher frame rate with the exact same webcam. The same goes for native solutions such as FaceOSC.
There has to be something wrong somewhere in the chain but I don't think the webcam is to blame.
from opencv4nodejs.
In this case the webcam should definitely be fine. Maybe this is just a configuration issue. You can by the way set properties for your VideoCapture, maybe this helps.
See the list of properties here:
https://docs.opencv.org/3.3.0/d4/d15/group__videoio__flags__base.html#gaeb8dd9c89c10a5c63c139bf7c4f5704d
You could try setting CAP_PROP_FPS
manually. CAP_PROP_BUFFERSIZE
sounds also interesting, maybe just buffer more frames in the capture. I never had the issue, thus I never had to touch these properties.
For example you can get and set properties by:
const fps = vCap.get(cv.CAP_PROP_FPS)
vCap.set(cv.CAP_PROP_FPS, 60)
from opencv4nodejs.
Awesome!
from opencv4nodejs.
I'm running into a similar issue with my webcam capture. It's very slow and its impacting my would be real time method. I am running the following code on my express server on an electron app inside of an interval running 30 times a second.
//defined
var vCap = new cv.VideoCapture(2);
vCap.set(cv.CAP_PROP_FRAME_HEIGHT,640);
vCap.set(cv.CAP_PROP_FRAME_WIDTH,960);
const fps = vCap.get(cv.CAP_PROP_FPS)
//loop
console.time("CAPTURE")
var frame = vCap.read();
console.timeEnd("CAPTURE")
console.time("ALGO")
const resizedImage = frame.resize(320, 480);
const gray = frame.bgrToGray();
const faces = classifier.detectMultiScale(gray, faceClassifierOpts).objects;
console.timeEnd("ALGO")
When I check the console to see the speed of this code, ALGO is only taking on average 11ms to complete. Yet, CAPTURE is outputting an average of 35ms. I've seen it go as high as 75ms. I tested 2 USB webcams and the built in webcam on my mac and the speeds are all roughly the same. Is there something I am doing wrong with my video capture?
I've also tried @djipco 's example with the async call, but all this allows me to do is set up a .then promise chain and the frame rate isn't any better.
from opencv4nodejs.
@justadudewhohacks bump
from opencv4nodejs.
Related Issues (20)
- Install failing at 55% MacOS Monterey HOT 2
- [!] RollupError: Could not resolve "../build/Release/opencv4nodejs.node" from "../build/Release/opencv4nodejs.node?commonjs-external" HOT 1
- Cannot find module 'C:\Users\pierr\Desktop\onnxjs\node_modules\opencv4nodejs\build\Release\opencv4nodejs' HOT 2
- Imencode - OpenCV Error: (!image.empty()) in imencode, in file... RPI4 HOT 1
- Correct the state of the `opencv4nodejs` package on the NPM. HOT 5
- canot load opencv4nodejs after install HOT 1
- Cannot find module 'D:\Web\opencv\node_modules\opencv4nodejs\build\Release\opencv4nodejs' HOT 2
- Error: CalcHist - object has no property: ranges HOT 1
- Can't resolve '../build/Release/opencv4nodejs.node' HOT 1
- Cannot find module /node_modules/@u4/opencv4nodejs/build/Release/opencv4nodejs'
- why no cv.compare or mat.compare function?
- how to catch mouse event on imshow
- ar
- Type error with mat.drawCountours()
- Unable to install opencv4nodejs HOT 1
- Unable to install in Ubuntu 22.05
- Ssim library
- Issues with node-gyp
- Issue with cv.Mat
- Issue with importing cv
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opencv4nodejs.