colonelparrot / jscanify Goto Github PK
View Code? Open in Web Editor NEWOpen-source Javascript mobile document scanner.
Home Page: https://colonelparrot.github.io/jscanify/
License: MIT License
Open-source Javascript mobile document scanner.
Home Page: https://colonelparrot.github.io/jscanify/
License: MIT License
During testing, I encountered a case where the top-right of my page was incorrectly evaluated as being near the top-left, resulting in a near triangular poly from getCornerPoints(contour)
. What seems to have happened is that there was a point in the contour that was just right of the centre, yet had a bigger distance than the right-hand corner of the page (although the page was at an almost 45 degree angle in the frame, which would be unusual).
Recommend applying a cv.approxPolyDP to the maxContour
prior to getCornerPoints(contour)
to reduce occurrences.
return this.getApproxPoly(maxContour);
}
/** refines a contour to get approximate poly
* @param {*} contour contour to refine
*/
getApproxPoly(contour) {
const peri = cv.arcLength(contour, true);
const approx = new cv.Mat();
cv.arcLength(contour, true) * 0.02
cv.approxPolyDP(contour, approx, 0.1 * peri, true);
return approx;
}
Hey!
I was trying to use the package on the FE side (React) but faced several issues. If I install it via npm (in my case pnpm, but I guess it doesn't matter) it fails with the error. It requires Node.js variable process.
If I use it via CDN, its dependency - opencv.js requires Node.js variable process.
Error: Uncaught ReferenceError: process is not defined
Could you please clarify if it's possible to use this lib in React? If you do - please clarify the steps to use it.
The React example didn't help...
I am trying to get the cornerPoints from a scanned image to see how wide the pdf atcually is.
However, when scanner.getCornerPoints(paperContour, image)
the console will throw an ambigous error in the opencv.js
file as follows uncaught (in promise) 23302432
.
Below is a minimal example which shows this error (image with pdf not included)
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Test</title>
<script src="https://docs.opencv.org/4.7.0/opencv.js" async></script>
<!-- warning: loading OpenCV can take some time. Load asynchronously -->
<script src="https://cdn.jsdelivr.net/gh/ColonelParrot/jscanify@master/src/jscanify.min.js"></script>
</head>
<body>
<img src="pic.jpeg" id="image" />
<script>
window.addEventListener("load", () => {
cv['onRuntimeInitialized'] = () => {
const scanner = new jscanify();
const image = document.getElementById("image");
const paperContour = cv.imread(image);
const resultCanvas = scanner.findPaperContour(paperContour);
const {
topLeftCorner,
topRightCorner,
bottomLeftCorner,
bottomRightCorner,
} = scanner.getCornerPoints(paperContour, image);
console.log("topLeftCorner",topLeftCorner);
console.log("topRightCorner",topRightCorner);
console.log("bottomLeftCorner",bottomLeftCorner);
console.log("bottomRightCorner",bottomRightCorner);
}
})
</script>
</body>
</html>
Every time I try, I get this message when loading and camera does not turn on:
Uncaught TypeError: Class extends value undefined is not a constructor or null
at node_modules/jsdom/lib/jsdom/virtual-console.js
node version:
v18.16.0
code for Scanner.vue component:
<script setup lang="ts">
import {onMounted, ref} from "vue";
import useScript from '../composables/useScript';
//const jscanify = require('jscanify');
//@ts-ignore
import jscanify from 'jscanify'
const openCVURL = 'https://docs.opencv.org/4.7.0/opencv.js';
let scanner:any = new jscanify();
const canvas: HTMLCanvasElement | null = <HTMLCanvasElement> document.getElementById("canvas");
const result: HTMLCanvasElement | null = <HTMLCanvasElement> document.getElementById("result");
const video: HTMLVideoElement | null = <HTMLVideoElement> document.getElementById("video");
onMounted(async () => {
await useScript(openCVURL);
const canvasCtx = canvas.getContext("2d");
const resultCtx = result.getContext("2d");
scanner = new jscanify();
navigator.mediaDevices.getUserMedia({ video: true }).then((stream) => {
video.srcObject = stream;
video.onloadedmetadata = () => {
video.play();
setInterval(() => {
canvasCtx!.drawImage(video, 0, 0);
const resultCanvas = scanner.highlightPaper(canvas);
resultCtx!.drawImage(resultCanvas, 0, 0);
}, 10);
};
});
})
</script>
<template>
<video id="video"></video>
<canvas id="canvas"></canvas>
<!-- original video -->
<canvas id="result"></canvas>
<!-- highlighted video -->
<h2>Document scanner component TODO</h2>
</template>
Distutils were removed in python 3.12 meaning the canvas dependancy fails to install.
Fix is to roll back python to 3.11, install then works without.
Just an FYI
It does not work well with pages with round corners. Otherwise works well.
Hi @ColonelParrot, thank you for this awesome library. I'm currently using it in a personal project, but I've noticed that the image quality is poor. I've attached the scanned document images below for reference.
I made a small modification to the extractPaper() method, but even before that, the quality was not satisfactory. Any suggestions or tips would be highly appreciated.
// Calculate the aspect ratio of the original document
const aspectRatio = img.cols / img.rows;
// Calculate the corresponding height to maintain the aspect ratio
const calculatedHeight = Math.round(resultWidth / aspectRatio);
I'm very impressed by jscanify and I really appreciate that you open sourced your work.
I have two examples where a cluttered background breaks the extractPaper
logic to find the paper in the image. I think this is an edge case (no pun intended) but perhaps these can help you future improve the algorithm.
As you know, capturing a photo is basically taking a still frame from the video stream.
When doing video, usually any project is compatible with the idea of wanting a 360, 720 or 1080 video... but when you're working with photos, you usually want the best possible photo resolution.
There is a "WebRTC resolution scanner" but it works pretty poorly. For example, on my iPhone 14, it gives me "fail" as a result in all resolutions, making it difficult for me to set up a stream with "the best resolution the mobile can offer".
And this may not be a problem when you use WebRTC to take a profile picture or a picture where good resolution is not required, but in the case of a document scanner, it is critical to have the best possible resolution to be able to OCR it.
How do you deal with this problem and how do you configure WebRTC to take the best possible mobile resolution?
I have tried with the "ideal" constraints but without good results.
Thanks in advance
how to set auto height or width to maintain the ratio of the image size instead of fixed width and height?
scanner.extractPaper(image, paperWidth, paperHeight)
I think this is the biggest problem right now. If you use a fixed value then it will only work for that particular aspect ratio. Any plans for implementing something similar to the methods outlined in the Whiteboard Scanning and Image Enhancement by Microsoft?
Appreciate it.
Hi,
thank you for this amazing project. I used it as a base to further improve the recognition algorithm as follows:
I would just like to share it, maybe it is useful for other people as well. Maybe you can test it as well and consider adding some changes to your project.
The note in the wiki implies that I can pass HTMLImageElement
, HTMLCanvasElement
and File
s for img or image parameters.
However findPaperContour(img)
requires a cv.Mat
.
Expectation:
const contour = scanner.findPaperContour(canvas); // Results in cv bind error
const corner_points = scanner.getCornerPoints(contour);
Reality:
const cvmat = cv.imread(canvas);
const contour = scanner.findPaperContour(cvmat);
const corner_points = scanner.getCornerPoints(contour);
When attempting to create a scanner object in browser environment following error is thrown:
Uncaught (in promise) TypeError: Cannot set property document of #<Window> which has only a getter at installDOM (jscanify-node.js:9:10) at new jscanify (jscanify-node.js:30:5) at scanner.svelte:11:9 at run (index.mjs:20:12) at Array.map (<anonymous>) at index.mjs:2101:58 at flush (index.mjs:1329:17) at init (index.mjs:2197:9) at new Root (root.svelte:20:25) at createProxiedComponent (svelte-hooks.js?v=6abbeb80:341:9)
Chrome Dev version: 115.0.5762.4 (Official Build) dev (arm64)
Used stack: Vite + React
Full example:
https://codesandbox.io/p/sandbox/dreamy-breeze-wjwo7h
I'm experiencing an issue when trying to import and use the jscanify library in a React application. While the CDN approach works fine, importing the library as a module using the import
statement causes an error.
import React, { useEffect, useRef, useState } from 'react';
import jscanify from 'jscanify';
export default function PaperScanner() {
const canvasRef = useRef(null);
const [photoData, setPhotoData] = useState(null);
useEffect(() => {
const scanner = new jscanify(); // <-- Error occurs here
}, []);
}
Error Message:
TypeError: Cannot set property document of # which has only a getter
I just noticed this error while testing this library. The full error reads:
Passing a number "-1" from JS side to C/C++ side to an argument of type "unsigned long", which is outside the valid range [0, 4294967295]!
This is on the Brave browser
I do actually see it working though, so it doesn't seem very important.
My code:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>Scanner</title>
<script src="https://docs.opencv.org/4.7.0/opencv.js" async></script>
<script src="https://cdn.jsdelivr.net/gh/ColonelParrot/jscanify@master/src/jscanify.min.js"></script>
</head>
<body>
<video id="video"></video> <canvas id="canvas"></canvas>
<!-- original video -->
<canvas id="result"></canvas>
<!-- highlighted video -->
<script type="module">
const scanner = new jscanify();
const canvasCtx = canvas.getContext("2d");
const resultCtx = result.getContext("2d");
navigator.mediaDevices.getUserMedia({ video: true }).then((stream) => {
video.srcObject = stream;
video.onloadedmetadata = () => {
video.play();
setInterval(() => {
canvasCtx.drawImage(video, 0, 0);
const resultCanvas = scanner.highlightPaper(canvas);
resultCtx.drawImage(resultCanvas, 0, 0);
}, 10);
};
});
</script>
</body>
</html>
Do you have the document detection in real time while using the camera to get the photo through MediaCapture API ?
thank you !
Camer freeze in IOS browsers
any solution please?
I am using the the starter js code but the highlighted image does not load.
Expected Usage
import jscanify from 'jscanify'
On the website it says:
It can run in the browser or on a server with NodeJS.
When importing the library in Node however, I get the following error:
Uncaught ReferenceError: document is not defined
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.