tony-xlh / vision-camera-dynamsoft-label-recognizer Goto Github PK

React Native Vision Camera Frame Processor Plugin of Dynamsoft Label Recognizer

License: MIT License

Java 35.08% JavaScript 2.80% Ruby 4.38% TypeScript 30.70% Swift 16.22% C 0.57% Objective-C 5.90% Objective-C++ 1.08% Kotlin 3.28%

vision-camera-dynamsoft-label-recognizer's Introduction

vision-camera-dynamsoft-label-recognizer

React Native Vision Camera Frame Processor Plugin of Dynamsoft Label Recognizer

label-recognizer-example.mp4

Versions

For vision-camera v2, use versions 0.x.

For vision-camera v3, use versions 1.x.

For vision-camera v4, use versions >= 2.0.0.

SDK Versions Used for Different Platforms

Product	Android	iOS
Dynamsoft Label Recognizer	2.x	2.x

Installation

yarn add vision-camera-dynamsoft-label-recognizer
cd ios && pod install

Add the plugin to your babel.config.js:

module.exports = {
   plugins: [['react-native-worklets-core/plugin']],
    // ...

Note: You have to restart metro-bundler for changes in the babel.config.js file to take effect.

Usage

Scan text with vision camera.

import { recognize } from 'vision-camera-dynamsoft-label-recognizer';

// ...
const frameProcessor = useFrameProcessor((frame) => {
  'worklet';
  const scanResult = recognize(frame);
}, []);

Scan text from base64-encoded static images.

const scanResult = await decodeBase64(base64);

License initialization (apply for a trial license).
```
await initLicense("your license");
```

Interfaces

Scanning configuration:

//the value is in percentage
export interface ScanRegion{
  left: number;
  top: number;
  width: number;
  height: number;
}

export interface ScanConfig{
  scanRegion?: ScanRegion;
  includeImageBase64?: boolean;
}

export interface CustomModelConfig {
  customModelFolder: string;
  customModelFileNames: string[];
}

You can use a custom model like a model for MRZ passport reading using the CustomModelConfig prop and update the template. You can find the MRZ model and template in the example.

You need to put the model folder in the assets folder for Android or the root for iOS.

About the result:

export interface ScanResult {
  results: DLRResult[];
  imageBase64?: string;
}

export interface DLRResult {
  referenceRegionName: string;
  textAreaName: string;
  pageNumber: number;
  location: Quadrilateral;
  lineResults: DLRLineResult[];
}

export interface Quadrilateral{
  points:Point[];
}

export interface Point {
  x:number;
  y:number;
}

export interface DLRLineResult {
  text: string;
  confidence: number;
  characterModelName: string;
  characterResults: DLRCharacherResult[];
  lineSpecificationName: string;
  location: Quadrilateral;
}

export interface DLRCharacherResult {
  characterH: string;
  characterM: string;
  characterL: string;
  characterHConfidence: number;
  characterMConfidence: number;
  characterLConfidence: number;
  location: Quadrilateral;
}

Supported Platforms

Android
iOS

Blog

Contributing

See the contributing guide to learn how to contribute to the repository and the development workflow.

License

MIT

vision-camera-dynamsoft-label-recognizer's People

Contributors

Stargazers

Watchers

Forkers

tminhtri92 xyztlp reyna168 majirosstefan

vision-camera-dynamsoft-label-recognizer's Issues

Frame Not detecting the labels

@xulihang @tony-xlh
What is the issue can you tell please as it is not recongizing text label
code:
import React, {useEffect, useState} from 'react';
import {View, Text, StyleSheet} from 'react-native';
import {
useCameraPermission,
useMicrophonePermission,
useCameraDevice,
Camera,
useFrameProcessor,
runAtTargetFps,
useCameraFormat
} from 'react-native-vision-camera';
import { recognize, ScanConfig, ScanRegion, DLRCharacherResult, DLRLineResult, DLRResult } from 'vision-camera-dynamsoft-label-recognizer';

import { Worklets, useSharedValue } from 'react-native-worklets-core';

const App = () => {

const [frameWidth, setFrameWidth] = React.useState(1280);
const [frameHeight, setFrameHeight] = React.useState(720);

const {
hasPermission: cameraPermission,requestPermission: requestCameraPermission} = useCameraPermission();
const {
hasPermission: microphonePermission, requestPermission: requestMicrophonePermission,} = useMicrophonePermission();
const cameraDevice = useCameraDevice('back');
const [loading, setLoading] = useState(true);

useEffect(() => {
if (!cameraPermission) {
requestCameraPermission();
}
if (!microphonePermission) {
requestMicrophonePermission();
}
}, [cameraPermission, microphonePermission]);

useEffect(() => {
if (cameraPermission && microphonePermission) {
setLoading(false);
}
}, [cameraPermission, microphonePermission]);

const format = useCameraFormat(cameraDevice, [
{ videoResolution: { width: 1280, height: 720 } },
{ fps: 10 }
])

const scanRegion:ScanRegion = {
left: 0,
top: 0,
width: 100,
height: 100
}

const updateFrameSize = (width:number, height:number) => {
if (width != frameWidth && height!= frameHeight) {
setFrameWidth(width);
setFrameHeight(height);
}
}

const updateFrameSizeJS = Worklets.createRunInJsFn(updateFrameSize);

const frameProcessor = useFrameProcessor((frame) => {
'worklet';

Explain
runAtTargetFps(1, () => {
'worklet'
updateFrameSizeJS(frame.width, frame.height);

let config:ScanConfig = {};

console.log("frame width:"+frame.width);
console.log("frame height:"+frame.height);
config.scanRegion = scanRegion;
config.includeImageBase64 = true;
let scanResult = recognize(frame,config);

let results:DLRResult[] = scanResult.results;
let lineResults:DLRLineResult[] = [];
for (let index = 0; index < results.length; index++) {
const result = results[index];
const lines = result?.lineResults;
if (lines) {
lines.forEach(line => {
lineResults.push(line);
});
}
}
console.log('result is ',results);

})
}, []);
return (

{loading ? (
Loading...
) : !cameraPermission || !microphonePermission ? (

Please grant camera and microphone permissions to use the app.

) : !cameraDevice ? (
No camera device available.
) : (
<Camera

frameProcessor={frameProcessor}
format={format}
pixelFormat='yuv'
style={styles.camera} device={cameraDevice} isActive={true} />

)}

);
};

const styles = StyleSheet.create({
container: {
flex: 1,
justifyContent: 'center',
alignItems: 'center',
},
text: {
fontSize: 24,
fontWeight: 'bold',
textAlign: 'center',
},
camera: {
flex: 1,
width: '100%',
},
});

export default App;

and this output
LOG frame width:1920
LOG frame height:1080
LOG {}

Frame Processor Error: Regular javascript function '' cannot be shared. Try decorating the function with the 'worklet' keyword to allow the javascript function to be used as a worklet., js engine: VisionCamer

Hi, thanks very much for this pluggin.

I'm persistently getting:

Frame Processor Error: Regular javascript function '' cannot be shared. Try decorating the function with the 'worklet' keyword to allow the javascript function to be used as a worklet., js engine: VisionCamera

When running the example code on a dev build (expo).

Any pointers would be greatly appreciated.

Tom