ugu11 / camera_tutorial Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
I have cloned your respository and try to run it. But i met an error when build on ios/android:
../../development/flutter/.pub-cache/hosted/pub.dartlang.org/ffi-0.1.3/lib/src/utf8.dart:63:33: Error: The getter 'addressOf' isn't defined for the class 'Utf8'.
- 'Utf8' is from 'package:ffi/src/utf8.dart' ('../../development/flutter/.pub-cache/hosted/pub.dartlang.org/ffi-0.1.3/lib/src/utf8.dart').
Try correcting the name to the name of an existing getter, or defining a getter or field named 'addressOf'.
String toString() => fromUtf8(addressOf);
^^^^^^^^^
../../development/flutter/.pub-cache/hosted/pub.dartlang.org/ffi-0.1.3/lib/src/utf8.dart:23:7: Error: Struct 'Utf8' is empty. Empty structs and unions are undefined behavior.
class Utf8 extends Struct {
^
../../development/flutter/.pub-cache/hosted/pub.dartlang.org/ffi-0.1.3/lib/src/utf16.dart:16:7: Error: Struct 'Utf16' is empty. Empty structs and unions are undefined behavior.
class Utf16 extends Struct {
^
../../development/flutter/.pub-cache/hosted/pub.dartlang.org/ffi-0.1.3/lib/src/allocation.dart:47:33: Error: Expected type 'T' to be a valid and instantiated subtype of 'NativeType'.
final int totalSize = count * sizeOf<T>();
^
Command PhaseScriptExecution failed with a nonzero exit code
Big thanks for your solution.
I looked at the code for your converter. C file,It is almost the same as the code I found from this issue(https://github.com/flutter/flutter/issues/26348#issuecomment-462321428), why is the code of c fast
Your code:
uint32_t *convertImage(uint8_t *plane0, uint8_t *plane1, uint8_t *plane2, int bytesPerRow, int bytesPerPixel, int width, int height){
int hexFF = 255;
int x, y, uvIndex, index;
int yp, up, vp;
int r, g, b;
int rt, gt, bt;
uint32_t *image = malloc(sizeof(uint32_t) * (width * height));
for(x = 0; x < width; x++){
for(y = 0; y < height; y++){
uvIndex = bytesPerPixel * ((int) floor(x/2)) + bytesPerRow * ((int) floor(y/2));
index = y*width+x;
yp = plane0[index];
up = plane1[uvIndex];
vp = plane2[uvIndex];
rt = round(yp + vp * 1436 / 1024 - 179);
gt = round(yp - up * 46549 / 131072 + 44 - vp * 93604 / 131072 + 91);
bt = round(yp + up * 1814 / 1024 - 227);
r = clamp(0, 255, rt);
g = clamp(0, 255, gt);
b = clamp(0, 255, bt);
image[getRotatedImageByteIndex(y, x, height)] = (hexFF << 24) | (b << 16) | (g << 8) | r;
}
}
return image;
}
my code
Future<Image> convertYUV420toImageColor(CameraImage image) async {
try {
final int width = image.width;
final int height = image.height;
final int uvRowStride = image.planes[1].bytesPerRow;
final int uvPixelStride = image.planes[1].bytesPerPixel;
print("uvRowStride: " + uvRowStride.toString());
print("uvPixelStride: " + uvPixelStride.toString());
// imgLib -> Image package from https://pub.dartlang.org/packages/image
var img = imglib.Image(width, height); // Create Image buffer
// Fill image buffer with plane[0] from YUV420_888
for(int x=0; x < width; x++) {
for(int y=0; y < height; y++) {
final int uvIndex = uvPixelStride * (x/2).floor() + uvRowStride*(y/2).floor();
final int index = y * width + x;
final yp = image.planes[0].bytes[index];
final up = image.planes[1].bytes[uvIndex];
final vp = image.planes[2].bytes[uvIndex];
// Calculate pixel color
int r = (yp + vp * 1436 / 1024 - 179).round().clamp(0, 255);
int g = (yp - up * 46549 / 131072 + 44 -vp * 93604 / 131072 + 91).round().clamp(0, 255);
int b = (yp + up * 1814 / 1024 - 227).round().clamp(0, 255);
// color: 0x FF FF FF FF
// A B G R
img.data[index] = (0xFF << 24) | (b << 16) | (g << 8) | r;
}
}
imglib.PngEncoder pngEncoder = new imglib.PngEncoder(level: 0, filter: 0);
List<int> png = pngEncoder.encodeImage(img);
muteYUVProcessing = false;
return Image.memory(png);
} catch (e) {
print(">>>>>>>>>>>> ERROR:" + e.toString());
}
return null;
}
Hello.
How can i convert your uint32_t *image to openCV Mat in c++?
I had converted but not successed
Here is my code in c++
cv::Mat intMat_RGBA = cv::Mat(cv::Size(oriImgWidth, oriImgHeight), CV_8UC4, image);
std::vector<cv::Mat> RGBA_channels;
cv::split(intMat_RGBA, RGBA_channels);
// remove the alpha channel:
RGBA_channels.pop_back();
std::reverse(RGBA_channels.begin(), RGBA_channels.end());
// and merge back to image:
cv::Mat intMat_BGR;
cv::merge(RGBA_channels, intMat_BGR);
phone: huawei Mate7
Android version: 6.0
CPU: Hisilicon Kirin 925
flutter run --release
using C elapsed time 35-46ms
using Dart elapsed time 431-593ms
why release slow?
import 'dart:ffi';
import 'dart:io';
import 'dart:typed_data';
import 'package:ffi/ffi.dart';
import 'package:flutter/material.dart';
import 'package:camera/camera.dart';
import 'package:image/image.dart' as imglib;
import 'ImagePreview.dart';
import 'image_converter.dart';
typedef convert_func = Pointer<Uint32> Function(
Pointer<Uint8>, Pointer<Uint8>, Pointer<Uint8>, Int32, Int32, Int32, Int32);
typedef Convert = Pointer<Uint32> Function(Pointer<Uint8>, Pointer<Uint8>, Pointer<Uint8>, int, int, int, int);
void main() {
runApp(MyApp());
}
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Camera App',
theme: ThemeData(
primarySwatch: Colors.blue,
// visualDensity: VisualDensity.adaptivePlatformDensity,
),
home: MyHomePage(title: 'Camera App'),
);
}
}
class MyHomePage extends StatefulWidget {
MyHomePage({Key key, this.title}) : super(key: key);
final String title;
@override
_MyHomePageState createState() => _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
CameraController _camera;
bool _cameraInitialized = false;
CameraImage _savedImage;
final DynamicLibrary convertImageLib =
Platform.isAndroid ? DynamicLibrary.open("libconvertImage.so") : DynamicLibrary.process();
Convert conv;
@override
void initState() {
super.initState();
_initializeCamera();
// Load the convertImage() function from the library
conv = convertImageLib.lookup<NativeFunction<convert_func>>('convertImage').asFunction<Convert>();
}
void _initializeCamera() async {
// Get list of cameras of the device
List<CameraDescription> cameras = await availableCameras();
// Create the CameraController
_camera = new CameraController(cameras[0], ResolutionPreset.medium);
_camera.initialize().then((_) async {
// Start ImageStream
await _camera.startImageStream((CameraImage image) => _processCameraImage(image));
setState(() {
_cameraInitialized = true;
});
});
}
void _processCameraImage(CameraImage image) async {
setState(() {
_savedImage = image;
});
}
List<int> _durations = [];
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text(widget.title),
),
body: Stack(
children: [
Center(
child: (_cameraInitialized)
? AspectRatio(
aspectRatio: _camera.value.aspectRatio,
child: CameraPreview(_camera),
)
: CircularProgressIndicator()),
Center(
child: CircularProgressIndicator(
backgroundColor: Colors.red,
))
],
),
floatingActionButton: Row(
mainAxisAlignment: MainAxisAlignment.spaceEvenly,
children: [
RaisedButton(
onPressed: () async {
// print('''
//
//tap FloatingActionButton get img
//''');
// The jump has animation, and all the pushing that you see is not going very fast
// Navigator.push(context, MaterialPageRoute(builder:
// (context) => new ImagePreview(img: img)));
await _convertImageByC();
},
child: Text('ConvertImageByC'),
),
RaisedButton(
onPressed: () async {
await _convertImageByDart();
},
child: Text('ConvertImageByDart'),
),
],
), // This trailing comma makes auto-formatting nicer for build methods.
floatingActionButtonLocation: FloatingActionButtonLocation.centerFloat,
);
}
_convertImageByC() {
Stopwatch stopwatch = Stopwatch();
stopwatch.start();
// Allocate memory for the 3 planes of the image
Pointer<Uint8> p = allocate(count: _savedImage.planes[0].bytes.length);
Pointer<Uint8> p1 = allocate(count: _savedImage.planes[1].bytes.length);
Pointer<Uint8> p2 = allocate(count: _savedImage.planes[2].bytes.length);
// Assign the planes data to the pointers of the image
Uint8List pointerList = p.asTypedList(_savedImage.planes[0].bytes.length);
Uint8List pointerList1 = p1.asTypedList(_savedImage.planes[1].bytes.length);
Uint8List pointerList2 = p2.asTypedList(_savedImage.planes[2].bytes.length);
pointerList.setRange(0, _savedImage.planes[0].bytes.length, _savedImage.planes[0].bytes);
pointerList1.setRange(0, _savedImage.planes[1].bytes.length, _savedImage.planes[1].bytes);
pointerList2.setRange(0, _savedImage.planes[2].bytes.length, _savedImage.planes[2].bytes);
// Call the convertImage function and convert the YUV to RGB
Pointer<Uint32> imgP = conv(p, p1, p2, _savedImage.planes[1].bytesPerRow, _savedImage.planes[1].bytesPerPixel,
_savedImage.width, _savedImage.height);
// Get the pointer of the data returned from the function to a List
List imgData = imgP.asTypedList((_savedImage.width * _savedImage.height));
stopwatch.stop();
int getImgDataTime = stopwatch.elapsedMilliseconds;
stopwatch.reset();
stopwatch.start();
// Generate image from the converted data
imglib.Image img = imglib.Image.fromBytes(_savedImage.height, _savedImage.width, imgData);
// Free the memory space allocated
// from the planes and the converted data
free(p);
free(p1);
free(p2);
free(imgP);
stopwatch.stop();
print(
'get imgData: ${getImgDataTime}ms, imglib.Image.fromBytes: ${stopwatch.elapsedMilliseconds}ms, total: ${getImgDataTime + stopwatch.elapsedMilliseconds}ms');
_durations.add(getImgDataTime + stopwatch.elapsedMilliseconds);
if (_durations.length == 10) {
int totalMilliseconds = 0;
for (var value in _durations) {
totalMilliseconds += value;
}
double averageMilliseconds = totalMilliseconds / 10;
print('Ten takes an average time: ${averageMilliseconds}ms');
_durations.clear();
}
}
_convertImageByDart() async {
Stopwatch stopwatch = Stopwatch();
stopwatch.start();
await convertYUV420toImageColor(_savedImage);
stopwatch.stop();
print('convertYUV420toImageColor time: ${stopwatch.elapsedMilliseconds}ms');
_durations.add(stopwatch.elapsedMilliseconds);
if (_durations.length == 10) {
int totalMilliseconds = 0;
for (var value in _durations) {
totalMilliseconds += value;
}
double averageMilliseconds = totalMilliseconds / 10;
print('Ten takes an average time: ${averageMilliseconds}ms');
_durations.clear();
}
}
static const shift = (0xFF << 24);
Future<List<int>> convertYUV420toImageColor(CameraImage image) async {
try {
final int width = image.width;
final int height = image.height;
final int uvRowStride = image.planes[1].bytesPerRow;
final int uvPixelStride = image.planes[1].bytesPerPixel;
// print("uvRowStride: " + uvRowStride.toString());
// print("uvPixelStride: " + uvPixelStride.toString());
// imgLib -> Image package from https://pub.dartlang.org/packages/image
var img = imglib.Image(width, height); // Create Image buffer
// Fill image buffer with plane[0] from YUV420_888
for (int x = 0; x < width; x++) {
for (int y = 0; y < height; y++) {
final int uvIndex = uvPixelStride * (x / 2).floor() + uvRowStride * (y / 2).floor();
final int index = y * width + x;
final yp = image.planes[0].bytes[index];
final up = image.planes[1].bytes[uvIndex];
final vp = image.planes[2].bytes[uvIndex];
// Calculate pixel color
int r = (yp + vp * 1436 / 1024 - 179).round().clamp(0, 255);
int g = (yp - up * 46549 / 131072 + 44 - vp * 93604 / 131072 + 91).round().clamp(0, 255);
int b = (yp + up * 1814 / 1024 - 227).round().clamp(0, 255);
// color: 0x FF FF FF FF
// A B G R
img.data[index] = shift | (b << 16) | (g << 8) | r;
}
}
return img.getBytes();
} catch (e) {
print(">>>>>>>>>>>> ERROR:" + e.toString());
}
return null;
}
}
Hey Hugand. Love your work. So I've been trying to send live frames to server after converting to base64. But the conversion from yuv to normal image for every image captured takes a lot of time. Is it possible to make it real-time? I just want to send every frame to the server and I can't convert yuv or bgra directly to base64 . Please help.
device: iPhone6s plus
iOS version: 13.3.1
Launching lib/main.dart on 干志雄的 iPhone in debug mode...
Found saved certificate choice "iPhone Developer: Guangjian Wang (N8AC8H42N4)". To clear, use "flutter config".
Signing iOS app for device deployment using developer identity: "iPhone Developer: Guangjian Wang (N8AC8H42N4)"
Running Xcode build...
Xcode build done. 13.1s
Installing and launching... 18.4s
Debug service listening on ws://localhost:1024/ws
Syncing files to device 干志雄的 iPhone...
flutter: ══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
flutter: The following ArgumentError was thrown building Builder:
flutter: Invalid argument(s): Failed to lookup symbol (dlsym(RTLD_DEFAULT, convertImage): symbol not found)
flutter:
flutter: The relevant error-causing widget was:
flutter: MaterialApp
flutter: file:///Users/ganzhixiong/Documents/yilisheng/TemperatureSensor/test/camera_tutorial-master.zip%20Folder/camera_tutorial-master/lib/main.dart:23:12
flutter:
flutter: When the exception was thrown, this was the stack:
flutter: #0 DynamicLibrary.lookup (dart:ffi-patch/ffi_dynamic_library_patch.dart:33:29)
flutter: #1 _MyHomePageState.initState (package:camera_tutorial/main.dart:57:28)
flutter: #2 StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4644:58)
flutter: #3 ComponentElement.mount (package:flutter/src/widgets/framework.dart:4480:5)
flutter: ... Normal element mounting (24 frames)
flutter: #27 Element.inflateWidget (package:flutter/src/widgets/framework.dart:3450:14)
flutter: #28 MultiChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:5951:32)
flutter: ... Normal element mounting (119 frames)
flutter: #147 Element.inflateWidget (package:flutter/src/widgets/framework.dart:3450:14)
flutter: #148 MultiChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:5951:32)
flutter: ... Normal element mounting (261 frames)
flutter: #409 Element.inflateWidget (package:flutter/src/widgets/framework.dart:3450:14)
flutter: #410 Element.updateChild (package:flutter/src/widgets/framework.dart:3218:18)
flutter: #411 RenderObjectToWidgetElement._rebuild (package:flutter/src/widgets/binding.dart:1129:16)
flutter: #412 RenderObjectToWidgetElement.mount (package:flutter/src/widgets/binding.dart:1100:5)
flutter: #413 RenderObjectToWidgetAdapter.attachToRenderTree.<anonymous closure> (package:flutter/src/widgets/binding.dart:1042:17)
flutter: #414 BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2607:19)
flutter: #415 RenderObjectToWidgetAdapter.attachToRenderTree (package:flutter/src/widgets/binding.dart:1041:13)
flutter: #416 WidgetsBinding.attachRootWidget (package:flutter/src/widgets/binding.dart:922:7)
flutter: #417 WidgetsBinding.scheduleAttachRootWidget.<anonymous closure> (package:flutter/src/widgets/binding.dart:903:7)
flutter: (elided 11 frames from class _RawReceivePortImpl, class _Timer, dart:async, and dart:async-patch)
flutter:
flutter: ════════════════════════════════════════════════════════════════════════════════════════════════════
Lost connection to device.
I met a problem with this function
Convert conv = convertImageLib
.lookup<NativeFunction<convert_func>>('convertImage')
.asFunction<Convert>();
[VERBOSE-2:ui_dart_state.cc(199)] Unhandled Exception: Invalid argument(s): Failed to lookup symbol (dlsym(RTLD_DEFAULT, convertImage): symbol not found)
#0 DynamicLibrary.lookup (dart:ffi-patch/ffi_dynamic_library_patch.dart:31:29)
#0 DynamicLibrary.lookup (dart:ffi-patch/ffi_dynamic_library_patch.dart:31:29)
#1 convertCameraImage
package:flutter_app/utils/cameraImage.util.dart:20
#2 _CameraState.handleStream.<anonymous closure>
package:flutter_app/main.dart:74
#3 _CameraState.handleStream.<anonymous closure>
package:flutter_app/main.dart:66
#4 CameraController.startImageStream.<anonymous closure>
package:camera/src/camera_controller.dart:395
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.