takuya-takeuchi / ncnndotnet Goto Github PK
View Code? Open in Web Editor NEWncnn wrapper written in C++ and C# for Windows, MacOS, Linux, iOS and Android
License: MIT License
ncnn wrapper written in C++ and C# for Windows, MacOS, Linux, iOS and Android
License: MIT License
On OSX, compiler says
/Users/t-takeuchi/Work/OpenSource/NcnnDotNet/src/NcnnDotNet.Native/ncnn/layer/layer.h:17:11: warning: 'auto' type specifier is a C++11 extension [-Wc++11-extensions]
Hello, I've tried just about everything I can think of and have tried so many different methods and combinations. I am unsure of what I am doing wrong here.. The only thing I can think of is the PixelFormat but I am unsure if that really matters when converted that way I am doing it.
First I grab a screenshot:
public Bitmap? ScreenGrab(Rectangle detectionBox)
{
if (_graphics == null || _screenCaptureBitmap == null || _screenCaptureBitmap.Width != detectionBox.Width || _screenCaptureBitmap.Height != detectionBox.Height)
{
_screenCaptureBitmap?.Dispose();
_screenCaptureBitmap = new Bitmap(detectionBox.Width, detectionBox.Height);
_graphics?.Dispose();
_graphics = Graphics.FromImage(_screenCaptureBitmap);
}
_graphics.CopyFromScreen(detectionBox.Left, detectionBox.Top, 0, 0, detectionBox.Size);
return _screenCaptureBitmap;
}
I have tried to convert the bitmap so many different ways and all of them have lead to a System.AccessViolationException.
This was my last attempt using OpenCVSharp's Extensions:
Bitmap? frame = ScreenGrab(detectionBox);
if (frame == null) return null;
//float[] inputArray = BitmapToFloatArray(frame);
//if (inputArray == null) return null;
OpenCvSharp.Mat mat = OpenCvSharp.Extensions.BitmapConverter.ToMat(frame);
using var NCmat = new NcnnDotNet.Mat(mat.Width, mat.Height, mat.Data);
_ex.Input("in0", NCmat);
using var outMat = new NcnnDotNet.Mat();
_ex.Extract("out0", outMat);
Debug.WriteLine(outMat.W);
The violation is on the Input bool on this line:
if (NativeMethods.net_Extractor_input(base.NativePtr, bytes, bytes.Length, @in.NativePtr) != 0)
Here are the model files:
yolov8n_ncnn_model.zip
Hello again
I was able to write a c# port of stable-diffusion-ncnn using your library and it works as intended.
However i noticed a big difference in terms of RAM usage
For example, the following code uses 5.5 GB of RAM
Net net = new Net();
net.LoadParam("./assets/UNetModel-fp16.param");
net.LoadModel("./assets/UNetModel-fp16.bin");
while the original one in c++ uses only 3GB
net.load_param("assets/UNetModel-fp16.param");
net.load_model("assets/UNetModel-fp16.bin");
Its not vital for me to reduce memory usage but out of curiosity, do you have any idea of what is causing this?
Thanks for your time
public static void Retina(NcnnDotNet.OpenCV.Mat src)
{
// Model from https://github.com/nihui/ncnn-assets/tree/master/models
Net net = new Net();
net.Opt.UseVulkanCompute = false;
net.LoadParam(@"Models\mnet.25-opt.param");
net.LoadModel(@"Models\mnet.25-opt.bin");
var imgW = src.Cols;
var imgH = src.Rows;
NcnnDotNet.Mat @in = NcnnDotNet.Mat.FromPixelsResize(
pixel: src.Data,
type: PixelType.Bgr2Rgb,
width: src.Cols,
height: src.Rows,
targetWidth: imgW,
targetHeight: imgH
);
NcnnDotNet.Extractor ex = net.CreateExtractor();
ex.Input("data", @in);
var scoreBlob = new NcnnDotNet.Mat();
var bboxBlob = new NcnnDotNet.Mat();
var landmarkBlob = new NcnnDotNet.Mat();
ex.Extract("face_rpn_cls_prob_reshape_stride32", scoreBlob);
ex.Extract("face_rpn_bbox_pred_stride32", bboxBlob);
ex.Extract("face_rpn_landmark_pred_stride32", landmarkBlob);
}
It's possible to have a way to get a Mat from a Bitmap or a byte[]?
Right now I need to create a Bitmap, use ToMap Extension from OpenCVSharp then I could use Mat.FromPixels.
Would be much simple if I could get the Mat use in NcnnDotNet directly.
hello,
i'm trying to port a ncnn project
i need to set some option like these
net.opt.use_winograd_convolution = false;
net.opt.use_sgemm_convolution = false;
i see that they are referenced in this repo but i can not access them
this may be a dumb question but is there a way to do that? thanks
Since 1fc2c7c, it does not work on linux machine except for android and arm.
Hi;
First of all, thank you for your effort developing this great library.
Currently I am working for face detection using UltraFaceDotNet and EmguCV. However I have problem in passing pointer to NcnnDotNet.OpenCV Mat. Below is my code:
UltraFace _ultraFace = UltraFace.Create(_param);
...
using Emgu.CV.Mat frame = new Emgu.CV.Mat();
_videoCapture.Retrieve(frame, 0);
using var inMat = NcnnDotNet.Mat.FromPixels(frame.Ptr, NcnnDotNet.PixelType.Bgr2Rgb, frame.Cols, frame.Rows);
var faceInfos = _ultraFace.Detect(inMat).ToArray();
RectangleF[] rectangles = new RectangleF[faceInfos.Length];
for (var j = 0; j < faceInfos.Length; j++)
{
var face = faceInfos[j];
var pt1 = new Point<float>(face.X1, face.Y1);
var pt2 = new Point<float>(face.X2, face.Y2);
float width = face.X2 - face.X1;
float height = face.Y2 - face.Y1;
rectangles[j] = new RectangleF(face.X1, face.Y1, width, height);
//Cv2.Rectangle(frame, pt1, pt2, new Scalar<double>(0, 255, 0), 2);
}
When I run the code, I got error
System.Exception: 'Capture error'
Inner Exception
AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt
Is there any way to solve the problem?
Or if Ncnndotnet.OpenCV Mat can received bitmap, then I can pass bitmap parameter from Emgu CV Mat.
Thank you
can you support arm cpu?
Thank you so much for the great library
I want to display Mat image on Winforms C# GUI. Could you help me create a function to convert Mat to Bitmap c#?
I gave some tries but i could not.
Thank you in advance
how using NcnnDotNet in android for license plate Cars
Hi @takuya-takeuchi
I just update to newest package from Nuget, try with Yolo3 sample and i get this error:
I also try other samples and get the same error.
Is this a bug?
Thanks
Hello,
sorry for the (perhaps dumb) question, but Im new to ncnn. I
m trying to port RealSR to c# and came to the following lines:
ncnn::ParamDict pd;
pd.set(0, 3);// bicubic
pd.set(1, 4.f);
pd.set(2, 4.f);
bicubic_4x->load_param(pd);
...but the ParamDict-class does not have any method / constructor to "fill" the dict...
Do I get something wrong here? Can anyone help me with thsi?
Thanks a lot
Carl
Hi,
I want to save images after inference, which is a NcnnDotNet.Mat object after inference. I noticed that there is a NcnnDotNet.C.Ncnn.MatToPixels()
method that could be the equivalent of the to_pixels() method in the ncnn. However, I got an error "Unable to convert NcnnDotNet.Mat to NcnnDotNet.C.Mat".
Is there any way that I can convert NcnnDotNet.Mat to NcnnDotNet.C.Mat, or should I save the image with another approach?
PS: I tried to save the byte data copied from NcnnDotNet.Mat.Data, but I can only get a meaningless picture.
Much appreciated.
Hi @takuya-takeuchi
I test Yolov3 sample with two versions of NcnnDotnet (only cpu).
With version 0.0.0.20230101 is more than 500 miliseconds
With version 0.0.0.20221224 is only 78 miliseconds
Both test on Windows 11 x64, Cpu version of NcnnDotNet, the same image size is 1000x754
Is there any changes in Api that can cause lower speed?
Thanks
Hi @takuya-takeuchi
I use VideoCapture from OpencvSharp4 to capture video like this:
OpenCvSharp.VideoCapture mCap = new OpenCvSharp.VideoCapture(0); OpenCvSharp.Mat mat = mCap.RetrieveMat();
I convert OpenCvSharp.Mat to NcnnDotNet.Ma like this:
NcnnDotNet.Mat mm = NcnnDotNet.Mat.FromPixels(mat.Data, NcnnDotNet.PixelType.Bgr2Rgb, mat.Cols, mat.Rows);
Then i convert back to Bitmap for testing result
System.Drawing.Bitmap bm2 = mm.ToBitmap(PixelType.Rgb, PixelFormat.Format24bppRgb);
bm2.Save("C:\bitmap.jpg");
Bit it gives me wrong bitmap
Can you please have a look on my code and point me what i was wrong?
Thank you very much
Always output vkEnumerateInstanceExtensionProperties failed -6
when running NcnnDotNet
Output directory always be likeexamples/YoloV3/bin/x64/Debug/netcoreapp3.1
.
It is occured by nuget/nuspec/build/NcnnDotNet.props
So we have to remove it from nuspec file?
At least, it occurs the following error on linux arm device.
$ dotnet run -c Release -r linux-arm64
/usr/share/dotnet/sdk/3.1.410/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(166,5): error NETSDK1032: The RuntimeIdentifier platform 'linux-arm64' and the PlatformTarget 'x64' must be compatible. [/home/t-takeuchi/Work/OpenSource/NcnnDotNet/examples/YoloV3/YoloV3.csproj]
When I run "pwsh BuildWindows.ps1", it said "Error: This platform is not support". Could you please upload NcnnDotNetNative.dll?
ncnn does have a C api now https://github.com/Tencent/ncnn/blob/master/src/c_api.h
Hi,
I'm trying porting example of yolov5
but there's a costom layer named YoloV5Focus
how to handle this case in c#
would you like to give some instructions on this?
thanks!
great work!
the performance improve a lot as its release note said
how to update native code of NcnnDotNet to the latest?
and there're some new examples added,
do you have some plan for that ?
thanks!
how to use? no doc.
i'm try use this lib for yolov7.
but show is error. how to use ?
doc???
ncnn uses YYYYMMDD as tag name and releases.
But, this format may be changed.
If version format is changed to X.Y.Z format, new version is treated as old.
So format should be 0.0.0.YYYYMMDD.
https://github.com/deepinsight/insightface/tree/master/detection/scrfd
After I load the SCRFD_2.5G algorithm model, an error is reported
so excited to find NcnnDotNet , It's great work!
is it possible to transfer byte array or stream to mat ?
In .net core website, I want to infer the file (IFormFile) uploaed without saving it.
how to make it ?
thx!
I want to put the models and params files into the project resources, and load them directly from memory.
Is there any way?
Thanks!
Hi @takuya-takeuchi ,
I got problem with running NcnnDotNet with camera.
I do not know if the Extractor should be created one time at application starting or re-create in every frame of camera?
Can you please have a look on this problem?
Thank you very much,
Hello the MobileNetV3SSDLite with the example link: 'https://github.com/xingjinglu/models_collection/tree/master/ncnn' is not working.
Hello @takuya-takeuchi
I'm running NcnnDorNet on windows 10 X64 and I have a customized model with 25 classes.
when I use NcnnDotNet ,Everything is going well and the bounding boxes are returned successfully expect the labels with id greater than 10. their label are returned 10.
However it doesn't happen for the labels with lower than 10.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.