Hi, thanks for this nice plugin. I just tried to use your ARKitExamp

Plugin crashes randomly if ARKitExample is used together with ARKitFaceTrackingConfiguration (Partially solved),about wtct-hungary/unityvision-ios

Comments (7)

borderlineinteractive commented on June 13, 2024 1

Hi Adam, Works very nicely! Thanks a lot for your help! Best wishes, Leif

On 23. Nov 2018, at 13:06, Adam Hegedus ***@***.***> wrote: Hi Leif, I added a new example for ARKit. I hope this approach solves your problem. It uses the texture that Unity's ARKit plugin generates from the capturedImage (CVPixelBuffer) of the current ARFrame. I want you to try this first before having to create a deep copy of the pixel buffer every frame in your client application. You can find it in ARKitExample2.cs. Try it out and let me know if it works for you. Best regards, Adam — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AYBTB0yIRcH0QeyKPUUjV4LrciFUHw30ks5ux-TXgaJpZM4YoC8e>.

from unityvision-ios.

adamhegedues commented on June 13, 2024

Hi there,

It seems to me that the CVPixelBuffer ARKit exposes gets deallocated before it arrives to the native vision plugin. Remember, The Unity ARKit plugin has no idea that UnityVision is reading its image buffer, so if Unity made changes how and when the buffer for the front facing camera gets released, that may cause problems for UnityVision.

There are ways in UnityVision to allocate a buffer that it owns. In the ARKitExample, UnityVision steals the buffer reference from ARKit, because this way we save a copy. But in the webcam example, an independent metal texture is used for classification. Also, I have written a managed wrapper for CVPixelBuffer. You can check it out in the core video example.

I may implement a way to duplicate native CVPixelBuffers, so you don't have to rely on the WebCamTexture API when you also use ARKit. You would then just make a copy of ARKit's buffer and its guaranteed that it won't get released before time. I just need to allocate some time for this. Meanwhile, you can use a webcam texture.

Please keep me updated!

from unityvision-ios.

borderlineinteractive commented on June 13, 2024

Thanks a lot! This clarifies what is going on and I should be able to fix this.

…

----------------------------------------- Priv.-Doz. Dr. Leif Dehmelt Max-Planck-Institute of Molecular Physiology and Technical University Dortmund Department for Systemic Cell Biology Room CP-02-157 Otto-Hahn-Str. 4a 44227 Dortmund Tel.: ++49-231-755-7057 Fax: ++49-231-133-2299 Website: http://www.mpi-dortmund.mpg.de/61312/Dehmelt

Am 19.11.2018 um 18:00 schrieb Adam Hegedus ***@***.***>: Hi there, It seems to me that the CVPixelBuffer ARKit exposes gets deallocated before it arrives to the native vision plugin. Remember, The Unity ARKit plugin has no idea that UnityVision is reading its image buffer, so if Unity made changes how and when the buffer for the front facing camera gets released, that may cause problems for UnityVision. There are ways in UnityVision to allocate a buffer that it owns. In the ARKitExample, UnityVision steals the buffer reference from ARKit, because this way we save a copy. But in the webcam example, an independent metal texture is used for classification. Also, I have written a managed wrapper for CVPixelBuffer. You can check it out in the core video example. I may implement a way to duplicate native CVPixelBuffers, so you don't have to rely on the WebCamTexture API when you also use ARKit. You would then just make a copy of ARKit's buffer and its guaranteed that it won't get released before time. I just need to allocate some time for this. Meanwhile, you can use a webcam texture. Please keep me updated! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

from unityvision-ios.

borderlineinteractive commented on June 13, 2024

Hi Adam, Thanks again for your help. Unfortunately, the WebcamTexture does not seem to work together with ARKit. On the other hand, starting from the unityArCamera object, I don't know how to safely access the cvPixelBufferPtr to generate a persistent copy of the cvPixelBuffer. I would be very helpful, if you could point me towards the right direction. Best wishes, Leif

…

On 19. Nov 2018, at 18:00, Adam Hegedus ***@***.***> wrote: Hi there, It seems to me that the CVPixelBuffer ARKit exposes gets deallocated before it arrives to the native vision plugin. Remember, The Unity ARKit plugin has no idea that UnityVision is reading its image buffer, so if Unity made changes how and when the buffer for the front facing camera gets released, that may cause problems for UnityVision. There are ways in UnityVision to allocate a buffer that it owns. In the ARKitExample, UnityVision steals the buffer reference from ARKit, because this way we save a copy. But in the webcam example, an independent metal texture is used for classification. Also, I have written a managed wrapper for CVPixelBuffer. You can check it out in the core video example. I may implement a way to duplicate native CVPixelBuffers, so you don't have to rely on the WebCamTexture API when you also use ARKit. You would then just make a copy of ARKit's buffer and its guaranteed that it won't get released before time. I just need to allocate some time for this. Meanwhile, you can use a webcam texture. Please keep me updated! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AYBTB8WOdcehiZlV4CVtVEki84QsBBwbks5uwuPGgaJpZM4YoC8e>.

from unityvision-ios.

adamhegedues commented on June 13, 2024

Hi Leif,

I added a new example for ARKit. I hope this approach solves your problem.
It uses the texture that Unity's ARKit plugin generates from the capturedImage (CVPixelBuffer) of the current ARFrame. I want you to try this first before having to create a deep copy of the pixel buffer every frame in your client application.

You can find it in ARKitExample2.cs. Try it out and let me know if it works for you.

Best regards,
Adam

from unityvision-ios.

Ilesh commented on June 13, 2024

Hi Adam,

I am working on the measurement application like the apple's Measure App. I have tried with ARKit but not getting any satisfying result. After looking at your demo I think it's useful for my application. Please find below link for my requirements.

Edge-detection-corner-detection-in-AR

I am waiting for your acknowledgement.

Thanks and regards,
Ilesh

from unityvision-ios.

adamhegedues commented on June 13, 2024

Hi Ilesh,

This plugin features rectangle detection which only gives you results reliably if a rectangular object is visible in full. You can try implementing your app with the current state of the plugin, but I don't guarantee anything. Also, since you don't need image classification, you'll probably need to strip out that feature from the plugin because the mlmodel is over a 100 MBs.

If I were you I'd probably look for a native edge detection solution for iOS and use this plugin as an example how to integrate it with Unity.

Best regards,
Adam

from unityvision-ios.

Plugin crashes randomly if ARKitExample is used together with ARKitFaceTrackingConfiguration (Partially solved) about unityvision-ios HOT 7 CLOSED

Comments (7)

Related Issues (10)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent