microsoft / azure-percept-advanced-development Goto Github PK

Describe the bug
While following the steps in Banana Tutorial under the section "Running the notebook" I am seeing this error appear:

"error": {
"code": "ServiceError",
"severity": null,
"message": "AzureMLCompute job failed.\nServiceError: runTaskLetTask failed because: libffi.so.7: cannot open shared object file: No such file or directory\n\tReason: Job failed with non-zero exit Code",
"messageFormat": null,
"messageParameters": null,
"referenceCode": null,
"detailsUri": null,
"target": null,
"details": [],
"innerError": null,
"debugInfo": null,
"additionalInfo": null
},
"correlation": {
"operation": "cf742550df05044dbf2b80b3397f31d7",
"request": "8ae5bc2bda1f4e59"
},
"environment": "australiaeast",
"location": "australiaeast",
"time": "2021-07-16T05:26:14.7993082+00:00",
"componentName": "execution-worker"

} <

The note book I am running is SemanticsSegmentationUNet.ipynb

Screenshots

Describe the bug
Code azure-percept-advanced-development/ssd.cpp at main · Microsoft/azure-percept-advanced-development (github.com) results in compile error when considering camera feed.

To Reproduce
recompile the azureeyemodule using the current code

Expected behavior
no compile error

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
We modified the following to make it work:

// before: auto pipeline = graph.compileStreaming(cv::compile_args(networks, kernels, cv::gapi::mx::mvcmdFile{ this->mvcmd })); 
auto pipeline = graph.compileStreaming(cv::gapi::mx::Camera::params(), cv::compile_args(networks, kernels, cv::gapi::mx::mvcmdFile{ this->mvcmd }));
...
// before: pipeline.setSource<cv::gapi::wip::GCaptureSource>(video_file_path);
pipeline.setSource(cv::gapi::wip::make_src<cv::gapi::mx::Camera>()); 
...
// before: pipeline.setSource<cv::gapi::wip::GCaptureSource>(0); 
pipeline.setSource(cv::gapi::wip::make_src<cv::gapi::mx::Camera>());

Hi,

great news with this development sdk.
However, I was wondering where you document the custom possibilities with the audio module.
Would be interested to get access to the audio stream.
Do you plan to have a similar module as with azureeymodule?

Thanks

format that can be converted to OpenVINO IR.

here for a compatiblity matrix.

Only certain ops/layers are supported.

There may be more, but these were ones I needed more info on. Not sure, perhaps the OpenVINO site is down temporarily, and this is only temporary... Worth testing.

While attempting the prerequisites, executing "run_workbench.ps1", I get this error.

`.\run_workbench.ps1 : File C:\percept\azure-percept-advanced-development\scripts\run_workbench.ps1 cannot be loaded.
The file C:\percept\azure-percept-advanced-development\scripts\run_workbench.ps1 is not digitally signed. You cannot
run this script on the current system. For more information about running scripts and setting execution policy, see
about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170.
At line:1 char:1

• .\run_workbench.ps1

•   + CategoryInfo          : SecurityError: (:) [], PSSecurityException
    + FullyQualifiedErrorId : UnauthorizedAccess`
  
  

How do I digitally sign this script for you?

If i create a brand new iothub with different name. how to i get azure percept to work again

i tried editing the device_connection_string in /etc/iotedge/config.yaml
it connects to the device in the new iothub but only has two modules.
edgeAgent and edgeHub

Firstly, thank you for great documents.

It seems like currently only C++ is supported to develop edge application.
In a future, will Percept support other language? (e.g. Python, Rust)

Thank you.

Step 1 of the tutorial fails.

It states "create a new folder in your [AML] workspace"

1. there is nothing called "workspace"
   but I do see a Azure Machine Learning (AML) Studio

2. There is no 'folder' option in the Create New menu

3. There really should be a link in step one to assist new developers to the correct 'workspace'.

Essential the entire tutorial is out of reach because step 1 cannot be completed.

Describe the bug

After submitting the training job:
run = experiment.submit(config)

The training script gives an error:
User program failed with HTTPError: HTTP Error 404: Not Found

Driver log attached:
70_driver_log.txt

I'm pretty new to this software, and I have been trying to do the bananas tutorial for a couple days now. I keep getting to the part where we must run the code (which I know takes hours), but it always ends up crashing after 7 hours.

Not sure if this is because I am doing something wrong, or maybe because I am on MacOS, but any help would be appreciated

It would be nice if you could provide the WebStreamModule sample code.

Describe the bug
Not sure if its a bug, my environment is not the same.
Repeated error message below when doing the myriad_compile step
E: [xLinkUsb] [ 367214] [myriad_compile] usb_find_device_with_bcd:266 Library has not been initialized when loaded

To Reproduce
Run up to Step 9 of https://github.com/microsoft/azure-percept-advanced-development/blob/main/machine-learning-notebooks/train-from-scratch/SemanticSegmentationUNet.ipynb
I'm doing the steps locally (not on a Azure ML remote compute instance). I have the model built locally.
Next, I was taking the model thru the conversion steps in the openvino container.
Container is openvino/ubuntu18_dev:2021.3
Prior steps, listed below, succeeded:

• pytorch model to onnx - OK
• onnx to IR - OK
• IR to blob - Fail

The exact command is

source /opt/intel/openvino_2021/bin/setupvars.sh 

/opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/myriad_compile \                                                                                                                    
8    -m intel/bananas.xml -o intel/bananas.blob -VPU_NUMBER_OF_SHAVES 8 -VPU_NUMBER_OF_CMX_SLICES 8 -ip U8 -op FP32 

Thank you for taking a look.

Update: seems harmless as it generated a blob file.

Describe the bug
I tried to find the OpenVINO version of the default object detection model, ssd_mobilenet_v2_coco.blob, but couldn't find it.

In README.md, it said the default SSD model is from here. However, I checked the labels, the model on Percept has 183 labels, but the model in OpenVINO has only 92 labels. Where is the OpenVINO version of the default ssd_mobilenet_v2_coco.blob?

To Reproduce
None

Expected behavior
The labels on Azure Percept should be the same as The labels in OpenVINO.

Screenshots
None

Logs
None

Additional context
None

When reading the document, we feel the following aspects are not clear enough. Some points mentioned below are not easy to find:

• An overview page that shows people e2e structure of IoT edge modules running on Azure Percept, including relationships between different modules, providing links to different resources.
• A well-structured documentation that gives instructions to developers and machine learning engineers (or data scientists) on how to add custom modules.
• A clear documentation that shows how to compile azureeyemodule end-to-end on Windows and Linux OS. Current document provided in github is not clear in the following aspects:
  • Differences between native compile and non-native compile?
  • When to use a native compile method and when to use non-native compile method?
  • how to revert (or fixed) if any steps go wrong? (reset?)
• The documentation doesn't point out modules to be stopped if re-deploy custom azureeyemodule is needed -- we need to stop both azureeyemodule and IoT agent
• The document doesn't tell the readers that rtsp:3000 is working when you customize it using a native way.
• The document doesn't state that the readers can check 8554 using a VLC player
• The document should clearly state expected drivers for external devices, for example, whether a driver for external monitor is expected.
• The document should be revisited and updated frequently based on the OpenVINO update, and check if the recommended approaches of converting models to blob are still valid.
• The document should state that it is possible to convert model on device using the built-in code provided by azureeyemodule, along with the required model files and format. Currently, when reading the code, we understand it is possible to do so, but no instructions or sample model files are provided.
• It would be nice if the document provided a reference link to OpenCV G-API
• It would be nice if the document shows the structure of the model zip file required by azureeyemodule.
• It would be nice if the hardware spec could also provide the GFLOPs besides TOPs. Or provide latency benchmark on the default models provided in azureeyemodule repo.

Seem azureeyemodule:2112-1 uses non-root permission and access uvc camera will get error.
Any suggestion for accessing uvc camera at azureeyemodule:2112-1?

Thank you.

Hi, is there a way to add an inference script (like when deploying a model in azure ml) to do per/postprocessing? if not is there a way to access the frames of the camera video stream?

what I want to achieve is to use a pre-trained model deployed in a container to detect an object in an image frame and then do some postprocessing and send the result with the image frame to another deployed container

Describe the bug
Following the PyTorch from Scratch Tutorial I get the following error, when trying to run detection through the mock-eye-module:

terminate called after throwing an instance of 'std::invalid_argument'  
  what():  stoi

The model used is pulled from openvino workbench zoo. The default ssd model to test the mock-eye-module container. Hence error should not be related to the model ssd_mobilenet_v2_coco.

Mock-eye-module by itself is compiling fine.

Based on simple debugging, error seems to be from the G-API when executed.

To Reproduce
Steps to reproduce the behavior:
Follow all steps in Prerequisites.

When converted model, and video person-bicycle-car-detection.mp4 downloaded.

I run the following on mac:

./scripts/compile_and_test.sh --video=test-artifacts/person-bicycle-car-detection.mp4 --weights=test-artifacts/ssd_mobilenet_v2_coco.bin --xml=test-artifacts/ssd_mobilenet_v2_coco.xml

I run the following on windows:

./scripts/compile_and_test.ps1 -ipaddr <your IP address> -xml test-artifacts/ssd_mobilenet_v2_coco.xml -video test-artifacts/person-bicycle-car-detection.mp4

Both return:

[setupvars.sh] OpenVINO environment initialized
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenCV: /opt/intel/openvino_2021.1.110/opencv (found version "4.5.0") found components:  gapi highgui 
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1") 
-- Checking for module 'gstreamer-1.0>=1.14'
--   Found gstreamer-1.0, version 1.16.2
-- Found InferenceEngine: /opt/intel/openvino/deployment_tools/inference_engine/lib/intel64/libinference_engine.so (Required is at least version "2.0") 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/openvino/tmp/build
Scanning dependencies of target mock_eye_app
[ 14%] Building CXX object CMakeFiles/mock_eye_app.dir/main.cpp.o
[ 28%] Building CXX object CMakeFiles/mock_eye_app.dir/kernels/ssd_kernels.cpp.o
[ 42%] Building CXX object CMakeFiles/mock_eye_app.dir/kernels/utils.cpp.o
[ 57%] Building CXX object CMakeFiles/mock_eye_app.dir/modules/device.cpp.o
[ 71%] Building CXX object CMakeFiles/mock_eye_app.dir/modules/objectdetection/object_detectors.cpp.o
[ 85%] Building CXX object CMakeFiles/mock_eye_app.dir/modules/parser.cpp.o
[100%] Linking CXX executable mock_eye_app
[100%] Built target mock_eye_app
Cannot open labelfile /home/openvino/tmp/labels.txt
Labels will not be available.
terminate called after throwing an instance of 'std::invalid_argument'
  what():  stoi

Expected behavior
Based on the documentation, a window should pop out and stream a video with object detection overplayed.

Additional context
Additional observation
The scripts compile_and_test.sh expects an "example" folder in the mock-eye-module, not included when cloning the repo.
The "mock-eye-module-debug" pulls "openvino/ubuntu18_runtime:latest" instead of "openvino/ubuntu18_runtime:2021.1".

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

I am trying to find a way to combine data from the vision module and data from the Bluetooth module. It is quite hard to determine the right approach for using the Bluetooth module. i was hoping to find any guidance on this here??