mit-cml / appinventor-extensions Goto Github PK

Reading the button status in the micro:bit Button extension crashes the app, but the same method in the massive component with all features works okay. Via @cavedunissin

We use float base 2 parsing in the various read/write calls for float. Some devices prefer to use float base 10 instead (BLE spec allows for both representations). We should add support for base 2 or base 10 parsing to make it easier to interact with devices using the base 10 representation.

And write up tutorial for it after the project is done

We should investigate how to allow instances of the BLE extension to share connections across screens to support those who design multiscreen apps.

Where (in whichever folder) save Native library (.so) files? During extension building.

A while back I created an example extension (also in this repo) for how to use a BluetoothLE extension as a communication channel for higher-level extensions, such as our Arduino101 and Microbit extensions. However, this results in a lot of replicated code as each extension has to duplicate the same functionality of receiving a BluetoothLE instance, setting up listeners, handling onPause, onResume, etc. What we should do instead is create in the BluetoothLE extension an abstract base class that these other extensions can extend. This base class would have all the now currently duplicated functionality and allow us to reconcile to a single code path. Further, as we make improvements to the BLE extension or its base class these other extensions would immediately benefit.

Please use the attached project.
MicrobitMagnetometer.aia.zip

Instructions:
http://iot.appinventor.mit.edu/assets/howtos/MIT_App_Inventor_Microbit_Magnetometer.pdf

There is no project for these two microbit parts. But I would like them tested to make sure they work with micorbit V2.

I'm creating 1 extension, which will need to write the manifest if you use the extension.
<meta-data android:name=\"com.google.android.gms.version\" android:value=\"10084000\"/>
how can add this line in extensions ?

Hi,

I'm getting a connection error when trying to connect my app to ESP32-C3 mcu.

I'm using the latest extension BluetoothLE.aix 20230728.
My phone is a Samsung S20+ with Android version 13 and latest software updates.
I've tested the code in the ESP32-C3 by connecting to it using nRF Connect app.

If I downgrade my extension to version 20201223, I can connect when using AI companion, but if I compile this to an apk and install, the app doesn't work (permission errors).

• [ x] Designer
• Blocks editor
• Companion
• Compiled apps
• Buildserver
• Debugging
• Other... (please describe)

Expected behavior

Here's an image of the code I'm using for testing purposes.

Note that there is no call to "gatt.writeDescriptor()" in the "unsubscribe" function called by the "UnregisterForValues" function of the BluetoothLE extension. This means that while the callback gets disabled and the App Inventor app does not receive any notifications from the BLE device any more, the BLE device still generates notifications in the background. The notification bit in the corresponding CCCD on the BLE device has not been cleared.

This is probably not the intended behavior. As BLE devices often try to save power wherever possible, being able to disable notifications on the device is important.

Note that the call to "gatt.writeDescriptor()" is used as expected in the "subscribe" function called by the "RegisterForX" functions of the BluetoothLE extension. Notifications are enabled by a write request to the corresponding CCCD.

I suggest to review the internals of the "unsubscribe" function and to include a call to "gatt.writeDescriptor()" such as to disable the notification bit in the CCCD.

If SerialOTG (USB) is open, you can open a new screen and it will be detected as being open, as well as being able to send and receive using it, even though it was started on a different screen. If you try to do this with bluetoothLE, it doesn't work. Even though you're connected, it will show it's not, and you can't connect again without disconnecting, which you also can't do on a different screen. So in order to use it on a different screen you have to disconnect before opening the new screen, then when the new screen opens you have to connect again. This makes it extremely difficult to do things like AES CTR encryption, which must always stay in sync. It also takes a long time to do the disconnect/reconnect and doing this can often be unreliable. Is it possible to make it work across screens without the massive headache?

Describe the bug
When lots of users start to use microbit in App Inventor, they get used to manipulate .hex code and don't realize that their .hex code disables blue tooth service accidentally. App Inventor works with Microbit through blue tooth LE extension, App crashes when it tries to connect to microbit through blue tooth LE while blue tooth service is not available in the microbit. We should display an error message and remind users to enable blue tooth service in the device when that happens instead of crashing the companion, which puzzles users.

Affects

• Designer
• Blocks editor
• [ x] Companion
• Compiled apps
• Buildserver
• Debugging
• Other... (please describe)

Expected behavior

Use the attached the .hex code and .aia project file. App Inventor should gracefully let users know that blue tooth service in microbit is not available instead of crashing.

Steps to reproduce

Upload the attached .hex code to microbit and use the attached . aia project file.
open the project file in App Inventor.
click the scan button
click the stop scanning button
Select your microbit device discovered by App Inventor
click connect
Companion disappears.

microbit-empty.hex.zip
MicrobitLED.aia.zip

Describe the bug

PersonalAudioClassifier extension only available currently to users in a project template file. Needs to be added to the extensions page for download.

Affects

• Designer
• Blocks editor
• Companion
• Compiled apps
• Buildserver
• [ x] Other... (please describe)

Expected behavior

Should be able to locate aix and import into an App Inventor project

Steps to reproduce

Describe the bug
Test the existing micro bit button project and upgrade it to work with micro bit V2 if needed.

Affects

• Designer
• Blocks editor
• Companion
• [x ] Compiled apps
• Buildserver
• Debugging
• Other... (please describe)
  MicrobitButton.aia.zip

Steps to reproduce

Please check this page for instructions to test the attached project.
http://iot.appinventor.mit.edu/assets/howtos/MIT_App_Inventor_Microbit_Button.pdf

Test attached projects to make sure they still work with micro bit V2.

MicrobitIOpin_button.aia.zip
MicrobitIOpin_RGBLED.aia.zip

Instructions:
http://iot.appinventor.mit.edu/assets/howtos/MIT_App_Inventor_Microbit_IOpin_RGBLED.pdf
http://iot.appinventor.mit.edu/assets/howtos/MIT_App_Inventor_Microbit_IOpin_button.pdf

Bug

I'm tryint to connect an app of AppInventor to my ESP32 and it doesn't work. It realease the error "Connection status was set to OS code 4" and nothing happens.

The same App can connect to other BLE devices (RN4870) that hold the same code, and I can connect to that ESP32 device with other apps. It's just a matter of AppInventor extension with ESP32-wrover.

Mention that I'm using Arduino Release v2.0.14 based on ESP-IDF v4.4.6

Describe the bug

Documentation for Microbit_Io_Pin_Simple is a placeholder. Tooltips need to be added to the source code in order to auto-generate a useful documentation page.

Describe the desired feature
Currently I feel the posenet video frame has hardcoded size of 300x250. Can these become properties for the posenet extension? At least expose the height and width on the extension if they are immutable.

https://github.com/mit-cml/appinventor-extensions/blob/extension/posenet/appinventor/components/src/edu/mit/appinventor/ai/posenet/assets/app.js

const videoWidth = 300; const videoHeight = 250;

Give an example of how this feature would be used
This can give flexibility to the sizing of user defined video display/canvas area. Making them at least immutable properties can help user scale the canvas accordingly when these posenet key points X,Y are only relative to the 300x250 orignal video size.

Why doesn't the current App Inventor system address this use case?

Why is this feature beneficial to App Inventor's educational mission?

Need Immediate assistance regarding how to use YOLO object detection pretrained model (Programmed in python) as well as custom trained model with MIT App Inventor, quick attention over this issue would be highly appreciated!

need to add this file

UsesMetaData.zip
Because I don't know how to add this.
Just like adding the file and its classes to the uses permissions as well and etc.
With this we will be able to get rid of the Dependency Trouble

If you compile an app with BLE and it is run on a version of Android prior to 5.0, we throw an exception in the constructor and this causes the app to crash. Instead, we should dispatch an error. We should also add an "IsAvailable" property to check whether the device has BLE support so that developers can provide feedback to the app user.

Hi there,

I want to use Look Extenxion with Pre-trained TensorFlow.js models,for example mobilenet model.
How can I do it?

BR

If one passes an array of mixed values, including strings, to a Write function in the BLE extension, the extension does not attempt to coerce it to a number and throws an exception in an attempt to cast String to Integer. We should instead test whether the item is a String and then use the appropriate parse method to convert it to a number. If that fails, then we can signal an error.

Describe the bug

The PAC extension originally worked with the template https://drive.google.com/open?id=19quTQ9UWVSQ3VfKKh9R-zBiE35wkKy93

Now, although you get a ClassifierReady event, it never triggers a GotClassification event.

Affects

• Designer
• Blocks editor
• Companion
• Compiled apps
• Buildserver
• Other... (please describe)

Expected behavior
With attached aia compiled and installed on device, you should see "Ready to authenticate" label at bottom of screen. When you click on record and record yourself saying "hello" or "goodbye" , the status label should display response from the classifier, and you should hear "Sorry you cannot open my diary".

voice_authentication_diary_complete.aia.zip

Steps to reproduce

Compile attached aia into apk (Companion manifest does not allow use of microphone with aia), install on device, when app is opened, you get "Ready to authenticate" label that signals classifier is ready. Then press Record button and record "hello" or "goodbye". You should see the status label at the bottom change to the response from classifier, but ... nothing happens.

It would be nice to be able to retrieve the list of classes from the Look extension for building apps.

From the IOT forum: Make the reporting of UUIDs consistent in their case. Some methods return lowercase UUIDs and others return uppercase UUIDs. It would be better if we were consistent in which version is returned.

Also create tutorial for it.

Instructions:
http://iot.appinventor.mit.edu/assets/howtos/MIT_App_Inventor_Microbit_Temperature.pdf

MicrobitTemperature.aia.zip

Describe the bug

Forum report: Facemesh extension appears to not support using the rear-facing camera.

Affects

• Designer
• Blocks editor
• Companion
• Compiled apps
• Buildserver
• Debugging
• Other... (please describe)

Expected behavior

We should support both the front-facing and rear-facing cameras.

Steps to reproduce

Describe the desired feature
Currently only the configuration properties of MinPartConfidence and MinPoseConfidence can be queried? Would like to see PoseConfidence and {Part}Confidence to be available for all parts.

Give an example of how this feature would be used

// the following callback only works when the pose detection passes MinPoseConfidence 
// so I would want to paint the canvas whether there's a pose inside the camera feed or not,
// and meanwhile I want the pose analysis to start only when the picture has an actual pose in it.
when posenetExtension.PoseUpdated {
    canvas.setBackgroundImageInBase64(posenetExtension.backgroundImage);
    if (posenetExtension.PoseConfidence > 0.4) {
        // analyze the pose
    }
}

Same thing goes for {Part}Confidence properties but that can wait.

Why doesn't the current App Inventor system address this use case?
As in the example, currently I have to set a low MinPoseConfidence so that I can paint the canvas's background image on every frame, that means I would have to compute inside the callback each time whether there's a real pose inside the frame or not. This is not good.

Why is this feature beneficial to App Inventor's educational mission?

Please make sure the attached micro bit LED project work with latest micro bit V2.
MicrobitLED.aia.zip

Instructions:
http://iot.appinventor.mit.edu/assets/howtos/MIT_App_Inventor_Microbit_LED.pdf

Is there a chance to return this function as array? In some situations there is few characteristics with the same UUID in the same service.

https://github.com/mit-cml/appinventor-extensions/blob/extension/bluetoothle/appinventor/components/src/edu/mit/appinventor/ble/BluetoothLEint.java#L2730

It would be nice to have function to request MTU change.

From the forum: BluetoothLE extension does not support detecting beacons by UUID. A good use case for why using MAC addresses rather than UUIDs is given. We should explore how we would want to update the extension to be more service oriented rather than hardware oriented.

In the BLE extension we rely on Guava's Lists utility class to perform operations on lists. This works well for the companion app, but causes apps to fail when compiled. Due to the way dex works, we cannot simply add a @UsesLibrary annotation to make this work because then apps built with components that rely on Guava would have errors. We can provide our own implementation to prevent collisions and allow built apps to work.

Creating an extension for microbit V2 speaker.

From the forum: Users would like an API on the BluetoothLE extension to access information about whether the characteristic is readable or writeable.

Are there any appinventor-extensions that support component creation?

Describe the bug

There is a thread on the community site regarding event of readstring is not running

Affects

• Designer
• Blocks editor
• [ x] Companion
• Compiled apps
• Buildserver
• Debugging
• Other... (please describe)

Please refer to https://community.appinventor.mit.edu/t/uart-with-micro-bit-and-app-inventor/8935/18 for details.
MicroBitReceiveText.aia.zip

Extension with UART service

com.bbc.microbit.profile-20200518.aix.zip

microbitRW_2.aia.zip

And write up the tutorial after the project is done.

Describe the bug
I'm interested in using posenet extension supplied on this page: https://mit-cml.github.io/extensions/ and especially I would like to mark the posenet points on the camera feed. I was not able to get the canvas to overlay on top of the webviewer so I have been trying to see how the sample apps accomplish that. However, an error always comes up from ai2.appinventor.mit.edu whenever I try to load the posted sample apps.

"This project was saved with a newer version of the App Inventor system. We will attempt to load the project, but there may be compatibility issues."

ai2.appinventor.mit.edu About info:
Built: February 18 2020 Version: nb182b
Use Companion: 2.57b or 2.57bu
Target Android SDK: 28 (Android 9.0 Pie)

Affects
Obviously, the sample code would not load properly. Almost all the code is missing. I'm seeing a horizontal arrangement with the canvas and the webviewer side by side. So I would not be able to guess how overlay can be implemented.

• Designer
• Blocks editor
• Companion
• Compiled apps
• Buildserver
• Other... (please describe)

Expected behavior
Either the code should load on the production version of ai2.appinventor.mit.edu or you can show me where the beta version that is compatible with the sample apps may be so I can try it out.

Steps to reproduce
Download PosenetTest.aia and TophatMe.aia from https://mit-cml.github.io/extensions/
Load these projects into ai2.appinventor.mit.edu and you see the error message posted above.

Describe the bug

When using the BLE extension, the app crashes on Android versions 12 and 13 because the lack of bluetooth_scan and bluetooth_connect permissions.

Affects

• Designer
• Blocks editor
• [ x] Companion
• [x ] Compiled apps
• Buildserver
• Debugging
• Other... (please describe)

Expected behavior

I am not sure if the permissions are requested by App Inventor in other cases. If thats the case: MIT AI2 Companion/build app requests for the required permissions.

Steps to reproduce

On an Android 12 or Android 13 device.

1. Import Bluetooth LE extension.
2. Scan devices (the app will crash on some devices at this point).
3. Connect to a BLE device (other devices will crash at this point).

Extra info:
I am using BBC micro:bit for my tests.

The bluetoothle extension doesn't seem to provide any capability to receive notifications and/or indications.

Indications and Notifications are a way for a GATT Client to subscribe to data provided by a GATT Server. A Notification is an unacknowledged message or update while an Indication is an acknowledged message or update.

Is there any plan to support notifications and indications?

From David Tseng:

Farzeen and I are working on Micro:bit IOpins, but it forced exit the app when Microbit.WritePinData is called -> java.lang.ClassCastException: gnu.mapping.SimpleSymbol cannot be cast to java.lang.Number

When checking the adb log, it seems like something wrong with BluetoothLEint.java (#694, #621). I attach the screenshot and adb log, please take a look of this.

內置圖片 1

​11-06 13:07:20.863  3236  3236 E AndroidRuntime: FATAL EXCEPTION: main
11-06 13:07:20.863  3236  3236 E AndroidRuntime: Process: edu.mit.appinventor.aicompanion3, PID: 3236
11-06 13:07:20.863  3236  3236 E AndroidRuntime: java.lang.ClassCastException: gnu.mapping.SimpleSymbol cannot be cast to java.lang.Number
11-06 13:07:20.863  3236  3236 E AndroidRuntime: 	at edu.mit.appinventor.ble.BluetoothLEint$BLEWriteOperation.write(BluetoothLEint.java:694)
11-06 13:07:20.863  3236  3236 E AndroidRuntime: 	at edu.mit.appinventor.ble.BluetoothLEint$BLEWriteOperation.run(BluetoothLEint.java:621)
11-06 13:07:20.863  3236  3236 E AndroidRuntime: 	at android.os.Handler.handleCallback(Handler.java:739)
11-06 13:07:20.863  3236  3236 E AndroidRuntime: 	at android.os.Handler.dispatchMessage(Handler.java:95)
11-06 13:07:20.863  3236  3236 E AndroidRuntime: 	at android.os.Looper.loop(Looper.java:168)
11-06 13:07:20.863  3236  3236 E AndroidRuntime: 	at android.app.ActivityThread.main(ActivityThread.java:5885)
11-06 13:07:20.863  3236  3236 E AndroidRuntime: 	at java.lang.reflect.Method.invoke(Native Method)
11-06 13:07:20.863  3236  3236 E AndroidRuntime: 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:797)
11-06 13:07:20.863  3236  3236 E AndroidRuntime: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:687)