The list command is useful but when listing commands for a very large suite, the results can extend off of one's screen.

I think this powerful test suite would be made even better with a more straightforward integration with Mbed's existing greentea testing system.

While this test suite was originally developed for integration tests of the BLE stack, if combined with the greentea testing suite it would provide an ideal way of performing integration tests on BLE services.

My vision for Mbed BLE is to eventually offer a number of standard BLE services (currently working on a more robust implementation of the old UARTService).

This is similar to what Mbed did in the past (and still does). Some of the previous standard Mbed BLE services were clearly lacking in terms of robustness, usability, and quality and were recently removed from Mbed master.

I think Mbed should provide common-use-case BLE services that are put through the same rigorous testing as the BLE stack.

I'm currently working on an integration test using this suite and the greentea suite for the above UARTService implementation. My integration test is set up so that a single DUT runs the greentea client (which hosts a GattServer with a UARTService). Using the greentea framework, I plan to call python host test scripts to interact with boards running the ble-cliapp. The host test scripts will manipulate the boards running ble-cliapp to interact with the greentea DUT over BLE.

I'm starting this issue to collect feedback on the proposed "standard services" idea and the integration testing strategy.

The change I really need is:

The official test_suite python package must be installable so that out-of-source python scripts can import things from the common directory. This would provide most of what the host tests need to interact with connected boards running the ble-cliapp.

Not only would this allow Mbed to provide standard BLE services that are thoroughly tested, it would provide an example of how customers can use these tools to do integration testing on their own end devices running the Mbed BLE stack.

@pan- @paul-szczepanek-arm Thoughts?

Is it possible to add only the added need package,
and keep the original mbed-os version ?

ex:

• mbed-os: beautifulsoup4>=4,<=4.6.3
• here: beautifulsoup4==4.9.1

Is it really mandatory ?

• mbed-os: mbed-ls>=1.5.1,<1.8
• here: mbed-ls==1.7.11

Why do you really need to override ?

etc...

Hi

Lines:
https://github.com/ARMmbed/mbed-os-bluetooth-integration-testsuite/blob/master/ble-cliapp/mbed_app.json#L27-L30

are not needed.

Thx

As mentioned in another issue, I am attempting to write integration tests using this framework with greentea for a UARTService implementation.

The first tests I am writing are simply loopback tests where the blecli-app is used to write data to the greentea DUT running a GattServer with a UARTService.

Currently, the writes are limited to the current MTU of the connection, which defaults to 20 bytes.

I found that the tests were timing out when the size of the random payload went above 13 bytes (oooh spooky 🎃). The timeout was evidently coming from the ble-cliapp serial connection as the logs showed this:

[1603472211.99][mbed_ble_test_suite.common.device]0x7f80291032b0: Sending command to client: "gattClient writeWithoutResponse 1 13 645F7A3275BDD46E4AE5E3ADE6AA"
[1603472211.99][mbed_ble_test_suite.common.serial_device]-->|0x7f80291032b0| gattClient writeWithoutResponse 1 13 645F7A3275BDD46E4AE5E3ADE6AA
[1603472212.99][mbed_ble_test_suite.common.device]0x7f80291032b0: Waiting for "retcode: 0" string... Timeout in 29 s

When the payload is 13 bytes and under, the logs look like this:

[1603472268.45][mbed_ble_test_suite.common.device]0x7fd869e2c2b0: Sending command to client: "gattClient writeWithoutResponse 1 13 B1BDDE7DA78EC881B0C27A89D6"
[1603472268.45][mbed_ble_test_suite.common.serial_device]-->|0x7fd869e2c2b0| gattClient writeWithoutResponse 1 13 B1BDDE7DA78EC881B0C27A89D6
[1603472268.46][mbed_ble_test_suite.common.serial_device]<--|0x7fd869e2c2b0| {"status": 0}
[1603472268.46][mbed_ble_test_suite.common.serial_device]<--|0x7fd869e2c2b0| retcode: 0
[1603472268.52][mbed_ble_test_suite.common.serial_device]<--|0x7fd869e2c2b0| 
[1603472268.53][mbed_ble_test_suite.common.serial_device]<--|0x7fd869e2c2b0| <<< {"connHandle": 1,"handle": 15,"type": "BLE_HVX_NOTIFICATION","data": "B1BDDE7DA78EC881B0C27A89D6"}
[1603472268.53][TEST][INF] received data: B1BDDE7DA78EC881B0C27A89D6

I will point a few things:

1. The hang is from the ble-cliapp. This is confirmed by the fact that the "above-13-bytes" case has no status/result code in the log.
2. If you inspect two commands, you will notice that the 13-byte command has 63 characters while the 14-byte command has 65 characters.

Due to point 2 my assumption is that somewhere in the code there is a buffer limited to 64 bytes that is causing this issue...

I appreciate any hints while I look for the cause.

Hi

Command:

mbed compile -t ARM -m DISCO_L475VG_IOT01A --profile toolchains_profile.json -f

Check link warnings:

Object file AdvertisingParameters.o is not unique! It could be made from: mbed-os\connectivity\FEATURE_BLE\source\gap\AdvertisingParameters.cpp source\Commands\parameters\AdvertisingParameters.cpp
Object file ConnectionParameters.o is not unique! It could be made from: source\Commands\parameters\ConnectionParameters.cpp mbed-os\connectivity\FEATURE_BLE\source\gap\ConnectionParameters.cpp
Object file DiscoveredCharacteristic.o is not unique! It could be made from: source\Commands\Serialization\DiscoveredCharacteristic.cpp mbed-os\connectivity\FEATURE_BLE\source\gatt\DiscoveredCharacteristic.cpp

In ble-cliapp

Command:
mbed compile -t ARM -m DISCO_L475VG_IOT01A --profile toolchains_profile.json

=> makes too many warnings

What about applying ARMmbed/mbed-os#10740 in ble-cliapp ?

In test_suite/common/ble_device.py shouldn't def flush(self, timeout: float = 0) -> [str]: be calling self.device.flush instead of self.device.send?

def flush(self, timeout: float = 0) -> [str]:
        lines = self.device.send(timeout)
        return self._filter_events(lines)

change to:

def flush(self, timeout: float = 0) -> [str]:
        lines = self.device.flush(timeout)
        return self._filter_events(lines)

Hi

Note that ble-x-nucleo-idb0xa1 has been deprecated by cordio-ble-x-nucleo-idb0xa1,
which is also deprecated in mbed-os-6 as it is now a COMPONENT

=> delete ble-cliapp/shields directory

When building with mbed-cli, the list command does not work for any BLE command suites available.

ie: Attempting gap list results in the following error message:

{"status": -1,"error": "invalid command name, you can get all the command name for this module by using the command 'list'"}

This is due to some missing configuration macros (maybe this is intended to be built with the new cmake-based build tools?)

Hi

There are currently 4 markers:

• ble41
• ble42
• ble50
• smoketest

Is it possible to define several marker for the same test?

• set ble41 for all ble42 tests ?
• set ble42 for all ble50 tests ?
• create new markers: "SingleDeviceNeeded", "2devices", "3devices" ?

Regards

the bluetooth driver has a bug exactly when a device (ios) connects to the peripheral (nordic chip) and the device sleeps and causing the connection to disconnect from one side only, the peripheral can not detect this disconnection and must get a hardware reset in order to be able to connect to it again.

It should detect the disconnection using a time or indication or something else.