Ticking off "Task #01 - Compile test" as the last commit on the release branch passed a murdock run that did not seem to skip any tests.

from release-specs.

Test Report

from release-specs.

Tests ran manually on iotlab-m3:

$ BOARD=iotlab-m3 IOTLAB_NODES=1 IOTLAB_DURATION=10 make iotlab-exp -C tests/tests_unittests
$ iotlab-experiment get -n
[...] m3-101.grenoble.iot-lab.info
$ BOARD=iotlab-m3 IOTLAB_NODE=m3-101.grenoble.iot-lab.info make all flash test -C tests/tests_unittests
$ BOARD=iotlab-m3 IOTLAB_NODE=(actually a different experiment) ./dist/tools/compile_and_test_for_board/compile_and_test_for_board.py . iotlab-m3

This is ongoing, and I'll tick off 02-tests / "Task #2 - Subset of tests on iotlab-m3" when that's done.

from release-specs.

I've run through the native tests (compile-and-test), with some failures -- but AFAICT they were all related to odd versions of compilers used here:

• Rust applications break; this is known as #18389 and generally we only support the nightly in CI -- but given it showed up here I better file for a backport.
• examples/ccn-lite-relay build failed with RIOT-OS/RIOT#18480
• Some just didn't work because I lacked depenencies locally
  • fuzzing was not tested for lack of an AFL compiler
  • For C++ tests I didn't have the right headers around
  • For several pkgs, C++ headers, Python libraries or programs (protoc) were missing.

Except for the 18480 issue, I consider these good enough. If that turns out to be a false positive, I'll tick the "build on native" box.

from release-specs.

Going through the native tests, they worked except

• RIOT-OS/RIOT#18487 (which I consider not critical)
• RIOT-OS/RIOT#18490 (which is about unrealistic expectations on the microsecond timer on native; not ciritcal)
• RIOT-OS/RIOT#17170 (tests/thread_float which is a known issue and highly dependent on the local setup on native)

I consider these tests good enough to tick the "native" box.

from release-specs.

At iotlab-m3, local building and testing-at-IoT-LAB worked except:

• where building lacked the C++ headers or other local dependencies (like on native)
• if (deviating from the documentation) nightly is not installed as the default Rust version, tests/rust_libs fails. That's OK for this release, but I'm taking the opportunity for RIOT-OS/RIOT#18500

A few tests failed (but hey, this is already running all runnable ones rather than "a subset of"; still):

• Several tests just failed once but worked in a rerun (eg. bench_msg_pingpong, bench_mutex_pingpong up to tests/deferred_yield_higher) ... looks a bit like I just had network trouble during that time in the first round
• tests/turo passed on second attempt, maybe in the same category as bench_msg_pingpong..deferred_yield_higher
• tests/buttons is not runnable on that board ("no buttons available"), indicating that the test should not be run there (TBD)
• tests/riotboot failed but AIU that's not applicable w/o a bootloader (which we don't have there -- TBD)
• examples/micropython fails, but we've had that on native already.

Up to be investigated are still:

• gnrc_rpl -- Makefile already says that automated tests only work on nightly -- TBD as with the above: check why compile_and_test_for_board disregards that)
• periph_timer_short_relative_set -- similarly, Makefile says it's OK, TBD

both of which fail in repeated tests on m3-100.grenoble.iot-lab.info.

(all done in manual runs like BOARD=iotlab-m3 IOTLAB_NODE=m3-100.grenoble.iot-lab.info make flash reset test -C tests/...)

With the above boxes ticked, some things are left to do in tests cleanup, but none of that is release critical; ticking the iotlab-m3 box.

from release-specs.

Closing in favor of #258

from release-specs.