Code Monkey home page Code Monkey logo

release-specs's People

Contributors

aabadie avatar benpicco avatar cgundogan avatar chrysn avatar cladmi avatar danpetry avatar fjmolinas avatar haukepetersen avatar jeandudey avatar jia200x avatar kaspar030 avatar kb2ma avatar kyc0o avatar leandrolanzieri avatar maribu avatar miri64 avatar mrkevinweiss avatar oleghahm avatar peterkietzmann avatar smlng avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

release-specs's Issues

02-tests/task 03 Release 2016.07 - RC1

Task # 03

Perform a subset of tests (!= subset from Task #02) on samr21-xpro node:

  • bitarithm_timings
  • bloom_bytes
  • coap
  • conn_ip
  • cpp11_condition_variable
  • cpp11_mutex
  • cpp11_thread
  • driver_adt7310 -> results in expected behaviour: Initializing ADT7310 sensor... [Failed]*
  • driver_at30tse75x
  • driver_at86rf2xx
  • driver_bh1750 -> results in expected behaviour: the lux value is always 0, since no BH1750FVI ambient light sensor is attached
  • driver_bmp180
  • driver_dht
  • driver_enc28j60 -> results in expected behaviour: ifconfig doesn't show up the ethernet device since its not attached
  • driver_encx24j600
  • driver_hdc1000
  • driver_hih6130 -> results in expected behaviour: Initializing HIH6130 sensor at I2C_0, address 0x27... [OK] Communication error: -1, since no HIH6130 sensor is attached
  • driver_ina220
  • driver_isl29020
  • driver_isl29125 -> results in expected behaviour: Initializing ISL29125 sensor at I2C_0... [Failed]
  • driver_kw2xrf
  • driver_l3g4200d
  • driver_lis3dh -> results in expected behaviour: Initializing LIS3DH sensor... [Failed]
  • driver_lis3mdl
  • driver_lps331ap
  • driver_lsm303dlhc -> results in expected behaviour: Initializing LSM303DLHC sensor at I2C_0... [Failed]
  • driver_mag3110
  • driver_mma8652
  • driver_mpl3115a2 -> results in expected behaviour: Initializing MPL3115A2 sensor at I2C_0... [Failed]
  • driver_mpu9150
  • driver_mq3
  • driver_nrf24l01p_lowlevel -> results in expected behaviour: initializing the transceiver it fails with all read registers are 0
  • driver_nrfmin
  • driver_nvram_spi
  • driver_pcd8544 -> runs but doesn't give feedback if a display is actually attached
  • driver_pir
  • driver_servo
  • driver_si70xx -> results in expected behaviour: Testing sensor communication...[Failed], since no SI7021 temperature and humidity sensor is attached
  • driver_srf02
  • driver_srf08
  • driver_tcs37727 -> results in expected behaviour: Initializing TCS37727 sensor at I2C_0... [Failed], since its not attached
  • driver_tmp006
  • driver_xbee
  • emb6
  • fault_handler
  • float
  • fmt_print
  • gnrc_ipv6_ext
  • gnrc_sixlowpan
  • irq
  • leds
  • libfixmath
  • lwip
  • malloc
  • minimal
  • msg_send_receive (fixed: RIOT-OS/RIOT#5701)
  • msg_try_receive
  • mutex_order
  • mutex_unlock_and_sleep
  • netdev2_test
  • netstats_l2
  • nhdp
  • periph_adc
  • periph_cpuid
  • periph_dac
  • periph_gpio
  • periph_hwrng
  • periph_i2c -> initialization of the i2c works
  • periph_pwm
  • periph_rtc
  • periph_rtt
  • periph_spi
  • periph_timer
  • periph_uart -> initialized UART device 1 with a 9800 baudrate
  • pipe
  • pkg_cmsis-dsp
  • pkg_jsmn
  • pkg_micro-ecc
  • pkg_u8g2
  • posix_semaphore -> TEST4 fails due to timer drift, i.e. ~265 us difference to expected 1s
  • posix_sleep
  • pthread
  • pthread_barrier
  • pthread_cleanup
  • pthread_condition_variable
  • pthread_cooperation
  • pthread_rwlock
  • pthread_tls
  • saul
  • sched_testing
  • shell
  • sizeof_tcb
  • slip
  • struct_tm_utility
  • thread_basic
  • thread_cooperation
  • thread_exit
  • thread_flags
  • thread_flood
  • thread_msg
  • thread_msg_avail
  • thread_msg_block_wo_queue
  • thread_msg_block_w_queue
  • thread_msg_seq
  • warn_conflict
  • xtimer_drift -> after a handfull min running drift/jitter max: ~170 us
  • xtimer_hang
  • xtimer_longterm
  • xtimer_msg
  • xtimer_msg_receive_timeout
  • xtimer_now64_continuity
  • xtimer_remove
  • xtimer_reset
  • xtimer_usleep_until
  • zep

list made manually by executing:

ls -1a | sed  -e 's/^/- [ ] /'

in RIOT/tests and removing unittests et al. from the list

Release 2016.10 - RC2

Release 2017.10 - RC1

02-tests/task 02 Release 2017.01 - RC2

Task # 02

Perform a subset of tests on iotlab-m3 node:

  • bitarithm_timings
  • bloom_bytes
  • board_calliope-mini
  • board_microbit (as expected: whitelist works, headers are missing for this board)
  • conn_ip
  • cpp11_condition_variable
  • cpp11_mutex
  • cpp11_thread
  • driver_adt7310
  • driver_at30tse75x (as expected: calling/initializing in shell returns an error)
  • driver_at86rf2xx
  • driver_bh1750
  • driver_bme280 (as expected: returns initialization error after boot)
  • driver_bmp180
  • driver_dht
  • driver_enc28j60 (as expected: with no adaptor connected the device cannot be used, Note: the ENC_RST GPIO pin must be set to a different pin for this test, e.g. 0,7, since it conflicts with the onboard transceiver sleep pin)
  • driver_encx24j600
  • driver_hdc1000
  • driver_hih6130 (as expected: without attached humidity sensor the communication fails)
  • driver_ina220
  • driver_io1_xplained
  • driver_isl29020
  • driver_isl29125
  • driver_jc42
  • driver_kw2xrf (💥 the transceiver is not attached so fails initializing it, RIOT ends up in a kernel panic after the failed initialization)
  • driver_l3g4200d
  • driver_lis3dh
  • driver_lis3mdl (as expected: whitelist works, initializing the missing magnetometer fails)
  • driver_lpd8808
  • driver_lps331ap
  • driver_lsm303dlhc
  • driver_mag3110
  • driver_mma8x5x
  • driver_mpl3115a2 (as expected: the pressure sensor is not attached so initialization fails)
  • driver_mpu9150
  • driver_mq3
  • driver_nrf24l01p_lowlevel (as expected: the transceiver is not attached so initialization fails and the register reads always return -1)
  • driver_nvram_spi
  • driver_pcd8544
  • driver_pir (as expected: no motion sensor is attached so initialization fails)
  • driver_pn532
  • driver_sdcard_spi
  • driver_servo (as expected: compilation fails since periph_pwm is not available on this board)
  • driver_si70xx
  • driver_srf02
  • driver_srf08 (as expected: no range sensor is mounted so we get no output)
  • driver_tcs37727
  • driver_tmp006
  • driver_xbee (as expected: no range sensor is mounted so we get no output)
  • emb6
  • fault_handler
  • float
  • fmt_print
  • gnrc_ipv6_ext
  • gnrc_sixlowpan (seems that pexpect does not run correctly for me, @cgundogan succesfully ran the test)
  • gnrc_sock_ip
  • gnrc_sock_udp
  • irq
  • leds
  • libfixmath
  • lwip (compiles, is flashable and ifconfig shows a device, but make test fails RIOT-OS/RIOT#6533, calling ps in the shell spams a wall of garbage in the shell)
  • lwip_sock_ip
  • malloc
  • minimal
  • mpu_stack_guard
  • msg_send_receive
  • msg_try_receive
  • mutex_order
  • mutex_unlock_and_sleep
  • netdev2_test
  • netstats_l2
  • nhdp
  • od (works as expected, testrunner doesn't work for non-native boards)
  • periph_adc
  • periph_cpuid
  • periph_dac (as expected: no DAC is mounted so we get no output)
  • periph_flashpage
  • periph_gpio
  • periph_hwrng (as expected: we get warned and linking fails due to missing objects for the absent hwrng)
  • periph_i2c
  • periph_pwm
  • periph_rtc (as expected: we get warned and build fails without a peripheral realtime clock)
  • periph_rtt
  • periph_spi
  • periph_timer
  • periph_uart
  • pipe
  • pkg_aversiveplusplus (as expected: without attached servos the probing for them fails)
  • pkg_cmsis-dsp
  • pkg_jsmn
  • pkg_libcoap RIOT-OS/RIOT#6536
  • pkg_micro-ecc
  • pkg_micro-ecc-with-hwrng
  • pkg_oonf_api RIOT-OS/RIOT#6537
  • pkg_tiny-asn1
  • pkg_u8g2
  • posix_semaphore (works as expected, testrunner doesn't work for non-native boards)
  • posix_sleep
  • pthread
  • pthread_barrier
  • pthread_cleanup
  • pthread_condition_variable
  • pthread_cooperation
  • pthread_rwlock
  • pthread_tls
  • saul
  • sched_testing
  • shell
  • sizeof_tcb
  • slip
  • struct_tm_utility
  • thread_basic (works as expected, testrunner doesn't work for non-native boards)
  • thread_cooperation
  • thread_exit
  • thread_flags
  • thread_flood
  • thread_msg
  • thread_msg_avail
  • thread_msg_block_wo_queue
  • thread_msg_block_w_queue
  • thread_msg_seq
  • warn_conflict
  • xtimer_drift
  • xtimer_hang
  • xtimer_longterm
  • xtimer_msg
  • xtimer_msg_receive_timeout (works as expected, testrunner doesn't work for non-native boards)
  • xtimer_now64_continuity
  • xtimer_periodic_wakeup
  • xtimer_remove
  • xtimer_reset
  • xtimer_usleep
  • zep

list made manually by executing:

ls -1a | sed  -e 's/^/- [ ] /'

in RIOT/tests and removing unittests et al. from the list

Release 2018.01 - RC1

Release 2015.09 - RC4

This issue lists the status of all tests for the Release Candidate 4 of the 2015.09 release.

Specs tested:

  • 01-CI
  • 02-Tests
    • Task 01:
      • tests/cpp11_* are failing on native, but work on iotlab-m3
      • Results of tests/malloc are ambigious
      • tests/msg_send_receive needs RIOT-OS/RIOT#4036
    • all other tests succeed on native
    • Task 02 and 03 succeed
  • 03-Single Hop IPv6 ICMP
    • Task 01: 1000 packets transmitted, 1000 received, 0% packet loss, time 10.06155388 s (and pktbuf is empty)
    • Task 02: 1000 packets transmitted, 1000 received, 0% packet loss, time 100.06267042 s
    • Task 03: 3600 packets transmitted, 3600 received, 0% packet loss, time 3600.06266335 s
  • 04-Single Hop 6LoWPAN ICMP
    • Task 01: 1000 packets transmitted, 1000 received, 0% packet loss, time 18.06298406 s
    • Task 02: 1000 packets transmitted, 1000 received, 0% packet loss, time 119.06712665 s (and pktbuf is empty)
    • Task 03: 1000 packets transmitted, 998 received, 1% packet loss, time 243.06217368 s
    • Task 04: 10000 packets transmitted, 9996 received, 1% packet loss, time 1222.06596852 s
  • 05-Single Hop Route
    • Task 01: 100 packets transmitted, 100 received, 0% packet loss, time 1.0619766 s
    • Task 02: 100 packets transmitted, 99 received, 1% packet loss, time 16.06761171 s
    • Task 03: 10 packets transmitted, 10 received, 0% packet loss, time 0.0693490 s
  • 06-Single Hop UDP
    • Task 01: tested with patched version in #7 (comment): Packets received: 999 (forgot to add od, so no pktbuf check possible, but should be alright)
  • 07-Multi Hop
    • Task 01:
window: [87, 104, 118, 144]
Setting route dead:beef::90 via fe80::3432:4833:46d8:8a16 for m3-87 ...success
Setting route :: via fe80::3432:4833:46d4:8822 for m3-104 ... success
Setting route dead:beef::90 via fe80::3432:4833:46df:8922 for m3-104 ...success
Setting route :: via fe80::3432:4833:46d8:8a16 for m3-118 ... success
Setting route dead:beef::90 via fe80::3432:4833:46da:9672 for m3-118 ...success
Setting route :: via fe80::3432:4833:46df:8922 for m3-144 ... success
Pinging (dead:beef::90) for node m3-87 ... success with 0% packet loss
Successfully pinged with 3 hops
SUCCESS
  • Task 02:
window: [162, 163, 165, 176]
Setting route dead:beef::b0 via fe80::3432:4833:46de:8922 for m3-162 ...success
Setting route :: via fe80::3432:4833:46d5:8a36 for m3-163 ... success
Setting route dead:beef::b0 via fe80::3432:4833:46d8:8726 for m3-163 ...success
Setting route :: via fe80::3432:4833:46de:8922 for m3-165 ... success
Setting route dead:beef::b0 via fe80::3432:4833:46d4:8d26 for m3-165 ...success
Setting route :: via fe80::3432:4833:46d8:8726 for m3-176 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Send UDP to dead:beef::b0 from node m3-162 ... success
Sent successfully with packet loss of 0.0%
Successfully communicated with UDP over 3 hops
SUCCESS
  • Task 03: SUCCESS with 0% - 10% packet loss
  • Task 04: Tested successfully manually.
  • Task 01:
    • Floodping with 1.000.000 packets and default payload from Linux to RIOT: 1000000 packets transmitted, 1000000 received, 0% packet loss, time 33466ms
    • Floodping with 100.000 packets and payload of 1kB from Linux to RIOT: 100000 packets transmitted, 100000 received, 0% packet loss, time 3684ms
    • Floodping with 100.000 pacekts and default payload from RIOT to Linux: 100000 packets transmitted, 100000 received, 0% packet loss, time 3.06685815 s
  • Task 02:
    • Floodping with 100.000 packets and default payload from Linux to RIOT: 100000 packets transmitted, 99648 received, 0% packet loss, time 1010348ms
    • Flooding with 100.000 packets and 1kB payload from RIOT to Linux: 100000 packets transmitted, 93395 received, 7% packet loss, time 21539.06641653 s
  • For Linux testing a patched version of radvd is required, otherwise it won't send the SLLAO in the router advertisements.

02-tests/task 02 Release 2016.10 - RC3

Task # 02

Perform a subset of tests on iotlab-m3 node:

  • bitarithm_timings
  • bloom_bytes OK but don't know if the output is right:
INFO # checking 10000 elements took 2145ms
INFO # 
INFO # 267 elements probably in the filter.
INFO # 9733 elements not in the filter.
INFO #  false positive rate.
INFO # 
INFO # All done!
  • coap
  • conn_ip
  • cpp11_condition_variable OK
  • cpp11_mutex
  • cpp11_thread OK
  • driver_adt7310
  • driver_at30tse75x
  • driver_at86rf2xx OK
  • driver_bh1750
  • driver_bmp180 OK Results in error as expected
  • driver_dht
  • driver_enc28j60
  • driver_encx24j600
  • driver_hdc1000 OK Results in error as expected
  • driver_hih6130
  • driver_ina220
  • driver_io1_xplained
  • driver_isl29020 OK
  • driver_isl29125
  • driver_kw2xrf
  • driver_l3g4200d OK
  • driver_lis3dh
  • driver_lis3mdl
  • driver_lpd8808
  • driver_lps331ap OK
  • driver_lsm303dlhc
  • driver_mag3110
  • driver_mma8652 OK Results in error as expected
  • driver_mpl3115a2
  • driver_mpu9150
  • driver_mq3 OK Lectures are 0 since there's no sensor
  • driver_nrf24l01p_lowlevel
  • driver_nrfmin
  • driver_nvram_spi
  • driver_pcd8544 OK
  • driver_pir
  • driver_servo
  • driver_si70xx OK Results in error as expected
  • driver_srf02
  • driver_srf08
  • driver_tcs37727 OK Results in error as expected
  • driver_tmp006
  • driver_xbee
  • emb6 OK
  • fault_handler
  • float
  • fmt_print OK
  • gnrc_ipv6_ext
  • gnrc_sixlowpan
  • gnrc_sock_ip OK
  • gnrc_sock_udp OK
  • irq
  • leds
  • libfixmath
  • lwip OK
  • malloc
  • minimal
  • mpu_stack_guard OK Fails because iotlab-m3 is not whitelisted
  • msg_send_receive OK
  • msg_try_receive
  • mutex_order
  • mutex_unlock_and_sleep OK
  • netdev2_test
  • netstats_l2
  • nhdp OK but fails in OS X
  • od OK
  • periph_adc
  • periph_cpuid
  • periph_dac OK compiles and runs but there's no DAC for iotlab-m3
  • periph_gpio
  • periph_hwrng
  • periph_i2c OK sensors work as expected
  • periph_pwm
  • periph_rtc
  • periph_rtt OK
  • periph_spi
  • periph_timer
  • periph_uart OK
  • pipe
  • pkg_aversiveplusplus OK
  • pkg_cmsis-dsp
  • pkg_jsmn OK
  • pkg_micro-ecc
  • pkg_tiny-asn1 OK
  • pkg_u8g2
  • posix_semaphore OK
  • posix_sleep
  • pthread
  • pthread_barrier OK
  • pthread_cleanup
  • pthread_condition_variable
  • pthread_cooperation OK
  • pthread_rwlock
  • pthread_tls
  • saul OK
  • sched_testing
  • shell
  • sizeof_tcb OK
  • slip
  • struct_tm_utility
  • thread_basic OK
  • thread_cooperation
  • thread_exit
  • thread_flags OK
  • thread_flood
  • thread_msg
  • thread_msg_avail OK
  • thread_msg_block_wo_queue
  • thread_msg_block_w_queue
  • thread_msg_seq OK
  • warn_conflict
  • xtimer_drift
  • xtimer_hang OK
  • xtimer_longterm
  • xtimer_msg
  • xtimer_msg_receive_timeout OK
  • xtimer_now64_continuity
  • xtimer_periodic_wakeup OK
  • xtimer_remove
  • xtimer_reset OK
  • xtimer_shift_on_compare
  • zep

This list is based on previous #23 with different test marked for this release. I also added the new ones which are systematically performed.

Release 2017.07 - RC2

Release 2015.09 - RC1

This issue lists the status of all tests for the Release Candidate 1 of the 2015.09 release.

Specs tested:

  • 01-CI
  • 02-Tests
  • [03-Single Hop IPv6 ICMP](https://github.com/RIOT-OS/Release-Specs/blob/master/specs/2015-09/ 03-single-hop-ipv6-icmp.md)
    • For task 01, one has manually delete the link-local address from the bridge interface.
    • All tasks succeeded without packet loss.
  • 04-Single Hop 6LoWPAN ICMP
    • Task 01: 1000 packets transmitted, 999 received, 1% packet loss, time 23.06321836 s
    • Task 02: 1000 packets transmitted, 1000 received, 0% packet loss, time 113.06589983 s
    • Task 03: 1000 packets transmitted, 992 received, 1% packet loss, time 280.06967510 s
    • Task 04: '10000 packets transmitted, 9994 received, 1% packet loss, time 1270.06575971 s`
  • 05-Single Hop Route
    • Task 01: 100 packets transmitted, 100 received, 0% packet loss, time 1.0618546 s
    • Task 02: 100 packets transmitted, 100 received, 0% packet loss, time 15.06920354 s
    • Task 03: 10 packets transmitted, 10 received, 0% packet loss, time 0.0692178 s
  • 06-Single Hop UDP
  • 07-Multi Hop
    • Task 01: 100 packets transmitted, 100 received, 0% packet loss, time 6.06885310 s
    • Task 02: Sent 100 UDP packets over 3 hops: 0% packet loss
  • 08-Interop
    • For Linux testing a patched version of radvd is required, otherwise it won't send the SLLAO in the router advertisements.

02-tests/task 01 Release 2017.01 - RC2

  • bitarithm_timings success
  • bloom_bytes success
  • cpp11_condition_variable success
  • cpp11_mutex success
  • cpp11_thread success
  • fault_handler success
  • float success
  • fmt_print success
  • gnrc_ipv6_ext
  • gnrc_sixlowpan
  • gnrc_sock_ip success
  • gnrc_sock_udp success
  • irq success
  • libfixmath success
  • libfixmath_unittests success
  • lwip_sock_ip success
  • malloc success
  • msg_send_receive success
  • msg_try_receive success
  • mutex_order success
  • mutex_unlock_and_sleep success
  • netstats_l2 success
  • od failed see #39 (comment)
  • periph_cpuid success
  • periph_hwrng success
  • periph_timer success
  • pipe success
  • pkg_u8g2 success
  • posix_semaphore success
  • posix_sleep success
  • pthread success
  • pthread_barrier success
  • pthread_cleanup success
  • pthread_condition_variable success
  • pthread_cooperation success
  • pthread_rwlock success
  • pthread_tls success
  • sched_testing success
  • sizeof_tcb success
  • slip
  • struct_tm_utility success
  • thread_basic success
  • thread_cooperation success
  • thread_exit success
  • thread_flags success
  • thread_flood success
  • thread_msg success
  • thread_msg_avail success
  • thread_msg_block_wo_queue success
  • thread_msg_block_w_queue success
  • thread_msg_seq success
  • xtimer_drift failed see (RIOT-OS/RIOT#6052)
  • xtimer_hang success
  • xtimer_longterm success
  • xtimer_msg success
  • xtimer_msg_receive_timeout success
  • xtimer_now64_continuity success
  • xtimer_periodic_wakeup success
  • xtimer_remove success
  • xtimer_reset success
  • xtimer_usleep success

Part of the script to create the list:

find "$BASEDIR" -maxdepth 2 -mindepth 2 -type f \
    \( -name 'main.c' -or -name 'main.cpp' \) \
    -exec grep -Eq '@ingroup[[:blank:]]+\<test' {} \; \
    -print | sed 's#^\(.*\)/[^/]*$#\1#' | sort > "$LIST"

Create the markdown:

sed -e 's#^.*/\([^/]*\)$#- [ ] \1#' "$LIST"

Release 2015.12 - RC3

Status of all tests for the Release Candidate 3 of the 2015.12 release.

Please check off all tests that performed successfully. Inline any errors/warnings you encounter in this list.
The names I assigned are tentative. Feel free to re-assign or execute tests that are not marked with your name.

Release 2016.04 - RC2

Release 2017.04 - RC2

This issue lists the status of all tests for the Release Candidate 2 of the 2017.04 release.

Specs tested:

Release 2016.10 - RC3

This issue lists the status of all tests for the Release Candidate 3 of the 2016.10

Specs tested:

Task 7: arbitrary error margin?

Why does Task 7 ask for a success rate of <10% for UDP packets but a success rate of <20% for ICMPv6 packets? What -- in the author's mind -- made UDP (uses 8 byte header, has checksum, might be header compressed) more successful at delivering packets than ICMPv6 (uses 8 byte header, has checksum, is not header compressed). Just the fact that one is compressed and the other not?

Release 2015.12 - RC4

Status of all tests for the Release Candidate 4 of the 2015.12 release.

Please check off all tests that performed successfully. Inline any errors/warnings you encounter in this list.
The names I assigned are tentative. Feel free to re-assign or execute tests that are not marked with your name.

02-tests/task 03 Release 2017.01 - RC2

Task # 03

Perform a subset of tests (!= subset from Task #02) on samr21-xpro node:

  • bitarithm_timings
  • bloom_bytes
  • board_calliope-mini
  • board_microbit
  • conn_ip
  • cpp11_condition_variable
  • cpp11_mutex
  • cpp11_thread
  • driver_adt7310
  • driver_at30tse75x
  • driver_at86rf2xx
  • driver_bh1750
  • driver_bme280
  • driver_bmp180
  • driver_dht
  • driver_enc28j60
  • driver_encx24j600
  • driver_hdc1000
  • driver_hih6130
  • driver_ina220
  • driver_io1_xplained
  • driver_isl29020
  • driver_isl29125
  • driver_jc42
  • driver_kw2xrf
  • driver_l3g4200d
  • driver_lis3dh
  • driver_lis3mdl
  • driver_lpd8808
  • driver_lps331ap
  • driver_lsm303dlhc
  • driver_mag3110
  • driver_mma8x5x
  • driver_mpl3115a2
  • driver_mpu9150
  • driver_mq3
  • driver_nrf24l01p_lowlevel
  • driver_nvram_spi
  • driver_pcd8544
  • driver_pir
  • driver_pn532
  • driver_sdcard_spi
  • driver_servo
  • driver_si70xx
  • driver_srf02
  • driver_srf08
  • driver_tcs37727
  • driver_tmp006
  • driver_xbee
  • emb6
  • fault_handler
  • float
  • fmt_print
  • gnrc_ipv6_ext
  • gnrc_sixlowpan
  • gnrc_sock_ip
  • gnrc_sock_udp
  • irq
  • leds
  • libfixmath
  • libfixmath_unittests
  • lwip
  • lwip_sock_ip
  • Makefile.tests_common
  • malloc
  • minimal
  • mpu_stack_guard
  • msg_send_receive
  • msg_try_receive
  • mutex_order
  • mutex_unlock_and_sleep
  • netdev2_test
  • netstats_l2
  • nhdp
  • od
  • periph_adc
  • periph_cpuid
  • periph_dac
  • periph_flashpage
  • periph_gpio
  • periph_hwrng
  • periph_i2c
  • periph_pwm
  • periph_rtc
  • periph_rtt
  • periph_spi
  • periph_timer
  • periph_uart
  • pipe
  • pkg_aversiveplusplus
  • pkg_cmsis-dsp
  • pkg_jsmn
  • pkg_libcoap
  • pkg_micro-ecc
  • pkg_micro-ecc-with-hwrng
  • pkg_oonf_api
  • pkg_tiny-asn1, (Repo currently down)
  • pkg_u8g2
  • posix_semaphore
  • posix_sleep
  • pthread
  • pthread_barrier
  • pthread_cleanup
  • pthread_condition_variable
  • pthread_cooperation
  • pthread_rwlock
  • pthread_tls
  • saul
  • sched_testing
  • shell
  • sizeof_tcb
  • slip
  • struct_tm_utility
  • thread_basic
  • thread_cooperation
  • thread_exit
  • thread_flags
  • thread_flood
  • thread_msg
  • thread_msg_avail
  • thread_msg_block_wo_queue
  • thread_msg_block_w_queue
  • thread_msg_seq
  • warn_conflict
  • xtimer_drift
  • xtimer_hang
  • xtimer_longterm
  • xtimer_msg
  • xtimer_msg_receive_timeout
  • xtimer_now64_continuity
  • xtimer_periodic_wakeup
  • xtimer_remove
  • xtimer_reset
  • xtimer_usleep
  • zep

list made manually by executing:

ls -1a | sed  -e 's/^/- [ ] /'

in RIOT/tests and removing unittests et al. from the list

02-tests/task 01 Release 2016.10 - RC3

  • bitarithm_timings
  • bloom_bytes
  • conn_ip
  • cpp11_condition_variable
  • cpp11_mutex
  • cpp11_thread
  • fault_handler
  • float
  • fmt_print
  • gnrc_ipv6_ext
  • gnrc_sixlowpan
  • gnrc_sock_ip
  • gnrc_sock_udp
  • irq
  • libfixmath
  • libfixmath_unittests
  • malloc
  • msg_send_receive
  • msg_try_receive
  • mutex_order
  • mutex_unlock_and_sleep
  • netstats_l2
  • od
  • periph_hwrng
  • periph_timer
  • periph_uart
  • pipe
  • pkg_u8g2
  • posix_semaphore
  • posix_sleep
  • pthread
  • pthread_barrier
  • pthread_cleanup
  • pthread_condition_variable
  • pthread_cooperation
  • pthread_rwlock
  • pthread_tls
  • sched_testing
  • sizeof_tcb
  • slip
  • struct_tm_utility
  • thread_basic
  • thread_cooperation
  • thread_exit
  • thread_flags
  • thread_flood
  • thread_msg
  • thread_msg_avail
  • thread_msg_block_wo_queue
  • thread_msg_block_w_queue
  • thread_msg_seq
  • xtimer_drift
  • xtimer_hang
  • xtimer_longterm
  • xtimer_msg
  • xtimer_msg_receive_timeout
  • xtimer_now64_continuity
  • xtimer_periodic_wakeup
  • xtimer_remove
  • xtimer_reset
  • zep

02-tests/task 01 Release 2016.10 - RC4

  • bitarithm_timings success
  • bloom_bytes success
  • conn_ip success
  • cpp11_condition_variable success
  • cpp11_mutex success
  • cpp11_thread success
  • fault_handler success
  • float success
  • fmt_print success
  • gnrc_ipv6_ext success
  • gnrc_sixlowpan success
  • gnrc_sock_ip success
  • gnrc_sock_udp success
  • irq success
  • libfixmath unsure 17894 calculations passed. 15 calculations had errors.
  • libfixmath_unittests success
  • malloc success
  • minimal success
  • msg_send_receive success
  • msg_try_receive success
  • mutex_order success
  • mutex_unlock_and_sleep success
  • netstats_l2 success
  • od success
  • periph_hwrng success
  • periph_timer success
  • periph_uart success
  • pipe success
  • pkg_u8g2 success
  • posix_semaphore success
  • posix_sleep success
  • pthread success
  • pthread_barrier success
  • pthread_cleanup success
  • pthread_condition_variable success
  • pthread_cooperation success
  • pthread_rwlock success
  • pthread_tls success
  • sched_testing success
  • sizeof_tcb success
  • slip (no UART available, so just compiled and checked ifconfig (no interfaces as expected))
  • struct_tm_utility success
  • thread_basic success
  • thread_cooperation success
  • thread_exit success
  • thread_flags success
  • thread_flood success
  • thread_msg success
  • thread_msg_avail success
  • thread_msg_block_wo_queue success
  • thread_msg_block_w_queue success
  • thread_msg_seq success
  • unittests success
  • xtimer_drift failed: freezes (see: #34 (comment))
  • xtimer_hang success
  • xtimer_longterm success
  • xtimer_msg success
  • xtimer_msg_receive_timeout success
  • xtimer_now64_continuity success
  • xtimer_periodic_wakeup success
  • xtimer_remove success
  • xtimer_reset success

Release 2015.09 - RC2

This issue lists the status of all tests for the Release Candidate 2 of the 2015.09 release.

Specs tested:

  • 01-CI
  • 02-Tests
  • 03-Single Hop IPv6 ICMP
    • Task 01: 1000 packets transmitted, 1000 received, 0% packet loss, time 10.06171217 s
    • Task 02: 1000 packets transmitted, 1000 received, 0% packet loss, time 100.06128933 s
    • Task 03: 3600 packets transmitted, 3600 received, 0% packet loss, time 3599.06917114 s
  • 04-Single Hop 6LoWPAN ICMP
    • Task 01: 1000 packets transmitted, 1000 received, 0% packet loss, time 18.06335692 s
    • Task 02: 1000 packets transmitted, 999 received, 1% packet loss, time 109.06740717 s
    • Task 03: 1000 packets transmitted, 993 received, 1% packet loss, time 276.06342833 s
    • Task 04: 10000 packets transmitted, 9998 received, 1% packet loss, time 1313.06892156 s
  • 05-Single Hop Route
    • Task 01: 100 packets transmitted, 100 received, 0% packet loss, time 1.0614627 s
    • Task 02: 100 packets transmitted, 100 received, 0% packet loss, time 16.068110 s
    • Task 03: 10 packets transmitted, 10 received, 0% packet loss, time 0.0692516 s
  • 06-Single Hop UDP
  • 07-Multi Hop
  • 08-Interop
    • For Linux testing a patched version of radvd is required, otherwise it won't send the SLLAO in the router advertisements.

Release 2017.01 - RC2

Release 2015.12 - RC1

This issue lists the status of all tests for the Release Candidate 1 of the 2015.12 release.

Specs tested:

Release 2017.01 - RC1

02-tests/task 01 Release 2016.10 - RC1

  • bitarithm_timings success
  • bloom_bytes success
  • conn_ip
  • cpp11_condition_variable success
  • cpp11_mutex success
  • cpp11_thread success
  • fault_handler success
  • float success
  • fmt_print success
  • gnrc_ipv6_ext
  • gnrc_sixlowpan success
  • gnrc_sock_ip success
  • gnrc_sock_udp success
  • irq success
  • libfixmath unsure 17894 calculations passed. 15 calculations had errors.
  • libfixmath_unittests success
  • malloc success
  • msg_send_receive success
  • msg_try_receive success
  • mutex_order success
  • mutex_unlock_and_sleep success
  • netstats_l2 success
  • od success
  • pipe success
  • pkg_u8g2 success
  • posix_semaphore success
  • posix_sleep success
  • pthread success
  • pthread_barrier success
  • pthread_cleanup success
  • pthread_condition_variable success
  • pthread_cooperation success
  • pthread_rwlock success
  • pthread_tls success
  • sched_testing success
  • sizeof_tcb success
  • slip
  • struct_tm_utility success
  • thread_basic success
  • thread_cooperation success
  • thread_exit success
  • thread_flags
  • thread_flood success
  • thread_msg success
  • thread_msg_avail success
  • thread_msg_block_wo_queue success
  • thread_msg_block_w_queue success
  • thread_msg_seq success
  • unittests success
  • xtimer_drift : FAILED stops after ~30-90 seconds (two different PCs)
  • xtimer_hang success
  • xtimer_longterm success
  • xtimer_msg success
  • xtimer_msg_receive_timeout success
  • xtimer_now64_continuity success
  • xtimer_periodic_wakeup success
  • xtimer_remove success
  • xtimer_reset success
  • zep

Release 2017.10 - RC2

Release 2016.10 - RC4

02-tests/task 01 Release 2016.07 - RC1

Task # 01

Execute all tests on native and check their output:

  • bitarithm_timings
  • bloom_bytes
  • coap
  • conn_ip
  • cpp11_condition_variable
  • cpp11_mutex
  • cpp11_thread
  • driver_adt7310
  • driver_at30tse75x
  • driver_at86rf2xx
  • driver_bh1750
  • driver_bmp180
  • driver_dht -> compiles without warning runs and obviously fails
  • driver_enc28j60
  • driver_encx24j600
  • driver_hdc1000
  • driver_hih6130
  • driver_ina220
  • driver_isl29020
  • driver_isl29125
  • driver_kw2xrf
  • driver_l3g4200d
  • driver_lis3dh
  • driver_lis3mdl
  • driver_lps331ap
  • driver_lsm303dlhc
  • driver_mag3110
  • driver_mma8652
  • driver_mpl3115a2
  • driver_mpu9150
  • driver_mq3
  • driver_nrf24l01p_lowlevel
  • driver_nrfmin
  • driver_nvram_spi
  • driver_pcd8544
  • driver_pir -> compiles without warning runs and obviously fails on GPIO access
  • driver_servo
  • driver_si70xx
  • driver_srf02
  • driver_srf08
  • driver_tcs37727
  • driver_tmp006
  • driver_xbee
  • emb6 -> automatically sets samr21-xpro when BOARD is not set, build with BOARD=native make for the expected build fail
  • fault_handler
  • float -> only shows/spams - on the screen
  • fmt_print
  • gnrc_ipv6_ext
  • gnrc_sixlowpan
  • irq
  • leds
  • libfixmath
  • lwip -> automatically sets iotlab-m3 when BOARD is not set, build with BOARD=native make
  • malloc -> only malloc tested yet since I don't want to wait for filling my whole RAM with 1024 byte chunks
  • minimal
  • msg_send_receive
  • msg_try_receive
  • mutex_order
  • mutex_unlock_and_sleep
  • netdev2_test
  • netstats_l2
  • nhdp
  • periph_adc
  • periph_cpuid
  • periph_dac
  • periph_gpio -> compiles without warning and runs
  • periph_hwrng
  • periph_i2c
  • periph_pwm
  • periph_rtc -> compiles without warning and runs: f: rtc_set_alarm: not implemented
  • periph_rtt
  • periph_spi
  • periph_timer
  • periph_uart -> compiles without warning and runs: UART used for STDIO (the shell): UART_DEV(-1)
  • pipe
  • pkg_cmsis-dsp
  • pkg_jsmn
  • pkg_micro-ecc
  • pkg_u8g2
  • posix_semaphore
  • posix_sleep
  • pthread
  • pthread_barrier
  • pthread_cleanup
  • pthread_condition_variable
  • pthread_cooperation
  • pthread_rwlock
  • pthread_tls
  • saul -> compiles without warning and runs: No SAUL devices present
  • sched_testing
  • shell
  • sizeof_tcb
  • slip (compiled and started, function not tested yet)
  • struct_tm_utility
  • thread_basic
  • thread_cooperation
  • thread_exit
  • thread_flags
  • thread_flood
  • thread_msg
  • thread_msg_avail
  • thread_msg_block_wo_queue
  • thread_msg_block_w_queue
  • thread_msg_seq
  • warn_conflict
  • xtimer_drift
  • xtimer_hang
  • xtimer_longterm
  • xtimer_msg
  • xtimer_msg_receive_timeout
  • xtimer_now64_continuity
  • xtimer_remove
  • xtimer_reset
  • xtimer_usleep_until
  • zep (compiled and started, function not tested yet)

list made manually performing:

ls -1a | sed  -e 's/^/- [ ] /'

in RIOT/tests

Release 2016.04 - RC3

This issue lists the status of all tests for the Release Candidate 3 of the 2016.04 release.

Specs tested:

02-tests/task 02 Release 2016.07 - RC1

Task # 02

Perform a subset of tests on iotlab-m3 node:

  • bitarithm_timings
  • bloom_bytes
  • coap
  • conn_ip sent 3 pings out (ping6)
  • cpp11_condition_variable
  • cpp11_mutex
  • cpp11_thread
  • driver_adt7310
  • driver_at30tse75x
  • driver_at86rf2xx
  • driver_bh1750
  • driver_bmp180
  • driver_dht -> results in expected behaviour: gets stuck at dht_read(dev, &temp, &hum), since no DHT 11/22 temperature and humidity sensor is attached
  • driver_enc28j60
  • driver_encx24j600
  • driver_hdc1000 -> results in expected behaviour: Initializing HDC1000 sensor at I2C_0... [Failed]
  • driver_hih6130
  • driver_ina220
  • driver_isl29020
  • driver_isl29125
  • driver_kw2xrf
  • driver_l3g4200d
  • driver_lis3dh
  • driver_lis3mdl
  • driver_lps331ap
  • driver_lsm303dlhc
  • driver_mag3110
  • driver_mma8652 -> results in expected behaviour: Initializing MMA8652 accelerometer at I2C_0... [Failed]
  • driver_mpl3115a2
  • driver_mpu9150
  • driver_mq3
  • driver_nrf24l01p_lowlevel
  • driver_nrfmin
  • driver_nvram_spi -> results in expected behaviour: Verifying contents... [Failed]
  • driver_pcd8544
  • driver_pir
  • driver_servo
  • driver_si70xx
  • driver_srf02
  • driver_srf08 -> results in expected behaviour: no values are printed, since no SRF08 ultrasonic sensor is attached
  • driver_tcs37727
  • driver_tmp006
  • driver_xbee -> results in expected behaviour: the device cannot be configured, since no xbee shield is attached
  • emb6
  • fault_handler
  • float -> only shows/spams - on the screen
  • fmt_print
  • gnrc_ipv6_ext
  • gnrc_sixlowpan
  • irq
  • leds
  • libfixmath
  • lwip
  • malloc
  • minimal
  • msg_send_receive
  • msg_try_receive
  • mutex_order
  • mutex_unlock_and_sleep
  • netdev2_test
  • netstats_l2
  • nhdp
  • periph_adc
  • periph_cpuid
  • periph_dac
  • periph_gpio
  • periph_hwrng
  • periph_i2c
  • periph_pwm
  • periph_rtc
  • periph_rtt
  • periph_spi
  • periph_timer
  • periph_uart
  • pipe
  • pkg_cmsis-dsp
  • pkg_jsmn
  • pkg_micro-ecc
  • pkg_u8g2
  • posix_semaphore
  • posix_sleep
  • pthread
  • pthread_barrier
  • pthread_cleanup
  • pthread_condition_variable
  • pthread_cooperation
  • pthread_rwlock
  • pthread_tls
  • saul
  • sched_testing
  • shell
  • sizeof_tcb
  • slip
  • struct_tm_utility
  • thread_basic
  • thread_cooperation
  • thread_exit
  • thread_flags
  • thread_flood
  • thread_msg
  • thread_msg_avail
  • thread_msg_block_wo_queue
  • thread_msg_block_w_queue
  • thread_msg_seq
  • warn_conflict
  • xtimer_drift -> I ran it for a handful minutes: max drift/jitter is about +-35 us (these high values occured ~ 4 times a min)
  • xtimer_hang
  • xtimer_longterm
  • xtimer_msg
  • xtimer_msg_receive_timeout
  • xtimer_now64_continuity
  • xtimer_remove
  • xtimer_reset
  • xtimer_usleep_until
  • zep

list made manually by executing:

ls -1a | sed  -e 's/^/- [ ] /'

in RIOT/tests and removing unittests et al. from the list

Release 2017.07 - RC1

02-tests/task 03 Release 2016.10 - RC4

Task #2.03

Perform a subset of tests on samr21-xpro node:

  • bitarithm_timings
  • bloom_bytes
  • coap
  • conn_ip
  • cpp11_condition_variable
  • cpp11_mutex
  • cpp11_thread
  • driver_at86rf2xx
  • driver_io1_xplained
  • fault_handler
  • float
  • fmt_print
  • gnrc_ipv6_ext
  • gnrc_sixlowpan
  • gnrc_sock_ip
  • gnrc_sock_udp
  • irq
  • leds
  • libfixmath
  • malloc
  • minimal
  • msg_send_receive
  • msg_try_receive
  • mutex_order
  • netdev2_test
  • netstats_l2
  • nhdp
  • periph_cpuid
  • periph_gpio
  • periph_rtc
  • periph_spi
  • periph_timer
  • pipe
  • pkg_cmsis-dsp
  • pkg_micro-ecc [failed as expected since required feature is missing]
  • pkg_u8g2
  • posix_sleep
  • pthread
  • pthread_cleanup
  • pthread_condition_variable
  • pthread_rwlock
  • pthread_tls
  • sched_testing
  • shell
  • slip
  • struct_tm_utility
  • thread_cooperation
  • thread_exit
  • thread_flood
  • thread_msg
  • thread_msg_avail
  • thread_msg_block_wo_queue
  • thread_msg_block_w_queue
  • warn_conflict
  • xtimer_drift
  • xtimer_longterm
  • xtimer_msg
  • xtimer_now64_continuity
  • xtimer_remove

This list is based on previous #24 with different test marked for this release. I also added the new ones which are systematically performed.

Release 2015.09 - RC5

This issue lists the status of all tests for the Release Candidate 5 of the 2015.09 release. Only native related specs and those which were not tested in #7 are required here.

Specs tested:

  • 01-CI
  • 02-Tests
  • 03-Single Hop IPv6 ICMP
    • Task 01: 1000 packets transmitted, 1000 received, 0% packet loss, time 10.06728395 s
    • Task 02: 1000 packets transmitted, 1000 received, 0% packet loss, time 100.06765821 s
    • Task 03: 3600 packets transmitted, 3600 received, 0% packet loss, time 3600.06916673 s
  • 05-Single Hop Route
    • Task 01: 100 packets transmitted, 100 received, 0% packet loss, time 1.0692095 s
    • Task 02: see #7
    • Task 03: 10 packets transmitted, 10 received, 0% packet loss, time 0.06102241 s
  • 08-Interop
    • Task 01: 1000000 packets transmitted, 1000000 received, 0% packet loss, time 202.06653731 s
    • Task 02: see #7
    • Task 03: no idea how to do pinging with Contiki, skipping.
    • Task 04: Floodpinging 100.000 packets with default payload from a Linux host over a Raspberry Pi border router to a RIOT powered IoT-LAB-M3: 100000 packets transmitted, 83292 received, 16% packet loss, time 1231630ms
    • Task 05: Pinging from a Linux host over a SAMR21 RIOT based border router to a RIOT powered IoT-LAB-M3, 100.000 packets with an interval of 50ms: 100000 packets transmitted, 99770 received, 0% packet loss, time 5055033ms
    • For Linux testing a patched version of radvd is required, otherwise it won't send the SLLAO in the router advertisements.

02-tests/task 02 Release 2016.10 - RC4

Task # 02

Perform a subset of tests on iotlab-m3 node:

  • bitarithm_timings
  • bloom_bytes OK but don't know if the output is right:
INFO # checking 10000 elements took 2145ms
INFO # 
INFO # 267 elements probably in the filter.
INFO # 9733 elements not in the filter.
INFO #  false positive rate.
INFO # 
INFO # All done!
  • coap
  • conn_ip
  • cpp11_condition_variable OK
  • cpp11_mutex
  • cpp11_thread OK
  • driver_adt7310
  • driver_at30tse75x
  • driver_at86rf2xx OK
  • driver_bh1750
  • driver_bmp180 OK Results in error as expected
  • driver_dht
  • driver_enc28j60
  • driver_encx24j600
  • driver_hdc1000 OK Results in error as expected
  • driver_hih6130
  • driver_ina220
  • driver_io1_xplained
  • driver_isl29020 OK
  • driver_isl29125
  • driver_kw2xrf
  • driver_l3g4200d OK
  • driver_lis3dh
  • driver_lis3mdl
  • driver_lpd8808
  • driver_lps331ap OK
  • driver_lsm303dlhc
  • driver_mag3110
  • driver_mma8652 OK Results in error as expected
  • driver_mpl3115a2
  • driver_mpu9150
  • driver_mq3 OK Lectures are 0 since there's no sensor
  • driver_nrf24l01p_lowlevel
  • driver_nrfmin
  • driver_nvram_spi
  • driver_pcd8544 OK Runs but no display to test
  • driver_pir
  • driver_servo
  • driver_si70xx OK Results in error as expected
  • driver_srf02
  • driver_srf08
  • driver_tcs37727 OK Results in error as expected
  • driver_tmp006
  • driver_xbee
  • emb6 OK
  • fault_handler
  • float
  • fmt_print OK
  • gnrc_ipv6_ext
  • gnrc_sixlowpan
  • gnrc_sock_ip OK
  • gnrc_sock_udp OK
  • irq
  • leds
  • libfixmath
  • lwip OK
  • malloc
  • minimal
  • mpu_stack_guard OK Shows failure because board is not whitelisted
  • msg_send_receive OK
  • msg_try_receive
  • mutex_order
  • mutex_unlock_and_sleep OK
  • netdev2_test
  • netstats_l2
  • nhdp OK But build in OS X fails
  • od OK
  • periph_adc
  • periph_cpuid
  • periph_dac OK But periph_dac module is not provided for iotlab-m3
  • periph_gpio
  • periph_hwrng
  • periph_i2c OK All drivers work as expected
  • periph_pwm
  • periph_rtc
  • periph_rtt OK
  • periph_spi
  • periph_timer
  • periph_uart OK
  • pipe
  • pkg_aversiveplusplus OK But no servos attached to board
  • pkg_cmsis-dsp
  • pkg_jsmn OK
  • pkg_micro-ecc
  • pkg_tiny-asn1 OK
  • pkg_u8g2
  • posix_semaphore OK
  • posix_sleep
  • pthread
  • pthread_barrier OK
  • pthread_cleanup
  • pthread_condition_variable
  • pthread_cooperation OK
  • pthread_rwlock
  • pthread_tls
  • saul OK
  • sched_testing
  • shell
  • sizeof_tcb OK
  • slip
  • struct_tm_utility
  • thread_basic OK
  • thread_cooperation
  • thread_exit
  • thread_flags OK
  • thread_flood
  • thread_msg
  • thread_msg_avail OK
  • thread_msg_block_wo_queue
  • thread_msg_block_w_queue
  • thread_msg_seq OK
  • warn_conflict
  • xtimer_drift
  • xtimer_hang OK
  • xtimer_longterm
  • xtimer_msg
  • xtimer_msg_receive_timeout OK
  • xtimer_now64_continuity
  • xtimer_periodic_wakeup OK
  • xtimer_remove
  • xtimer_reset OK
  • xtimer_shift_on_compare
  • zep

This list is based on previous #23 with different test marked for this release. I also added the new ones which are systematically performed.

Task 6: Questionable test parameters

When testing for the last release I wondered why UDP over 6LoWPAN has less error margin (<5%) than ICMPv6 over 6LoWPAN (<10%). Especially since Task 6 asks for way harder to deliver 1kB, while some of Task 5's specs only ask for 0B. Is there a specific reason for these 5%?

The other questionable thing more from a timing constraint is: Why does the delay need to be 1s? For testing it is really a long amount of time to send 1000 packets (~17min), while the tests would be as significant if the interval between them would be 50ms or 100ms.

02-tests/task 01 Release 2016.10 - RC2

  • bitarithm_timings success
  • bloom_bytes success
  • conn_ip
  • cpp11_condition_variable success
  • cpp11_mutex success
  • cpp11_thread success
  • fault_handler success
  • float success
  • fmt_print success
  • gnrc_ipv6_ext
  • gnrc_sixlowpan success
  • gnrc_sock_ip success
  • gnrc_sock_udp success
  • irq success
  • libfixmath unsure 17894 calculations passed. 15 calculations had errors.
  • libfixmath_unittests success
  • malloc success
  • msg_send_receive success
  • msg_try_receive success
  • mutex_order success
  • mutex_unlock_and_sleep success
  • netstats_l2 success
  • od
  • pipe success
  • pkg_u8g2 success
  • posix_semaphore success
  • posix_sleep success
  • pthread success
  • pthread_barrier success
  • pthread_cleanup success
  • pthread_condition_variable success
  • pthread_cooperation success
  • pthread_rwlock success
  • pthread_tls success
  • sched_testing success
  • sizeof_tcb success
  • slip
  • struct_tm_utility success
  • thread_basic success
  • thread_cooperation success
  • thread_exit success
  • thread_flags
  • thread_flood success
  • thread_msg success
  • thread_msg_avail success
  • thread_msg_block_wo_queue success
  • thread_msg_block_w_queue success
  • thread_msg_seq success
  • unittests success
  • xtimer_drift
  • xtimer_hang success
  • xtimer_longterm
  • xtimer_msg success
  • xtimer_msg_receive_timeout success
  • xtimer_now64_continuity success
  • xtimer_periodic_wakeup success
  • xtimer_remove success
  • xtimer_reset success
  • zep

02-tests/task 01 Release 2017.07 - RC2

  • bitarithm_timings
  • bloom_bytes
  • cbor
  • conn_can
  • cpp11_condition_variable
  • cpp11_mutex
  • cpp11_thread
  • evtimer_msg
  • evtimer_underflow
  • fault_handler
  • float
  • fmt_print
  • gnrc_ipv6_ext
  • gnrc_sixlowpan
  • gnrc_sock_dns
  • gnrc_sock_ip
  • gnrc_sock_udp
  • gnrc_udp
  • irq
  • libfixmath
  • libfixmath_unittests
  • lwip
  • lwip_sock_ip
  • lwip_sock_tcp
  • lwip_sock_udp
  • malloc
  • msg_avail
  • msg_send_receive
  • msg_try_receive
  • mutex_order
  • mutex_unlock_and_sleep
  • netstats_l2
  • od
  • periph_cpuid
  • periph_hwrng
  • periph_timer
  • pipe
  • pkg_fatfs
  • pkg_minmea
  • pkg_u8g2
  • posix_semaphore
  • posix_sleep
  • ps_schedstatistics ???
  • pthread
  • pthread_barrier
  • pthread_cleanup
  • pthread_condition_variable
  • pthread_cooperation
  • pthread_rwlock
  • pthread_tls
  • rmutex
  • saul
  • sched_testing
  • sizeof_tcb
  • slip
  • sntp
  • ssp
  • struct_tm_utility
  • thread_basic
  • thread_cooperation
  • thread_exit
  • thread_flags
  • thread_flags_xtimer
  • thread_flood
  • thread_msg
  • thread_msg_block_wo_queue
  • thread_msg_block_w_queue
  • thread_msg_seq
  • trickle
  • xtimer_drift
  • xtimer_hang
  • xtimer_longterm
  • xtimer_msg
  • xtimer_msg_receive_timeout
  • xtimer_now64_continuity
  • xtimer_periodic_wakeup
  • xtimer_remove
  • xtimer_reset
  • xtimer_usleep

Part of the script to create the list:

find "$BASEDIR" -maxdepth 2 -mindepth 2 -type f \
    \( -name 'main.c' -or -name 'main.cpp' \) \
    -exec grep -Eq '@ingroup[[:blank:]]+\<test' {} \; \
    -print | sed 's#^\(.*\)/[^/]*$#\1#' | sort > "$LIST"

Create the markdown:

sed -e 's#^.*/\([^/]*\)$#- [ ] \1#' "$LIST"

02-tests/task 01 Release 2017.10 - RC2

  • bitarithm_timings success
  • bloom_bytes success
  • buttons not blacklisted
  • cbor success
  • conn_can
  • cpp11_condition_variable success
  • cpp11_mutex success
  • cpp11_thread success
  • driver_apa102 not blacklisted
  • driver_dht not blacklisted
  • driver_grove_ledbar false positive not blacklisted
  • driver_lpd8808 not blacklisted
  • driver_my9221 false positive not blacklisted
  • driver_pir not blacklisted
  • driver_xbee not blacklisted
  • evtimer_msg success
  • evtimer_underflow unclear result
  • fault_handler success
  • float success
  • fmt_print success
  • gnrc_ipv6_ext
  • gnrc_ipv6_nib success
  • gnrc_ipv6_nib_6ln success
  • gnrc_ndp2 success
  • gnrc_netif2
  • gnrc_sixlowpan
  • gnrc_sock_dns
  • gnrc_sock_ip success
  • gnrc_sock_udp success
  • gnrc_udp
  • irq success
  • leds not blacklisted
  • libfixmath success
  • libfixmath_unittests success
  • lwip
  • lwip_sock_ip success
  • lwip_sock_tcp success
  • lwip_sock_udp success
  • malloc success
  • minimal not blacklisted
  • msg_avail success
  • msg_send_receive success
  • msg_try_receive success
  • mutex_order success
  • mutex_unlock_and_sleep success
  • netstats_l2
  • od success
  • periph_cpuid success
  • periph_gpio not blacklisted
  • periph_hwrng success
  • periph_rtc not blacklisted
  • periph_timer success
  • periph_uart not blacklisted
  • pipe success
  • pkg_fatfs success
  • pkg_microcoap
  • pkg_minmea success
  • pkg_u8g2 success
  • posix_semaphore success
  • posix_sleep success
  • ps_schedstatistics unclear result
  • pthread success
  • pthread_barrier success
  • pthread_cleanup success
  • pthread_condition_variable success
  • pthread_cooperation success
  • pthread_rwlock success
  • pthread_tls success
  • rmutex success
  • saul not blacklisted
  • sched_testing success
  • sizeof_tcb success
  • slip
  • sntp
  • ssp success
  • struct_tm_utility success
  • thread_basic success
  • thread_cooperation success
  • thread_exit success
  • thread_flags success
  • thread_flags_xtimer success
  • thread_flood success
  • thread_msg success
  • thread_msg_block_wo_queue success
  • thread_msg_block_w_queue success
  • thread_msg_seq success
  • trickle success
  • xtimer_drift
  • xtimer_hang success
  • xtimer_longterm success
  • xtimer_msg success
  • xtimer_msg_receive_timeout success
  • xtimer_now64_continuity success
  • xtimer_periodic_wakeup success
  • xtimer_remove success
  • xtimer_reset success
  • xtimer_usleep success
  • xtimer_usleep_short success

All performed tests have a checkmark and are either success, failed (with short comment) or have an unclear result (at least for me).

02-tests/task 01 Release 2017.10 - RC1

  • bitarithm_timings success
  • bloom_bytes success
  • buttons not blacklisted
  • cbor success
  • conn_can
  • cpp11_condition_variable success
  • cpp11_mutex success
  • cpp11_thread success
  • driver_apa102 not blacklisted
  • driver_dht not blacklisted
  • driver_grove_ledbar false positive not blacklisted
  • driver_lpd8808 not blacklisted
  • driver_pir not blacklisted
  • driver_xbee not blacklisted
  • evtimer_msg success
  • evtimer_underflow unclear result
  • fault_handler success
  • float success
  • fmt_print success
  • gnrc_ipv6_ext
  • gnrc_ipv6_nib success
  • gnrc_ndp2 success
  • gnrc_netif2
  • gnrc_sixlowpan
  • gnrc_sock_dns
  • gnrc_sock_ip success
  • gnrc_sock_udp success
  • gnrc_udp
  • irq success
  • leds not blacklisted
  • libfixmath success
  • libfixmath_unittests success
  • lwip
  • lwip_sock_ip
  • lwip_sock_tcp success
  • lwip_sock_udp success
  • malloc success
  • minimal not blacklisted
  • msg_avail success
  • msg_send_receive success
  • msg_try_receive success
  • mutex_order success
  • mutex_unlock_and_sleep
  • netstats_l2
  • od success
  • periph_cpuid success
  • periph_gpio not blacklisted
  • periph_hwrng success
  • periph_rtc not blacklisted
  • periph_timer success
  • periph_uart not blacklisted
  • pipe success
  • pkg_fatfs failed, mkfs error, rtc warning
  • pkg_microcoap
  • pkg_minmea unclear result
  • pkg_u8g2 success
  • posix_semaphore failed expect timeout
  • posix_sleep success
  • ps_schedstatistics unclear result
  • pthread success
  • pthread_barrier success
  • pthread_cleanup success
  • pthread_condition_variable success
  • pthread_cooperation success
  • pthread_rwlock success
  • pthread_tls success
  • rmutex success
  • saul not blacklisted
  • sched_testing success
  • sizeof_tcb success
  • slip
  • sntp
  • ssp success
  • struct_tm_utility success
  • thread_basic success
  • thread_cooperation success
  • thread_exit success
  • thread_flags success
  • thread_flags_xtimer success
  • thread_flood success
  • thread_msg success
  • thread_msg_block_wo_queue success
  • thread_msg_block_w_queue success
  • thread_msg_seq success
  • trickle success
  • xtimer_drift
  • xtimer_hang success
  • xtimer_longterm
  • xtimer_msg success
  • xtimer_msg_receive_timeout success
  • xtimer_now64_continuity success
  • xtimer_periodic_wakeup success
  • xtimer_remove success
  • xtimer_reset success
  • xtimer_usleep success
  • xtimer_usleep_short success

Release 2016.07 - RC1

This issue lists the status of all tests for the Release Candidate 1 of the 2016.07 release.

Specs tested:

C++ buildproblems with arm-none-eabi-g++ v6

Building for ARM-Cortex platforms currently fails for me with:

arm-none-eabi-g++ \
        -DRIOT_FILE_RELATIVE=\"tests/cpp11_condition_variable/main.cpp\" \
        -DRIOT_FILE_NOPATH=\"main.cpp\" \
        -Werror -mcpu=cortex-m3 -mlittle-endian -mthumb -mfloat-abi=soft -mno-thumb-interwork -ffunction-sections -fdata-sections -fno-builtin -fshort-enums -ggdb -g3 -Os -fno-delete-null-pointer-checks -Werror=strict-prototypes -Werror=old-style-definition -fno-common -Wall -include '/home/oleg/git/RIOT/tests/cpp11_condition_variable/bin/cc2538dk/riotbuild/riotbuild.h' -std=c++11 -isystem /usr/arm-none-eabi/include/nano -isystem /usr/arm-none-eabi/include -I/home/oleg/git/RIOT/core/include -I/home/oleg/git/RIOT/drivers/include -I/home/oleg/git/RIOT/sys/include -I/home/oleg/git/RIOT/cpu/cc2538/include -I/home/oleg/git/RIOT/boards/cc2538dk/include -I/home/oleg/git/RIOT/cpu/cortexm_common/include -I/home/oleg/git/RIOT/sys/libc/include -I/home/oleg/git/RIOT/sys/cpp11-compat/include  -MD -MP -c -o /home/oleg/git/RIOT/tests/cpp11_condition_variable/bin/cc2538dk/cpp11_condition_variable/main.o /home/oleg/git/RIOT/tests/cpp11_condition_variable/main.cpp
In file included from /usr/arm-none-eabi/include/c++/6.2.0/ext/string_conversions.h:41:0,
                 from /usr/arm-none-eabi/include/c++/6.2.0/bits/basic_string.h:5402,
                 from /usr/arm-none-eabi/include/c++/6.2.0/string:52,
                 from /home/oleg/git/RIOT/tests/cpp11_condition_variable/main.cpp:21:
/usr/arm-none-eabi/include/c++/6.2.0/cstdlib:75:25: fatal error: stdlib.h: No such file or directory
 #include_next <stdlib.h>
                         ^

However:

`# ls -lah /usr/arm-none-eabi/include/c++/6.2.0/stdlib.h
-rw-r--r-- 1 root root 2.2K Aug 31 19:52 /usr/arm-none-eabi/include/c++/6.2.0/stdlib.h

Release 2016.10 - RC1

Release 2017.04 - RC1

Release 2016.04 - RC1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.