Release 2017.10 - RC2 about release-specs HOT 41 CLOSED

How do we note down who is testing what? Our old approach (putting it into the description) lead to problems in RC1. For now I do it the way we used to, because I hope I'm careful enough to not overwrite collisions accidentally ;-).

from release-specs.

The errors from RC1 are fixed. All performed tests 02-tests Task 01 were successful.

from release-specs.

@A-Paul thanks!

@miri64 lets note this our names in this list as usual, but only names and tick-marks. If we make sure, that we keep the time editing (press edit, enter name, press save) to a minium, we should be good. Also everyone should re-check after some minutes to make sure that entries still exist... For bugs/issues/todos we keep using the google spreadsheet.

from release-specs.

Just looked through everything that was merged since RC1: we only touched

• multiple tests
• the cc2536/cc26x0
• cbor (knwon issue)
• fatfs pkg
• ds1307 driver

IMHO we mainly need to re-test everything regarding these modules (mainly task 02). But I don't see the need to do ALL tests again, and would suggest that we only do some quick 'probing' for all tasks > 2. Do you all agree with this approach?

from release-specs.

to add to @haukepetersen's important notes:
reload the page before doing tick marks. Each marking is an edit in the sense and overrides previous states probably made by others.

from release-specs.

@cgundogan yes absolutely, thanks!

from release-specs.

@smlng: would you mind to 'play' with the cc2536 and the cc26x0 using RC2 a little bit to confirm them working?
@miri64: could you confirm the ds1307 once more?
@A-Paul: when you re-did task 02.01, you also verified that the fatfs pkg is working, correct?

from release-specs.

IMHO we mainly need to re-test everything regarding these modules (mainly task 02). But I don't see the need to do ALL tests again, and would suggest that we only do some quick 'probing' for all tasks > 2. Do you all agree with this approach?

I agree. I will do some probing for the multi hop tests, but I don't expect any changes from the previous results.

from release-specs.

Nice. Any changes from the previous results would indeed be very strange :-)

from release-specs.

@smlng: would you mind to 'play' with the cc2536 and the cc26x0 using RC2 a little bit to confirm them working?

Yes, will do!

from release-specs.

Task 04.05 and 04.06 succeeded as before, however packet loss is still above the desired boundaries. That is, for ping payload above 50B its mostly 30 to 50 % loss with smaller payloads sometimes < 10% but often more like 25%.

This needs further investigation on the cause, i.e. board, driver, or transceiver related or something general.

from release-specs.

cc2650 is working, too.

from release-specs.

@smlng: thanks! I propose to take this as a known issue into the release. Is there already an open issue for this?

from release-specs.

I just tested the tasks with samr21-xpro and PhyNode (pba-d-01-kw2x) and discovered that pings with payloads below 100B work, even with better results regarding packet loss than the remotes (so this might be board related). But starting from 100B nothing gets through, while this worked with the remotes though with high packet loss. This seems to be related to the PhyNode.

from release-specs.

If I remember correctly, there were fragmentation issues with the phynode knwon some time ago. I have no overview how that turned out, @miri64 do you know something here?

from release-specs.

Yes, this is still an issue RIOT-OS/RIOT#4822

from release-specs.

Fragmentation is also an issue for CC2538 (where pings work, but with higher packet loss) and 100B payload should still fit in one, or not?

from release-specs.

No, I don't think so. It depends a little bit on how the cc2538 driver advertises its MAC header length to the upper layer but e.g. for at86rf2xx the switch-over point is at 96B (or even lower) IIRC. There it just advertises the maximum possible MAC payload (without security information), so with both 8 byte addresses, both 2 byte PAN IDs a 2 byte FCF and a 2 byte FCS you get only 112B out of your 127B total MAC frame length. This is rounded down to the nearest divisible of 8 (because 6Lo fragmentations works on a unit of 8 bytes) and you get 96 bytes.

from release-specs.

you get only 112B out of your 127B total MAC frame length. This is rounded down to the nearest divisible of 8 (because 6Lo fragmentations works on a unit of 8 bytes) and you get 96 bytes.

😕 112/8 = 14?

from release-specs.

How did I get to 112...? I think I meant 102 ... But even that is wrong: 127 - 2 * 8 - 2 * 2 - 2 - 2 = 103. So 103 was meant ;-).

from release-specs.

(Also: you can check that your self by incrementing the payload size slowly and sniffing the packets, if it starts to be 6Lo fragmentation packets the threshold is reached ;-))

from release-specs.

04-task7 and 04-task8, done with respectively 0.8% and 0.7% packet loss. I also confirm that the XBee gets a global prefix from a RPI + OpenLabs.

from release-specs.

perfect.

Tasks 02 is also done.

from release-specs.

Since despite me believing so RIOT-OS/RIOT#7750 wasn't merged into the release Task 1.1 failed due to a full disk. I run it now on a different machine in the background with 7750 cherry-picked, but the results will most likely only be there in the evening.

from release-specs.

@aabadie I think you destroyed the post above right now....

from release-specs.

I think you destroyed the post above right now....

how ?

from release-specs.

Well propably not by reloading before editing. I had a lot of ticked tests than reloaded and they disappeared after an edit of yours ;-)

from release-specs.

(including my taking of the tests)

from release-specs.

How exactly to manage circumventing the warning that the content had changed I don't know ;-)

from release-specs.

Ah, ok I see, sorry. One solution is to replay your tests ;)

from release-specs.

For tests that run >1h that is not acceptable :P

from release-specs.

@miri64 your last tick removed my ticks 😩

from release-specs.

gnarf I reloaded before :(

from release-specs.

Sorry

from release-specs.

04-task1, 04-task2, 04-task3 and 04-task4 are done (clearly less than 1% packet loss)

from release-specs.

@A-Paul: when you re-did task 02.01, you also verified that the fatfs pkg is working, correct?

Every checkmark in #50 is full repeated test, based on the new RC2 branch. So, yes.
To prevent further uncertainties I've added a comment.

from release-specs.

@miri64: could you confirm the ds1307 once more?

Sorry saw that only now :-/ Will do tomorrow!

from release-specs.

Multi-hop and single-hop tests are all good. I encountered a bug in the trickle implementation (for RPL), but it's fixed and merged/backported already.

from release-specs.

Thanks everyone! As I see it, we should be good with testing RC2, and I'd say we are very close to getting the release out. As soon as RIOT-OS/RIOT#7874 is merged I would like to go ahead and tag the release. Any objections?

from release-specs.

Any objections?

no objection from my side!

from release-specs.

Release is out -> done.

from release-specs.