App Channels Test Cases about fdc3-conformance-framework HOT 14 CLOSED

@Joe-Dunleavy @robmoffat
First off be careful about joinChannel vs. getOrCreateChannel - these are APP Channel tests and should NOT involve join anywhere (rather retrieve the channel with getOrCreateChannel). Hence, step definition should have the word join removed (replace "joinChannel" with "retrieve channel").

These tests were probably copied from system channels and then updated for app channels, but without correcting for some small differences. As you are working with the channel object retrieved (rather than the fdc3 global) there is no way to "add the listener first, then join the channel". Hence, AC Basic Usage 2 is indeed not valid.

Here is a corrected set of the basic tests - I can't edit the issue so one of you should:

• AC Basic Usage 1 Perform above test
• AC Basic Usage 2 Do the app B steps first to populate the channel with context, check that A will receive the context after adding a context listener

from fdc3-conformance-framework.

Re:

• AC Filtered Context 2: You are correct its not valid (as you can't add a listener before retrieving channel); it and the table should be deleted.
• AC Filtered Context 3, AC Unsubscribe, AC Filtered Context 4,AC Filtered Context 5: can stay, but should follow / be based off AC Filtered Context 1 - note they are all applying a single edit to AC Filtered Context 1, rather than the test right before them in the list
• Do renumber the tests

Regarding AC Invalid Broadcast 1 -- fdc3.broadcast() expects a Context object to be passed to it, so passing it an object that doesn't have a type field or that has an invalid structure isn't possible unless casting it to any first. I don't think this is what you had in mind though?

Its not possible to pass a non-context object in TYpescript and have your code compile. But it is possible in javascript and the behavior in that situation needs testing. The Standard is not super clear on the behavior when invalid objects are passed, but it is clear that only objects with a type matching the context listener should be delivered when a type was specified. Hence, that's whats being tested and there is no equivalent test under basic usage (where no type is passed for the context listener).

from fdc3-conformance-framework.

I've removed AC Filtered Context 2 and renumbered accordingly.

from fdc3-conformance-framework.

@Joe-Dunleavy For the avoidance of doubt I by 'filtered context' we mean cases where the first argument to addContextListener is not null. When that's the case, context of other types should not be delivered to the listener.

The two tests you quote are the unfiltered unsubscribe and filtered unsubscribe tests respectively, yes.

Also, I haven't yet read any test code - all my comments relate to the test descriptions in this issue which appear to have errors in them - probably from when they were converted from notes to issues.

We will certainly do a full code review for you - I'm not going to get to it this week however. I'll probably involve colleagues working on our FDC3 implementation if I can to get as many eyes as possible on them. It might be worth us meeting beforehand to record a run-through to speed up review - ultimately its going to need a number of reviews from FDC3 maintainers/implementors as part of acceptance as the conformance test ;-)

from fdc3-conformance-framework.

Hi @robmoffat , I am currently implementing theses tests.
Some observations:

• AC Basic Usage 2 is no different than the first test. I presume you meant "add the listener first, then join the channel"?.
• With the app channels it isn't possible to add the listener before creating/joining the channel first as the channel object needs to be created first before the listener can be added.
• It also isn't possible to broadcast to an app channel that hasn't been created/joined first for the reason mentioned above => AC Basic usage 3
• The filtered contexts -- starting from AC Filtered Context 2, are identical to the System channel tests that I implemented last week. I've changed them so that they are implemented using an app channel instead. Also, as mentioned above, for these tests the channel needs to be created/joined before adding the listener.

from fdc3-conformance-framework.

One more thing that needs clarifying.

RegardingAC Invalid Broadcast 1 -- fdc3.broadcast() expects a Context object to be passed to it, so passing it an object that doesn't have a type field or that has an invalid structure isn't possible unless casting it to any first. I don't think this is what you had in mind though?

from fdc3-conformance-framework.

@kriswest

from fdc3-conformance-framework.

AC Context History tests are correct (use getCurrentContext), although watch out for the context type names fdc3.instrument rather than fdc.instrument

from fdc3-conformance-framework.

@kriswest Thanks for clarifying.
I have now implemented these and should be ready for PR at some point today.

One last point I'd like to mention. For AC Filtered Context 4 -- looking through the documentation it doesn't look like there is a fdc3.leaveCurrentChannel(); equivalent for app channels.

Am I correct in thinking that the only way to leave an app channel is to switch to a different channel?
in that case, the AC Filtered Context 4 test is also not possible.

from fdc3-conformance-framework.

@Joe-Dunleavy

One last point I'd like to mention. For AC Filtered Context 4 -- looking through the documentation it doesn't look like there is a fdc3.leaveCurrentChannel(); equivalent for app channels.

Correct, there is no leaveCurrentChannel analog. Rather AC Filtered Context 4 should repeat AC Unsubscribe but with the filtered context listener and ensure context stops arriving.

Am I correct in thinking that the only way to leave an app channel is to switch to a different channel?
in that case, the AC Filtered Context 4 test is also not possible.

No, you can retrieve and work with as many channel objects simultaneously as you like. You can remove context listeners on those objects to stop receiving messages from them. Hence, I would re-write AC Filtered Context 4 as AC Filtered Context Unsubscribe

FYI I didn't review these issues directly, although I did help dictate the original cases they were based on (in another doc Rob has). I think the issues here probably all stemmed from copying the system/user use cases and then missing a few of the underlying differences between them. If there are any other areas that might have similar problems let me know to go review (I'd assume not though as this is the one area of (near) duplication).

from fdc3-conformance-framework.

The overlap with small but important differences between user/system channels and app channels is one of the toughest/most dangerous parts of FDC3 for comprehension unfortunately. I know half a dozen people that have read those parts of the standard a dozen times without spotting key differences. That will no doubt bleed over into compliance test results...

from fdc3-conformance-framework.

@robmoffat if you add me to the team for this repo I can add the other corrections to this issue myself and do so for any others I see. Otherwise, there's a few more above to correct, all covered in this thread, e.g.:

Correct, there is no leaveCurrentChannel analog. Rather AC Filtered Context 4 should repeat AC Unsubscribe but with the filtered context listener and ensure context stops arriving.

from fdc3-conformance-framework.

Hi @kriswest, apologies if some of the tests are wrong.

I hadn't realised until your last comment that a "filtered context" is the context type filter that is applied to the listener. Therefore some of the contexts in these tests are being filtered when they shouldn't be. I'm fixing this now in both the app channel and user channel tests.

As for the AC Filtered Context 4 test for app channels (testing unsubscribe() for both filtered and unfiltered contexts), this should be covered in the following two tests:

• "Should not receive context when listening for all context types then unsubscribing an app channel before app B broadcasts to that channel"
• "Should not receive context when unsubscribing an app channel before app B broadcasts the listened type to that channel"

Just re-read your comments and could not spot any other issues so if there are any more issues, it is possible that I have misunderstood something that was said.

If you would like to have a chat on teams, I can walk you through what I've done with these tests and we can clear up any misunderstandings. I am free until 3:30, or after 4:30pm. Or if it's easier for you to apply the required changes yourself that's fine with me.

from fdc3-conformance-framework.

@robmoffat if you add me to the team for this repo

invite on it's way @kriswest

from fdc3-conformance-framework.