READ RESPONSE with timestamped SenML about leshan HOT 22 OPEN

This is another part of LWM2M specification which is unclear to me for which I have been unable to obtain clarification.

See relative OMA and Leshan issue about that :

• OpenMobileAlliance/OMA_LwM2M_for_Developers#538 (comment)
• OpenMobileAlliance/OMA_LwM2M_for_Developers#567
• #1479

More generally, I tried several times to interact with the OMA authors to find a solution to this global situation where we failed to get an answer to our LWM2M misunderstanding, without success ... I'm not even able to get answer using private mail ... this is like this since several years now and I must confess the situation become really frustrating 😞.

Let's try to better understand together what could means supporting ReadResponse with timestamp.

Currently, I understand a read request is about sending current value on the device.
With this is mind, it's hard to me to understand how timestamp can be used on Read.

Do you have use case in mind ? What exactly did IoTerop SDK ?

from leshan.

(In your log there is something strange : [path:/3300/0/5700](http://path/3300/0/5700)], I don't get what this is this http is that a Leshan log bug ?)

from leshan.

(In your log there is something strange : [path:/3300/0/5700](http://path/3300/0/5700)], I don't get what this is this http is that a Leshan log bug ?)

This is not bug - this log is from our backend 😉 , but in leshan server we also observing this error:

Caused by: org.eclipse.leshan.core.node.codec.CodecException: 
Unable to decode node[path:/3303/0/5700] : value should not be timestamped

We agree that specification does not tell anything specific on READ response, excepted a mention that " ...are not supported by the LwM2M Server MUST be silently ignored."

But specification in the SenML chapter does not make any difference, and says "When a LwM2M Client is supporting the SenML JSON data format and the format is used to transport Object Instance(s), multiple resource and single resource values for both "Read" and "Write" operations, SenML JSON payload MUST use the format defined in this section. This format MAY be used also for transporting a single value of a Resource." and it does not says that it is forbidden to use "t" or "bt" on READ response, so it should be allowed.

from leshan.

We agree that specification does not tell anything specific on READ response, excepted a mention that " ...are not supported by the LwM2M Server MUST be silently ignored."

The whole sentence is :
"Any optional resources included in the "Read" operation response that are not supported by the LwM2M Server MUST be silently ignored.." (source : LWM2M-v1.1.1@core§6.3.1. Read Operation)

So this talk about optional resource from LWM2M model that server could not know so this is not relevant here.

it does not says that it is forbidden to use "t" or "bt" on READ response, so it should be allowed.

Even if I understand your argument, it can also be used to say that t and bt can be used in SENML for Write request because "this is not forbidden so this is allowed". But for me this doesn't make sense to put timestamp in write request. So I don't think this is a strong argument.

So from my point of view specification/authors doesn't help us to know if this should be supported and how this should be supported.

My current point of view on read is that it doesn't have so much sense but it seems we disagree so I would like to better understand you.

Is it possible to allow such payload with an embedded timestamp from READ responses ?

Could you precise the behavior/API changes you would like to see in Leshan ?
and also you didn't answer to : Do you have use case in mind ? What exactly did IoTerop SDK ? (I mean what is the meaning of this timestramp ?)

from leshan.

This topis is transferred from Jarek to me 🙂

From Technical point of view:
When there was only lwm2m 1.0, senML transport format did not exists in lwm2m, and the only way to send a "measure timestamp" was to use specific timestamp lwm2m resource (ex: object 3300 resource 5518).
But with lwm2m 1.1 senML transport format allows a device to send the "measure timestamp" at any time, and for all objects, (not only the one where a specific "timestamp" resource exists). That is why also in Orange use case we also need to get the "measure timestamp" when doing just a READ.

From business case point of view:
Battery-powered devices allow for adjusting sensor sampling frequency and transmission frequency to optimize power consumption. Message and measurement timestamps may differ. The timestamp of the measurement returned by a READ is important because it allows us to determine the age of the measurement taken by the device. This helps us to better correlate it with the analysis context. For instance, if an electricity meter value received at 5pm corresponds to a measurement taken at midday, it is normal to have only half the daily electricity consumption.
This is the case for instance with Adeunis Comfort Serenity device: https://www.adeunis.com/en/produit/temperature-humidity-co2-voc-iot-sensor/

from leshan.

When there was only lwm2m 1.0, senML transport format did not exists in lwm2m, and the only way to send a "measure timestamp" was to use specific timestamp lwm2m resource (ex: object 3300 resource 5518).

Not exactly true, I think we can consider we have exactly same "question" with old JSON format (application/vnd.oma.lwm2m+json, 11543)
So this could also be a lwm2m 1.0 question.

But with lwm2m 1.1 senML transport format allows a device to send the "measure timestamp" at any time, and for all objects, (not only the one where a specific "timestamp" resource exists)

I disagree on that point. 🙂
The specification says nothing clear which allow to do that.
I don't think that "this is not forbidden so this is allowed" is a good argument here. I explained why above : #1553 (comment)
And I think this is reasonable to think that time is supposed to be used only with historical representation.
(see : OpenMobileAlliance/OMA_LwM2M_for_Developers#538 (comment))

Now, I understand that we can have different interpretation as the specification is pretty ambiguity and I also understand your use case.

So now let's try to imagine how we could add support of that in Leshan
Would it be OK if we support it like we support it with Observe but you can only have 1 timestamped node ?

So if we read a resource, you get a only 1 TimestampedLwM2mNode with a LwM2mResource as node.
That's also mean if you read an object you get only 1 TimestampedLwM2mNode with a LwM2mObject as node.
So timestamp should be the same for the whole object.

With an example with :

// A) This is OK, because there is 1 timestamp for the whole node
[
    { "bn":"/6/0/","n":"0","v":-21.0, "bt":1706888656},
    {"n":"1","v":141.0},
    {"n":"5","v":1706888656}
]
// B) This is not OK because there is several measurement timestamp in the same node : 
[
    { "bn":"/6/0/","n":"0","v":-21.0, "bt":1706888656}, 
    {"n":"1","v":141.0, "t":"10"}, 
    {"n":"5","v":1706888656}
]

If we want to support A) with current state of Leshan this should be doable.
If we want to support B) this will break a lot of code and probably will need lot of re-design... I'm not sure we should do that...

from leshan.

I wanted to ask what is your point of view on how to simulate it?

from leshan.

I'm not sure I get what you mean ?

from leshan.

I mean what is your opinion on how should I implement it, can I make changes at client side or should I create some test at the begining?

from leshan.

Should we first agree about what we want to try to implement ?
I didn't get if A) is fine for you ? or if you need B) ? #1553 (comment)

I mean what is your opinion on how should I implement it

I understand you want to work on this now ? Does it mean that you will stop to work on write_attribute ?

from leshan.

Sorry, I didn't mention it earlier. We would like to implement A) - only with 1 timestamp.
I will work on this issue and also on write_attribute

from leshan.

Sorry, I didn't mention it earlier. We would like to implement A) - only with 1 timestamp.

OK so we could try to implement that :

1. changing ReadResponse API. (inspired by ObserveResponse)
2. implement the feature at server side (optionally with integration tests using LockStepLwM2mClient)
3. decide IF we want to add a way to make leshan client send timestamp on read request and how it could looks like

But before to start to work on this,
As Leshan is probably still pretty new for you, are you sure its a good idea for you to work on 2 task at same time ? Should it be better to focus on 1 then once it's done move on a new one ?

from leshan.

Maybe it's good idea. Because of our priorities I'll switch to this issue and when I finish I'll back to write attributes, ok?

from leshan.

Because of our priorities I'll switch to this issue and when I finish I'll back to write attributes, ok?

Ok, no problem.

from leshan.

2. implement the feature at server side (optionally with integration tests using LockStepLwM2mClient)

Is it possible to use LockStepLwM2mClient test without modification at client side?

from leshan.

LockStepLwM2mClient is not really based on LeshanClient. It is mainly a way to receive and send CoAP message + some LWM2M facilities.

from leshan.

Can you help me with test read response using LockStepLwM2mClient? I looked through observe and observe composite tests (there is a hack with triggering notification which I can't make in read case). I don't know how to simulate read response with time stamp, could you give me some tips or example?

from leshan.

I didn't investigate too much but maybe something like :

// Do all the stuff about registering

// Server send Read Request
 
// Wait for server READ request and store CoAP Token and MID
client.expectRequest().storeToken("TKN").storeMID("MID").go();

// Send custom READ Response
// here you need to create the Coap Response and its payload on your own (you can not use LWM2M API)
// I see that maybe it miss a way to add content format an so maybe you need to add it.
client.sendResponse(Type.ACK, ResponseCode.CONTENT).loadMID("MID").loadToken("TKN")
                .payload(givenServerEndpointProvider).go();               

Let me know if help is enough.

from leshan.

Hi,
I still have a problem, currently response is a null. Here is a link to my changes and test in LockStepTest:
https://github.com/JaroslawLegierski/leshan/blob/d044723a764e21a1cad7b330a23576c5625d1be5/leshan-integration-tests/src/test/java/org/eclipse/leshan/integration/tests/lockstep/LockStepTest.java

Maybe I'm still doing something wrong.

from leshan.

I look at it and as I explain at #1553 (comment), you should probably first send the Read Request before to call expectRequest because this is sync API. There is an async API to send request. (server.send())

Let me know if this solve your problem.

from leshan.

Could you take a look at my changes, I think it is almost finished. Here is a link:

master...JaroslawLegierski:leshan:read_response_senml

from leshan.

I take a very quick look at it.

Some remarks at first sight :

1. Following #1553 (comment), I think that ReadResponse constructor should have a TimestampedLwM2mNode instead of List<TimestampedLwM2mNode> as argument?
2. Please do not use merge commit, use cherrypick or rebase instead.

If you want more details review, you can open a PR (but please no merge commit)

from leshan.