let's better focus on the core ZonedDateTime aspect

Ok, sure. In your case, since, it looks like, you can choose the format in which to send data, right? Then, I think a good choice depending on the circumstance is either an Instant for representing specific moments (+ an optional TimeZone if the way the datetimes are displayed should also be controlled) or LocalDateTime + TimeZone for representing "the moment when the wall clock in the given time zone displays this". What do you think?

I can't fully choose the format because we have an existing system and already pre-existing official specifications that we have to support.

Actively juggling with LocalDateTime and TimeZone as separate values is definitely not enough.

• How do you compare dates?
• How do you parse and format dates?
• How do you serialize/deserialize? This is almost always a single field in the serialized data (e.g. JSON).
• How do you make sure no mistakes happen because people forget to take the TimeZone into account in the whole codebase with tons of developers of varying experience?

We had to build our own ZonedDateTime to deal with this and I'm pretty sure everyone else is doing the same thing because this is a very common use-case. Other libraries already provide it out of the box and I think this library should do it, too.

from kotlinx-datetime.

Which does result in the loss of information

from kotlinx-datetime.

So not only specifically for OpenAPI, but in general to represent RFC3339 Datetimes (which is meant to be the Internet standard).
You can of course use an Instant here, but that means loosing the offset information. I don't have a specific use-case at hand right now, but I'm sure there was a reason RFC3339 was designed this way and I can think of some cases where the offset could be useful (for example if the server already converts it to the user local time as configured in the server or this information could also represent the local time of the server). I don't know if it's smart to use this, but as in this specific case it's often about existing APIs, this could render APIs unusable. You might find some more use-cases in the sources of the RFC.
Also as far as I'm aware Instant also can't parse RFC3339.

from kotlinx-datetime.

One specific use-case - our API deals with public transport departure times (bus, metro, etc) in various cities. Public transport departure times (like flights) are always displayed in the local city time. The API returns times conforming to RFC3339 such as 2020-07-31T09:16:15+02:00. If these are parsed and represented as an Instant, we need an additional redundant data point - the zone offset - to convert to and display LocalDateTimes to the user.

This does not imply that it is necessary to have a kotlinx.datetime.OffsetDateTime, but at the very least it would be useful to have a parsing function that returns a LocalDateTime and ZoneOffset from an RFC3339 time.

from kotlinx-datetime.

Fixed in recent release: https://github.com/Kotlin/kotlinx-datetime/releases/tag/v0.2.0

But parsing it to an instant the time zone is getting lost isn't it?

from kotlinx-datetime.

My use-case for OffsetDateTime is to parse data from a database. So serialization and deserialization are sufficient. But Ideally we can override them automatically thorugh kotlinx

from kotlinx-datetime.

I compared how java.time handles that. We're discussing the DST point too much and let's better focus on the core ZonedDateTime aspect.

I think aligning with java's behavior makes a lot of sense instead of rolling your own and dealing with a lot of inconsistent behavior between different services.

LocalDateTime + TimeZone for representing "the moment when the wall clock in the given time zone displays this".

This seems to basically be how Java implements it and is also how I work around it right now.

from kotlinx-datetime.

FWIW: Crystal's equivalent of Instant always carries a location property to designate the time zone instance. Internal represenatation is normalized to UTC, so for example comparison between instances with different locations doesn't require conversion. The time zone information is only used for interaction with a human-readable datetime representation (parsing, stringfigication, calendrical arithmetics).
This has proven to be very practical and enables use cases like the one described here without adding too much complexity, and actually simplifying the API because you don't need to pass a TimeZone instance to every method which operates time-zone aware.

from kotlinx-datetime.

Came here searching for a multiplatform replacement for java.time.ZonedDateTime. It's really surprising that kotlinx-datetime isn't equally able to parse an ISO-8601 and/or a RFC-3339 date string that includes a non-zulu time-zone. Or am I getting it wrong?

from kotlinx-datetime.

We do want to support returning the parsed offsets, but the exact implementation is unclear: maybe it's sufficient to just return a Pair, maybe something heavier, like ZonedDateTime, needs to be introduced, or something in-between would be the best choice.

To decide this, we need to consider the specific use cases. @harry248, could you please share yours? @justasm already mentioned that an API they're dealing with returns the instants with a time zone that also indicates how the instant should be displayed to the end-user—this is a good point, thanks!

from kotlinx-datetime.

@dkhalanskyjb It's exactly the same scenario in our case.

from kotlinx-datetime.

If you want to keep the API as small as possible then here’s some provocative food for thought: If you had ZonedDateTime instead of Instant maybe people wouldn’t be missing Instant because ZonedDateTime in UTC handles that use-case already.

The question is where you want to make distinctions explicit via types and maybe where you want to provide common base types to allow abstracting the distinction away.

The use-case for instant is different: It should be use to record past events that may not change (e.g. the Unix timestamp cannot change) while ZonedDateTimes can (e.g. if laws change), which is why they should be used to record things like future appointments, so maybe not impossible to use a common type, you have to be extra careful to not break those functionalities.

from kotlinx-datetime.

Yes, I think you are right. What I was thinking of for Instants is something like a "last modified" in a database or something similar, where it's more important to have a (easily comparable and transferrable) timestamp, where you care about the distance and order (which is way easier without timezones) and not where it was recorded. Examples would be caches, logs if you log in UTC, distributed systems, etc.. when face the user you'd probably just convert those to Local datetime using the user's local time zone.

Edit: A thing where past time zones might change would be if you had a library for example that wasn't updated (in time). While this is probably rather rare and a bug, Instants (often serialized as unix time) just provide that security against that, for use cases, where it's required.

from kotlinx-datetime.

Because it's a much more (on purpose) constrained and a more simple class. I mean why do we want UInt when we have Int? It's artificially constrained and simpler, and that makes life easier. Also see my list above. It can provide certain guarantees that ZonedDateTime can't.

from kotlinx-datetime.

So basically you'd want the java Temporal thing that is like a real mess IMO it's so generalized that it's actually pretty useless again and not very indicative of what's going on.

from kotlinx-datetime.

rules for validating the European Union's COVID vaccination certificates:

These can be parsed already with Instant, only the offset will be lost. Is it important in this case?

We deserialize the JSON from the backend and serialize it to the DB and I'd personally prefer to always store 100% exactly the original information instead of Instant which means loss of information. In a PR review at least I'd mark the use of Instant as future bug potential - even if right now it might not cause any problems. The future can bring crazy new requirements.

I'd like to emphasize that unintended loss of information is a VERY BAD thing and causes very difficult bugs or impossible problems later. No matter in which context, you need to be careful about keeping the original information content.

Well ZonedDateTimes may not even refer to an unambiguous point of time. See the time change. If you need to refer to an exact point of time you need a Instant + TimeZone, which would be different from ZonedDateTime.

When is Instant + TimeZone not equivalent to a ZonedDateTime?

Just a nitpick: if the time zone DB can get refreshed while the process is running you need LocalDateTime + TimeZone because otherwise you lose stability / reproducibility under serialization.

I carefully read all of the above and noticed some things that (I think) are technically incorrect. Let me try to also summarize the state of things.

ZonedDateTime

As implemented in Java, (almost always) equivalent to Instant + TimeZone.

• (Almost always) automatically adapts LocalDateTime so that the Instant + TimeZone pair is represented correctly.

• (Almost always) adapts LocalDateTime if it happens to fall into a time gap.

Do you have an example? My impression is that it adapts the Instant to ensure that the LocalDateTime+TimeZone is stable. Even in the ZonedDateTime source code it's private final LocalDateTime dateTime; and the documentation says:

"""
A ZonedDateTime holds state equivalent to three separate objects, a LocalDateTime, a ZoneId and the resolved ZoneOffset. The offset and local date-time are used to define an instant when necessary. The zone ID is used to obtain the rules for how and when the offset changes. The offset cannot be freely set, as the zone controls which offsets are valid.
"""

To be even more precise, it's actually LocalDateTime+Offset or LocalDateTime+ZoneId depending on what input you provide. So Java's ZonedDateTime is a supertype of, let's call them, OffsetDateTime and RegionDateTime (zone id).

In effect, ZonedDateTime is Pair<Instant, TimeZone>, but

• With some additional behavior like comparisons also taking LocalDateTime corresponding to the instant into account. Is there evidence that we do need that additional behavior?

I actually dislike that behavior. From an end-user point of view the Instant is what matters for comparisons and ordering.

For special sorting use-cases one can still construct an artificial ordering, but I don't remember ever needing this.

• With more streamlined arithmetic due to not requiring specifying a time zone explicitly. The fact that two parameters often go together into functions is not enough to throw them into one object

The same argument can be made for date + time. Still we do it because otherwise things become messy (how do you even serialize into a single field if they're separate?).

• I thought that LocalDateTime + TimeZone is a valid use case, but after a night of sleep, I don't anymore. If we assume that the zone rules may change, then the ZonedDateTime object will be invalidated and recreate its LocalDateTime part, losing some information. If we assume that the rules will not change, OffsetDateTime is also fine for representing this. So, the specific behavior of ZonedDateTime, it seems, is never useful for this.

That can be exactly what you want because people in the affected country keep thinking in LocalDateTime, not in Instant. If a country shifts its base zone offset (which does happen) or moves the DST switch around then your calendar events (meetings, concerts, etc.) should stay unmodified in LocalDateTime and auto-correct the underlying Instant because people in that country keep going to work at the same LocalDateTime and even airports might adapt their active hours (some disallow flights after midnight for example). That's why Java's ZonedDateTime is not Instant+TimeZone, but LocalDateTime+TimeZone.

For the past a ZonedDateTime can be mapped 1:1 to Instant+TimeZone because zone information should never change for the past. ZonedDateTime additionally handles the future more correctly. That's the difference.

Issues with introducing ZonedDateTime:

• Unclear what to do with broken objects, where LocalDateTime + UtcOffset + TimeZone can't be validly combined.

• Baroque comparisons.

• Only as portable as the TimeZone object is.

• Duplicates the functionality of Instant, which will lead to confusion about which one to use.

• Error-prone when representing a LocalDateTime in the future in an implicit time zone, due to possible rules change.

That's not correct. It's pretty reliable at representing LocalDateTime and that's the intended use-case. This is also why it's not duplicating the functionality of Instant and like I said above it's very important to prevent loss of information under serialization. The only valid comparison that is left is ZonedDateTime vs. Pair<LocalDateTime, TimeZone> but to me it's pretty obvious that people will prefer ZonedDateTime.

It's interesting that neither ZonedDateTime nor OffsetDateTime is fully suitable for representing a future event with the ground truth being a LocalDateTime in a specific time zone. Looks like some form of a pair of LocalDateTime and TimeZone may be even more useful than a proper ZonedDateTime.

Oh the irony.😄 Sounds like we agree.

from kotlinx-datetime.

It would probably be useful to use an ZoneOffset for this, but it cannot be (directly) parsed currently

from kotlinx-datetime.

conversion from and to java.time is also easy (using the example):

import kotlinx.datetime.toJavaLocalDateTime
import kotlinx.datetime.toJavaZoneOffset
import kotlinx.datetime.toKotlinLocalDateTime
import kotlinx.datetime.toKotlinZoneOffset

public fun java.time.OffsetDateTime.toKotlinOffsetDateTime(): OffsetDateTime {
    return OffsetDateTime.of(
        toLocalDateTime().toKotlinLocalDateTime(),
        toOffsetTime().offset.toKotlinZoneOffset()
    )
}

public fun OffsetDateTime.toJavaOffsetDateTime(): java.time.OffsetDateTime {
    return java.time.OffsetDateTime.of(dateTime.toJavaLocalDateTime(), offset.toJavaZoneOffset())
}

from kotlinx-datetime.

So did I get it right that you need such a type to represent deserialized values of the built-in date-time OpenAPI format?
Why is it important to preserve the offset information in this case? What can happen if the Instant type is used instead?

from kotlinx-datetime.

Fixed in recent release: https://github.com/Kotlin/kotlinx-datetime/releases/tag/v0.2.0
Sorry, I mistaken this for another issue.

from kotlinx-datetime.

The parsed time zone is not returned to the user, yes.

from kotlinx-datetime.

Then unfortunately this is not a solution / fix for us, as we have to keep the time zone.

from kotlinx-datetime.

I've had one (rare & weak) use case for OffsetDateTime so far, just recently:
I had to parse an ISO 8601 timestamp from a 3rd party API. While I could parse it as Instant I also needed to know the offset in that case because that API doesn't provide a proper time zone. However I've asked them to include it as that would make more sense and be more reliable than using offsets.

That use case is more or less #90 (comment) where also the API isn't providing an ideal data format. It should probably provide a local datetime + time zone instead of datetime merged with an offset.

In general over many different projects there hasn't been any instance where I've needed ZonedDateTime.
In most cases where a TimeZone is needed it's stored and passed around separately anyway.
For example for each insurance policy we store a TimeZone that's used for all LocalDate(Time) instances affecting that insurance. No need for ZonedDateTime here. TimeZone is typically part of a context (e.g. the insurance policy, a calendar entry, a geographical location) and thus not linked directly to individual Date(Time) instances. That was true for all projects so far.

An offset I've never needed except for the very rare use case mentioned initially.

from kotlinx-datetime.

Our use-case is to be able to losslessly serialize/deserialize and work with arbitrary future datetimes with zone and DST information like java.time.ZonedDateTime already does:

1. offset only (e.g. +02:00)
2. zone id only (e.g. Europe/Berlin)
3. zone id with a flag for DST vs. standard (e.g. [Europe/Berlin][DST], [Europe/Berlin][ST] or maybe +02:00[Europe/Berlin])

Only (3) can correctly represent a future event/datetime if the zone's offset and DST handling gets changed unexpectedly. Using separate LocalDateTime and TimeZone and dst flags is unnecessarily complicated and error-prone.

Finally, with such a flexible ZonedDateTime maybe there is no need for a separate OffsetDateTime (I never had a need for that at least).

from kotlinx-datetime.

Hi @wkornewald!

Only (3) can correctly represent a future event/datetime if the zone's offset and DST handling gets changed unexpectedly.

Could you elaborate on this? I don't see why this is true. Consider two cases here:

• The instant is what's important. For example, we may need to run something exactly 2000 hours later. In this case, this information is encoded as an Instant, and optionally a TimeZone to convert to LocalDateTime for human readability.
• The clock readings is what's important. For example, something, like a meeting or a train departure, needs to happen on June 12th, 13:30. In this case, the right thing to do is to transfer the LocalDateTime + TimeZone only: it doesn't matter if it's DST or not, what offset it is, etc., the only thing that matters is that the date (which is the ground truth in this case) map correctly to the Instant at that instant.

Am I missing some case here where LocalDateTime + TimeZone + ZoneOffset are all important?

In general, IIRC, this is the first time someone asks that we handle the DST in a specific manner.

from kotlinx-datetime.

Note that point (3) is just an edge case to be able to represent a precise umanbiguous clock readings time. My main suggestion is to have a ZonedDateTime that at least fulfills (1) and (2), but additionally being able to losslessly represent serialized string-based dates produced by java.time would be nice.

Regarding (3): If you want to trigger an event at an exact point in clock readings time then you have two different interpretations. How do you distinguish the point in time before the DST change vs. the time after the change? 02:30 could mean two points in time.

from kotlinx-datetime.

How do you distinguish the point in time before the DST change vs. the time after the change? 02:30 could mean two points in time.

Tricky.

• If we assume that the DST dates do not suddenly change, we can use Instant to represent this: the translation from Instant to LocalDateTime won't be ambiguous.
• If we assume that the DST dates may suddenly change and the datetime is important, then we want to adapt to the changes. How do we do that?
  • If some datetime used to be ambiguous, but after the rules are changed, it is no longer ambiguous, then the DST flag should be ignored. Seems fairly obvious: since the datetime is important, we don't want to do anything that won't preserve it.
  • If some datetime used to be unambiguous, but after the rules are changed, it becomes ambiguous, then, yes, we could use the DST flag to choose the correct side of the overlap, but would this do any good? My worry is that this choice would be rather arbitrary. If someone used, say, a date picker to choose 12:15 on a particular day and it happened to be DST, but then the rules changed so that 12:15 occurs twice on that day, does the existing DST flag really say anything about what the user would have wanted in this case? I don't think so.
  • If some datetime used to be ambiguous (for example, 12:15 when the clocks are moved from 13:00 to 12:00), and after the rules are changed, it stays ambiguous (for example, the clocks are moved from 12:30 to 11:30), then, yes, the DST flag would help. Now, this (to me) seems really obscure and not worthy of a special case. I don't really know if such an event ever happened even, and wouldn't be surprised if it hasn't.

from kotlinx-datetime.

Imagine a „simple“ calendar app that wants to allow users to specify the DST flag when an event occurs at some ambiguous time. The exact day or time when the switch to DST occurs might change in the future and an Instant can’t deal with that, so the only option would be ZonedDateTime with a nullable DST Boolean (part of TimeZone?).

from kotlinx-datetime.

But you're right, this is pretty obscure and indeed maybe not worth it - though then we can't be fully compatible with everything representable by java.time then. Anyway, my main request was having a ZonedDateTime where the zone can be either an id or an offset and I just continued to think of obscure edge cases and then the DST question came up, so I compared how java.time handles that. We're discussing the DST point too much and let's better focus on the core ZonedDateTime aspect. 😄

from kotlinx-datetime.

Imagine a „simple“ calendar app that wants to allow users to specify the DST flag

Can't really imagine that. I'd expect the calendar app to provide a time picker, which, if the author of the calendar so chose, would have some times repeated.

we can't be fully compatible with everything representable by java.time then

Oh, I think it's not a viable goal at all, given how vast the Java Time API is. We want this library to be simple and discoverable, not all-encompassing.

let's better focus on the core ZonedDateTime aspect

Ok, sure. In your case, since, it looks like, you can choose the format in which to send data, right? Then, I think a good choice depending on the circumstance is either an Instant for representing specific moments (+ an optional TimeZone if the way the datetimes are displayed should also be controlled) or LocalDateTime + TimeZone for representing "the moment when the wall clock in the given time zone displays this". What do you think?

from kotlinx-datetime.

If you want to keep the API as small as possible then here’s some provocative food for thought: If you had ZonedDateTime instead of Instant maybe people wouldn’t be missing Instant because ZonedDateTime in UTC handles that use-case already.

The question is where you want to make distinctions explicit via types and maybe where you want to provide common base types to allow abstracting the distinction away.

from kotlinx-datetime.

You mean that laws can change the meaning of the past for a ZonedDateTime (in UTC)?

from kotlinx-datetime.

You mean that laws can change the meaning of the past for a ZonedDateTime (in UTC)?

Yes. For UTC this might not be a problem, but for other timezones it might (see the efforts on DST). Just remember to think about all the use cases before trying to implement something.

from kotlinx-datetime.

That’s the point I was trying to provoke. ZonedDateTime in UTC can handle the Instant use-case.

Personally, I’d only still want to have the distinction of both types so I can enforce the UTC timezone via types where that is critical.

from kotlinx-datetime.

That’s the point I was trying to provoke. ZonedDateTime in UTC can handle the Instant use-case.

Personally, I’d only still want to have the distinction of both types so I can enforce the UTC timezone via types where that is critical.

Well I that case you can just use a LocalDateTime, because the Timezone is always UTC and they just define an interface that both implement.

from kotlinx-datetime.

I've had a look at java and they both implement Temporal. It implements some changing of parts, addition, subtraction and time difference. Not having a compare function (and maybe even no common interface at all) might make sense so people have to explicitly have to convert to Instant so the dev has to understand that they are comparing two types for different purposes and basically has to opt-in to this conversion rather than have it happen implicitly.
That way the type could be pretty light weight any basically only be a wrapper for LocalDateTime and TimeZone, with comparison, equals (maybe 2 functions because you'd probably want the default equals to check if the time zones and the date time is the same) and a difference calculation or so.

from kotlinx-datetime.

I've had a look at java and they both implement Temporal. It implements some changing of parts, addition, subtraction and time difference. Not having a compare function (and maybe even no common interface at all) might make sense so people have to explicitly have to convert to Instant so the dev has to understand that they are comparing two types for different purposes and basically has to opt-in to this conversion rather than have it happen implicitly. That way the type could be pretty light weight any basically only be a wrapper for LocalDateTime and TimeZone, with comparison, equals (maybe 2 functions because you'd probably want the default equals to check if the time zones and the date time is the same) and a difference calculation or so.

Our implementation is actually a rather lightweight wrapper (a few hundred lines) with support for parsing/formatting, handling offset vs. zone id, comparison, etc. And I also agree that for some operations it could be better to make the type difference explicit. That would need some more thought...

We have even more complicated use-cases because we work with medical data that can contain arbitrarily precise datetimes (year, year-month, year-month-day, full datetime without zone or with zone) and we had to abstract even over that, so you can e.g. sort by date easily or format to a human-friendly string etc. Anyway, that's a different topic.

The use-case for instant is different: It should be use to record past events that may not change (e.g. the Unix timestamp cannot change) while ZonedDateTimes can (e.g. if laws change), which is why they should be used to record things like future appointments, so maybe not impossible to use a common type, you have to be extra careful to not break those functionalities.

I've thought about this some more and would like to challenge several assumptions. Do you have any examples which show that it's not an extremely obscure edge case to have timezone details of the past get changed?

We can also look at it this way: If you create an event for a future ZonedDateTime and then two years later the event moves to the past, do you suddenly have to convert it to an Instant, so it still refers to the same actual point in time? I've never heard of something strange like that. People just keep the ZonedDateTime and happily represent the past without any problems.

Finally, there's always this implicit assumption in the whole discussion and the README of this project:

• Instant: past only
• LocalDateTime+TimeZone: future (only?)

But the reality is more like

• ZonedDateTime: past and future
• Instant: mostly past - but only if you have to enforce UTC (e.g. because third-party APIs/schemas don't support it) or if you really need to normalize everything to UTC (but this can become a mistake for end-user data - I've had my share of mistakes with choosing Instant instead of ZonedDateTime)

So the real question for deciding between ZonedDateTime and Instant is: Do you know the original timezone and do your storage schema and communication APIs support timezones? If the answer is no then you pick Instant. For everything else you pick ZonedDateTime.

In most user-facing apps I see practically no use for Instant. In fact, in some previous projects where people started with Instant it happened that we later found out that this was a mistake and ZonedDateTime would have allowed an important analysis (e.g. when the user wakes up, when he takes some medicine, etc.). DST changes happen and users travel between countries. In many cases it's useful to not just look at the data in UTC, but actually know the local time and the timezone of all events even in the past. Of course there are also cases for Instant like storing backend metrics. But I think this library is making a mistake by putting so much emphasis on Instant. ZonedDateTime should be your default choice for everything - past and future. Every use of Instant needs deliberate thought because you can't go back once you've lost the timezone information.

from kotlinx-datetime.

Right, so maybe the rule of thumb is

• ZonedDateTime: user data
• Instant: machine data (logs, metrics, cache expiry, etc)

The opinions in this discussion probably just reflect what kind of data each person works with the most.

from kotlinx-datetime.

Basically in the end we'd need both anyway and they probably shouldn't be mixed without explicit conversion (so no common API on the public API). Both should retain their properties and I guess for most users it's not important how it's implemented under the hood as long as it's fast and it works.

from kotlinx-datetime.

First of all, thank you both for the discussion! It will really be of huge help when/if we arrive at some conclusion.

LocalDateTime + TimeZone for representing "the moment when the wall clock in the given time zone displays this".

This seems to basically be how Java implements it and is also how I work around it right now.

Not really. ZonedDateTime is equivalent to Instant + TimeZone. LocalDateTime + TimeZone is not enough: if some LocalDateTime happens twice in a given time zone, LocalDateTime + TimeZone won't be enough to distinguish between them.

Of course, LocalDateTime + FixedOffsetTimeZone, for example, is unambiguous.

Actively juggling with LocalDateTime and TimeZone as separate values is definitely not enough.

How do you compare dates?

If you need to compare moments in time, compare Instant values. If you just want to see which clock reading goes first, compare LocalDateTime. The former is more useful.

Comparing ZonedDateTime is like comparing Instant values, but much more complex with no clear benefit. First, ZonedDateTimes are compared as Instant values. If they are equal, then they are compared as LocalDateTime values. If those are equal, the identifiers of the time zones are lexicographically compared.

I think what they were going for is to establish some order on the ZonedDateTime values, but is it really any more practical than just comparing Instant values? Why?

How do you parse and format dates?

That's a real issue. We're working on the parsing/formatting API. However, I don't see how ZonedDateTime vs separately using TimeZone relates to this.

How do you serialize/deserialize? This is almost always a single field in the serialized data (e.g. JSON).

I assume you mean a single primitive field. Could you explain where this requirement comes from? I don't see why "2021-12-14T09:48:54.486500+01:00[Europe/Berlin]" is better than {"moment":"2021-12-14T08:49:40.958189Z","zone":"Europe/Berlin"}. The string already has these components, they are just made explicit in the JSON structure.

If anything, ZonedDateTime is at a disadvantage here. Unless every participant is careful, the timezone databases between them may go out of sync, that is, they may have different opinions about the time zone rules. What should happen with "2021-12-14T09:48:54.486500+01:00[Europe/Berlin]" on deserialization if your system for some reason thinks that, on 2021-12-14, the offset in Berlin is +03:00? Who knows what will happen? On the other hand, Instant + TimeZone is completely transparent, and it's obvious what will happen on desync: if the TimeZone is used to create a LocalDateTime, that LocalDateTime will be different across the machines.

In fact, ZonedDateTime typically deals with invalid inputs the same way, by adjusting the LocalDateTime part.

How do you make sure no mistakes happen because people forget to take the TimeZone into account in the whole codebase with tons of developers of varying experience?

We don't provide LocalDateTime arithmetic, so there's not much you can do without it aside from parsing and printing. If one wants to, say, advance a datetime by two days, it is done with Instant. I don't think there's anything in our library that doesn't require a time zone when it needs to be taken into account.

If you want to keep the API as small as possible [...]

This is a non-goal. If this was a goal, then, on the extreme, we would have a mishmash omnipotent datetime class which, depending on the context, would represent a period, or a date, or a moment in time, as NSDateComponents does. Instead, we want something simple and discoverable; omnipotent objects are not known to be that.

[...] then here’s some provocative food for thought: If you had ZonedDateTime instead of Instant maybe people wouldn’t be missing Instant because ZonedDateTime in UTC handles that use-case already.

Given how several datetime libraries already did this, we, of course, thought about this and decided against it. Instant is simple, predictable, and portable; ZonedDateTime is neither of those, and it didn't look like it brought anything compared to a tandem of Instant + TimeZone aside from parsing/formatting.

The use-case for instant is different: It should be use to record past events that may not change (e.g. the Unix timestamp cannot change) while ZonedDateTimes can (e.g. if laws change)

Not the case. Instant is used for all events when their specific moment is known. For example, if I set the microwave to two minutes from now, the moment when it beeps will be represented as an Instant. If the paper submission deadline is "2021-11-19, 23:59:59 anywhere-on-earth", it's an Instant, because it is a specific moment, the meaning of which is obvious to everyone, regardless of their time zone.

If, however, a friend says "let's meet tomorrow at 2 p.m.", this is a LocalDateTime: regardless of what time zone it is and whether there's a time gap or a time overlap that morning, we'll still going to meet when the clock in our town shows 2 p.m. Unless that moment falls into a time gap, of course, but then we'll have to reschedule the meeting in any case, can't be helped.

If, however, a friend says "call me tomorrow at the time when it's 2 p.m. in New York", it's an Instant again, as "the next time it's 2 p.m. in New York" is a specific moment.

Well in that case you can just use a LocalDateTime because the Timezone is always UTC and they just define an interface that both implement.

Not really. In our library, LocalDateTime is not specified to be in UTC; it is in an unspecified time zone. If it were UTC, we would be able to implement arithmetic on it, as it would be well-defined. However, since people rarely use LocalDateTime to represent the clock readings in UTC, and instead it's almost always used to represent the clock readings in some unspecified time zone, providing arithmetic operations would lead to bugs when the implicit time zone does differ from UTC.

If you create an event for a future ZonedDateTime and then two years later the event moves to the past, do you suddenly have to convert it to an Instant, so it still refers to the same actual point in time? I've never heard of something strange like that.

Again, this depends on how the moment is defined. If you need something to happen on "2023-12-14, 10:48 MSK", then you need something to happen on "2023-12-14, 10:48" + "MSK", so LocalDateTime + TimeZone. If you need something to happen an exact amount of seconds into the future (which is unlikely in the stated case of two years into the future), it's an Instant.

Finally, there's always this implicit assumption in the whole discussion and the README of this project:

• Instant: past only
• LocalDateTime+TimeZone: future (only?)

Oh. Could you list the parts of the README that led to this assumption (preferably in a separate issue) so we could fix this? The intention is for Instant to represent moments in time, and for LocalDateTime to represent clock readings, regardless of whether it's the past or the future.

Of course, if the meeting was at 2 p.m. yesterday, we already know which time zone rules were in effect and can represent this as an Instant, but if the meeting is scheduled to happen at 2 p.m. exactly two years from now, we can't be sure that the time zone rules won't change, and so can't calculate the number of seconds until that event beforehand. So, there is a class of things that can be represented as an Instant in the past but not the future, but I'd argue this is not the defining properties of Instant and LocalDateTime.

So the real question for deciding between ZonedDateTime and Instant is: Do you know the original timezone and do your storage schema and communication APIs support timezones? If the answer is no then you pick Instant. For everything else you pick ZonedDateTime.

Since Instant + TimeZone is equivalent to ZonedDateTime, I don't see why this question would even arise.

In fact, in some previous projects where people started with Instant it happened that we later found out that this was a mistake and ZonedDateTime would have allowed an important analysis

This is very interesting. Do you know of some functionality of ZonedDateTime that can't be easily implemented with Instant + TimeZone? Please do share an example! This would make the argument for introducing ZonedDateTime much more strong.

DST changes happen and users travel between countries. In many cases, it's useful to not just look at the data in UTC, but actually know the local time and the timezone of all events even in the past.

Why do you need to know the timezone of all events in the past? Isn't it enough to know precisely when they happened? If you also need to know where someone was at that moment, this is a separate piece of data and has nothing to do with the time of the event.

Also, since users travel between countries, Instant becomes that much more useful. "The headache started ten hours ago; also, since the user has to take a pill every 48 hours, the next time they should do it is in 8 hours". These are statements that are nicely represented as Instant values, and to show them to the user, you can use the time zone of the place where the user currently resides. Storing a time zone alongside these would be error-prone: it would be easy to accidentally use the stored time zone and not the current one to display the data.

ZonedDateTime should be your default choice for everything - past and future. Every use of Instant needs deliberate thought because you can't go back once you've lost the timezone information.

Could you elaborate on when the time zone is an inherent trait of some moment in time and not just side information?

where you care about the distance and order (which is way easier without timezones) and not where it was recorded

When do you care in which time zone something was recorded? Time zones don't map neatly to geography, after all, so… what meaningful things can one do by knowing the time zone?

when face the user you'd probably just convert those to Local datetime using the user's local time zone.

Exactly! When would you not do that?

Right, so maybe the rule of thumb is

• ZonedDateTime: user data
• Instant: machine data (logs, metrics, cache expiry, etc)

Could you elaborate on this as well? Do users want to see "this happened at 2021-11-19, 23:59:59 in the MSK time zone"?

Both should retain their properties

The issue is, we don't have both yet, we only have Instant, and we're trying to decide what good, if any, could come out of also introducing a ZonedDateTime.

from kotlinx-datetime.

How do you serialize/deserialize? This is almost always a single field in the serialized data (e.g. JSON).

I assume you mean a single primitive field. Could you explain where this requirement comes from? I don't see why "2021-12-14T09:48:54.486500+01:00[Europe/Berlin]" is better than {"moment":"2021-12-14T08:49:40.958189Z","zone":"Europe/Berlin"}. The string already has these components, they are just made explicit in the JSON structure.

We have existing APIs, third-party APIs and official medical formats that we have to support. We can't tell third-parties and standards bodies to change their formats and we won't refactor a multi-million LOC project (which is using ZoneDateTime for all user data).

Also, 99% of the APIs in the world use a single field instead of splitting it up because it is in fact more convenient to work with a single object. The convenience of a primary use-case is the main point.

Also, you can do date arithmetic more easily on a combined object and e.g. add one month and get the same local time - adjusted for DST changes.

Most programming languages, frameworks, ORMs and even OpenAPI/Swagger work with a single value for ZonedDateTime. We won't change the official OpenAPI specification, the official Email format, the official HTTP header formats and instruct the whole world to switch to a different structure. Practically every standard uses ZonedDateTime, so it should be easy to work with that kind of data.

Splitting everything up into two values definitely is not the easiest solution imaginable.

DST changes happen and users travel between countries. In many cases, it's useful to not just look at the data in UTC, but actually know the local time and the timezone of all events even in the past.

Why do you need to know the timezone of all events in the past? Isn't it enough to know precisely when they happened? If you also need to know where someone was at that moment, this is a separate piece of data and has nothing to do with the time of the event.

Also, since users travel between countries, Instant becomes that much more useful. "The headache started ten hours ago; also, since the user has to take a pill every 48 hours, the next time they should do it is in 8 hours". These are statements that are nicely represented as Instant values, and to show them to the user, you can use the time zone of the place where the user currently resides. Storing a time zone alongside these would be error-prone: it would be easy to accidentally use the stored time zone and not the current one to display the data.

Depending on where you travel the "in 8 hours" could mean "while you're asleep". That definitely won't happen for people in the real world. If someone has to do something every time he wakes up or before going to sleep then these actions shift with the time zone. However, for analysis it's still important to additionally know the absolute time differences, so LocalDateTime alone is not enough and neither is Instant alone. We really need ZonedDateTime to get the full picture.

ZonedDateTime should be your default choice for everything - past and future. Every use of Instant needs deliberate thought because you can't go back once you've lost the timezone information.

Could you elaborate on when the time zone is an inherent trait of some moment in time and not just side information?

• Flight departure and arrival are mostly described in local time in certain time zones and people want to have the full information.
• Calendar invites often contain a time zone, so you know in advance when the conference or meeting happens even in the local time, so you can do your travel planning.
• I've often looked at the time zone information in an email, so I can make sure we schedule a time that works for everyone.
• For analytics it can be important to also analyze at which local time users do something in addition to analyzing metrics in UTC.
• The example above.

It's really good that the people who define specifications don't just say "use UTC for everything", but instead they usually add time zone information.

Good API design optimizes for the most common use-cases and makes them fool-proof and trivially easy. Apparently ZonedDateTime is a very common use-case (almost every official standard uses it), so the point of having a ZonedDateTime is to make this as easy as possible.

I don't know how else to explain it. Your suggestion sounds to me like having separate date + time (instead of Instant) because it's "just as easy". No it's not.

from kotlinx-datetime.

existing APIs

No argument here. It's obvious that we'll need to provide parsing/formatting that recognizes time zones. The question is, do we have to implement a whole ZonedDateTime? If ZonedDateTime is needed just for compatibility, we could instead parse/format a Pair<Instant, TimeZone>, for example. If there are some other use cases, maybe adding ZonedDateTime is the right answer. Or maybe there's some other data structure that's even better at representing the use cases.

Also, 99% of the APIs in the world use a single field instead of splitting it up because it is in fact more convenient to work with a single object.

If this were true, then everyone would send everything as a single object. The reason they don't is twofold:

• Everyone would have to support parsing this object that encompasses everything;
• Logical separation of concerns: if things are separate, they shouldn't be mixed together.

ZonedDateTime ticks the "everyone has to support parsing/formatting it" box, but is the Instant + TimeZone more than just the sum of its parts?

Also, you can do date arithmetic more easily on a combined object and e.g. add one month and get the same local time - adjusted for DST changes.

In our library, we provide Instant arithmetic that also allows this.

LocalDateTime alone is not enough and neither is Instant alone. We really need ZonedDateTime to get the full picture.

This argument is not sufficient: maybe you actually need both LocalDateTime and Instant, but separately. For cleaner code, if nothing else. LocalDateTime would represent the user-visible time of the action, whereas Instant would be the actual moment in time when the action happened.

However, you later say that LocalDateTime can be also useful for analysis in your case. This does call for using ZonedDateTime, I agree: the user sees the LocalDateTime part, you see what the user saw, and also the Instant—sounds like it's convenient to bundle everything together.

It's really good that the people who define specifications don't just say "use UTC for everything", but instead they usually add time zone information.

It's not really viable to use UTC for everything. For example, you wouldn't be able to conveniently schedule a meeting using just UTC in a way that adapted to the time zone changes.

Good API design optimizes for the most common use-cases and makes them fool-proof and trivially easy.

Sure, this is what we are going for.

Apparently ZonedDateTime is a very common use-case (almost every official standard uses it)

Maybe it does indicate a common use case, maybe it only indicates a historical coincidence. In any case, the ease of representing something does not indicate the ease of using something.

so the point of having a ZonedDateTime is to make this as easy as possible.

Yeah, we will need to do something about compatibility. The question is, what exactly.

Your suggestion sounds to me like having separate date + time (instead of Instant) because it's "just as easy". No it's not.

I'm not claiming that it's just as easy. I'm claiming that it's not clear whether it's just as easy, more difficult, or easier. I can see arguments for why using ZonedDateTime is more difficult, and in this discussion, I'm trying to understand why using ZonedDateTime could be easier than using Instant + TimeZone.

Flight departure and arrival

I think those are represented naturally as "they happen at these moments; also, the time zones of the source and the destination are this and that".

Calendar invites often contain a time zone

This one is good, I agree that there's some model of the ground truth that invites the use of ZonedDateTime: if the meeting time is represented as a LocalDateTime, but should be accessed in other time zones as well, then, yeah, it makes sense to know the LocalDateTime and the time zone simultaneously.

We'll need to look at how calendar meetings are actually defined in real systems: whether they use LocalDateTime as the source of truth, and if so, whether they handle the situations when the time zone of the meeting or the time zone of the viewer changes their rules.

time zone information in an email

"Happened at a given moment; also, the sender was in that time zone; also, it was at that particular moment for the sender". I think there's a pattern emerging in your use cases: we want to bundle everything together if we need to know the local date-time for someone in a particular time zone, but are also interested in the real moment.

Thanks, this indeed looks like a common use case that's better covered by a ZonedDateTime than Instant + TimeZone! However, I don't see still any evidence that there's a need to do arithmetic on this, for example. It looks like the only use cases we've discussed are for sending, receiving, and changing the representation of some instants. Maybe we could implement it in some way that's less cumbersome than a full-fledged ZonedDateTime.

from kotlinx-datetime.

No argument here. It's obvious that we'll need to provide parsing/formatting that recognizes time zones. The question is, do we have to implement a whole ZonedDateTime? If ZonedDateTime is needed just for compatibility, we could instead parse/format a Pair\u003CInstant, TimeZone>, for example. If there are some other use cases, maybe adding ZonedDateTime is the right answer. Or maybe there's some other data structure that's even better at representing the use cases.

I'd say that even for this case it'd be worth it to add at least a rather lightweight wrapper. I'd pre really confused to see an API that takes a Pair<LocalDateTime, TimeZone>. Everybody would ask what's this doing here? Of course everyone could implement their own wrapper object, but then we'd have potentially 10s of wrapper objects and we'd land in conversion hell, where we get one object from API 1 and need to convert it to a pair and then to another object from API 2. That's just a horrible experience.

Also Instant and TimeZone doesn't work, because if you'd schedule let's say a meeting in a year and now some rules may change. You expect it's always e.g. at 13:00, because that's what you set. Now you use an Instant and suddenly it's at 14:00. That... seems not like your user would expect it. They don't want to reschedule everything by an hour because of a change. For future events that are important that they happen in exactly 1 Year, so 365 days, x hours, n seconds, you'd need an instant, where the time zone isn't really part of the Date itself, but an additional indicator. And then you'd also need a wrapper for conversation to an instant for example, but what's important is that the conversation to Instant and TimeZone may not be lossless. If you'd pick a date now in 10 years, save it as RFC3339 and Instant + TimeZone, in a few years those two values would refer to a different time, because an Instant is absolute, but the ZonedDateTime should be relative to the calendar, otherwise it shouldn't be parsed from an RFC3339 (which is the source of this issue), because that could fundamentally change the underlying value. In practice of course this would only happen if you'd change rules while the code is running or serialize both components, but I'd argue that the altering of the meaning in this way should be an explicit conversion.

from kotlinx-datetime.

Calendar invites often contain a time zone

This one is good, I agree that there's some model of the ground truth that invites the use of ZonedDateTime: if the meeting time is represented as a LocalDateTime, but should be accessed in other time zones as well, then, yeah, it makes sense to know the LocalDateTime and the time zone simultaneously.

We'll need to look at how calendar meetings are actually defined in real systems: whether they use LocalDateTime as the source of truth, and if so, whether they handle the situations when the time zone of the meeting or the time zone of the viewer changes their rules.

My calendar software, for example, shows LocalDateTime by default, but it has a checkbox to define the time zone of the start date and time zone of the end date.

time zone information in an email

"Happened at a given moment; also, the sender was in that time zone; also, it was at that particular moment for the sender". I think there's a pattern emerging in your use cases: we want to bundle everything together if we need to know the local date-time for someone in a particular time zone, but are also interested in the real moment.

Yes, we store all end-user data like that, so we can later add new features that can take both pieces of information into account.

Thanks, this indeed looks like a common use case that's better covered by a ZonedDateTime than Instant + TimeZone! However, I don't see still any evidence that there's a need to do arithmetic on this, for example. It looks like the only use cases we've discussed are for sending, receiving, and changing the representation of some instants. Maybe we could implement it in some way that's less cumbersome than a full-fledged ZonedDateTime.

The arithmetic becomes necessary once you start to store all end-user data and other data with with ZonedDateTime. Then you need to calculate e.g. if a minimum number of days have passed or if the current date is within the validity range.

But even if it's not directly end-user data, here is an extract from the certlogic rules for validating the European Union's COVID vaccination certificates:

{"EngineVersion":"0.7.5","ValidTo":"2030-06-01T00:00:00Z","ValidFrom":"2021-08-19T00:00:00+02:00","Engine":"CERTLOGIC","Version":"1.1.0", ...

See a snapshot of the full file here: https://github.com/Digitaler-Impfnachweis/covpass-android/blob/main/covpass-sdk/src/main/assets/covpass-sdk/eu-rules.json

As you can see, the ValidTo and ValidFrom use different time zones. In other places we also do arithmetic over ZonedDateTime to check if a certain minimum amount of time has passed, etc. This project is just a little bit too small to show all the possible real-world complexities and unfortunately I can't show you any examples from the main projects. But it's pretty normal to have some object's createdOn in ZonedDateTime and then in the UI show the duration like "1 hour ago" which already requires arithmetic - and again we wouldn't want to manually convert to Instant to be able to do arithmetic.

BTW, in many cases I've seen offset-based ZonedDateTime, but I've also had zone id based ZonedDateTime. So, ZonedDateTime would need to handle both cases transparently (at least my own implementation of that class does).

A combined offset + zone id might be too much of an edge case, I don't know. Personally, I'd even add that because java.time surely has it for a reason and it's needed to handle DST correctly and e.g. auto-correct zone database changes (and possibly even missing updates which you talked about as an edge case in your previous message).

from kotlinx-datetime.

and again we wouldn't want to manually convert to Instant to be able to do arithmetic.

You should. Because if not using an offset, but a full-fledged time, there's no guarantee that's after a server has been restarted for example the "in 2 hours", will still be true. There might be a change of rules (I'm aware this wouldn't be that close usually, but samples with huge numbers aren't easy to understand) and now it's "in 1 hour" or "in 3 hours". Yes this should be a rare occasion, but I think it absolutely makes sense to do, because edge cases in datetimes are an absolute PITA. It just doesn't make sense to compare an Instant (right now) to a ZonedDateTime (calendar date) directly. A calendar is an artificial conversation scheme where as an Instant is a (as long as the atomic clocks work fine) basically absolute point in time. Therefore to compare them you'd have to implicitly convert them and I don't think an implicit conversion would be a good idea. If you are dealing with past times it should be safe to convert to an Instant and a TimeZone which would be a separate class, which in turn would be safe to compare to an Instant because it's unambiguous unlike ZonedDateTime, maybe we'd need a different issue for that.

from kotlinx-datetime.

If I do the toInstant conversion by hand then how is that different from the ZonedDateTime class doing the conversion automatically? The problem stays the same.

Anyway, I wasn’t even talking about comparison/arithmetic over ZonedDateTime and Instant but between two ZonedDateTime objects.

from kotlinx-datetime.

Also, why are we even discussing extreme edge cases? Are you running servers for weeks/months without updates? Then missing time zone updates are the least of your problems and you should be caring about security updates (e.g. log4j) a lot more.

Also, time zone changes usually don’t happen over night. Time zone updates can be created months in advance, so there’s enough time to update all machines.

And if you have IoT devices operating without internet/updates for prolonged periods you can store ZonedDateTime with offset + zone ID and maybe even DST flag and auto-correct the data once the update is applied.

from kotlinx-datetime.

Also, why are we even discussing extreme edge cases? Are you running servers for weeks/months without updates? Then missing time zone updates are the least of your problems and you should be caring about security updates (e.g. log4j) a lot more.

Also, time zone changes usually don’t happen over night. Time zone updates can be created months in advance, so there’s enough time to update all machines.

A) yes some machines run without app updates for a long time, offline machines and also for this to cause no pain at all everything has to be update instantly all at the same time.
B) This also happens if you update in time, because there's not limit, that disallows you to create ZonedDateTimes months and years ahead.
C) Edge cases like this (will) have caused real consequences two times now (see year 2000, less of a pain than we thought, but still caused headaches and some failures; upcoming is the Unix epoch debacle, we've know about this decades ahead, but there's probably still some systems that will fail), we should learn from that and maybe plan now an do it properly now and not do something again that "probably will work fine".

from kotlinx-datetime.

If I do the toInstant conversion by hand then how is that different from the ZonedDateTime class doing the conversion automatically? The problem stays the same.

Yes but you opt-in to that conversion. Similar how you can convert an Instant to a LocalDateTime and discard some information.

Anyway, I wasn’t even talking about comparison/arithmetic over ZonedDateTime and Instant but between two ZonedDateTime objects.

Well ZonedDateTimes may not even refer to an unambiguous point of time. See the time change. If you need to refer to an exact point of time you need a Instant + TimeZone, which would be different from ZonedDateTime. I think we are kinda mixing some use-cases here, I'll do a bit of research and try to define and distinguish both use-cases, so we can split it into two issues with seperate discussions.

from kotlinx-datetime.

Most programming languages, frameworks, ORMs and even OpenAPI/Swagger work with a single value for ZonedDateTime. We won't change the official OpenAPI specification, the official Email format, the official HTTP header formats and instruct the whole world to switch to a different structure. Practically every standard uses ZonedDateTime, so it should be easy to work with that kind of data.

@wkornewald Are you sure these are ZonedDateTimes? AFAIK, Email format headers and HTTP headers do not store a timezone identifier, but rather only an offset from UTC. So this makes them OffsetDateTimes which is what was originally requested in this issue.
Could you show examples of ZonedDateTime in OpenAPI, ORMs, frameworks?

from kotlinx-datetime.

Here are the two use-cases, I also mixed those, so we should probably split it to clear it up:

OffsetDateTime(?) (this issue)

Standards

• RFC3339
• ISO 8601

Characteristics

LocalDateTime (offset from UTC)/Instant with a fixed Offset, no time zone identifier allowed (which would be thing that could change, offsets can't).

Good at

• Representing fixed moments relative to now/semi-absolute
• Basically everything an Instant is good at with an added offset
• Comparing them with each other
• Establishing the order of events
  • Method of comparison has to be clearly and explicitly communicated to the developer

Bad at

• Some kinds of future dates like calendar dates, that have to change with a named time zone
• Doing calculations

Problems

• Leap seconds?
• Clearly communicate comparisons.

ZonedDateTime

Standards

Probably exist, I don't know any off my head

Characteristics

LocalDateTime with a named TimeZone (these can change) (or LocalDateTime (UTC)/Instant with a fixed Offset?).

Good at

• Representing things like a user would expect in a calendar (that change change with laws that change).

Bad at

• Representing an absolute point of time
• Being unambiguous, they may require rules/interpretation
• Comparisons
• Establishing the order of events (Instants) with its natural comparable order (if it would have any)

Problems

• Same as OffsetDateTime
• Time changes (DST / no-DST)
• Changes of laws

from kotlinx-datetime.

@ilya-g I always tried to explicitly say that the ZonedDateTime would handle offset and ID dynamically based on the amount of information available. And depending on how precise we want to represent time we could also still consider even a combination of both offset and ID at the same time.

Basically, what I’d like to see is an object that can precisely identify the local and global time at the same time and adapt to the precision that is available.

from kotlinx-datetime.

@cromefire To add to above, OffsetDateTime, bad at:

• establishing the order of events (Instants) with its natural comparable order (if it would have any)
• arithmetic operations between several OffsetDateTimes: usually such operations need a single time zone or offset, and several OffsetDateTimes provide several sources of what can be considered a time zone information.

from kotlinx-datetime.

establishing the order of events (Instants) with its natural comparable order (if it would have any)

I think OffsetDateTimes are fine with that (actually good), because they have a static offset so they are unambiguous. That's a Problem for ZonedDateTimes though.

arithmetic operations between several OffsetDateTimes: usually such operations need a single time zone or offset, and several OffsetDateTimes provide several sources of what can be considered a time zone information.

I think comparative operations (<, >, ==, in-rage/between) are fine, but calculating ones (-, +) are a not a good idea.

from kotlinx-datetime.

comparative operations (<, >, ==, in-rage/between) are fine

@cromefire Consider a range 2020-01-01T00:00+02 .. 2020-01-01T01:00+02. Does the value 2020-01-01T01:00+03 which represents the same instant as the start of the range lies in this range?
Can you tell the same about the value 2020-01-01T00:00+02 being in the range 2020-01-01T01:00+03 .. 2020-01-01T02:00+03?

The natural order of OffsetDateTime breaks an intuitive expectation that the result of comparison of two OffsetDateTimes should be the same as the result of comparison of Instants they correspond to. That's why in Java, OffsetDateTime additionally has functions isBefore/isAfter/isEqual. Having the similar ones together with the <, >, == operators in Kotlin may be quite confusing.

from kotlinx-datetime.

comparative operations (<, >, ==, in-rage/between) are fine

@cromefire Consider a range 2020-01-01T00:00+02 .. 2020-01-01T01:00+02. Does the value 2020-01-01T01:00+03 which represents the same instant as the start of the range lies in this range? Can you tell the same about the value 2020-01-01T00:00+02 being in the range 2020-01-01T01:00+03 .. 2020-01-01T02:00+03?

I would say no for the first. And yes for the second (assuming inclusive). (all assuming I calculated correctly in my head and basically just comparing the absolute points in time (e.g. UTC / Instant))

The natural order of OffsetDateTime breaks an intuitive expectation that the result of comparison of two OffsetDateTimes should be the same as the result of comparison of Instants they correspond to. That's why in Java, OffsetDateTime additionally has functions isBefore/isAfter/isEqual. Having the similar ones together with the <, >, == operators in Kotlin may be quite confusing.

Well I could certainly agree with having having explicit methods instead of operators, might make more sense. So I think they are good for representing a series of events in a timeline, but yes I think we could put in explicit methods instead of doing something implicit.

from kotlinx-datetime.

So do we have a preference already? Maybe I should summarize the points again because after @cromefire's summary you both made some additional important points.

Support in practice

Depending on the database, API and official standards you might have to work with either.

• Offset seems to be more popular with APIs and standards.
• For databases it's more mixed.
  • I don't have the impression that PostgreSQL's zone abbreviations are 100% equal to offets, so it seems to prefer zone IDs or maybe the combined abbreviation+offset+DST representation (that's two kinds of offsets combined? plus a DST flag?).
  • Elasticsearch can represent almost anything, including the offset + zone id combination (+01:00[Europe/Paris]).
  • MySQL seems to only support offsets.

Of course at the database level, depending on your DB choice, you could use two separate fields if the level of precision is important. With third-party APIs and official standards you have no choice, so this means offsets most of the time.

Precision and correctness / lack of ambiguity:

This is mostly about mapping 1:1 to Instant.

• Offsets work pretty well for the past, but are error-prone for the future.
• Zone IDs work okay-ish for the future and past. They're ambiguous during DST transitions, but outside of that there is no ambiguitiy and the zone id has a well-defined offset for every point in time, so they can be mapped 1:1 to Instant (except for a 1h DST winter->summer overlap every year).

Neither is 100% precise and unambiguous all of the time.

Only the combination of offset + zone id (+02:00[Europe/Paris]) could be relatively precise and unambiguous, but it's not always supported. /rant Though, somehow a DST offset, like Europe/Paris+1 for summer time and Europe/Paris+0 for winter time, instead of a UTC-based offset (+02:00[Europe/Paris] for summer time) feels more correct and even less ambiguous (e.g. when the country moves its whole base UTC offset by one hour exactly during the winter -> summer DST transition period).

Comparisons and arithmetic

Here it's not always enough to just map 1:1 to Instant. You need to know the time zone rules.

• Comparisons: Both work well, but offets win because there are no ambiguities at all.
• Difference calculations (i.e. duration between two datetimes): both work well, but offsets win again.
• Adding a duration to a date: Zone IDs work pretty well. Offsets fail because they can't deal with DST-caused offset changes (like @cromefire also noted). But in many cases people use this in combination with comparisons (now > createdOn + 1 hour) in which case DST doesn't matter. So, this is mostly important in reminder and calendar applications where you need a "repeats every ..." feature for events happening within a specific zone ID (not offset).

What's the solution?

This is my personal wish list:

• Flexibility in the precision and format. Based on what the backend API and/or official spec and/or database allows, any of these can be used:
  • Offset: +02:00
  • Zone ID: Europe/Paris
  • DST shift (or UTC offset?) + Zone ID: Europe/Paris+1 (summer time) vs. Europe/Paris+0 (winter time) and/or +02:00[Europe/Paris] (UTC offset mostly for better compatibility with certain databases and APIs)
• Arithmetic works for all of those, but if you want correct DST handling you must use offset + zone id or at least zone id (and risk ambiguitity in rare cases).
• Comparisons are always exactly based on the corresponding Instant.
• Serialization and formatting/parsing allow lossless conversion of the flexible format (if you have only an offset it will serialize to that). Alternatively, specific formats can be chosen (e.g. ISO8601 or MM/dd/yy etc.).
• Maybe in a separate i18n lib and outside of the scope of this lib: Formatting/parsing with common language-specific translations like SHORT, MEDIUM, SEMILONG, LONG, etc.

Overall, this would still make ZonedDateTime just a relatively simple wrapper around LocalDateTime + mandatory TimeZone (= RegionTimeZone or FixedOffsetTimeZone) + optional DST offset.

from kotlinx-datetime.

Only the combination of offset + zone id (+02:00[Europe/Paris]) could be precise and pretty unambiguous, but it's not always supported.

This makes basically nothing better, because now you have to choose which one to use if they conflict, which isn't really that much better... The only thing it would help with is having just a normal offset and using the timezones just to display the timezones to the user, but I don't think one would really need a type for that. There's also no standard I know of that uses this.

Offsets fail because they can't deal with DST-caused offset changes

Well because an offset isn't a time zone and isn't trying to be one it doesn't know DST by design. There might even be multiple time zones per offset, it's an irreversible conversion. It a pretty severe limitation of the format.

To proceed in general I'd recommend fast-tracking OffsetDateTime (even if it's just a lighter weight implementation for now, features can always be added), because I think it's pretty straight forward now and then discussing ZonedDateTime which is infinitely more complex. If no one is interested I can probably do I a PR for OffsetDateTime in January, if I can keep it in mind until then.

from kotlinx-datetime.

Only the combination of offset + zone id (+02:00[Europe/Paris]) could be precise and pretty unambiguous, but it's not always supported.

This makes basically nothing better, because now you have to choose which one to use if they conflict, which isn't really that much better... The only thing it would help with is having just a normal offset and using the timezones just to display the timezones to the user, but I don't think one would really need a type for that. There's also no standard I know of that uses this.

A DST offset instead of UTC offset should be able to solve this, though. In case of an invalid DST offset you just fall back to the only possible time at that point.

Offsets fail because they can't deal with DST-caused offset changes

Well because an offset isn't a time zone and isn't trying to be one it doesn't know DST by design. There might even be multiple time zones per offset, it's an irreversible conversion. It a pretty severe limitation of the format.

To proceed in general I'd recommend fast-tracking OffsetDateTime (even if it's just a lighter weight implementation for now, features can always be added), because I think it's pretty straight forward now and then discussing ZonedDateTime which is infinitely more complex. If no one is interested I can probably do I a PR in January, if I can remember it until then.

Then I'd rather suggest a ZonedDateTime that takes a TimeZone (i.e. either RegionTimeZone or FixedOffsetTimeZone). This way you're at least flexible and you can represent the future correctly. And this also allows to still add an optional DST or UTC offset at a later point without breaking anything or introducing yet another DateTime class.

from kotlinx-datetime.

Then I'd rather suggest a ZonedDateTime that takes a TimeZone (i.e. either RegionTimeZone or FixedOffsetTimeZone). This way you're at least flexible and you can represent the future correctly. And this also allows to still add an optional DST or UTC offset at a later point without breaking anything or introducing yet another DateTime class.

Yes, that's how most implementations of something like that seem to work, I man supporting both types isn't much more complicated than RegionTimeZone I suppose.

But let's pause this here and implement OffsetDateTime first and then continue with ZonedDateTime.

from kotlinx-datetime.

Yes, that's how most implementations of something like that seem to work, I man supporting both types isn't much more complicated than RegionTimeZone I suppose.

Mostly the parse() and toString() methods and arithmetic with durations need special handling based on RegionTimeZone vs. FixedOffsetTimeZone. I think the whole rest of the class could be identical for both cases.

from kotlinx-datetime.

Then I'd rather suggest a ZonedDateTime that takes a TimeZone (i.e. either RegionTimeZone or FixedOffsetTimeZone). This way you're at least flexible and you can represent the future correctly. And this also allows to still add an optional DST or UTC offset at a later point without breaking anything or introducing yet another DateTime class.

Yes, that's how most implementations of something like that seem to work, I man supporting both types isn't much more complicated than RegionTimeZone I suppose.

But let's pause this here and implement OffsetDateTime first and then continue with ZonedDateTime.

But why do you need OffsetDateTime if you can represent them with an almost equally simple ZonedDateTime? I mean, why should we have multiple classes which only confuse other developers?

from kotlinx-datetime.

Because it's a much more (on purpose) constrained and a more simple class. I mean why do we want UInt when we have Int? It's artificially constrained and simpler, and that makes life easier. Also see my list above. It can provide certain guarantees that ZonedDateTime can't.

Hmm maybe we could later introduce a sealed interface/class for the TimeZone base type and have the two specific types derive from that, so if you parse a random string you work with the base type but if you want to enforce an offset or zone id you can work with the subtypes.

from kotlinx-datetime.

No I want a ZonedDateTime (with subtypes OffsetDateTime and RegionDateTime - sharing like 90% of the API) that can contain either an offset or a zone id so I can process an arbitrary input (imagine the JSON data can have either an offset or zone id) which gets converted to the more specific subtype (which I could match with a when expression or just keep using the base type), then work with that and format it back to the original input format without any loss. And if I explicitly want to only allow an offset I instead use the subtype‘s parser which only allows offset.

from kotlinx-datetime.

sharing like 90% of the API

Well that's what I meant, I don't think they will share much API if you don't build something super generic. You should use a toOffsetDateTime() in that case. ZonedDateTime will most like likely have methods that don't make a lot of sense on OffsetDateTime, and vice versa.

IMO it's like having LocalTime be a specific type of LocalDateTime. They are related in a way (like most DateTime types), but not quite that similar.

from kotlinx-datetime.

Temporal is much more arbitrary. No matter if you use an OffsetDateTime or RegionDateTime, the base type ZonedDateTime is a LocalDateTime with offset information (even if it's a zone id based RegionDateTime you can get the offset). This is a lot of information which means you can safely

• of course parse/format and serialize/deserialize which is useful on its own => no loss of information
• calculate meaningful (to humans, not mathematicians) durations between two ZonedDateTimes (x - y) => "x hours ago"
• compare (x > y) => sort the data
• compare even under arithmetic (x > y + 12 hours) =>check if a limit has been reached

There are also minimum guarantees:

• if you only use it for the past it reliably represents an Instant (so it's at least as good as OffsetDateTime for the createdOn/modifiedOn use-case)
• adding a duration behaves at least as precisely as OffsetDateTime (it will not handle DST and future zone changes)

In practice you could say that it's very useful to humans and it's also the way humans would think about these date strings if they just read them.

Temporal is much worse because you don't even know if it's a date or time or instant. That's too arbitrary.

And I can tell from experience that sometimes date fields can have even more precision differences and in one object contain just the year while another has a zoned date time for the same field (or year-month or date etc.) and we have to deal with that somehow and do comparisons and ordering and human-friendly formatting. Still the result is pretty useful to humans as long as you don't mix in a time-only value (which is why Temporal is useless). As long as you build an information hierarchy which strictly adds extra information at each step you can do useful stuff at each information level.

from kotlinx-datetime.

Also Instant and TimeZone doesn't work

This argument can be used to show that ZonedDateTime also won't work, because it will also adapt its LocalDateTime part. So, Pair<Instant, TimeZone> wins this round: in the case of the pair, the behavior is clear and predictable.

java.time.ZonedDateTime.parse("2021-12-15T10:38:25.793402647+10:00[Europe/Moscow]").toString() ==
    "2021-12-15T03:38:25.793402647+03:00[Europe/Moscow]"

rules for validating the European Union's COVID vaccination certificates:

These can be parsed already with Instant, only the offset will be lost. Is it important in this case?

Well ZonedDateTimes may not even refer to an unambiguous point of time. See the time change. If you need to refer to an exact point of time you need a Instant + TimeZone, which would be different from ZonedDateTime.

When is Instant + TimeZone not equivalent to a ZonedDateTime?

I carefully read all of the above and noticed some things that (I think) are technically incorrect. Let me try to also summarize the state of things.

ZonedDateTime

As implemented in Java, (almost always) equivalent to Instant + TimeZone.

• (Almost always) automatically adapts LocalDateTime so that the Instant + TimeZone pair is represented correctly.
• (Almost always) adapts LocalDateTime if it happens to fall into a time gap.

I say "almost always" because is possible to construct a ZonedDateTime that stores arbitrary LocalDateTime, TimeZone, and offset, using ZonedDateTime.ofLenient. I propose that we ignore these malformed things for now, at least until a use case comes up.

So, for ZonedDateTime, Instant and TimeZone work as the source of truth, from which it then recalculates LocalDateTime and the offset as needed.

Arithmetic on ZonedDateTime works the same way the arithmetic on Instant does, but without specifying a time zone, as it comes bundled.

In effect, ZonedDateTime is Pair<Instant, TimeZone>, but

• With some additional behavior like comparisons also taking LocalDateTime corresponding to the instant into account. Is there evidence that we do need that additional behavior?
• With more streamlined arithmetic due to not requiring specifying a time zone explicitly. The fact that two parameters often go together into functions is not enough to throw them into one object

We've established only one use case for ZonedDateTime:

• Parsing/formatting for interoperability with the APIs that also include the time zone in the output.
• I thought that LocalDateTime + TimeZone is a valid use case, but after a night of sleep, I don't anymore. If we assume that the zone rules may change, then the ZonedDateTime object will be invalidated and recreate its LocalDateTime part, losing some information. If we assume that the rules will not change, OffsetDateTime is also fine for representing this. So, the specific behavior of ZonedDateTime, it seems, is never useful for this.

Good sides of ZonedDateTime:

• The use cases will be covered. Maybe there's a way to cover them without introducing ZonedDateTime.
• That's about it?

Issues with introducing ZonedDateTime:

• Unclear what to do with broken objects, where LocalDateTime + UtcOffset + TimeZone can't be validly combined.
• Baroque comparisons.
• Only as portable as the TimeZone object is.
• Duplicates the functionality of Instant, which will lead to confusion about which one to use.
• Error-prone when representing a LocalDateTime in the future in an implicit time zone, due to possible rules change.

OffsetDateTime

Equivalent to LocalDateTime + UtcOffset. This information is also enough to recreate an Instant, so, in effect, this is also equivalent to Instant + UtcOffset.

Use cases:

• Parsing/formatting for interoperability with the APIs for which the offset is a significant piece of information on its own (so Instant's parsing can't be used).
• If the offset is already known, representing the combined pieces of information:
  • The LocalDateTime somewhere else;
  • The approximate state of the day-night cycle somewhere else;
  • The absolute moment.

Good sides:

• Portable.
• No ambiguity.
• Mostly simple and predictable.

Issues:

• Again, comparisons. LocalDateTime and UtcOffset don't seem to meaningfully combine here any better in this regard than they would in a Pair<Instant, UtcOffset>.
• Error-prone when representing a LocalDateTime in the future in an implicit time zone, due to possible rules change.

To summarize: yes, we established that there are some strings that include the time zone that we want to be able to parse; we also know that there are some strings that represent Instant, but also place meaning in the offsets themselves. I propose that we no longer reiterate this, but instead, concentrate on finding some use cases for ZonedDateTime and OffsetDateTime that are not just parsing/formatting or "an Instant, but also a TimeZone/ZoneOffset because it's convenient for them to be sent together".

It's interesting that neither ZonedDateTime nor OffsetDateTime is fully suitable for representing a future event with the ground truth being a LocalDateTime in a specific time zone. Looks like some form of a pair of LocalDateTime and TimeZone may be even more useful than a proper ZonedDateTime.

from kotlinx-datetime.

Do you have an example? My impression is that it adapts the Instant to ensure that the LocalDateTime+TimeZone is stable.

Yep, sure.

ZonedDateTime.of(LocalDateTime.parse("2021-03-27T02:16:20"), ZoneId.of("Europe/Berlin")).plusDays(1).plusDays(1).toString() ==
"2021-03-28T03:16:20+02:00[Europe/Berlin]"

Because an intermediate computation (plusDays(1)) fell into the gap, all future results get shifted by an hour in the LocalDateTime.

it's actually LocalDateTime+Offset or LocalDateTime+ZoneId depending on what input you provide

Not the case. I've shown before that, on deserialization, the LocalDateTime part gets changed to preserve the Instant, see above. Now I've shown also that the gap gets corrected.

The same argument can be made for date + time.

It can't because 1) waiting for an hour at 23:31 will start the next day 2) the set of valid times depends on the date.

should stay unmodified in LocalDateTime and auto-correct the underlying Instant because people in that country keep going to work at the same LocalDateTime and even airports might adapt their active hours (some disallow flights after midnight for example). That's why Java's ZonedDateTime is not Instant+TimeZone, but LocalDateTime+TimeZone.

Your argument: "A is good, and ZonedDateTime is good, therefore ZonedDateTime is A", whereas my argument is "ZonedDateTime is not A, A is good, therefore ZonedDateTime is not good". It's easy to observe that ZonedDateTime is not A.

It's pretty reliable at representing LocalDateTime and that's the intended use-case.

Not true.

from kotlinx-datetime.

Use #163 for ZonedDateTime specific discussion.

from kotlinx-datetime.

FWIW, in the design of the new Temporal date/time API in ECMAScript (aka JavaScript), we opted not to offer an OffsetDateTime type. (Although we do have a ZonedDateTime type, as discussed in #163 (comment).)

The reason we didn't offer OffsetDateTime was that this type made it easy for developers to perform anti-patterns like performing non-DST-safe arithmetic, measuring the length of a "day" that would be inaccurate near DST transitions, or constructing invalid local date/time values like times skipped by Spring DST transitions. We also wanted to discourage developers from non-DST regions from assuming that their country's UTC offset would never change, because politics is unpredictable! Finally, we didn't want to increase the complexity of an already-large family of types. Time zones are complicated enough without giving developers two different timezone-enabled types to decide between!

There are valid use cases for OffsetDateTime (like having an ergonomic way to parse and model an Instant string while retaining its original values for date, time, and offset) but the workarounds for these use cases were easy enough, and the use cases unusual enough, that we weren't too concerned about a few extra lines of code being required to accomplish them.

In Temporal, we allow developers to create ZonedDateTime instances using a numeric offset like +05:00 as the time zone. We do not, however, allow developers to parse Instant strings into ZonedDateTime. If someone wants a ZonedDateTime with a numeric offset, then they have to parse the string into an Instant and then convert it to a ZonedDateTime using the offset time zone. This extra step prevents accidentally creating ZonedDateTime instances that can't perform DST-safe math.

Note that I'm not suggesting that Kotlin needs a ZonedDateTime type. Your current solution, now that I understand it seems OK. But if you do add ZonedDateTime then I'd suggest some skepticism about whether OffsetDateTime is actually needed.

from kotlinx-datetime.