Parsing LocalTime from Unix seconds

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Parsing LocalTime from Unix seconds

Marc Busqué
In GHCi

```
:m +Data.Time
parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe UTCTime
-- => Just 2018-08-31 03:00:06 UTC
parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe LocalTime
-- => Just 1970-01-01 00:00:00
```

Why? ¯\(°_o)/¯

Marc Busqué
http://waiting-for-dev.github.io/about/
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Parsing LocalTime from Unix seconds

Alexander V Vershilov
Hi Marc,

The best way of answering such questions is to check the source code.
Hackage provides
a nice way of doing that -  click on 'Source' near the instance that
you are interested in:

https://hackage.haskell.org/package/time-1.6.0.1/docs/Data-Time-Format.html#t:ParseTime

And you'll see the implementation

```

instance ParseTime LocalTime where
    buildTime l xs = LocalTime <$> (buildTime l xs) <*> (buildTime l xs)
```

That builds time from `Day` and `TimeOfDay` passing your parse string
to each of those.
Then you can check ParseTime instance of Day:

https://hackage.haskell.org/package/time-1.6.0.1/docs/src/Data.Time.Format.Parse.html#line-331

I'm not providing it here, as it's quite big, but the main point is
that `s` is ignored so in that case
Day appear to be:

```
            rest (YearMonth m:_) = let
                d = safeLast 1 [x | MonthDay x <- cs]
                in fromGregorianValid y m d
```
with y=m=d=1

if you continue the process for TimeOfDay you'll find that `s` is
ignored there as well, and
`midnight = TimeOfDay 0 0 0` is returned in that case.

So it appeared that LocalTime consists of the components that ignore
your parse string and return
default value instead.

I don't know if that is intended behaviour or not, but for me it makes
more sense to parse to UTCTime/POSIXTime
and then convert into LocalTime, in case if you get seconds as input.

Hope that helps.


On Thu, 6 Sep 2018 at 13:42, Marc Busqué <[hidden email]> wrote:

>
> In GHCi
>
> ```
> :m +Data.Time
> parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe UTCTime
> -- => Just 2018-08-31 03:00:06 UTC
> parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe LocalTime
> -- => Just 1970-01-01 00:00:00
> ```
>
> Why? ¯\(°_o)/¯
>
> Marc Busqué
> http://waiting-for-dev.github.io/about/_______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.



--
Alexander
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Parsing LocalTime from Unix seconds

Marc Busqué
Thanks Alexander for your answer.

I opened an issue in `time` repository:

https://github.com/haskell/time/issues/104

It seems it is intended behaviour, but I think it is inconsistent and it
makes difficult to parse from a string when you don't know beforehand
the format it has.

In the issue link there is the developed version of my answer... :)

Marc Busqué
http://waiting-for-dev.github.io/about/

On Fri, 14 Sep 2018, Alexander V Vershilov wrote:

> Hi Marc,
>
> The best way of answering such questions is to check the source code.
> Hackage provides
> a nice way of doing that -  click on 'Source' near the instance that
> you are interested in:
>
> https://hackage.haskell.org/package/time-1.6.0.1/docs/Data-Time-Format.html#t:ParseTime
>
> And you'll see the implementation
>
> ```
>
> instance ParseTime LocalTime where
>    buildTime l xs = LocalTime <$> (buildTime l xs) <*> (buildTime l xs)
> ```
>
> That builds time from `Day` and `TimeOfDay` passing your parse string
> to each of those.
> Then you can check ParseTime instance of Day:
>
> https://hackage.haskell.org/package/time-1.6.0.1/docs/src/Data.Time.Format.Parse.html#line-331
>
> I'm not providing it here, as it's quite big, but the main point is
> that `s` is ignored so in that case
> Day appear to be:
>
> ```
>            rest (YearMonth m:_) = let
>                d = safeLast 1 [x | MonthDay x <- cs]
>                in fromGregorianValid y m d
> ```
> with y=m=d=1
>
> if you continue the process for TimeOfDay you'll find that `s` is
> ignored there as well, and
> `midnight = TimeOfDay 0 0 0` is returned in that case.
>
> So it appeared that LocalTime consists of the components that ignore
> your parse string and return
> default value instead.
>
> I don't know if that is intended behaviour or not, but for me it makes
> more sense to parse to UTCTime/POSIXTime
> and then convert into LocalTime, in case if you get seconds as input.
>
> Hope that helps.
>
>
> On Thu, 6 Sep 2018 at 13:42, Marc Busqué <[hidden email]> wrote:
>>
>> In GHCi
>>
>> ```
>> :m +Data.Time
>> parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe UTCTime
>> -- => Just 2018-08-31 03:00:06 UTC
>> parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe LocalTime
>> -- => Just 1970-01-01 00:00:00
>> ```
>>
>> Why? ¯\(°_o)/¯
>>
>> Marc Busqué
>> http://waiting-for-dev.github.io/about/_______________________________________________
>> Haskell-Cafe mailing list
>> To (un)subscribe, modify options or view archives go to:
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> Only members subscribed via the mailman list are allowed to post.
>
>
>
> --
> Alexander
>
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Parsing LocalTime from Unix seconds

Olaf Klinke
In reply to this post by Marc Busqué
Is the result of parsing Unix seconds to LocalTime even well-defined? A LocalTime could be in any time zone, while Unix epoch is relative to a certain time-point in UTC. Hence the parse result must depend on what time zone the LocalTime is referring to.

Olaf
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Parsing LocalTime from Unix seconds

Tobias Dammers
On Fri, Sep 14, 2018 at 10:11:46PM +0200, Olaf Klinke wrote:
>
> Is the result of parsing Unix seconds to LocalTime even well-defined?
> A LocalTime could be in any time zone, while Unix epoch is relative to
> a certain time-point in UTC. Hence the parse result must depend on
> what time zone the LocalTime is referring to.

It's even worse - unix time may also refer to the host's local timezone,
although these days almost everything sets the RTC to UTC, and corrects
for local timezone on the fly (simply because this allows for somewhat
sane handling of DST and such).

Also, unix time may represent either actual seconds elapsed since epoch, or
"logical" seconds since epoch (ignoring leap seconds, such that midnight
is always a multiple of 86400 seconds).


However, once you pick a convention for the second part, you can convert
unix timestamps to some "local time without timezone info" data
structure (e.g. LocalTime); whether you assume UTC or any timezone
becomes relevant when you want to convert that data structure into a
data structure that does contain timezone info explicitly (like
ZonedTime) or implicitly (like UTCTime).

https://hackage.haskell.org/package/time-1.9.2/docs/Data-Time.html
provides a nice overview of the various date/time types defined in the
`time` package, and what they mean. The relevant ones here are:

- UTCTime: idealized (ignoring leap seconds) absolute moment in time
- LocalTime: moment in time, in some unspecified timezone
- ZonedTime: moment in time, in a specific timezone

And the "morally correct" way of parsing unix time into anything would
be to go through LocalTime first, then either convert to UTCTime if the
unix timestamp may be interpreted as being in UTC, or attaching the
correct timezone, yielding a ZonedTime. All this assuming that you can
afford to ignore leap seconds one way or another.
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Parsing LocalTime from Unix seconds

Viktor Dukhovni


> On Sep 17, 2018, at 2:01 AM, Tobias Dammers <[hidden email]> wrote:
>
> Also, unix time may represent either actual seconds elapsed since epoch, or
> "logical" seconds since epoch (ignoring leap seconds, such that midnight
> is always a multiple of 86400 seconds).

That would be a violation of the specification:

  http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_16

On all extant systems Unix Time is based on an 86400-second day, regardless
of any leap seconds.  The RTC clock has nothing to do with this, the epoch
time is defined as an interval.  It only becomes fuzzy when leap seconds
are being handled, as different systems may handle the leap second in somewhat
different ways.  Since the epoch time is quantized anyhow to a 1s granularity,
you can expect the epoch time reported by different systems to differ by +/-1s
normally, even with clocks reasonably well synchronized, and +/-2s when leap
seconds are being added if they're not both using the same adjustment algorithm
(say NTP leap second smearing).

--
        Viktor.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Parsing LocalTime from Unix seconds

Neil Mayhew
In reply to this post by Tobias Dammers
On 2018-09-17 12:01 AM, Tobias Dammers wrote:
… you can convert unix timestamps to some "local time without timezone info" data structure (e.g. LocalTime); whether you assume UTC or any timezone becomes relevant when you want to convert that data structure into a data structure that does contain timezone info explicitly (like ZonedTime) or implicitly (like UTCTime).

I don't think this is possible. If you want to measure seconds since 1970-01-01 00:00:00 local time (which may actually be 1969 in UTC) you have to know the timezone at the start (ie in 1970) and at the end (ie after the seconds have elapsed). That requires knowing whether DST applies at either end, and whether the location was rezoned during the intervening time. This requires IO and the conversion from seconds is no longer a pure operation.

IHMO, the only sane way to interpret seconds-since-the-epoch is using UTC.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Parsing LocalTime from Unix seconds

Joachim Durchholz
Am 17.09.2018 um 21:29 schrieb Neil Mayhew:

> On 2018-09-17 12:01 AM, Tobias Dammers wrote:
>> … you can convert unix timestamps to some "local time without timezone
>> info" data structure (e.g. LocalTime); whether you assume UTC or any
>> timezone becomes relevant when you want to convert that data structure
>> into a data structure that does contain timezone info explicitly (like
>> ZonedTime) or implicitly (like UTCTime).
>
> I don't think this is possible. If you want to measure seconds since
> 1970-01-01 00:00:00 local time (which may actually be 1969 in UTC) you
> have to know the timezone at the start (ie in 1970) and at the end (ie
> after the seconds have elapsed). That requires knowing whether DST
> applies at either end, and whether the location was rezoned during the
> intervening time.

Actually you need only know the time offset at start and end of
interval. Any intervening changes cancel out.

You do need to know whether the location had its time zone changed
(which happened multiple times in some areas I believe).

> This requires IO and the conversion from seconds is no
> longer a pure operation.

Eek. IO just to handle timezone is a really bad idea.
Just use a timestamp-with-timezone data type for the parameters and
you're back with pure functions.

> IHMO, the only sane way to interpret seconds-since-the-epoch is using UTC.

Now that's true. From the point of view in the paragraph above, simply
because it's lacking the data needed for any other interpretation.

Note that there are excellent designs for time representation arounds. A
good one would have to cover instants (multiple representations),
intervals (with and without leap seconds), time zones, and calendars.
 From my daily Java work, the one that works best is the Jodatime
library, documented at http://www.joda.org/joda-time/. If anybody wishes
to check whether it has good ideas worth stealing, take a look at the
Javadoc at http://www.joda.org/joda-time/apidocs/index.html.

If you want to be really thorough, you can also study JSR-310, which was
supposed to be a slightly improve version of Jodatime, AND part of the
Java standard library. That work stopped when it was "good enough",
which isn't good enough. Still, the differences between Jodatime and
JSR-310 may be interesting in themselves (they may show areas where the
Jodatime author believed his work could be improved, as he was deeply
involved in the JSR as well).
Anyway: the JSR-310 classes are documented at
https://docs.oracle.com/javase/8/docs/api/index.html, in the java.time
package and subpackages.

Regards,
Jo
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Parsing LocalTime from Unix seconds

Stefan Monnier
In reply to this post by Tobias Dammers
> It's even worse - unix time may also refer to the host's local timezone,

I don't know any unix-style system which does that.
Any detail on where/when this could happen?


        Stefan

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Parsing LocalTime from Unix seconds

Tobias Dammers
In reply to this post by Viktor Dukhovni
On Mon, Sep 17, 2018 at 09:12:38AM -0400, Viktor Dukhovni wrote:

>
>
> > On Sep 17, 2018, at 2:01 AM, Tobias Dammers <[hidden email]> wrote:
> >
> > Also, unix time may represent either actual seconds elapsed since epoch, or
> > "logical" seconds since epoch (ignoring leap seconds, such that midnight
> > is always a multiple of 86400 seconds).
>
> That would be a violation of the specification:
>
>   http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_16

Ah yes, you are right. Problem is, while Unix systems to day use Unix
time this way, not all other software systems do, or they may assume
that "current unix timestamp - some unix timestamp in the past =
exact number of seconds elapsed between timestamps". Which doesn't hold
in the presence of leap seconds, no matter how you implement them. In
other words, people use Unix timestamps (both the word and actual
implementations) sloppily.
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Parsing LocalTime from Unix seconds

Viktor Dukhovni


> On Sep 19, 2018, at 8:52 AM, Tobias Dammers <[hidden email]> wrote:
>
>>> Also, unix time may represent either actual seconds elapsed since epoch, or
>>> "logical" seconds since epoch (ignoring leap seconds, such that midnight
>>> is always a multiple of 86400 seconds).
>>
>> That would be a violation of the specification:
>>
>>  http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_16
>
> Ah yes, you are right. Problem is, while Unix systems to day use Unix
> time this way, not all other software systems do, or they may assume
> that "current unix timestamp - some unix timestamp in the past =
> exact number of seconds elapsed between timestamps". Which doesn't hold
> in the presence of leap seconds, no matter how you implement them. In
> other words, people use Unix timestamps (both the word and actual
> implementations) sloppily.

That may be so, but Haskell's libraries should not cater to broken implementations.

UNIX epoch time is the time elapsed since 19700101T00:00:00+0000 - number of leap
seconds added since 19700101T00:00:00+0000.

In principle this means that the epoch time repeats during the leap second.
In practice smearing is the better approach taken by many systems with the
clock remaining monotone, but briefly running more slowly (for added leap
seconds).

So you're not guaranteed precise synchronization with any particular clock, but
there is a deterministic conversion from epoch time to local time that does not
need to concern itself with leap seconds.  This conversion will never produce
a 60th second, but could produce a 59th second that never existed if there
are ever negative leap seconds.

--
        Viktor.
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.