Stop using "Int" for microsecond delays in "base"

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
32 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Stop using "Int" for microsecond delays in "base"

Paul Johnson-2
The "base" library has the "threadDelay" primitive, which takes an Int
argument in microseconds.  Unfortunately this means that the longest
delay you can get on a 32 bit machine with GHC is just under 36 minutes
(2^31 uSec), and a hypothetical compiler that only used 30 bit integers
(as per the standard) would get under 10 minutes.  It is a bit tricky to
write general-purpose libraries with this.  I think that there should be a

    type Delay = Int64

declaration, and that threadDelay and related functions should take that
as an argument type.

Paul.

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Henning Thielemann-4
Paul Johnson schrieb:
> The "base" library has the "threadDelay" primitive, which takes an Int
> argument in microseconds.  Unfortunately this means that the longest
> delay you can get on a 32 bit machine with GHC is just under 36 minutes
> (2^31 uSec), and a hypothetical compiler that only used 30 bit integers
> (as per the standard) would get under 10 minutes.  It is a bit tricky to
> write general-purpose libraries with this.

Isn't it just a

waitLong :: Integer -> IO ()
waitLong n =
   let stdDelay = 10^7
       (q,r) = divMod n stdDelay
   in  replicateM_ (fromInteger q) (threadDelay (fromInteger stdDelay))
        >> threadDelay (fromInteger r)


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Bas van Dijk-2
On 26 March 2011 20:16, Henning Thielemann
<[hidden email]> wrote:

> Paul Johnson schrieb:
>> The "base" library has the "threadDelay" primitive, which takes an Int
>> argument in microseconds.  Unfortunately this means that the longest
>> delay you can get on a 32 bit machine with GHC is just under 36 minutes
>> (2^31 uSec), and a hypothetical compiler that only used 30 bit integers
>> (as per the standard) would get under 10 minutes.  It is a bit tricky to
>> write general-purpose libraries with this.
>
> Isn't it just a
>
> waitLong :: Integer -> IO ()
> waitLong n =
>   let stdDelay = 10^7
>       (q,r) = divMod n stdDelay
>   in  replicateM_ (fromInteger q) (threadDelay (fromInteger stdDelay))
>        >> threadDelay (fromInteger r)
>
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/libraries
>

Or take the one from concurrent-extra:

http://hackage.haskell.org/packages/archive/concurrent-extra/0.6.0.1/doc/html/Control-Concurrent-Thread-Delay.html

However, I agree that it would be nice to reduce the need for these
functions by having a larger sized Delay type.

I don't think it's that hard to change. When using the threaded rts
the threadDelay function is defined by the GHC event manager which
stores time in seconds since the epoch (1970) in a Double. It gets the
current time from the gettimeofday function[1]. When not using the
threaded rts the threadDelay function is defined inside the rts. I'm
not sure how hard it is to change that one.

Bas

[1] http://linux.die.net/man/2/gettimeofday

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Edward Kmett-2
On Sat, Mar 26, 2011 at 3:44 PM, Bas van Dijk <[hidden email]> wrote:
Or take the one from concurrent-extra:

http://hackage.haskell.org/packages/archive/concurrent-extra/0.6.0.1/doc/html/Control-Concurrent-Thread-Delay.html

I'll admit the main problem that I have with the threadDelay in concurrent-extra is the gratuitously unicode source file. 

I wound up having to reimplement it inside a library of mine to avoid adding a dependency on that extension and the unrelated dependency on stm. =(

(Technically, I implemented it directly first, then found concurrent-extra and wasn't willing to refactor to use yours as a result of those.)

You are of course free to implement things as you see fit, but little things like that do impact adoption.

-Edward

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Paul Johnson-2
In reply to this post by Henning Thielemann-4
On 26/03/11 19:16, Henning Thielemann wrote:

> Paul Johnson schrieb:
>> The "base" library has the "threadDelay" primitive, which takes an Int
>> argument in microseconds.  Unfortunately this means that the longest
>> delay you can get on a 32 bit machine with GHC is just under 36 minutes
>> (2^31 uSec), and a hypothetical compiler that only used 30 bit integers
>> (as per the standard) would get under 10 minutes.  It is a bit tricky to
>> write general-purpose libraries with this.
> Isn't it just a
>
> waitLong :: Integer ->  IO ()
> waitLong n =
>     let stdDelay = 10^7
>         (q,r) = divMod n stdDelay
>     in  replicateM_ (fromInteger q) (threadDelay (fromInteger stdDelay))
>          >>  threadDelay (fromInteger r)
>
>
True, but:

1: Low level stuff like this belongs in the library, not the
application.  I'm writing Haskell, not C!

2: I actually want to use timeout, which inherits the same limitation.  
Right now it looks like the simplest thing to do would be to copy the
existing timeout code but use "waitLong" instead of "wait".  But I
shouldn't have to do that.

Paul.

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Bas van Dijk-2
In reply to this post by Edward Kmett-2
On 27 March 2011 07:59, Edward Kmett <[hidden email]> wrote:

> On Sat, Mar 26, 2011 at 3:44 PM, Bas van Dijk <[hidden email]> wrote:
>>
>> Or take the one from concurrent-extra:
>>
>>
>> http://hackage.haskell.org/packages/archive/concurrent-extra/0.6.0.1/doc/html/Control-Concurrent-Thread-Delay.html
>
> I'll admit the main problem that I have with the threadDelay in
> concurrent-extra is the gratuitously unicode source file.
> I wound up having to reimplement it inside a library of mine to avoid adding
> a dependency on that extension and the unrelated dependency on stm. =(
> (Technically, I implemented it directly first, then found concurrent-extra
> and wasn't willing to refactor to use yours as a result of those.)
> You are of course free to implement things as you see fit, but little things
> like that do impact adoption.
> -Edward

Note that 'delay' uses the 'threadDelay' function internally. AFAIK,
this function is only supported in GHC. So to build the module you
need GHC anyway which already supports the UnicodeSyntax language
extension (since 6.12.1).

I do agree that the dependency on stm is unfortunate. I will see if I
can put the modules Control.Concurrent.Thread.Delay and
Control.Concurrent.Timeout into a separate package. Any naming
suggestions? unbounded-delays or something?

Bas

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Bas van Dijk-2
In reply to this post by Paul Johnson-2
On 27 March 2011 11:15, Paul Johnson <[hidden email]> wrote:
> 2: I actually want to use timeout, which inherits the same limitation.
>  Right now it looks like the simplest thing to do would be to copy the
> existing timeout code but use "waitLong" instead of "wait".  But I shouldn't
> have to do that.

concurrent-extra to the rescue again:

http://hackage.haskell.org/packages/archive/concurrent-extra/0.6.0.1/doc/html/Control-Concurrent-Timeout.html

But note that I may move this to its own package soon (as in later today).

Bas

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Bas van Dijk-2
On 27 March 2011 11:21, Bas van Dijk <[hidden email]> wrote:
> But note that I may move this to its own package soon (as in later today).

Done: http://code.haskell.org/~basvandijk/code/unbounded-delays/

I plan to release it this evening or tomorrow.

Bas

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

unicode was: Re: Stop using "Int" for microsecond delays in "base"

Christian Maeder-2
In reply to this post by Bas van Dijk-2
Am 27.03.2011 11:17, schrieb Bas van Dijk:

> On 27 March 2011 07:59, Edward Kmett<[hidden email]>  wrote:
>> On Sat, Mar 26, 2011 at 3:44 PM, Bas van Dijk<[hidden email]>  wrote:
>>>
>>> Or take the one from concurrent-extra:
>>>
>>>
>>> http://hackage.haskell.org/packages/archive/concurrent-extra/0.6.0.1/doc/html/Control-Concurrent-Thread-Delay.html
>>
>> I'll admit the main problem that I have with the threadDelay in
>> concurrent-extra is the gratuitously unicode source file.
>> I wound up having to reimplement it inside a library of mine to avoid adding
>> a dependency on that extension and the unrelated dependency on stm. =(
>> (Technically, I implemented it directly first, then found concurrent-extra
>> and wasn't willing to refactor to use yours as a result of those.)
>> You are of course free to implement things as you see fit, but little things
>> like that do impact adoption.
>> -Edward
>
> Note that 'delay' uses the 'threadDelay' function internally. AFAIK,
> this function is only supported in GHC. So to build the module you
> need GHC anyway which already supports the UnicodeSyntax language
> extension (since 6.12.1).

With this argument you should use many more ghc extensions. The
dependency on base-unicode-symbols is unfortunate, too!

If this dependency was supposed to advertise your base-unicode-symbols
package I consider this dependency as spam. Basic coding style should
ASCII only.

Cheers Christian

>
> I do agree that the dependency on stm is unfortunate. I will see if I
> can put the modules Control.Concurrent.Thread.Delay and
> Control.Concurrent.Timeout into a separate package. Any naming
> suggestions? unbounded-delays or something?
>
> Bas

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Edward Kmett-2
In reply to this post by Bas van Dijk-2
Thanks!

On Sun, Mar 27, 2011 at 8:06 AM, Bas van Dijk <[hidden email]> wrote:
On 27 March 2011 11:21, Bas van Dijk <[hidden email]> wrote:
> But note that I may move this to its own package soon (as in later today).

Done: http://code.haskell.org/~basvandijk/code/unbounded-delays/

I plan to release it this evening or tomorrow.

Bas

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Bas van Dijk-2
I just released the package:

http://hackage.haskell.org/package/unbounded-delays

I kept UnicodeSyntax but dropped the dependency on
base-unicode-symbols. I hope this is a satisfying compromise.

I also made a new major release of concurrent-extra that removes the
respected modules:

http://hackage.haskell.org/package/concurrent-extra-0.7

This release also supports (but doesn't require) the new cabal
test-suite feature. To test the package simply:

$ cabal configure --enable-tests
$ cabal build

now run the test suite either manually (and get nice colourful output) using:

$ dist/build/test-concurrent-extra/test-concurrent-extra

or using cabal (and get only a summary of the test output):

$ cabal test

Regards,

Bas

On 28 March 2011 20:01, Edward Kmett <[hidden email]> wrote:

> Thanks!
>
> On Sun, Mar 27, 2011 at 8:06 AM, Bas van Dijk <[hidden email]> wrote:
>>
>> On 27 March 2011 11:21, Bas van Dijk <[hidden email]> wrote:
>> > But note that I may move this to its own package soon (as in later
>> > today).
>>
>> Done: http://code.haskell.org/~basvandijk/code/unbounded-delays/
>>
>> I plan to release it this evening or tomorrow.
>>
>> Bas
>>
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://www.haskell.org/mailman/listinfo/libraries
>
>

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Daniel Peebles
I kept UnicodeSyntax but dropped the dependency on
base-unicode-symbols. I hope this is a satisfying compromise.

I don't mind, myself, but lots of people are rather picky about being haskell<year>-compliant. Do you intend to submit this for general adoption or is it just a stopgap until the base threadDelay is improved?

Thanks,
Dan

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Bas van Dijk-2
On 29 March 2011 00:56, Daniel Peebles <[hidden email]> wrote:
>> I kept UnicodeSyntax but dropped the dependency on
>> base-unicode-symbols. I hope this is a satisfying compromise.
>
> I don't mind, myself, but lots of people are rather picky about being
> haskell<year>-compliant. Do you intend to submit this for general adoption
> or is it just a stopgap until the base threadDelay is improved?

When the base threadDelay and timeout switch to Integer based periods
I will of course deprecate this package. Until that time people can
use this package.

Bas

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Wolfgang Jeltsch-2
In reply to this post by Paul Johnson-2
Am Samstag, den 26.03.2011, 17:20 +0000 schrieb Paul Johnson:

> The "base" library has the "threadDelay" primitive, which takes an Int
> argument in microseconds.  Unfortunately this means that the longest
> delay you can get on a 32 bit machine with GHC is just under 36 minutes
> (2^31 uSec), and a hypothetical compiler that only used 30 bit integers
> (as per the standard) would get under 10 minutes.  It is a bit tricky to
> write general-purpose libraries with this.  I think that there should be a
>
>     type Delay = Int64
>
> declaration, and that threadDelay and related functions should take that
> as an argument type.
>
> Paul.

Maybe, we should stop using Int altogether. Why not use Integer if we
want integers, and try to implement Integer and its operations more
efficiently? Int8, Int16, Int32, and Int64 are good when doing system
programming, CInt is good when interfacing with C, but what is Int good
for?

Best wishes,
Wolfgang


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Daniel Fischer
On Wednesday 30 March 2011 14:31:33, Wolfgang Jeltsch wrote:
>
> Maybe, we should stop using Int altogether. Why not use Integer if we
> want integers, and try to implement Integer and its operations more
> efficiently?

Even getting near the performance of GMP with its probably hundreds of man-
years of optimisation gone in is hard, beating it is a formidable task.
Or were you thinking of making the calls to GMP faster?
If that could be done, that would be very nice.

> Int8, Int16, Int32, and Int64 are good when doing system
> programming, CInt is good when interfacing with C, but what is Int good
> for?

Speed. If you have computations you know won't overflow, it's good to have
the native-sized Int and need not throttle things on 64-bit systems by
using Int32 or on 32-bit systems by using Int64.

But in general, I agree that it would be good to use Integer instead of Int
more often.

Cheers,
Daniel

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Roel van Dijk-3
Maybe we could use something like stdint.h [1]. You could use a type
like int_fast16_t to ensure a minimum width of 16 while still getting
the benefits of native ints (if possible).

1 - http://en.wikipedia.org/wiki/Stdint.h

On 30 March 2011 15:20, Daniel Fischer <[hidden email]> wrote:

> On Wednesday 30 March 2011 14:31:33, Wolfgang Jeltsch wrote:
>>
>> Maybe, we should stop using Int altogether. Why not use Integer if we
>> want integers, and try to implement Integer and its operations more
>> efficiently?
>
> Even getting near the performance of GMP with its probably hundreds of man-
> years of optimisation gone in is hard, beating it is a formidable task.
> Or were you thinking of making the calls to GMP faster?
> If that could be done, that would be very nice.
>
>> Int8, Int16, Int32, and Int64 are good when doing system
>> programming, CInt is good when interfacing with C, but what is Int good
>> for?
>
> Speed. If you have computations you know won't overflow, it's good to have
> the native-sized Int and need not throttle things on 64-bit systems by
> using Int32 or on 32-bit systems by using Int64.
>
> But in general, I agree that it would be good to use Integer instead of Int
> more often.
>
> Cheers,
> Daniel

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Evan Laforge
In reply to this post by Wolfgang Jeltsch-2
> Maybe, we should stop using Int altogether. Why not use Integer if we
> want integers, and try to implement Integer and its operations more
> efficiently? Int8, Int16, Int32, and Int64 are good when doing system
> programming, CInt is good when interfacing with C, but what is Int good
> for?

It's good for big data structures since it can be unpacked into the
constructor.  I've seen space usage go down by 1/3 after an Integer ->
Int switch.  Integer is a sum type so there's an extra two words of
overhead (if not mistaken, one for the tag, and one for the
indirection since it can't unpack).

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Johan Tibell-2
On Wed, Mar 30, 2011 at 7:56 PM, Evan Laforge <[hidden email]> wrote:
> It's good for big data structures since it can be unpacked into the
> constructor.  I've seen space usage go down by 1/3 after an Integer ->
> Int switch.  Integer is a sum type so there's an extra two words of
> overhead (if not mistaken, one for the tag, and one for the
> indirection since it can't unpack).

Yes. Integer is terrible for performance.

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Max Bolingbroke-2
On 30 March 2011 19:00, Johan Tibell <[hidden email]> wrote:
> Yes. Integer is terrible for performance.

But in this case it's unlikely to make a difference because the whole
point of threadDelay is to slow the program down, so who cares if we
deref one more pointer :-)

Cheers,
Max

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Stop using "Int" for microsecond delays in "base"

Johan Tibell-2
On Wed, Mar 30, 2011 at 8:28 PM, Max Bolingbroke
<[hidden email]> wrote:
> On 30 March 2011 19:00, Johan Tibell <[hidden email]> wrote:
>> Yes. Integer is terrible for performance.
>
> But in this case it's unlikely to make a difference because the whole
> point of threadDelay is to slow the program down, so who cares if we
> deref one more pointer :-)

Sure. In this particular case.

What do we do if the user passes some humongous number to threadDelay?
The underlying syscall doesn't support it, so I guess we would have to
call that syscall repeatedly.

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
12