What can be UNPACK'ed?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

What can be UNPACK'ed?

Tom Ellis
What types can be UNPACK'ed?  From my vague understanding of what UNPACK
does, it can only be types whose size is bounded by a known value, so whilst

    data Foo = Foo {-# UNPACK #-} !Float {-# UNPACK #-} !Int

is fine, presumably

    data Foo = Foo {-# UNPACK #-} ![Float]

is not fine.  Or is it?  Technically it could be possible to reserve enough
space in Foo to store either a [] or a (:) Float [Float].  And if that is
possible could we also UNPACK polymorphic fields?

    data Storable a => Foo a = Foo {-# UNPACK #-} a

or

    data Foo a where
        Foo :: Storable a => {-# UNPACK #-} a -> Foo a

if either DatatypeContexts worked or my imaginary GADT UNPACK syntax worked
(I'm not sure if either does).

What are the limits of what we could reasonably get to UNPACK in GHC?

Thanks,

Tom
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What can be UNPACK'ed?

Oliver Charles-3
Afaik, you couldn't UNPACK a list in the past because it's a sum type, and sum types could not be unpacked. That has changed with the latest GHC, so maybe that now works.

I'm pretty certain you can't unpack polymorphic fields, and this is why libraries such as `ad` offer a polymorphic version, and a version specialized to a particular type:

http://hackage.haskell.org/package/ad-4.3.3/docs/Numeric-AD-Mode-Forward.html - polymorphic
http://hackage.haskell.org/package/ad-4.3.3/docs/Numeric-AD-Mode-Forward-Double.html - specialized to Double and significantly faster

Does that help?

On Fri, Jul 28, 2017 at 2:48 PM Tom Ellis <[hidden email]> wrote:
What types can be UNPACK'ed?  From my vague understanding of what UNPACK
does, it can only be types whose size is bounded by a known value, so whilst

    data Foo = Foo {-# UNPACK #-} !Float {-# UNPACK #-} !Int

is fine, presumably

    data Foo = Foo {-# UNPACK #-} ![Float]

is not fine.  Or is it?  Technically it could be possible to reserve enough
space in Foo to store either a [] or a (:) Float [Float].  And if that is
possible could we also UNPACK polymorphic fields?

    data Storable a => Foo a = Foo {-# UNPACK #-} a

or

    data Foo a where
        Foo :: Storable a => {-# UNPACK #-} a -> Foo a

if either DatatypeContexts worked or my imaginary GADT UNPACK syntax worked
(I'm not sure if either does).

What are the limits of what we could reasonably get to UNPACK in GHC?

Thanks,

Tom
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What can be UNPACK'ed?

Tom Ellis
Thanks, that's helpful info, but what I'm really after is a definitive
condition under which fields can be UNPACK'ed.  The docs are very vague:

    https://downloads.haskell.org/~ghc/7.0.2/docs/html/users_guide/pragmas.html

On Fri, Jul 28, 2017 at 03:01:02PM +0000, Oliver Charles wrote:

> Afaik, you couldn't UNPACK a list in the past because it's a sum type, and
> sum types could not be unpacked. That has changed with the latest GHC, so
> maybe that now works.
>
> I'm pretty certain you can't unpack polymorphic fields, and this is why
> libraries such as `ad` offer a polymorphic version, and a version
> specialized to a particular type:
>
> http://hackage.haskell.org/package/ad-4.3.3/docs/Numeric-AD-Mode-Forward.html
> - polymorphic
> http://hackage.haskell.org/package/ad-4.3.3/docs/Numeric-AD-Mode-Forward-Double.html
> - specialized to Double and significantly faster
>
> Does that help?
>
> On Fri, Jul 28, 2017 at 2:48 PM Tom Ellis <
> [hidden email]> wrote:
>
> > What types can be UNPACK'ed?  From my vague understanding of what UNPACK
> > does, it can only be types whose size is bounded by a known value, so
> > whilst
> >
> >     data Foo = Foo {-# UNPACK #-} !Float {-# UNPACK #-} !Int
> >
> > is fine, presumably
> >
> >     data Foo = Foo {-# UNPACK #-} ![Float]
> >
> > is not fine.  Or is it?  Technically it could be possible to reserve enough
> > space in Foo to store either a [] or a (:) Float [Float].  And if that is
> > possible could we also UNPACK polymorphic fields?
> >
> >     data Storable a => Foo a = Foo {-# UNPACK #-} a
> >
> > or
> >
> >     data Foo a where
> >         Foo :: Storable a => {-# UNPACK #-} a -> Foo a
> >
> > if either DatatypeContexts worked or my imaginary GADT UNPACK syntax worked
> > (I'm not sure if either does).
> >
> > What are the limits of what we could reasonably get to UNPACK in GHC?
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What can be UNPACK'ed?

amindfv
In reply to this post by Tom Ellis

>  Technically it could be possible to reserve enough
> space in Foo to store either a [] or a (:) Float [Float].  

What if it's an infinite list? Do you just want to unpack the first cons and not the whole list?

Tom

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What can be UNPACK'ed?

Brandon Allbery
On Sun, Jul 30, 2017 at 5:29 PM, <[hidden email]> wrote:
>  Technically it could be possible to reserve enough
> space in Foo to store either a [] or a (:) Float [Float].

What if it's an infinite list? Do you just want to unpack the first cons and not the whole list?

Isn't the example exactly the first of those?

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What can be UNPACK'ed?

Tom Ellis
In reply to this post by amindfv
On Sun, Jul 30, 2017 at 04:29:19PM -0500, [hidden email] wrote:
> >  Technically it could be possible to reserve enough space in Foo to
> > store either a [] or a (:) Float [Float].
>
> What if it's an infinite list? Do you just want to unpack the first cons
> and not the whole list?

Yes indeed.  That's what I meant by

    store either a [] or a (:) Float [Float]

(And I'm not saying I want to, just that it could be done.)
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What can be UNPACK'ed?

Olaf Klinke
In reply to this post by Tom Ellis
How is UNPACK and BangPatterns related to WHNF? I used to think that unpacking sort of forces the constructors up to the level where the bangs reside. Hence unpacking a tuple makes sense, whereas unpacking a linked list does not save much memory.

-- Olaf
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What can be UNPACK'ed?

Nick Smallbone-3
In reply to this post by Tom Ellis
Tom Ellis <[hidden email]> writes:

> Thanks, that's helpful info, but what I'm really after is a definitive
> condition under which fields can be UNPACK'ed.  The docs are very vague:
>
>     https://downloads.haskell.org/~ghc/7.0.2/docs/html/users_guide/pragmas.html

As far as I know, the restrictions for UNPACKing a field are as follows:
   * The field's type must have only one constructor
   * The type must not be just a type variable
     (i.e., the constructor must be known at compile time)
   * The field must be strict
     (this is because otherwise, UNPACKing changes the semantics of the
     data type you are declaring by making it more strict)

So, for example, lists cannot be UNPACKed (because they have two
constructors, nil and cons). Nor can polymorphic fields (because there
may or may not be only one constructor).

To apply this to primitive types, you have to know how they are defined.
For example, Int can be UNPACKed because it has only one constructor:

  data Int = I# Int#

But Integer cannot be UNPACKed because it has several constructors:

  data Integer  = S#                !Int#
                | Jp# {-# UNPACK #-} !BigNat
                | Jn# {-# UNPACK #-} !BigNat

In practice, most primitive types apart from Integer can be UNPACKed -
but to be sure you have to look at the standard library source code
(or just pay attention to the compiler warnings).

Nick

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What can be UNPACK'ed?

Tom Ellis
On Mon, Jul 31, 2017 at 11:44:20PM +0200, Nick Smallbone wrote:
> Tom Ellis <[hidden email]> writes:
> > Thanks, that's helpful info, but what I'm really after is a definitive
> > condition under which fields can be UNPACK'ed.  The docs are very vague:
> >
> >     https://downloads.haskell.org/~ghc/7.0.2/docs/html/users_guide/pragmas.html
>
> As far as I know, the restrictions for UNPACKing a field are as follows:

Ah, very nice, thanks Nick.

>    * The field must be strict
>      (this is because otherwise, UNPACKing changes the semantics of the
>      data type you are declaring by making it more strict)

That makes sense.

>    * The type must not be just a type variable
>      (i.e., the constructor must be known at compile time)

Eventually I'd like to see if we can relax this condition but first ...

>    * The field's type must have only one constructor

Do you know the rationale for this?  It seems like there should be no
problem unpacking a constructor tag plus payload, even if the payload size
varies, as long as the maximum payload size in known.

Tom
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What can be UNPACK'ed?

Brandon Allbery
On Mon, Jul 31, 2017 at 5:58 PM, Tom Ellis <[hidden email]> wrote:
On Mon, Jul 31, 2017 at 11:44:20PM +0200, Nick Smallbone wrote:
> Tom Ellis <[hidden email]> writes:
> > Thanks, that's helpful info, but what I'm really after is a definitive
> > condition under which fields can be UNPACK'ed.  The docs are very vague:
> >
> >     https://downloads.haskell.org/~ghc/7.0.2/docs/html/users_guide/pragmas.html
>
> As far as I know, the restrictions for UNPACKing a field are as follows:

Ah, very nice, thanks Nick.

I think this might be in the Commentary somewhere, as I've seen essentially that list. 

>    * The type must not be just a type variable
>      (i.e., the constructor must be known at compile time)

Eventually I'd like to see if we can relax this condition but first ...

>    * The field's type must have only one constructor

Do you know the rationale for this?  It seems like there should be no
problem unpacking a constructor tag plus payload, even if the payload size
varies, as long as the maximum payload size in known.

UNPACK means unbox, and the constructor tag is part of the box. I suspect something (gc?) assumes that a constructor tag is always the first thing in a memory chunk, so having a constructor tag unpacked into the middle of something else causes problems (e.g. it will try to gc the middle of the containing memory object).

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: What can be UNPACK'ed?

Nick Smallbone-3
In reply to this post by Tom Ellis
Tom Ellis <[hidden email]> writes:

> On Mon, Jul 31, 2017 at 11:44:20PM +0200, Nick Smallbone wrote:
>>    * The type must not be just a type variable
>>      (i.e., the constructor must be known at compile time)
>
> Eventually I'd like to see if we can relax this condition but first ...

(Take everything I say here with a pinch of salt: I'm not a GHC developer.)

IIUC, relaxing this would be difficult because GHC compiles using type
erasure. For example, if I have a type

   data Whatever a = Whatever !a

then values of type Whatever Int and Whatever Float are represented in
exactly the same way at runtime. However, if we were to unpack the 'a',
then they would internally have to be turned into two different types:
   data WhateverInt = Whatever Int#
   data WhateverFloat = Whatever Float#
This would in turn mean that a polymorphic function
   f :: Whatever a -> ...
would have to be compiled into two separate versions, one for Whatever
Int and one for Whatever Float, which is something that GHC doesn't do.

>>    * The field's type must have only one constructor
>
> Do you know the rationale for this?  It seems like there should be no
> problem unpacking a constructor tag plus payload, even if the payload size
> varies, as long as the maximum payload size in known.

One thing is that UNPACK right now can be implemented just by a simple
program transformation. When you declare a datatype with an unpacked
field, such as
  data T a b = C {-# UNPACK #-} !Int {-# UNPACK #-} !(a, b),
then GHC secretly replaces the datatype with the unpacked version, in
this case:
  data T a b = UnpackedC Int# a b
Then all uses of the constructor C are rewritten to use UnpackedC instead.
The resulting program is still type-correct - in fact, it is still a
normal Haskell (well, GHC core) program. This transformation happens
quite early on during compilation and AIUI the rest of the compiler
doesn't have to know about unpacking at all.

If you try to unpack a field that has multiple constructors this way,
it's not clear what the unpacked datatype should look like - I suppose
it'd need to be some kind of union. So at least it's not immediately
obvious how to do this...

Nick

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Loading...