Unboxed Vectors of newtype'd values

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Unboxed Vectors of newtype'd values

Ben Gamari
One of the reasons I find Haskell so well suited to my work is its
ability to easily work with large quantities of data. In particular, I
find myself using Data.Vector.Unbox quite frequently.

Another of my reasons for using Haskell is the type safety it
provides. I find myself using newtypes very frequently to force myself
to think about invariants that a more weakly typed language would allow
me to simply ignore.

Sadly, these two features don't interact particularly well. While the
Data.Vector.Unbox documentation claims that "Implementing unboxed
vectors for new data types can be very easy", it then goes on to list an
abridged version of the Complex instance---dozens of lines of
code. While this code certainly isn't difficult to write, it is time
consuming, error-prone, and, above else, utterly mind deadeningly dull
(making it quite uncharacteristic for Haskell). So dull that I generally
avoid newtypes at all cost in code that might need to use unboxed
vectors. This boilerplate is largely due to Vector's use of type
families as this precludes the use of (the otherwise quite cunning)
GeneralizedNewtypeDeriving to automatically derive the necessary
instances.

What can be done to fix this unfortunate state of affairs? The obvious
solution here seems to be Template Haskell, but this seems a bit of an
unfortunate hack around what might be a deficiency in the type families
mechanism (or at least this application of it). The newtype package
provides a nice mechanism to pack and unpack newtypes, but providing
blanket Unbox instances for Newtype instances seems like an awful idea
(and, frankly, I'm not sure how this would work with type
families). Other than these two possibilities I am at a loss. Thoughts?
I'd appreciate any ideas folks could offer.

Cheers,

- Ben


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Jake McArthur

I don't have a proposal, but I'd like to echo my support for thinking about this problem. It is very annoying to use my own types with unboxed vectors.

On May 30, 2012 12:28 AM, "Ben Gamari" <[hidden email]> wrote:
One of the reasons I find Haskell so well suited to my work is its
ability to easily work with large quantities of data. In particular, I
find myself using Data.Vector.Unbox quite frequently.

Another of my reasons for using Haskell is the type safety it
provides. I find myself using newtypes very frequently to force myself
to think about invariants that a more weakly typed language would allow
me to simply ignore.

Sadly, these two features don't interact particularly well. While the
Data.Vector.Unbox documentation claims that "Implementing unboxed
vectors for new data types can be very easy", it then goes on to list an
abridged version of the Complex instance---dozens of lines of
code. While this code certainly isn't difficult to write, it is time
consuming, error-prone, and, above else, utterly mind deadeningly dull
(making it quite uncharacteristic for Haskell). So dull that I generally
avoid newtypes at all cost in code that might need to use unboxed
vectors. This boilerplate is largely due to Vector's use of type
families as this precludes the use of (the otherwise quite cunning)
GeneralizedNewtypeDeriving to automatically derive the necessary
instances.

What can be done to fix this unfortunate state of affairs? The obvious
solution here seems to be Template Haskell, but this seems a bit of an
unfortunate hack around what might be a deficiency in the type families
mechanism (or at least this application of it). The newtype package
provides a nice mechanism to pack and unpack newtypes, but providing
blanket Unbox instances for Newtype instances seems like an awful idea
(and, frankly, I'm not sure how this would work with type
families). Other than these two possibilities I am at a loss. Thoughts?
I'd appreciate any ideas folks could offer.

Cheers,

- Ben


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Bryan O'Sullivan
On Wed, May 30, 2012 at 5:19 AM, Jake McArthur <[hidden email]> wrote:

I don't have a proposal, but I'd like to echo my support for thinking about this problem. It is very annoying to use my own types with unboxed vectors.

Likewise, if anyone is listening :-) 

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Johan Tibell-2
On Wed, May 30, 2012 at 9:50 AM, Bryan O'Sullivan <[hidden email]> wrote:
> Likewise, if anyone is listening :-)

Here too.

I'd like some compiler support for abstracting over unboxed values in
general and e.g. generate Unbox instances automatically. Then we could
have other unboxed structures than Vector.

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Roman Leshchinskiy
In reply to this post by Ben Gamari
On 30/05/2012, at 05:27, Ben Gamari wrote:

> Sadly, these two features don't interact particularly well. While the
> Data.Vector.Unbox documentation claims that "Implementing unboxed
> vectors for new data types can be very easy", it then goes on to list an
> abridged version of the Complex instance---dozens of lines of
> code. While this code certainly isn't difficult to write, it is time
> consuming, error-prone, and, above else, utterly mind deadeningly dull
> (making it quite uncharacteristic for Haskell). So dull that I generally
> avoid newtypes at all cost in code that might need to use unboxed
> vectors. This boilerplate is largely due to Vector's use of type
> families as this precludes the use of (the otherwise quite cunning)
> GeneralizedNewtypeDeriving to automatically derive the necessary
> instances.


I completely agree that the current situation is less than ideal but don't really have a good solution. Frankly, the easiest way of defining Unbox instances for newtypes is probably a preprocessor macro. I use a lot of those in vector to reduce boilerplate and they work well. I'll provide one for newtypes in the next release.

As you say, TH is another possibility but one I'm rather wary of.

Roman



_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Daniel Peebles
It seems like the most elegant long-term solution here is to make GeneralizedNewtypeDeriving more cunning. Is there some fundamental reason it can't work with the type families involved?

But in the short term, I agree and TH is probably best.

-Dan

On Wed, May 30, 2012 at 3:41 PM, Roman Leshchinskiy <[hidden email]> wrote:
On 30/05/2012, at 05:27, Ben Gamari wrote:

> Sadly, these two features don't interact particularly well. While the
> Data.Vector.Unbox documentation claims that "Implementing unboxed
> vectors for new data types can be very easy", it then goes on to list an
> abridged version of the Complex instance---dozens of lines of
> code. While this code certainly isn't difficult to write, it is time
> consuming, error-prone, and, above else, utterly mind deadeningly dull
> (making it quite uncharacteristic for Haskell). So dull that I generally
> avoid newtypes at all cost in code that might need to use unboxed
> vectors. This boilerplate is largely due to Vector's use of type
> families as this precludes the use of (the otherwise quite cunning)
> GeneralizedNewtypeDeriving to automatically derive the necessary
> instances.


I completely agree that the current situation is less than ideal but don't really have a good solution. Frankly, the easiest way of defining Unbox instances for newtypes is probably a preprocessor macro. I use a lot of those in vector to reduce boilerplate and they work well. I'll provide one for newtypes in the next release.

As you say, TH is another possibility but one I'm rather wary of.

Roman



_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Reiner Pope
In case anyone missed it, Liyang HU wrote a TH deriver for Unbox instances[1]. 

Also, I have tried using GHC's new generic deriving[2] to derive instances - this is almost possible, but is blocked by a GHC issue[3]. IMHO generic deriving would be the best solution (once it works), because it works for product types as well as newtypes, and requires no extra GHC assistance or TH.


Cheers,
Reiner

On 30 May 2012 22:03, Daniel Peebles <[hidden email]> wrote:
It seems like the most elegant long-term solution here is to make GeneralizedNewtypeDeriving more cunning. Is there some fundamental reason it can't work with the type families involved?

But in the short term, I agree and TH is probably best.

-Dan


On Wed, May 30, 2012 at 3:41 PM, Roman Leshchinskiy <[hidden email]> wrote:
On 30/05/2012, at 05:27, Ben Gamari wrote:

> Sadly, these two features don't interact particularly well. While the
> Data.Vector.Unbox documentation claims that "Implementing unboxed
> vectors for new data types can be very easy", it then goes on to list an
> abridged version of the Complex instance---dozens of lines of
> code. While this code certainly isn't difficult to write, it is time
> consuming, error-prone, and, above else, utterly mind deadeningly dull
> (making it quite uncharacteristic for Haskell). So dull that I generally
> avoid newtypes at all cost in code that might need to use unboxed
> vectors. This boilerplate is largely due to Vector's use of type
> families as this precludes the use of (the otherwise quite cunning)
> GeneralizedNewtypeDeriving to automatically derive the necessary
> instances.


I completely agree that the current situation is less than ideal but don't really have a good solution. Frankly, the easiest way of defining Unbox instances for newtypes is probably a preprocessor macro. I use a lot of those in vector to reduce boilerplate and they work well. I'll provide one for newtypes in the next release.

As you say, TH is another possibility but one I'm rather wary of.

Roman



_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries



_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Ben Gamari
Reiner Pope <[hidden email]> writes:

> In case anyone missed it, Liyang HU wrote a TH deriver for Unbox
> instances[1].
>
Thanks! I don't know how I missed this. This is quite sufficient for
most of my purposes.

> Also, I have tried using GHC's new generic deriving[2] to derive instances
> - this is almost possible, but is blocked by a GHC issue[3]. IMHO generic
> deriving would be the best solution (once it works), because it works for
> product types as well as newtypes, and requires no extra GHC assistance or
> TH.
>
This will be quite nice and it sounds like it should cover nearly all of
the cases I'm interested in. I'm not terribly familiar with generics
support so I'm not sure whether this is possible or not, but it would be
great if there were a way to specify whether an SoA or AoS
representation is derived. This way one could write nearly cache-optimal
code while keeping the mechanics of data layout nicely contained.

Cheers,

- Ben


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: Unboxed Vectors of newtype'd values

Simon Peyton Jones
In reply to this post by Johan Tibell-2
| I'd like some compiler support for abstracting over unboxed values in
| general and e.g. generate Unbox instances automatically. Then we could
| have other unboxed structures than Vector.

Could someone post an example or two of the problem being described here?

Simon

| -----Original Message-----
| From: [hidden email] [mailto:libraries-
| [hidden email]] On Behalf Of Johan Tibell
| Sent: 30 May 2012 17:57
| To: Bryan O'Sullivan
| Cc: Jake McArthur; [hidden email]
| Subject: Re: Unboxed Vectors of newtype'd values
|
| On Wed, May 30, 2012 at 9:50 AM, Bryan O'Sullivan <[hidden email]>
| wrote:
| > Likewise, if anyone is listening :-)
|
| Here too.
|
| I'd like some compiler support for abstracting over unboxed values in
| general and e.g. generate Unbox instances automatically. Then we could
| have other unboxed structures than Vector.
|
| _______________________________________________
| Libraries mailing list
| [hidden email]
| http://www.haskell.org/mailman/listinfo/libraries



_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Bas van Dijk-2
On 4 June 2012 12:43, Simon Peyton-Jones <[hidden email]> wrote:
> | I'd like some compiler support for abstracting over unboxed values in
> | general and e.g. generate Unbox instances automatically. Then we could
> | have other unboxed structures than Vector.
>
> Could someone post an example or two of the problem being described here?

The following is some boring code I wrote a while back at work for
storing UUIDs[1] in unboxed vectors:

{-# LANGUAGE MultiParamTypeClasses, TypeFamilies #-}

import Control.Monad (liftM)
import Data.Word (Word32)

import Data.UUID (UUID)
import qualified Data.UUID as UUID

import qualified Data.Vector.Unboxed         as VU
import qualified Data.Vector.Generic         as VG
import qualified Data.Vector.Generic.Mutable as VGM

newtype instance VU.MVector s UUID =
        MV_UUID (VU.MVector s (Word32, Word32, Word32, Word32))

newtype instance  VU.Vector UUID =
        V_UUID (VU.Vector (Word32, Word32, Word32, Word32))

instance VU.Unbox UUID

instance VGM.MVector VU.MVector UUID where
  {-# INLINE basicLength #-}
  {-# INLINE basicUnsafeSlice #-}
  {-# INLINE basicOverlaps #-}
  {-# INLINE basicUnsafeNew #-}
  {-# INLINE basicUnsafeReplicate #-}
  {-# INLINE basicUnsafeRead #-}
  {-# INLINE basicUnsafeWrite #-}
  {-# INLINE basicClear #-}
  {-# INLINE basicSet #-}
  {-# INLINE basicUnsafeCopy #-}
  {-# INLINE basicUnsafeGrow #-}
  basicLength (MV_UUID v) =
    VGM.basicLength v

  basicUnsafeSlice i n (MV_UUID v) =
    MV_UUID $ VGM.basicUnsafeSlice i n v

  basicOverlaps (MV_UUID v1) (MV_UUID v2) =
    VGM.basicOverlaps v1 v2

  basicUnsafeNew n =
    MV_UUID `liftM` VGM.basicUnsafeNew n

  basicUnsafeReplicate n uuid =
    MV_UUID `liftM` VGM.basicUnsafeReplicate n (UUID.toWords uuid)

  basicUnsafeRead (MV_UUID v) i =
    fromQuadruple `liftM` VGM.basicUnsafeRead v i

  basicUnsafeWrite (MV_UUID v) i uuid =
    VGM.basicUnsafeWrite v i (UUID.toWords uuid)

  basicClear (MV_UUID v) =
    VGM.basicClear v

  basicSet (MV_UUID v) uuid =
    VGM.basicSet v (UUID.toWords uuid)

  basicUnsafeCopy (MV_UUID v1) (MV_UUID v2) =
    VGM.basicUnsafeCopy v1 v2

  basicUnsafeMove (MV_UUID v1) (MV_UUID v2) =
    VGM.basicUnsafeMove v1 v2

  basicUnsafeGrow (MV_UUID v) n =
    MV_UUID `liftM` VGM.basicUnsafeGrow v n

instance VG.Vector VU.Vector UUID where
  {-# INLINE basicUnsafeFreeze #-}
  {-# INLINE basicUnsafeThaw #-}
  {-# INLINE basicLength #-}
  {-# INLINE basicUnsafeSlice #-}
  {-# INLINE basicUnsafeIndexM #-}
  {-# INLINE elemseq #-}
  basicUnsafeFreeze (MV_UUID v) =
    V_UUID `liftM` VG.basicUnsafeFreeze v

  basicUnsafeThaw (V_UUID v) =
    MV_UUID `liftM` VG.basicUnsafeThaw v

  basicLength (V_UUID v) =
    VG.basicLength v

  basicUnsafeSlice i n (V_UUID v) =
    V_UUID $ VG.basicUnsafeSlice i n v

  basicUnsafeIndexM (V_UUID v) i =
    fromQuadruple `liftM` VG.basicUnsafeIndexM v i

  basicUnsafeCopy (MV_UUID mv) (V_UUID v) =
    VG.basicUnsafeCopy mv v

  elemseq _ uuid z = VG.elemseq (undefined :: VU.Vector a) a
                   $ VG.elemseq (undefined :: VU.Vector a) b
                   $ VG.elemseq (undefined :: VU.Vector a) c
                   $ VG.elemseq (undefined :: VU.Vector a) d z
      where
        (a,b,c,d) = UUID.toWords uuid

fromQuadruple :: (Word32, Word32, Word32, Word32) -> UUID
fromQuadruple (a,b,c,d) = UUID.fromWords a b c d

Regards,

Bas

[1] http://hackage.haskell.org/packages/archive/uuid/1.2.5/doc/html/Data-UUID.html

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Johan Tibell-2
My desire probably doesn't overlap much with what others have
described in this thread, apologies. I *think* they want generalized
newtype deriving to work for Unbox instances.

I want to be able to write definition like this (pseudo code):

  data UnpackedList a = forall a. Unbox a => Cons {-# UNPACK #-} !a
(UnpackedList a) | Nil

and have GHC generate appropriate data type definitions at call sites.
In other words, I want polymorphic unpacking to work as long as the
unpacked field is unboxable (i.e. is member of some Unobx type class.)

I believe I've described this desire to you before. Unfortunately it
seems like a really hard problem.

-- Johan

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: Unboxed Vectors of newtype'd values

Simon Peyton Jones
In reply to this post by Bas van Dijk-2
I see the boring code, but I don't see what you *want*!  I'm guessing you want "generalised newtype deriving" but why does that not work?

S

| -----Original Message-----
| From: Bas van Dijk [mailto:[hidden email]]
| Sent: 04 June 2012 20:41
| To: Simon Peyton-Jones
| Cc: Johan Tibell; Bryan O'Sullivan; Jake McArthur; [hidden email]
| Subject: Re: Unboxed Vectors of newtype'd values
|
| On 4 June 2012 12:43, Simon Peyton-Jones <[hidden email]> wrote:
| > | I'd like some compiler support for abstracting over unboxed values
| > | in general and e.g. generate Unbox instances automatically. Then we
| > | could have other unboxed structures than Vector.
| >
| > Could someone post an example or two of the problem being described
| here?
|
| The following is some boring code I wrote a while back at work for
| storing UUIDs[1] in unboxed vectors:
|
| {-# LANGUAGE MultiParamTypeClasses, TypeFamilies #-}
|
| import Control.Monad (liftM)
| import Data.Word (Word32)
|
| import Data.UUID (UUID)
| import qualified Data.UUID as UUID
|
| import qualified Data.Vector.Unboxed         as VU
| import qualified Data.Vector.Generic         as VG
| import qualified Data.Vector.Generic.Mutable as VGM
|
| newtype instance VU.MVector s UUID =
| MV_UUID (VU.MVector s (Word32, Word32, Word32, Word32))
|
| newtype instance  VU.Vector UUID =
| V_UUID (VU.Vector (Word32, Word32, Word32, Word32))
|
| instance VU.Unbox UUID
|
| instance VGM.MVector VU.MVector UUID where
|   {-# INLINE basicLength #-}
|   {-# INLINE basicUnsafeSlice #-}
|   {-# INLINE basicOverlaps #-}
|   {-# INLINE basicUnsafeNew #-}
|   {-# INLINE basicUnsafeReplicate #-}
|   {-# INLINE basicUnsafeRead #-}
|   {-# INLINE basicUnsafeWrite #-}
|   {-# INLINE basicClear #-}
|   {-# INLINE basicSet #-}
|   {-# INLINE basicUnsafeCopy #-}
|   {-# INLINE basicUnsafeGrow #-}
|   basicLength (MV_UUID v) =
|     VGM.basicLength v
|
|   basicUnsafeSlice i n (MV_UUID v) =
|     MV_UUID $ VGM.basicUnsafeSlice i n v
|
|   basicOverlaps (MV_UUID v1) (MV_UUID v2) =
|     VGM.basicOverlaps v1 v2
|
|   basicUnsafeNew n =
|     MV_UUID `liftM` VGM.basicUnsafeNew n
|
|   basicUnsafeReplicate n uuid =
|     MV_UUID `liftM` VGM.basicUnsafeReplicate n (UUID.toWords uuid)
|
|   basicUnsafeRead (MV_UUID v) i =
|     fromQuadruple `liftM` VGM.basicUnsafeRead v i
|
|   basicUnsafeWrite (MV_UUID v) i uuid =
|     VGM.basicUnsafeWrite v i (UUID.toWords uuid)
|
|   basicClear (MV_UUID v) =
|     VGM.basicClear v
|
|   basicSet (MV_UUID v) uuid =
|     VGM.basicSet v (UUID.toWords uuid)
|
|   basicUnsafeCopy (MV_UUID v1) (MV_UUID v2) =
|     VGM.basicUnsafeCopy v1 v2
|
|   basicUnsafeMove (MV_UUID v1) (MV_UUID v2) =
|     VGM.basicUnsafeMove v1 v2
|
|   basicUnsafeGrow (MV_UUID v) n =
|     MV_UUID `liftM` VGM.basicUnsafeGrow v n
|
| instance VG.Vector VU.Vector UUID where
|   {-# INLINE basicUnsafeFreeze #-}
|   {-# INLINE basicUnsafeThaw #-}
|   {-# INLINE basicLength #-}
|   {-# INLINE basicUnsafeSlice #-}
|   {-# INLINE basicUnsafeIndexM #-}
|   {-# INLINE elemseq #-}
|   basicUnsafeFreeze (MV_UUID v) =
|     V_UUID `liftM` VG.basicUnsafeFreeze v
|
|   basicUnsafeThaw (V_UUID v) =
|     MV_UUID `liftM` VG.basicUnsafeThaw v
|
|   basicLength (V_UUID v) =
|     VG.basicLength v
|
|   basicUnsafeSlice i n (V_UUID v) =
|     V_UUID $ VG.basicUnsafeSlice i n v
|
|   basicUnsafeIndexM (V_UUID v) i =
|     fromQuadruple `liftM` VG.basicUnsafeIndexM v i
|
|   basicUnsafeCopy (MV_UUID mv) (V_UUID v) =
|     VG.basicUnsafeCopy mv v
|
|   elemseq _ uuid z = VG.elemseq (undefined :: VU.Vector a) a
|                    $ VG.elemseq (undefined :: VU.Vector a) b
|                    $ VG.elemseq (undefined :: VU.Vector a) c
|                    $ VG.elemseq (undefined :: VU.Vector a) d z
|       where
|         (a,b,c,d) = UUID.toWords uuid
|
| fromQuadruple :: (Word32, Word32, Word32, Word32) -> UUID fromQuadruple
| (a,b,c,d) = UUID.fromWords a b c d
|
| Regards,
|
| Bas
|
| [1]
| http://hackage.haskell.org/packages/archive/uuid/1.2.5/doc/html/Data-
| UUID.html

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: Unboxed Vectors of newtype'd values

Simon Peyton Jones
In reply to this post by Johan Tibell-2
I'm sorry I'm still struggling.  In your example below you bind 'a' twice, once with the "data UnpackedList a" and once with the "forall a".  Did you intend an existential?

Also I don't know what you mean by "generate appropriate type definitions at call sites".

I do remember a dinner conversation in Tokyo where I couldn't figure out how to implement what you wanted, but I've forgotten the details.  Maybe it's worth a wiki page to explain -- or maybe not if its un-implementable.

Anyway thanks for distinguishing the two threads.  If you can clarify the generalised newtype deriving problem too that would be great.

Simon

| -----Original Message-----
| From: Johan Tibell [mailto:[hidden email]]
| Sent: 04 June 2012 20:58
| To: Bas van Dijk
| Cc: Simon Peyton-Jones; Bryan O'Sullivan; Jake McArthur;
| [hidden email]
| Subject: Re: Unboxed Vectors of newtype'd values
|
| My desire probably doesn't overlap much with what others have described
| in this thread, apologies. I *think* they want generalized newtype
| deriving to work for Unbox instances.
|
| I want to be able to write definition like this (pseudo code):
|
|   data UnpackedList a = forall a. Unbox a => Cons {-# UNPACK #-} !a
| (UnpackedList a) | Nil
|
| and have GHC generate appropriate data type definitions at call sites.
| In other words, I want polymorphic unpacking to work as long as the
| unpacked field is unboxable (i.e. is member of some Unobx type class.)
|
| I believe I've described this desire to you before. Unfortunately it
| seems like a really hard problem.
|
| -- Johan



_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Johan Tibell-2
On Tue, Jun 5, 2012 at 12:06 AM, Simon Peyton-Jones
<[hidden email]> wrote:
> I'm sorry I'm still struggling.  In your example below you bind 'a' twice, once with the "data UnpackedList a" and once with the "forall a".  Did you intend an existential?

I didn't intend the existential. I haven't made any real progress on
my unpacking problem since we last talked so lets let it rest for now.
I'll try to find some time to write down a wiki page so we at least
know where we got stuck last time. Perhaps someone reading it will
have an aha moment that will let us move forward.

> Also I don't know what you mean by "generate appropriate type definitions at call sites".

If I write:

    module A

    data UnpackedList a = Unbox a => Cons {-# UNPACK #-} !a
(UnpackedList a) | Nil
    { -# INLINABLE_DATA UnpackedList #-}  -- Like INLINABLE, but for data types

    module B

    f :: UnpackedList Int -> ...

I'd like GHC to generate

    data UnpackedListInt = ConsInt {-# UNPACK #-} !Int UnpackedListInt | NilInt

and a specialized version of f, using that type, in B. It's a bit like
the call-site specialization afforded by the INLINABLE pragma, but
applied also to data type definitions.

You might protest and say that I could just just use type families,
but they don't work well if my type has more than one type parameter
(e.g. Map), as I need to write O(n^2) instances:

instance Map Int Int where
    type Map = MapIntInt ...{-# UNPACK #-} !Int {-# UNPACK #-} !Int...

instance Map Int Char where
    type Map = MapIntChar ...

instance Map Int (Int, Int) where
   type Map = MapIntIntInt ...

That's 50,625 instances to just cover the basic types (e.g. Int, Word,
etc) and pairs. The reason you'd want call-site generation of data
types (ala C++ templates) is that a real program would only use no
more than 100 (for a big program) of the potentially infinite number
of combinations.

> Anyway thanks for distinguishing the two threads.  If you can clarify the generalised newtype deriving problem too that would be great.

I'll leave it to others. I'm not quite sure what the issue is.

-- Johan

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Bryan O'Sullivan
In reply to this post by Simon Peyton Jones
On Tue, Jun 5, 2012 at 12:02 AM, Simon Peyton-Jones <[hidden email]> wrote:
I see the boring code, but I don't see what you *want*!  I'm guessing you want "generalised newtype deriving" but why does that not work?

Yes, I (and I assume Bas) want generalised newtype deriving to work, but it doesn't.

I want to write something very simple:

{-# LANGUAGE GeneralizedNewtypeDeriving #-}
newtype Foo = Foo Int
    deriving (Eq, Show, Unbox)

But with the above, GHC says:

    No instances for (M.MVector MVector Foo, G.Vector Vector Foo)
      arising from the 'deriving' clause of a data type declaration

And then we begin the journey that eventually gets us to Bas's rather wordy code.

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Andres Löh
Hi Bryan.

> Yes, I (and I assume Bas) want generalised newtype deriving to work, but it
> doesn't.
>
> I want to write something very simple:
>
> {-# LANGUAGE GeneralizedNewtypeDeriving #-}
> newtype Foo = Foo Int
>     deriving (Eq, Show, Unbox)
>
> But with the above, GHC says:
>
>     No instances for (M.MVector MVector Foo, G.Vector Vector Foo)
>       arising from the 'deriving' clause of a data type declaration

Yes, because these are superclasses of Unbox. So can't you simply say this:

> newtype Foo = Foo Int
>   deriving (Eq, Show, Unbox, M.MVector MVector, G.Vector Vector)

?

Cheers,
  Andres

--
Andres Löh, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Daniel Peebles
Yep, that works! Sweet!

On Tue, Jun 5, 2012 at 3:54 PM, Andres Löh <[hidden email]> wrote:
Hi Bryan.

> Yes, I (and I assume Bas) want generalised newtype deriving to work, but it
> doesn't.
>
> I want to write something very simple:
>
> {-# LANGUAGE GeneralizedNewtypeDeriving #-}
> newtype Foo = Foo Int
>     deriving (Eq, Show, Unbox)
>
> But with the above, GHC says:
>
>     No instances for (M.MVector MVector Foo, G.Vector Vector Foo)
>       arising from the 'deriving' clause of a data type declaration

Yes, because these are superclasses of Unbox. So can't you simply say this:

> newtype Foo = Foo Int
>   deriving (Eq, Show, Unbox, M.MVector MVector, G.Vector Vector)

?

Cheers,
 Andres

--
Andres Löh, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: Unboxed Vectors of newtype'd values

Simon Peyton Jones
In reply to this post by Bryan O'Sullivan

So Andres has explained how to do what Johan asks.  Does that mean that Bas’s problem is solved?

 

S

 

From: Bryan O'Sullivan [mailto:[hidden email]]
Sent: 05 June 2012 18:57
To: Simon Peyton-Jones
Cc: Bas van Dijk; Johan Tibell; Jake McArthur; [hidden email]
Subject: Re: Unboxed Vectors of newtype'd values

 

On Tue, Jun 5, 2012 at 12:02 AM, Simon Peyton-Jones <[hidden email]> wrote:

I see the boring code, but I don't see what you *want*!  I'm guessing you want "generalised newtype deriving" but why does that not work?

 

Yes, I (and I assume Bas) want generalised newtype deriving to work, but it doesn't.

 

I want to write something very simple:

 

{-# LANGUAGE GeneralizedNewtypeDeriving #-}

newtype Foo = Foo Int

    deriving (Eq, Show, Unbox)

 

But with the above, GHC says:

 

    No instances for (M.MVector MVector Foo, G.Vector Vector Foo)

      arising from the 'deriving' clause of a data type declaration

 

And then we begin the journey that eventually gets us to Bas's rather wordy code.


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Bryan O'Sullivan
On Tue, Jun 5, 2012 at 1:45 PM, Simon Peyton-Jones <[hidden email]> wrote:

So Andres has explained how to do what Johan asks.  Does that mean that Bas’s problem is solved?


I think so - thanks, Andres! I believe that Johan's problem is another beast entirely. 

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Unboxed Vectors of newtype'd values

Johan Tibell-2
On Tue, Jun 5, 2012 at 1:46 PM, Bryan O'Sullivan <[hidden email]> wrote:
> I think so - thanks, Andres! I believe that Johan's problem is another beast
> entirely.

Just to make it clear why I butted into this thread to begin with
(apologies for that.)

I also want a deriving Unbox mechanism, but with a more magic Unbox
class (e.g. one that knew enough about the representation of a data
type to unpack it.) That was the connection.

-- Johan

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
12