primitive library

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

primitive library

Haskell - Libraries mailing list

Andrew, David

I’ve seen a lot of traffic about the primitive library, in which you two are playing a leading role.

Clearly something interesting is going on, but I have not been paying enough attention to work out what.  Maybe lots of unrelated things?  Maybe a handful of closely related things?

Would you consider putting out a summary (to libraries and ghc-devs) to give an overview of the main threads, and any driving motivations.  Why has all this blown up now?

Meanwhile, thank you for being so active.

Simon


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: primitive library

Andrew Martin
I'll get something together soon that explains my motivations. The short summary is that the types in primitive unpack better that their counterparts in vector (meaning, they unpack into one machine word instead of three). I started using these types in internal data structures in libraries I would write (since I didn't need slicing), and then just started noticing stuff I thought it would be nice to add.

David noticed that a bunch of the typeclass instances were broken. I'd been working on a library quickcheck-classes that I use to test instances at my work place. The primitive library was missing a test suite, so I used it to test all the instances to ensure that the fixes David had written were correct. In the process, I found more broken instances and fixed them. The life lesson here is that property testing is important.

PrimArray is an interesting story. Both winterland1989 and I had independently written libraries that did the same exact thing: implement a typed interface to ByteArray that keeps track of the element type. This makes PrimArray much safer to work with than ByteArray. Eventually, winterland's initial PR to bring this to primitive stalled. It implemented several other features (some of which may still eventually get added), but it's scope was large enough that no maintainer was able to feel comfortable approving it. More recently, I took at stab at doing the same thing, but I only added PrimArray, and after a lot of feedback from David, Carter, and Ryan, it got merged in.

This got longer than I thought it would. I'll work on something that talks more about motivations and features soon.

On Mon, Apr 23, 2018 at 5:48 AM, Simon Peyton Jones <[hidden email]> wrote:

Andrew, David

I’ve seen a lot of traffic about the primitive library, in which you two are playing a leading role.

Clearly something interesting is going on, but I have not been paying enough attention to work out what.  Maybe lots of unrelated things?  Maybe a handful of closely related things?

Would you consider putting out a summary (to libraries and ghc-devs) to give an overview of the main threads, and any driving motivations.  Why has all this blown up now?

Meanwhile, thank you for being so active.

Simon




--
-Andrew Thaddeus Martin

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: primitive library

David Feuer
I think the areas in the primitive library demonstrate a part of the array design space that seems to have gone relatively unexplored in Haskell: plain old vectors. Unlike the vector library, there is no stream fusion framework. Unlike the array library, there is no class-based reliance on fold/build fusion.

The (flawed) addition of numerous class instances a couple versions ago made primitive arrays seem much more viable to end users as an alternative to their heavier counterparts. I have become interested in fixing the mistakes that were made, seeing just how much performance we can squeeze out (and what limitations we run into), and fleshing out the API.

On Mon, Apr 23, 2018, 9:20 AM Andrew Martin <[hidden email]> wrote:
I'll get something together soon that explains my motivations. The short summary is that the types in primitive unpack better that their counterparts in vector (meaning, they unpack into one machine word instead of three). I started using these types in internal data structures in libraries I would write (since I didn't need slicing), and then just started noticing stuff I thought it would be nice to add.

David noticed that a bunch of the typeclass instances were broken. I'd been working on a library quickcheck-classes that I use to test instances at my work place. The primitive library was missing a test suite, so I used it to test all the instances to ensure that the fixes David had written were correct. In the process, I found more broken instances and fixed them. The life lesson here is that property testing is important.

PrimArray is an interesting story. Both winterland1989 and I had independently written libraries that did the same exact thing: implement a typed interface to ByteArray that keeps track of the element type. This makes PrimArray much safer to work with than ByteArray. Eventually, winterland's initial PR to bring this to primitive stalled. It implemented several other features (some of which may still eventually get added), but it's scope was large enough that no maintainer was able to feel comfortable approving it. More recently, I took at stab at doing the same thing, but I only added PrimArray, and after a lot of feedback from David, Carter, and Ryan, it got merged in.

This got longer than I thought it would. I'll work on something that talks more about motivations and features soon.

On Mon, Apr 23, 2018 at 5:48 AM, Simon Peyton Jones <[hidden email]> wrote:

Andrew, David

I’ve seen a lot of traffic about the primitive library, in which you two are playing a leading role.

Clearly something interesting is going on, but I have not been paying enough attention to work out what.  Maybe lots of unrelated things?  Maybe a handful of closely related things?

Would you consider putting out a summary (to libraries and ghc-devs) to give an overview of the main threads, and any driving motivations.  Why has all this blown up now?

Meanwhile, thank you for being so active.

Simon




--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: primitive library

Haskell - Libraries mailing list
In reply to this post by Andrew Martin

This got longer than I thought it would. I'll work on something that talks more about motivations and features soon.

 

That would be great, thanks.  It would also mean that more people would use the new stuff, sooner.

 

Simon

 

From: Andrew Martin <[hidden email]>
Sent: 23 April 2018 14:20
To: Simon Peyton Jones <[hidden email]>
Cc: David Feuer <[hidden email]>; ghc-devs <[hidden email]>; Haskell Libraries <[hidden email]>
Subject: Re: primitive library

 

I'll get something together soon that explains my motivations. The short summary is that the types in primitive unpack better that their counterparts in vector (meaning, they unpack into one machine word instead of three). I started using these types in internal data structures in libraries I would write (since I didn't need slicing), and then just started noticing stuff I thought it would be nice to add.

 

David noticed that a bunch of the typeclass instances were broken. I'd been working on a library quickcheck-classes that I use to test instances at my work place. The primitive library was missing a test suite, so I used it to test all the instances to ensure that the fixes David had written were correct. In the process, I found more broken instances and fixed them. The life lesson here is that property testing is important.

 

PrimArray is an interesting story. Both winterland1989 and I had independently written libraries that did the same exact thing: implement a typed interface to ByteArray that keeps track of the element type. This makes PrimArray much safer to work with than ByteArray. Eventually, winterland's initial PR to bring this to primitive stalled. It implemented several other features (some of which may still eventually get added), but it's scope was large enough that no maintainer was able to feel comfortable approving it. More recently, I took at stab at doing the same thing, but I only added PrimArray, and after a lot of feedback from David, Carter, and Ryan, it got merged in.

 

This got longer than I thought it would. I'll work on something that talks more about motivations and features soon.

 

On Mon, Apr 23, 2018 at 5:48 AM, Simon Peyton Jones <[hidden email]> wrote:

Andrew, David

I’ve seen a lot of traffic about the primitive library, in which you two are playing a leading role.

Clearly something interesting is going on, but I have not been paying enough attention to work out what.  Maybe lots of unrelated things?  Maybe a handful of closely related things?

Would you consider putting out a summary (to libraries and ghc-devs) to give an overview of the main threads, and any driving motivations.  Why has all this blown up now?

Meanwhile, thank you for being so active.

Simon



 

--

-Andrew Thaddeus Martin


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries