Storing lifted and unlifted pointer types in the same Array#

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Storing lifted and unlifted pointer types in the same Array#

Reiner Pope
Hi libraries@,

In GHC.Prim, it is documented that Array# and ArrayArray# have the same runtime representation. I'd like to exploit this to put unlifted and lifted pointers into the same array (for example, unlifted pointers at even indices and lifted pointers at odd indices). The reason I want to do this is to save some array header words, and also to improve the cache locality of my data, so that a read of a (lifted, unlifted) pointer pair only needs to touch a single cache line rather than two.

I believe it is safe to do this (put unlifted and lifted pointers in the same array), as long as I always read at the same liftedness and type that I ran the write at for that index. Can you confirm?

Specifically, I'm expecting to use a code pattern like the following, to read an unlifted value from an Array#, by coercing the Array# to ArrayArray#, then doing the unlifted read there:

coerceToArrayArray :: Array# a -> ArrayArray#
coerceToArrayArray = unsafeCoerce#

coerceToUnlifted :: forall (b :: TYPE 'UnliftedRep). ArrayArray# -> b
coerceToUnlifted = unsafeCoerce#

indexUnliftedFromArray ::
  forall a (b :: TYPE 'UnliftedRep). Array# a -> Int# -> b
indexUnliftedFromArray arr i =
  coerceToUnlifted (indexArrayArrayArray# (coerceToArrayArray arr) i)

Similarly, I'm expecting to do writes by coercing the array to a MutableArrayArray# and doing the writes to that.

Reiner

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: Storing lifted and unlifted pointer types in the same Array#

Haskell - Libraries mailing list

. I'd like to exploit this to put unlifted and lifted pointers into the same array

 

I think this’ll be ok provided both are pointers. You cannot put an Int# in place of a pointer (whether the latter is lifted or unlifted) or you’ll crash the GC.

 

So your type for coerceToUnlifted looks too general to me.

 

But I have not devoted enough time to be certain.

 

Hope this helps a little

 

Simon

 

 

From: Libraries [mailto:[hidden email]] On Behalf Of Reiner Pope
Sent: 17 November 2017 05:38
To: [hidden email]
Subject: Storing lifted and unlifted pointer types in the same Array#

 

Hi libraries@,

 

In GHC.Prim, it is documented that Array# and ArrayArray# have the same runtime representation. I'd like to exploit this to put unlifted and lifted pointers into the same array (for example, unlifted pointers at even indices and lifted pointers at odd indices). The reason I want to do this is to save some array header words, and also to improve the cache locality of my data, so that a read of a (lifted, unlifted) pointer pair only needs to touch a single cache line rather than two.

 

I believe it is safe to do this (put unlifted and lifted pointers in the same array), as long as I always read at the same liftedness and type that I ran the write at for that index. Can you confirm?

 

Specifically, I'm expecting to use a code pattern like the following, to read an unlifted value from an Array#, by coercing the Array# to ArrayArray#, then doing the unlifted read there:

 

coerceToArrayArray :: Array# a -> ArrayArray#

coerceToArrayArray = unsafeCoerce#

 

coerceToUnlifted :: forall (b :: TYPE 'UnliftedRep). ArrayArray# -> b

coerceToUnlifted = unsafeCoerce#

 

indexUnliftedFromArray ::

  forall a (b :: TYPE 'UnliftedRep). Array# a -> Int# -> b

indexUnliftedFromArray arr i =

  coerceToUnlifted (indexArrayArrayArray# (coerceToArrayArray arr) i)

 

Similarly, I'm expecting to do writes by coercing the array to a MutableArrayArray# and doing the writes to that.

 

Reiner


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Storing lifted and unlifted pointer types in the same Array#

winter
I have been thinking about this problem some time ago, and I think it should be possible. What Reiner wants is a way to store boxed values(that means they may be unlifted / lifted mixed)  into the same array. For example, to implement HMAT efficiently, we may want to mix `ArrayArray# Node`(which are pointers pointing to next level) and some boxed value `a` into a same `ArrayArray# Node`.  But I don’t think the API proposed by Reiner is ideal, adding new primitive types like `MixedArray# a b / MixedArray# a b c …` may works better.

Note currently we can store unlifted boxed value already, primitive package use unsafeCoerce# to achieve this, i.e. the `PrimUnlifted` class. But we can’t mix boxed unlifted value and boxed lifted value.

Cheers~
Winter





On 17 Nov 2017, at 7:13 PM, Simon Peyton Jones via Libraries <[hidden email]> wrote:

. I'd like to exploit this to put unlifted and lifted pointers into the same array
 
I think this’ll be ok provided both are pointers. You cannot put an Int# in place of a pointer (whether the latter is lifted or unlifted) or you’ll crash the GC.
 
So your type for coerceToUnlifted looks too general to me.
 
But I have not devoted enough time to be certain.
 
Hope this helps a little
 
Simon
 
 
From: Libraries [[hidden email]] On Behalf Of Reiner Pope
Sent: 17 November 2017 05:38
To: [hidden email]
Subject: Storing lifted and unlifted pointer types in the same Array#
 

Hi libraries@,

 

In GHC.Prim, it is documented that Array# and ArrayArray# have the same runtime representation. I'd like to exploit this to put unlifted and lifted pointers into the same array (for example, unlifted pointers at even indices and lifted pointers at odd indices). The reason I want to do this is to save some array header words, and also to improve the cache locality of my data, so that a read of a (lifted, unlifted) pointer pair only needs to touch a single cache line rather than two.

 

I believe it is safe to do this (put unlifted and lifted pointers in the same array), as long as I always read at the same liftedness and type that I ran the write at for that index. Can you confirm?

 

Specifically, I'm expecting to use a code pattern like the following, to read an unlifted value from an Array#, by coercing the Array# to ArrayArray#, then doing the unlifted read there:

 

coerceToArrayArray :: Array# a -> ArrayArray#

coerceToArrayArray = unsafeCoerce#

 

coerceToUnlifted :: forall (b :: TYPE 'UnliftedRep). ArrayArray# -> b

coerceToUnlifted = unsafeCoerce#

 

indexUnliftedFromArray ::

  forall a (b :: TYPE 'UnliftedRep). Array# a -> Int# -> b

indexUnliftedFromArray arr i =

  coerceToUnlifted (indexArrayArrayArray# (coerceToArrayArray arr) i)

 

Similarly, I'm expecting to do writes by coercing the array to a MutableArrayArray# and doing the writes to that.

 

Reiner

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries