Proposal: Move primitive-Data.Primitive.Addr API into base

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
87 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
@danny... storable is only for fixed size pinned memory values

On Mon, Oct 29, 2018 at 8:31 PM Dannyu NDos <[hidden email]> wrote:
It is Storable that makes Ptr operate as typed pointers. I also think Ptr on its own doesn't have that much about its argument type, so I'm +1 on the proposal.

Btw, shouldn't every type be storable? In current Haskell, Maybes or (->)s aren't Storable, yet in C++, arrays of std::optionals or std::functions are well-defined.

As an exception, for Void, I agree that they must remain not Storable since it has no values.

2018년 10월 30일 (화) 08:31, Carter Schonwald <[hidden email]>님이 작성:
the parametricity isn't for when you know things, its for saying "these are possibly different or possibly the same, dont let me mix them up though, cause I dont know yet"


On Mon, Oct 29, 2018 at 7:03 PM Daniel Cartwright <[hidden email]> wrote:
I'm not sure that argument applies at all, when talking about _incorrect_ usages of Ptr. Sure, Addr probably shouldn't be used when there is meaningful type information/value to recover, but neither should Ptr be used when there is none.

The argument being made is not to make 'better', per se, and there definitely won't be a 'mathematical statement' about this, but it certainly may be made clearer - in my opinion, the usages of 'Ptr' that i've already brought up are inherently unclear because of the bogus phantom type associated with 'Ptr'. The illustration of this begs no code that doesn't already exist in corelibs.


On Mon, Oct 29, 2018 at 6:19 PM Carter Schonwald <[hidden email]> wrote:
to zoom out: what code is improved? what code is made better/clearer? No one has articulated this clearly. 

The one example of Addr being used in Vector.Storable.Mutable is not an argument in favor of using Addr. Its an argument against it existing.

i'm looking for evidence, in the form of code i can look at then say "yes, this is better code" when comparing the two. Or a mathematical statement of "what is made better"

[hidden email] , @Daniel , do you have one?

when i'm writing complicated code, MORE polymorphism helps me usually.

I can write some code like the following and even though I'm using it with Int at argument,
I *Know* that i'm not mixing up arguments/values that i write as different types. I cannot do this with Address!

{-# SPECIALIZE INLINE computeStarts :: [(Int,Int)]->Int->Int ->[(Int,Int)] #-}
computeStarts:: (Enum a, Ord a, Num b )=>[(a,b)]-> a -> a -> [(a,b)]

parametricity (even when constrained by type classes) is a powerful and foundational tool for good programming in haskell and similar languages

there has been nothing stated here that successfully articulates a good reason to forgo/discourage parametricity as an engineering tool. for thats what Addr is. 
A datatype thats never safe in isolation, and discourages using parametricity to write correct software.

a very strong case is needed to forgo parametricity. 




On Mon, Oct 29, 2018 at 5:33 PM David Feuer <[hidden email]> wrote:
Good point! Call it nominal then.

On Mon, Oct 29, 2018, 5:24 PM Carter Schonwald <[hidden email]> wrote:
absolutely false, represeentational equality of the  type a in `Ptr a` does not mean the memory representation at the corresponding address is the same.
(it sometimes is true, but memory packing/alignment details in structs in C  for otherwise equivlanet structs should rule this out)

aka, `a` being representationally equal to `b` via haskell newtypes does not mean the memory representation at `Ptr a`, and `Ptr b` are the same. a trivial example is when
host and network byte order aren't the same (eg big vs little endian memory encodings)

On Mon, Oct 29, 2018 at 12:28 PM David Feuer <[hidden email]> wrote:
What? Of course you can dereference it. You dereference it, getting a
value of type `Void`,
and apply absurd to get whatever you want in the world. This, of
course, is utter nonsense,
unless *having* the Ptr Void means that something has already gone
wrong. It's pretty
hard for me to imagine a situation where this is actually what you
want. A Ptr () isn't nonsense.
It is not terrible to use Ptr () to represent an Addr, but I wonder if
it sends the wrong message.
By the way: there's another argument for having Addr in base for now.
We would really
*like* for Ptr's parameter to have a *representational* role, but we
*don't* want to require
unsafeCoerce to cast Ptrs. The solution to that in the current role system:

    data Addr = Addr Addr#

    newtype Ptr a = Ptr_ Addr
    type role Ptr representational

    pattern Ptr :: Addr# -> Ptr a
    pattern Ptr addr# = Ptr_ (Addr addr#)

    -- Allow users to reveal coercibility of pointer types locally
    ptrCoercion :: Coercion (Ptr a) (Ptr b)
    ptrCoercion = Coercion

    castPtr :: Ptr a -> Ptr b
    castPtr = coerceWith ptrCoercion -- (or the now-free unwrap-rewrap
definition)


So even if we don't *expose* Addr in base, we should almost certainly *define*
it there.
On Mon, Oct 29, 2018 at 12:11 PM Carter Schonwald
<[hidden email]> wrote:
>
> The point , hahah, of a Ptr void is that you can’t dereference it.  But you certainly can cast it and do address arithmetic on it!!
>
>
>
> On Mon, Oct 29, 2018 at 10:10 AM David Feuer <[hidden email]> wrote:
>>
>> On Mon, Oct 29, 2018, 10:05 AM Sven Panne <[hidden email]> wrote:
>>>
>>> Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright <[hidden email]>:
>>>>
>>>> 'Ptr Void' is not a pointer to a value of type 'Void'; there are no values of type 'Void': this type is nonsensical.
>>>
>>>
>>> That's the whole point, and it actually makes sense: If you see "Ptr Void", you can't do much with it, apart from passing it around or using castPtr on it. This is exactly what should be achieved by using "Ptr Void" in an API. This is basically the same as "void *" in C/C++.
>>
>>
>> No, it does not make sense. The approximate equivalent of C's void* is Ptr Any. Ptr Void promises to give you anything you want on dereference, which is nonsense.
>>
>>>
>>> You can't store or read "()", so the same holds as for Void (which didn't exist when the FFI was created IIRC).
>>
>>
>> Sure you can. Storing () does nothing and reading it gives (). Our () is somewhat similar to C's void return type.
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Dannyu NDos
You mean Nothing and (Just 0) have actually different sizes? That's so weird for a former C++ programmer.

2018년 10월 30일 (화) 09:35, Carter Schonwald <[hidden email]>님이 작성:
@danny... storable is only for fixed size pinned memory values

On Mon, Oct 29, 2018 at 8:31 PM Dannyu NDos <[hidden email]> wrote:
It is Storable that makes Ptr operate as typed pointers. I also think Ptr on its own doesn't have that much about its argument type, so I'm +1 on the proposal.

Btw, shouldn't every type be storable? In current Haskell, Maybes or (->)s aren't Storable, yet in C++, arrays of std::optionals or std::functions are well-defined.

As an exception, for Void, I agree that they must remain not Storable since it has no values.

2018년 10월 30일 (화) 08:31, Carter Schonwald <[hidden email]>님이 작성:
the parametricity isn't for when you know things, its for saying "these are possibly different or possibly the same, dont let me mix them up though, cause I dont know yet"


On Mon, Oct 29, 2018 at 7:03 PM Daniel Cartwright <[hidden email]> wrote:
I'm not sure that argument applies at all, when talking about _incorrect_ usages of Ptr. Sure, Addr probably shouldn't be used when there is meaningful type information/value to recover, but neither should Ptr be used when there is none.

The argument being made is not to make 'better', per se, and there definitely won't be a 'mathematical statement' about this, but it certainly may be made clearer - in my opinion, the usages of 'Ptr' that i've already brought up are inherently unclear because of the bogus phantom type associated with 'Ptr'. The illustration of this begs no code that doesn't already exist in corelibs.


On Mon, Oct 29, 2018 at 6:19 PM Carter Schonwald <[hidden email]> wrote:
to zoom out: what code is improved? what code is made better/clearer? No one has articulated this clearly. 

The one example of Addr being used in Vector.Storable.Mutable is not an argument in favor of using Addr. Its an argument against it existing.

i'm looking for evidence, in the form of code i can look at then say "yes, this is better code" when comparing the two. Or a mathematical statement of "what is made better"

[hidden email] , @Daniel , do you have one?

when i'm writing complicated code, MORE polymorphism helps me usually.

I can write some code like the following and even though I'm using it with Int at argument,
I *Know* that i'm not mixing up arguments/values that i write as different types. I cannot do this with Address!

{-# SPECIALIZE INLINE computeStarts :: [(Int,Int)]->Int->Int ->[(Int,Int)] #-}
computeStarts:: (Enum a, Ord a, Num b )=>[(a,b)]-> a -> a -> [(a,b)]

parametricity (even when constrained by type classes) is a powerful and foundational tool for good programming in haskell and similar languages

there has been nothing stated here that successfully articulates a good reason to forgo/discourage parametricity as an engineering tool. for thats what Addr is. 
A datatype thats never safe in isolation, and discourages using parametricity to write correct software.

a very strong case is needed to forgo parametricity. 




On Mon, Oct 29, 2018 at 5:33 PM David Feuer <[hidden email]> wrote:
Good point! Call it nominal then.

On Mon, Oct 29, 2018, 5:24 PM Carter Schonwald <[hidden email]> wrote:
absolutely false, represeentational equality of the  type a in `Ptr a` does not mean the memory representation at the corresponding address is the same.
(it sometimes is true, but memory packing/alignment details in structs in C  for otherwise equivlanet structs should rule this out)

aka, `a` being representationally equal to `b` via haskell newtypes does not mean the memory representation at `Ptr a`, and `Ptr b` are the same. a trivial example is when
host and network byte order aren't the same (eg big vs little endian memory encodings)

On Mon, Oct 29, 2018 at 12:28 PM David Feuer <[hidden email]> wrote:
What? Of course you can dereference it. You dereference it, getting a
value of type `Void`,
and apply absurd to get whatever you want in the world. This, of
course, is utter nonsense,
unless *having* the Ptr Void means that something has already gone
wrong. It's pretty
hard for me to imagine a situation where this is actually what you
want. A Ptr () isn't nonsense.
It is not terrible to use Ptr () to represent an Addr, but I wonder if
it sends the wrong message.
By the way: there's another argument for having Addr in base for now.
We would really
*like* for Ptr's parameter to have a *representational* role, but we
*don't* want to require
unsafeCoerce to cast Ptrs. The solution to that in the current role system:

    data Addr = Addr Addr#

    newtype Ptr a = Ptr_ Addr
    type role Ptr representational

    pattern Ptr :: Addr# -> Ptr a
    pattern Ptr addr# = Ptr_ (Addr addr#)

    -- Allow users to reveal coercibility of pointer types locally
    ptrCoercion :: Coercion (Ptr a) (Ptr b)
    ptrCoercion = Coercion

    castPtr :: Ptr a -> Ptr b
    castPtr = coerceWith ptrCoercion -- (or the now-free unwrap-rewrap
definition)


So even if we don't *expose* Addr in base, we should almost certainly *define*
it there.
On Mon, Oct 29, 2018 at 12:11 PM Carter Schonwald
<[hidden email]> wrote:
>
> The point , hahah, of a Ptr void is that you can’t dereference it.  But you certainly can cast it and do address arithmetic on it!!
>
>
>
> On Mon, Oct 29, 2018 at 10:10 AM David Feuer <[hidden email]> wrote:
>>
>> On Mon, Oct 29, 2018, 10:05 AM Sven Panne <[hidden email]> wrote:
>>>
>>> Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright <[hidden email]>:
>>>>
>>>> 'Ptr Void' is not a pointer to a value of type 'Void'; there are no values of type 'Void': this type is nonsensical.
>>>
>>>
>>> That's the whole point, and it actually makes sense: If you see "Ptr Void", you can't do much with it, apart from passing it around or using castPtr on it. This is exactly what should be achieved by using "Ptr Void" in an API. This is basically the same as "void *" in C/C++.
>>
>>
>> No, it does not make sense. The approximate equivalent of C's void* is Ptr Any. Ptr Void promises to give you anything you want on dereference, which is nonsense.
>>
>>>
>>> You can't store or read "()", so the same holds as for Void (which didn't exist when the FFI was created IIRC).
>>
>>
>> Sure you can. Storing () does nothing and reading it gives (). Our () is somewhat similar to C's void return type.
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Edward Kmett-2
In reply to this post by Dannyu NDos
On Mon, Oct 29, 2018 at 8:31 PM Dannyu NDos <[hidden email]> wrote:
Btw, shouldn't every type be storable? In current Haskell, Maybes or (->)s aren't Storable, yet in C++, arrays of std::optionals or std::functions are well-defined.

Consider [a]. There is no fixed size representation of an entire list of elements. There is no supplementary memory management of auxillary stuff being managed by Storable instances. 

std::function holds onto its own data off to the side for the contexts of each functor.

We have StablePtrs which give you a way to marshal any Haskell data type out to C, including functions if you actually need it. However, it pins the haskell object down on the heap, because we have no insight into what happens to it once it goes out into C.

Maybe is just one extreme where you _could_ make up some convention and tagging scheme and store it in-place, same with Either. However, Storable makes no judgment about how to compose storables even for products, because there are many reasonable schemes for packing (or not) the data structures involved, and no real hope of ensuring you comply with whatever the host compiler's ABI might be.

-Edward

As an exception, for Void, I agree that they must remain not Storable since it has no values.

2018년 10월 30일 (화) 08:31, Carter Schonwald <[hidden email]>님이 작성:
the parametricity isn't for when you know things, its for saying "these are possibly different or possibly the same, dont let me mix them up though, cause I dont know yet"


On Mon, Oct 29, 2018 at 7:03 PM Daniel Cartwright <[hidden email]> wrote:
I'm not sure that argument applies at all, when talking about _incorrect_ usages of Ptr. Sure, Addr probably shouldn't be used when there is meaningful type information/value to recover, but neither should Ptr be used when there is none.

The argument being made is not to make 'better', per se, and there definitely won't be a 'mathematical statement' about this, but it certainly may be made clearer - in my opinion, the usages of 'Ptr' that i've already brought up are inherently unclear because of the bogus phantom type associated with 'Ptr'. The illustration of this begs no code that doesn't already exist in corelibs.


On Mon, Oct 29, 2018 at 6:19 PM Carter Schonwald <[hidden email]> wrote:
to zoom out: what code is improved? what code is made better/clearer? No one has articulated this clearly. 

The one example of Addr being used in Vector.Storable.Mutable is not an argument in favor of using Addr. Its an argument against it existing.

i'm looking for evidence, in the form of code i can look at then say "yes, this is better code" when comparing the two. Or a mathematical statement of "what is made better"

[hidden email] , @Daniel , do you have one?

when i'm writing complicated code, MORE polymorphism helps me usually.

I can write some code like the following and even though I'm using it with Int at argument,
I *Know* that i'm not mixing up arguments/values that i write as different types. I cannot do this with Address!

{-# SPECIALIZE INLINE computeStarts :: [(Int,Int)]->Int->Int ->[(Int,Int)] #-}
computeStarts:: (Enum a, Ord a, Num b )=>[(a,b)]-> a -> a -> [(a,b)]

parametricity (even when constrained by type classes) is a powerful and foundational tool for good programming in haskell and similar languages

there has been nothing stated here that successfully articulates a good reason to forgo/discourage parametricity as an engineering tool. for thats what Addr is. 
A datatype thats never safe in isolation, and discourages using parametricity to write correct software.

a very strong case is needed to forgo parametricity. 




On Mon, Oct 29, 2018 at 5:33 PM David Feuer <[hidden email]> wrote:
Good point! Call it nominal then.

On Mon, Oct 29, 2018, 5:24 PM Carter Schonwald <[hidden email]> wrote:
absolutely false, represeentational equality of the  type a in `Ptr a` does not mean the memory representation at the corresponding address is the same.
(it sometimes is true, but memory packing/alignment details in structs in C  for otherwise equivlanet structs should rule this out)

aka, `a` being representationally equal to `b` via haskell newtypes does not mean the memory representation at `Ptr a`, and `Ptr b` are the same. a trivial example is when
host and network byte order aren't the same (eg big vs little endian memory encodings)

On Mon, Oct 29, 2018 at 12:28 PM David Feuer <[hidden email]> wrote:
What? Of course you can dereference it. You dereference it, getting a
value of type `Void`,
and apply absurd to get whatever you want in the world. This, of
course, is utter nonsense,
unless *having* the Ptr Void means that something has already gone
wrong. It's pretty
hard for me to imagine a situation where this is actually what you
want. A Ptr () isn't nonsense.
It is not terrible to use Ptr () to represent an Addr, but I wonder if
it sends the wrong message.
By the way: there's another argument for having Addr in base for now.
We would really
*like* for Ptr's parameter to have a *representational* role, but we
*don't* want to require
unsafeCoerce to cast Ptrs. The solution to that in the current role system:

    data Addr = Addr Addr#

    newtype Ptr a = Ptr_ Addr
    type role Ptr representational

    pattern Ptr :: Addr# -> Ptr a
    pattern Ptr addr# = Ptr_ (Addr addr#)

    -- Allow users to reveal coercibility of pointer types locally
    ptrCoercion :: Coercion (Ptr a) (Ptr b)
    ptrCoercion = Coercion

    castPtr :: Ptr a -> Ptr b
    castPtr = coerceWith ptrCoercion -- (or the now-free unwrap-rewrap
definition)


So even if we don't *expose* Addr in base, we should almost certainly *define*
it there.
On Mon, Oct 29, 2018 at 12:11 PM Carter Schonwald
<[hidden email]> wrote:
>
> The point , hahah, of a Ptr void is that you can’t dereference it.  But you certainly can cast it and do address arithmetic on it!!
>
>
>
> On Mon, Oct 29, 2018 at 10:10 AM David Feuer <[hidden email]> wrote:
>>
>> On Mon, Oct 29, 2018, 10:05 AM Sven Panne <[hidden email]> wrote:
>>>
>>> Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright <[hidden email]>:
>>>>
>>>> 'Ptr Void' is not a pointer to a value of type 'Void'; there are no values of type 'Void': this type is nonsensical.
>>>
>>>
>>> That's the whole point, and it actually makes sense: If you see "Ptr Void", you can't do much with it, apart from passing it around or using castPtr on it. This is exactly what should be achieved by using "Ptr Void" in an API. This is basically the same as "void *" in C/C++.
>>
>>
>> No, it does not make sense. The approximate equivalent of C's void* is Ptr Any. Ptr Void promises to give you anything you want on dereference, which is nonsense.
>>
>>>
>>> You can't store or read "()", so the same holds as for Void (which didn't exist when the FFI was created IIRC).
>>
>>
>> Sure you can. Storing () does nothing and reading it gives (). Our () is somewhat similar to C's void return type.
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

chessai .
In reply to this post by Carter Schonwald
[19:26:50] <chessai_> hPutBuf :: Handle -> Ptr a -> Int -> IO ()
[19:26:50] <chessai_> hPutBuf :: Handle -> Addr -> Int -> IO ()
[19:26:55] <carter> ok
[19:27:15] <carter> maybe i wrote the buffer with bytes
[19:27:18] <carter> or maybe i did words?
[19:27:32] <carter> hputbuff dont care
[19:27:45] <hvr> the problem I see is rather that hPutBuf is a sub-optimal type-sig
[19:28:04] <hvr> it could refer to Storable
[19:28:19] <chessai_> Storable's Ptr usage is another example
[19:28:23] <hvr> but it doesn't, hence why it seems to be a bit awkward
[19:28:35] <carter> chessai_:  whats better there?
[19:28:36] <hvr> so I'd rather blame that it isn't aware of Storable
[19:28:45] <chessai_> hvr: makes sense
[19:28:51] <chessai_> carter: what's better where?
[19:29:27] <carter> what would you feel a better wrapper around storable would be?
[19:29:36] <carter> addr ruins having good type inference there afaict
[19:30:02] <chessai_> peekByteOff :: Ptr b -> Int -> IO a
[19:30:09] <chessai_> peekByteOff :: Addr -> Int -> IO a
[19:30:26] <chessai_> what is 'b' doing there? it's not used in any meaningful way by peekByteOff
[19:30:29] <chessai_> it's free
[19:30:39] <hvr> chessai_: so, another option besides making hGetBuf 'Storable'-aware would be to use 'Ptr Word8' imho
[19:31:00] <hvr> as that would imho be more in line with low-levelness of that operation which is clearly about 8-bit addressable mem
[19:31:10] <hvr> (it even says so in the haddocks)
[19:31:24] <carter> chessai_:  ok, thats definitely a bad api in storable
[19:31:32] <carter> seems liek that should have b and a the same in a higher level wrapper
[19:31:43] <chessai_> or no b at all
[19:32:03] <chessai_> hvr: i think i like both of those options
[19:32:22] <carter> pokeElemOff :: Ptr a -> Int -> a -> IO () --- way better than peak
[19:32:23] <chessai_> hvr: not sure about how i weigh them
[19:32:34] <carter> peak should be Ptr a -> Int -> IO a
[19:32:35] <hvr> and yes, peekByteOff had a weird type-sig which nobody dared to fix probably out of fear of breakage
[19:33:12] <carter> hvr: lets add safePeekByteOff :: Ptr a -> Int -> IO a ?
[19:33:15] <carter> or sane
[19:33:23] <hvr> chessai_: but do take into account that if you want to use 'Addr' there, you're going to have a lot more of justification to do, as this is Haskell2010 territory
[19:33:46] <chessai_> hvr: true. a lot of 'Ptr' usage seems to be part of the report
[19:34:20] <carter> chessai_:  you're giving good examples of apis that are currently bad
[19:34:28] <carter> but it seems like they're ones we can work together to make better
[19:34:44] <carter> and Addr vs Ptr a is your way of articulating "this API smells bad"
[19:35:15] <carter> so i agree with the smell, just not with the cure :)
[19:35:20] <carter> at least atm
[19:35:31] <chessai_> carter: i am glad we agree on the smell

On Mon, Oct 29, 2018 at 7:31 PM Carter Schonwald <[hidden email]> wrote:
the parametricity isn't for when you know things, its for saying "these are possibly different or possibly the same, dont let me mix them up though, cause I dont know yet"


On Mon, Oct 29, 2018 at 7:03 PM Daniel Cartwright <[hidden email]> wrote:
I'm not sure that argument applies at all, when talking about _incorrect_ usages of Ptr. Sure, Addr probably shouldn't be used when there is meaningful type information/value to recover, but neither should Ptr be used when there is none.

The argument being made is not to make 'better', per se, and there definitely won't be a 'mathematical statement' about this, but it certainly may be made clearer - in my opinion, the usages of 'Ptr' that i've already brought up are inherently unclear because of the bogus phantom type associated with 'Ptr'. The illustration of this begs no code that doesn't already exist in corelibs.


On Mon, Oct 29, 2018 at 6:19 PM Carter Schonwald <[hidden email]> wrote:
to zoom out: what code is improved? what code is made better/clearer? No one has articulated this clearly. 

The one example of Addr being used in Vector.Storable.Mutable is not an argument in favor of using Addr. Its an argument against it existing.

i'm looking for evidence, in the form of code i can look at then say "yes, this is better code" when comparing the two. Or a mathematical statement of "what is made better"

[hidden email] , @Daniel , do you have one?

when i'm writing complicated code, MORE polymorphism helps me usually.

I can write some code like the following and even though I'm using it with Int at argument,
I *Know* that i'm not mixing up arguments/values that i write as different types. I cannot do this with Address!

{-# SPECIALIZE INLINE computeStarts :: [(Int,Int)]->Int->Int ->[(Int,Int)] #-}
computeStarts:: (Enum a, Ord a, Num b )=>[(a,b)]-> a -> a -> [(a,b)]

parametricity (even when constrained by type classes) is a powerful and foundational tool for good programming in haskell and similar languages

there has been nothing stated here that successfully articulates a good reason to forgo/discourage parametricity as an engineering tool. for thats what Addr is. 
A datatype thats never safe in isolation, and discourages using parametricity to write correct software.

a very strong case is needed to forgo parametricity. 




On Mon, Oct 29, 2018 at 5:33 PM David Feuer <[hidden email]> wrote:
Good point! Call it nominal then.

On Mon, Oct 29, 2018, 5:24 PM Carter Schonwald <[hidden email]> wrote:
absolutely false, represeentational equality of the  type a in `Ptr a` does not mean the memory representation at the corresponding address is the same.
(it sometimes is true, but memory packing/alignment details in structs in C  for otherwise equivlanet structs should rule this out)

aka, `a` being representationally equal to `b` via haskell newtypes does not mean the memory representation at `Ptr a`, and `Ptr b` are the same. a trivial example is when
host and network byte order aren't the same (eg big vs little endian memory encodings)

On Mon, Oct 29, 2018 at 12:28 PM David Feuer <[hidden email]> wrote:
What? Of course you can dereference it. You dereference it, getting a
value of type `Void`,
and apply absurd to get whatever you want in the world. This, of
course, is utter nonsense,
unless *having* the Ptr Void means that something has already gone
wrong. It's pretty
hard for me to imagine a situation where this is actually what you
want. A Ptr () isn't nonsense.
It is not terrible to use Ptr () to represent an Addr, but I wonder if
it sends the wrong message.
By the way: there's another argument for having Addr in base for now.
We would really
*like* for Ptr's parameter to have a *representational* role, but we
*don't* want to require
unsafeCoerce to cast Ptrs. The solution to that in the current role system:

    data Addr = Addr Addr#

    newtype Ptr a = Ptr_ Addr
    type role Ptr representational

    pattern Ptr :: Addr# -> Ptr a
    pattern Ptr addr# = Ptr_ (Addr addr#)

    -- Allow users to reveal coercibility of pointer types locally
    ptrCoercion :: Coercion (Ptr a) (Ptr b)
    ptrCoercion = Coercion

    castPtr :: Ptr a -> Ptr b
    castPtr = coerceWith ptrCoercion -- (or the now-free unwrap-rewrap
definition)


So even if we don't *expose* Addr in base, we should almost certainly *define*
it there.
On Mon, Oct 29, 2018 at 12:11 PM Carter Schonwald
<[hidden email]> wrote:
>
> The point , hahah, of a Ptr void is that you can’t dereference it.  But you certainly can cast it and do address arithmetic on it!!
>
>
>
> On Mon, Oct 29, 2018 at 10:10 AM David Feuer <[hidden email]> wrote:
>>
>> On Mon, Oct 29, 2018, 10:05 AM Sven Panne <[hidden email]> wrote:
>>>
>>> Am Mo., 29. Okt. 2018 um 14:27 Uhr schrieb Daniel Cartwright <[hidden email]>:
>>>>
>>>> 'Ptr Void' is not a pointer to a value of type 'Void'; there are no values of type 'Void': this type is nonsensical.
>>>
>>>
>>> That's the whole point, and it actually makes sense: If you see "Ptr Void", you can't do much with it, apart from passing it around or using castPtr on it. This is exactly what should be achieved by using "Ptr Void" in an API. This is basically the same as "void *" in C/C++.
>>
>>
>> No, it does not make sense. The approximate equivalent of C's void* is Ptr Any. Ptr Void promises to give you anything you want on dereference, which is nonsense.
>>
>>>
>>> You can't store or read "()", so the same holds as for Void (which didn't exist when the FFI was created IIRC).
>>
>>
>> Sure you can. Storing () does nothing and reading it gives (). Our () is somewhat similar to C's void return type.
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Sven Panne-2
I am not sure if everybody fully comprehends what Storable is all about: It is meant as the lowest-level building block in an Addr-free world (remember: Addr is a GHCism and is *not* mentioned anywhere in the report) to put a few well-defined simple Haskell types into memory or read them from there. Its explicit non-goals are:

   * Achieve 100% type safety. In the presence of raw memory access, castPtr, C calls etc. this would be a total illusion. Forcing API users to sprinkle tons of castPtr at every possible place over their code wouldn't improve safety at all, it would only hurt readability.

   * Handle more complicated sum/product types. How would you do this? Respect your native ABI (i.e. automatically handle padding/alignment)? Tightly packed? Or even handle a foreign ABI? Your own ABI? Some funny encoding like OpenGL's packed data types? Etc. etc. You can build all of those things in a layer above Storable, probably introducing other type classes or some marshaling DSLs.

   * Portability of the written values. This is more in the realm of serialization libraries.

More concretely:

Am Di., 30. Okt. 2018 um 14:34 Uhr schrieb Daniel Cartwright <[hidden email]>:
[19:26:50] <chessai_> hPutBuf :: Handle -> Ptr a -> Int -> IO () [...]

The signature for this is actually perfect: hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Forcing castPtr Kung Fu here wouldn't buy you anything: The buffer will probably contain a wild mix of Haskell values or even no Haskell values at all, but that doesn't matter. Whatever you pass as "a" or whatever you cast from/to is probably a lie from the typing perspective. At this level this is no problem at all.
 
[19:30:02] <chessai_> peekByteOff :: Ptr b -> Int -> IO a
[19:30:09] <chessai_> peekByteOff :: Addr -> Int -> IO a
[19:30:26] <chessai_> what is 'b' doing there? it's not used in any meaningful way by peekByteOff [...]

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
 
[19:32:22] <carter> pokeElemOff :: Ptr a -> Int -> a -> IO () --- way better than peak  [...]

Yes, because this is intended to be used for *arrays* of values of the same type. Note "Elem" vs. "Byte".
 
[19:33:12] <carter> hvr: lets add safePeekByteOff :: Ptr a -> Int -> IO a ? [...]

This signature doesn't make sense, see above: Shifting a pointer by an arbitrary amount of bytes will probably change the type of what you're pointing to. If you shift by units of the underlying type, well, that's peekElemOff.
 
[19:35:31] <chessai_> carter: i am glad we agree on the smell

I don't have the full chat log, but I think I don't even agree on the smell, at least not at the places I've seen... :-) 



_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

chessai .
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?

On Tue, Oct 30, 2018 at 10:10 AM Sven Panne <[hidden email]> wrote:
I am not sure if everybody fully comprehends what Storable is all about: It is meant as the lowest-level building block in an Addr-free world (remember: Addr is a GHCism and is *not* mentioned anywhere in the report) to put a few well-defined simple Haskell types into memory or read them from there. Its explicit non-goals are:

   * Achieve 100% type safety. In the presence of raw memory access, castPtr, C calls etc. this would be a total illusion. Forcing API users to sprinkle tons of castPtr at every possible place over their code wouldn't improve safety at all, it would only hurt readability.

   * Handle more complicated sum/product types. How would you do this? Respect your native ABI (i.e. automatically handle padding/alignment)? Tightly packed? Or even handle a foreign ABI? Your own ABI? Some funny encoding like OpenGL's packed data types? Etc. etc. You can build all of those things in a layer above Storable, probably introducing other type classes or some marshaling DSLs.

   * Portability of the written values. This is more in the realm of serialization libraries.

More concretely:

Am Di., 30. Okt. 2018 um 14:34 Uhr schrieb Daniel Cartwright <[hidden email]>:
[19:26:50] <chessai_> hPutBuf :: Handle -> Ptr a -> Int -> IO () [...]

The signature for this is actually perfect: hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Forcing castPtr Kung Fu here wouldn't buy you anything: The buffer will probably contain a wild mix of Haskell values or even no Haskell values at all, but that doesn't matter. Whatever you pass as "a" or whatever you cast from/to is probably a lie from the typing perspective. At this level this is no problem at all.
 
[19:30:02] <chessai_> peekByteOff :: Ptr b -> Int -> IO a
[19:30:09] <chessai_> peekByteOff :: Addr -> Int -> IO a
[19:30:26] <chessai_> what is 'b' doing there? it's not used in any meaningful way by peekByteOff [...]

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
 
[19:32:22] <carter> pokeElemOff :: Ptr a -> Int -> a -> IO () --- way better than peak  [...]

Yes, because this is intended to be used for *arrays* of values of the same type. Note "Elem" vs. "Byte".
 
[19:33:12] <carter> hvr: lets add safePeekByteOff :: Ptr a -> Int -> IO a ? [...]

This signature doesn't make sense, see above: Shifting a pointer by an arbitrary amount of bytes will probably change the type of what you're pointing to. If you shift by units of the underlying type, well, that's peekElemOff.
 
[19:35:31] <chessai_> carter: i am glad we agree on the smell

I don't have the full chat log, but I think I don't even agree on the smell, at least not at the places I've seen... :-) 



_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

chessai .
Also, just to be clear, no one is talking about changing Storable, that seems terrible at this point. Your argument still makes sense to make outside of the context of wanting to change Storable, i just want to make sure it's reiterated that this is not being suggested.

On Tue, Oct 30, 2018 at 10:18 AM Daniel Cartwright <[hidden email]> wrote:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?

On Tue, Oct 30, 2018 at 10:10 AM Sven Panne <[hidden email]> wrote:
I am not sure if everybody fully comprehends what Storable is all about: It is meant as the lowest-level building block in an Addr-free world (remember: Addr is a GHCism and is *not* mentioned anywhere in the report) to put a few well-defined simple Haskell types into memory or read them from there. Its explicit non-goals are:

   * Achieve 100% type safety. In the presence of raw memory access, castPtr, C calls etc. this would be a total illusion. Forcing API users to sprinkle tons of castPtr at every possible place over their code wouldn't improve safety at all, it would only hurt readability.

   * Handle more complicated sum/product types. How would you do this? Respect your native ABI (i.e. automatically handle padding/alignment)? Tightly packed? Or even handle a foreign ABI? Your own ABI? Some funny encoding like OpenGL's packed data types? Etc. etc. You can build all of those things in a layer above Storable, probably introducing other type classes or some marshaling DSLs.

   * Portability of the written values. This is more in the realm of serialization libraries.

More concretely:

Am Di., 30. Okt. 2018 um 14:34 Uhr schrieb Daniel Cartwright <[hidden email]>:
[19:26:50] <chessai_> hPutBuf :: Handle -> Ptr a -> Int -> IO () [...]

The signature for this is actually perfect: hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Forcing castPtr Kung Fu here wouldn't buy you anything: The buffer will probably contain a wild mix of Haskell values or even no Haskell values at all, but that doesn't matter. Whatever you pass as "a" or whatever you cast from/to is probably a lie from the typing perspective. At this level this is no problem at all.
 
[19:30:02] <chessai_> peekByteOff :: Ptr b -> Int -> IO a
[19:30:09] <chessai_> peekByteOff :: Addr -> Int -> IO a
[19:30:26] <chessai_> what is 'b' doing there? it's not used in any meaningful way by peekByteOff [...]

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
 
[19:32:22] <carter> pokeElemOff :: Ptr a -> Int -> a -> IO () --- way better than peak  [...]

Yes, because this is intended to be used for *arrays* of values of the same type. Note "Elem" vs. "Byte".
 
[19:33:12] <carter> hvr: lets add safePeekByteOff :: Ptr a -> Int -> IO a ? [...]

This signature doesn't make sense, see above: Shifting a pointer by an arbitrary amount of bytes will probably change the type of what you're pointing to. If you shift by units of the underlying type, well, that's peekElemOff.
 
[19:35:31] <chessai_> carter: i am glad we agree on the smell

I don't have the full chat log, but I think I don't even agree on the smell, at least not at the places I've seen... :-) 



_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Sven Panne-2
In reply to this post by chessai .
Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright <[hidden email]>:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.
 
If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?
Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way.  As I mentioned yesterday , that’s been lacking here.  Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne <[hidden email]> wrote:
Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright <[hidden email]>:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.
 
If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?
Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Andrew Martin
Look at the definition of SerializedCompact (in ghc-compact):

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Ptr a, Word)]
      , serializedCompactRoot :: Ptr a
      }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Addr, Word)]
      , serializedCompactRoot :: Addr
      }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.


On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald <[hidden email]> wrote:
Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way.  As I mentioned yesterday , that’s been lacking here.  Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne <[hidden email]> wrote:
Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright <[hidden email]>:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.
 
If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?
Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


--
-Andrew Thaddeus Martin

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
This example seems more a discussion about how the ghc-compact lib could benefit from better docs. 

On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin <[hidden email]> wrote:
Look at the definition of SerializedCompact (in ghc-compact):

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Ptr a, Word)]
      , serializedCompactRoot :: Ptr a
      }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Addr, Word)]
      , serializedCompactRoot :: Addr
      }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.


On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald <[hidden email]> wrote:
Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way.  As I mentioned yesterday , that’s been lacking here.  Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne <[hidden email]> wrote:
Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright <[hidden email]>:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.
 
If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?
Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...
_______________________________________________


--
-Andrew Thaddeus Martin

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
There’s no class constraint here, so I would instead assume I need to read the paper / docs and source code for how heap objects are encoded here. Changing the type to Addr vs Ptr a or Ptr void is a red herring / irrelevant 

On Tue, Oct 30, 2018 at 12:09 PM Carter Schonwald <[hidden email]> wrote:
This example seems more a discussion about how the ghc-compact lib could benefit from better docs. 

On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin <[hidden email]> wrote:
Look at the definition of SerializedCompact (in ghc-compact):

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Ptr a, Word)]
      , serializedCompactRoot :: Ptr a
      }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Addr, Word)]
      , serializedCompactRoot :: Addr
      }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.


On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald <[hidden email]> wrote:
Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way.  As I mentioned yesterday , that’s been lacking here.  Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne <[hidden email]> wrote:
Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright <[hidden email]>:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.
 
If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?
Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...
_______________________________________________


--
-Andrew Thaddeus Martin

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
Anyways. This proposal is dead.  

Changing this internal stuff from Ptr to Addr doesn’t make an argument that the code is better for it.  That has not be made.  

Saying ptr a is a lie is about as informative as saying virtual memory is a lie. Or the cpu caches are a lie. 

Evidence works.  Saying something is a lie doesn’t.  

On Tue, Oct 30, 2018 at 12:11 PM Carter Schonwald <[hidden email]> wrote:
There’s no class constraint here, so I would instead assume I need to read the paper / docs and source code for how heap objects are encoded here. Changing the type to Addr vs Ptr a or Ptr void is a red herring / irrelevant 

On Tue, Oct 30, 2018 at 12:09 PM Carter Schonwald <[hidden email]> wrote:
This example seems more a discussion about how the ghc-compact lib could benefit from better docs. 

On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin <[hidden email]> wrote:
Look at the definition of SerializedCompact (in ghc-compact):

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Ptr a, Word)]
      , serializedCompactRoot :: Ptr a
      }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Addr, Word)]
      , serializedCompactRoot :: Addr
      }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.


On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald <[hidden email]> wrote:
Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way.  As I mentioned yesterday , that’s been lacking here.  Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne <[hidden email]> wrote:
Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright <[hidden email]>:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.
 
If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?
Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...
_______________________________________________


--
-Andrew Thaddeus Martin

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

David Feuer
Thankfully, Carter, you're not the sole arbiter of whether a library proposal is dead. The discussion can surely continue without you.

On Tue, Oct 30, 2018, 12:22 PM Carter Schonwald <[hidden email]> wrote:
Anyways. This proposal is dead.  

Changing this internal stuff from Ptr to Addr doesn’t make an argument that the code is better for it.  That has not be made.  

Saying ptr a is a lie is about as informative as saying virtual memory is a lie. Or the cpu caches are a lie. 

Evidence works.  Saying something is a lie doesn’t.  

On Tue, Oct 30, 2018 at 12:11 PM Carter Schonwald <[hidden email]> wrote:
There’s no class constraint here, so I would instead assume I need to read the paper / docs and source code for how heap objects are encoded here. Changing the type to Addr vs Ptr a or Ptr void is a red herring / irrelevant 

On Tue, Oct 30, 2018 at 12:09 PM Carter Schonwald <[hidden email]> wrote:
This example seems more a discussion about how the ghc-compact lib could benefit from better docs. 

On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin <[hidden email]> wrote:
Look at the definition of SerializedCompact (in ghc-compact):

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Ptr a, Word)]
      , serializedCompactRoot :: Ptr a
      }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Addr, Word)]
      , serializedCompactRoot :: Addr
      }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.


On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald <[hidden email]> wrote:
Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way.  As I mentioned yesterday , that’s been lacking here.  Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne <[hidden email]> wrote:
Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright <[hidden email]>:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.
 
If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?
Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...
_______________________________________________


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
Repeating it’s a lie isn’t an argument.  

Writing code and comparing it is a way to make a case. I even put some up on phabticator.  Then decided it’s actually not a useful piece of code.  

I’ve spoke with a number of folks privately and I’m arriculating the lack of suport and clear case. 

I’m open to being wrong and that this proposal is great.  But changing a type signature isn’t show casing what code is improved. It’s just a proof that the change can be made. 

I need evidence.  As does everyone :)

On Tue, Oct 30, 2018 at 12:25 PM David Feuer <[hidden email]> wrote:
Thankfully, Carter, you're not the sole arbiter of whether a library proposal is dead. The discussion can surely continue without you.

On Tue, Oct 30, 2018, 12:22 PM Carter Schonwald <[hidden email]> wrote:
Anyways. This proposal is dead.  

Changing this internal stuff from Ptr to Addr doesn’t make an argument that the code is better for it.  That has not be made.  

Saying ptr a is a lie is about as informative as saying virtual memory is a lie. Or the cpu caches are a lie. 

Evidence works.  Saying something is a lie doesn’t.  

On Tue, Oct 30, 2018 at 12:11 PM Carter Schonwald <[hidden email]> wrote:
There’s no class constraint here, so I would instead assume I need to read the paper / docs and source code for how heap objects are encoded here. Changing the type to Addr vs Ptr a or Ptr void is a red herring / irrelevant 

On Tue, Oct 30, 2018 at 12:09 PM Carter Schonwald <[hidden email]> wrote:
This example seems more a discussion about how the ghc-compact lib could benefit from better docs. 

On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin <[hidden email]> wrote:
Look at the definition of SerializedCompact (in ghc-compact):

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Ptr a, Word)]
      , serializedCompactRoot :: Ptr a
      }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Addr, Word)]
      , serializedCompactRoot :: Addr
      }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.


On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald <[hidden email]> wrote:
Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way.  As I mentioned yesterday , that’s been lacking here.  Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne <[hidden email]> wrote:
Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright <[hidden email]>:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.
 
If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?
Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...
_______________________________________________


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

David Feuer
Private off-list discussions are not, were never, and will never be a part of the proposal process, and for good reasons. Mentioning them only makes you look like you're trying to use your connections to bypass the process. Please don't.

On Tue, Oct 30, 2018, 12:36 PM Carter Schonwald <[hidden email]> wrote:
Repeating it’s a lie isn’t an argument.  

Writing code and comparing it is a way to make a case. I even put some up on phabticator.  Then decided it’s actually not a useful piece of code.  

I’ve spoke with a number of folks privately and I’m arriculating the lack of suport and clear case. 

I’m open to being wrong and that this proposal is great.  But changing a type signature isn’t show casing what code is improved. It’s just a proof that the change can be made. 

I need evidence.  As does everyone :)

On Tue, Oct 30, 2018 at 12:25 PM David Feuer <[hidden email]> wrote:
Thankfully, Carter, you're not the sole arbiter of whether a library proposal is dead. The discussion can surely continue without you.

On Tue, Oct 30, 2018, 12:22 PM Carter Schonwald <[hidden email]> wrote:
Anyways. This proposal is dead.  

Changing this internal stuff from Ptr to Addr doesn’t make an argument that the code is better for it.  That has not be made.  

Saying ptr a is a lie is about as informative as saying virtual memory is a lie. Or the cpu caches are a lie. 

Evidence works.  Saying something is a lie doesn’t.  

On Tue, Oct 30, 2018 at 12:11 PM Carter Schonwald <[hidden email]> wrote:
There’s no class constraint here, so I would instead assume I need to read the paper / docs and source code for how heap objects are encoded here. Changing the type to Addr vs Ptr a or Ptr void is a red herring / irrelevant 

On Tue, Oct 30, 2018 at 12:09 PM Carter Schonwald <[hidden email]> wrote:
This example seems more a discussion about how the ghc-compact lib could benefit from better docs. 

On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin <[hidden email]> wrote:
Look at the definition of SerializedCompact (in ghc-compact):

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Ptr a, Word)]
      , serializedCompactRoot :: Ptr a
      }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Addr, Word)]
      , serializedCompactRoot :: Addr
      }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.


On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald <[hidden email]> wrote:
Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way.  As I mentioned yesterday , that’s been lacking here.  Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne <[hidden email]> wrote:
Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright <[hidden email]>:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.
 
If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?
Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...
_______________________________________________


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
David: 
Please don’t make this personal. 

Either way you’re definitely correct that this is a discussion list to inform clc decisions and or applicable library maintainers.  

I’m speaking as a maintainer.of vector and primitive 

On Tue, Oct 30, 2018 at 12:38 PM David Feuer <[hidden email]> wrote:
Private off-list discussions are not, were never, and will never be a part of the proposal process, and for good reasons. Mentioning them only makes you look like you're trying to use your connections to bypass the process. Please don't.

On Tue, Oct 30, 2018, 12:36 PM Carter Schonwald <[hidden email]> wrote:
Repeating it’s a lie isn’t an argument.  

Writing code and comparing it is a way to make a case. I even put some up on phabticator.  Then decided it’s actually not a useful piece of code.  

I’ve spoke with a number of folks privately and I’m arriculating the lack of suport and clear case. 

I’m open to being wrong and that this proposal is great.  But changing a type signature isn’t show casing what code is improved. It’s just a proof that the change can be made. 

I need evidence.  As does everyone :)

On Tue, Oct 30, 2018 at 12:25 PM David Feuer <[hidden email]> wrote:
Thankfully, Carter, you're not the sole arbiter of whether a library proposal is dead. The discussion can surely continue without you.

On Tue, Oct 30, 2018, 12:22 PM Carter Schonwald <[hidden email]> wrote:
Anyways. This proposal is dead.  

Changing this internal stuff from Ptr to Addr doesn’t make an argument that the code is better for it.  That has not be made.  

Saying ptr a is a lie is about as informative as saying virtual memory is a lie. Or the cpu caches are a lie. 

Evidence works.  Saying something is a lie doesn’t.  

On Tue, Oct 30, 2018 at 12:11 PM Carter Schonwald <[hidden email]> wrote:
There’s no class constraint here, so I would instead assume I need to read the paper / docs and source code for how heap objects are encoded here. Changing the type to Addr vs Ptr a or Ptr void is a red herring / irrelevant 

On Tue, Oct 30, 2018 at 12:09 PM Carter Schonwald <[hidden email]> wrote:
This example seems more a discussion about how the ghc-compact lib could benefit from better docs. 

On Tue, Oct 30, 2018 at 11:48 AM Andrew Martin <[hidden email]> wrote:
Look at the definition of SerializedCompact (in ghc-compact):

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Ptr a, Word)]
      , serializedCompactRoot :: Ptr a
      }

The type variables on those Ptrs are a lie. The SerializedCompact needs that phantom type variable to keep track of what type of object in a compact region is represented by these blocks. But, the types of its two fields are extremely misleading. When I first saw this type, I was led to believe that I could call `peek` on the `serializedCompactRoot` field and get a value of type `a`. You cannot actually do that since it isn't actually a pointer to a value of that type. More correct would be:

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Addr, Word)]
      , serializedCompactRoot :: Addr
      }

This better documents what's actually going on here. We don't have a bunch of pointers to `a` values in a list. We value memory addresses that refer to arbitrary fragments of an object on the heap.


On Tue, Oct 30, 2018 at 11:32 AM Carter Schonwald <[hidden email]> wrote:
Daniel : at this point you’re not show casing examples of why Address leads to better code than Pointer a

A good library proposal makes code *better* in an unambiguous way.  As I mentioned yesterday , that’s been lacking here.  Or any improvements are ones sven , myself and others don’t see. :)

On Tue, Oct 30, 2018 at 10:41 AM Sven Panne <[hidden email]> wrote:
Am Di., 30. Okt. 2018 um 15:18 Uhr schrieb Daniel Cartwright <[hidden email]>:
> hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size.
Seems like a good use for Addr?

Nope, not at all: Using a free type variable like "a" in such a situation is *the* common idiom to specify that the function doesn't care about the type, just like "length" doesn't care about the type of the elements. If you don't like that idiom, fine, but it has been officially enshrined in the standard and people are using it for decades (well, almost) without any problems, at least I haven't heard of them.

What would using Addr buy us? Addr is non-standard, and you would have to use some cast from a Ptr somehow (the world outside GHC's innards is using Ptr, not Addr). This would buy you exactly zero safety or clarity, and it would only introduce API friction.
 
If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
This also seems like a good use for Addr?
Nope again, see above.

I still fail to see what problem this whole proposal is actually trying to solve...
_______________________________________________


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Sven Panne-2
In reply to this post by Andrew Martin
Am Di., 30. Okt. 2018 um 16:48 Uhr schrieb Andrew Martin <[hidden email]>:
Look at the definition of SerializedCompact (in ghc-compact):

    data SerializedCompact a = SerializedCompact
      { serializedCompactBlockList :: [(Ptr a, Word)]
      , serializedCompactRoot :: Ptr a
      }
[...]

I haven't used the ghc-compact or compact packages yet, so I can't really comment on the details here. The question boils down to: Why does Compact/SerializedCompact have a type parameter at all? Quickly browsing over the Hadoock documentation of both packages doesn't make that clear to me, but as I said: I can't comment on the design decision of someone else's API if I don't fully understand the use cases and tradeoffs involved.

In any case: This example has exactly *nothing* to do with the proposal at hand. If the API above uses "a" for some good reason/convenience etc. it's perfectly OK, if not, it should better use "Ptr ()" or "Ptr Void". This is perhaps an example of incomplete documentation and/or suboptimal API design, but nothing more. Using Addr here wouldn't improve usability or safety, it would just introduce a non-standard data type for nothing.

I could show you tons of APIs out in the wild where e.g. lists are used, but where they are actually not really the right data structure for the problem at hand. Nevertheless, this is not an argument against lists per se, this is a problem of the API which abuses them.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Edward Kmett-2
In reply to this post by Sven Panne-2
One messy sketch of how this could proceed would be to make additional Addr-based analogues of the report specified type-unrelated "Ptr a" functions and install them side by side with the existing functions. This would behave much like how we have both readsPrec and readPrec for standard vs. GHC specific Read internals. With MINIMAL pragmas it should basically become transparent if they are mutually defined. Since generally Storable dictionaries aren't built out of compositions of other Storable dictionaries, growing the class shouldn't do any measurable harm to performance. We haven't been shy about adding new members to report-specified classes like Bits. This doesn't strike me as much different.

At some point in the future we can work out if it is worth it to get the report fixed up by incorporating the Addr-based API as the default and make the _other_ the legacy.

-Edward


On Tue, Oct 30, 2018 at 10:11 AM Sven Panne <[hidden email]> wrote:
I am not sure if everybody fully comprehends what Storable is all about: It is meant as the lowest-level building block in an Addr-free world (remember: Addr is a GHCism and is *not* mentioned anywhere in the report) to put a few well-defined simple Haskell types into memory or read them from there. Its explicit non-goals are:

   * Achieve 100% type safety. In the presence of raw memory access, castPtr, C calls etc. this would be a total illusion. Forcing API users to sprinkle tons of castPtr at every possible place over their code wouldn't improve safety at all, it would only hurt readability.

   * Handle more complicated sum/product types. How would you do this? Respect your native ABI (i.e. automatically handle padding/alignment)? Tightly packed? Or even handle a foreign ABI? Your own ABI? Some funny encoding like OpenGL's packed data types? Etc. etc. You can build all of those things in a layer above Storable, probably introducing other type classes or some marshaling DSLs.

   * Portability of the written values. This is more in the realm of serialization libraries.

More concretely:

Am Di., 30. Okt. 2018 um 14:34 Uhr schrieb Daniel Cartwright <[hidden email]>:
[19:26:50] <chessai_> hPutBuf :: Handle -> Ptr a -> Int -> IO () [...]

The signature for this is actually perfect: hPutBuf doesn't care about what stuff has been written into the given buffer, it just cares about its start and its size. Forcing castPtr Kung Fu here wouldn't buy you anything: The buffer will probably contain a wild mix of Haskell values or even no Haskell values at all, but that doesn't matter. Whatever you pass as "a" or whatever you cast from/to is probably a lie from the typing perspective. At this level this is no problem at all.
 
[19:30:02] <chessai_> peekByteOff :: Ptr b -> Int -> IO a
[19:30:09] <chessai_> peekByteOff :: Addr -> Int -> IO a
[19:30:26] <chessai_> what is 'b' doing there? it's not used in any meaningful way by peekByteOff [...]

If you have a pointer pointing to something and shift that pointer by some bytes, you are probably pointing to something completely different, so of course "b" and "a" have nothing to do with each other. So peekByteOff intentionally ignores "b".
 
[19:32:22] <carter> pokeElemOff :: Ptr a -> Int -> a -> IO () --- way better than peak  [...]

Yes, because this is intended to be used for *arrays* of values of the same type. Note "Elem" vs. "Byte".
 
[19:33:12] <carter> hvr: lets add safePeekByteOff :: Ptr a -> Int -> IO a ? [...]

This signature doesn't make sense, see above: Shifting a pointer by an arbitrary amount of bytes will probably change the type of what you're pointing to. If you shift by units of the underlying type, well, that's peekElemOff.
 
[19:35:31] <chessai_> carter: i am glad we agree on the smell

I don't have the full chat log, but I think I don't even agree on the smell, at least not at the places I've seen... :-) 


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Anthony Cowley

Edward Kmett writes:

> One messy sketch of how this could proceed would be to make additional
> Addr-based analogues of the report specified type-unrelated "Ptr a"
> functions and install them side by side with the existing functions. This
> would behave much like how we have both readsPrec and readPrec for standard
> vs. GHC specific Read internals. With MINIMAL pragmas it should basically
> become transparent if they are mutually defined. Since generally Storable
> dictionaries aren't built out of compositions of other Storable
> dictionaries, growing the class shouldn't do any measurable harm to
> performance.

This doesn't sound right to me. I'm probably an outlier, but I rely heavily on composite Storable instances in vinyl and Frames and general FFI situations. I don't know if a more common usage like an ad hoc record relying on the Storable instances of its fields would be greatly impacted, but this needs some investigation. More generally, I'm in agreement with the points Sven has been making in this discussion, and wouldn't like to see such an unresolved debate be resolved by imposing a performance penalty on everyone as a compromise.

> We haven't been shy about adding new members to
> report-specified classes like Bits. This doesn't strike me as much
> different.

It is similar, but I think composite Storable instances are much more common than composite Bits instances.

Anthony

>
> At some point in the future we can work out if it is worth it to get the
> report fixed up by incorporating the Addr-based API as the default and make
> the _other_ the legacy.
>
> -Edward
>
>
> On Tue, Oct 30, 2018 at 10:11 AM Sven Panne <[hidden email]> wrote:
>
>> I am not sure if everybody fully comprehends what Storable is all about:
>> It is meant as the lowest-level building block in an Addr-free world
>> (remember: Addr is a GHCism and is *not* mentioned anywhere in the report)
>> to put a few well-defined simple Haskell types into memory or read them
>> from there. Its explicit non-goals are:
>>
>>    * Achieve 100% type safety. In the presence of raw memory access,
>> castPtr, C calls etc. this would be a total illusion. Forcing API users to
>> sprinkle tons of castPtr at every possible place over their code wouldn't
>> improve safety at all, it would only hurt readability.
>>
>>    * Handle more complicated sum/product types. How would you do this?
>> Respect your native ABI (i.e. automatically handle padding/alignment)?
>> Tightly packed? Or even handle a foreign ABI? Your own ABI? Some funny
>> encoding like OpenGL's packed data types? Etc. etc. You can build all of
>> those things in a layer above Storable, probably introducing other type
>> classes or some marshaling DSLs.
>>
>>    * Portability of the written values. This is more in the realm of
>> serialization libraries.
>>
>> More concretely:
>>
>> Am Di., 30. Okt. 2018 um 14:34 Uhr schrieb Daniel Cartwright <
>> [hidden email]>:
>>
>>> [19:26:50] <chessai_> hPutBuf :: Handle -> Ptr a -> Int -> IO () [...]
>>>
>>
>> The signature for this is actually perfect: hPutBuf doesn't care about
>> what stuff has been written into the given buffer, it just cares about its
>> start and its size. Forcing castPtr Kung Fu here wouldn't buy you anything:
>> The buffer will probably contain a wild mix of Haskell values or even no
>> Haskell values at all, but that doesn't matter. Whatever you pass as "a" or
>> whatever you cast from/to is probably a lie from the typing perspective. At
>> this level this is no problem at all.
>>
>>
>>> [19:30:02] <chessai_> peekByteOff :: Ptr b -> Int -> IO a
>>> [19:30:09] <chessai_> peekByteOff :: Addr -> Int -> IO a
>>> [19:30:26] <chessai_> what is 'b' doing there? it's not used in any
>>> meaningful way by peekByteOff [...]
>>>
>>
>> If you have a pointer pointing to something and shift that pointer by some
>> bytes, you are probably pointing to something completely different, so of
>> course "b" and "a" have nothing to do with each other. So peekByteOff
>> intentionally ignores "b".
>>
>>
>>> [19:32:22] <carter> pokeElemOff :: Ptr a -> Int -> a -> IO () --- way
>>> better than peak  [...]
>>>
>>
>> Yes, because this is intended to be used for *arrays* of values of the
>> same type. Note "Elem" vs. "Byte".
>>
>>
>>> [19:33:12] <carter> hvr: lets add safePeekByteOff :: Ptr a -> Int -> IO a
>>> ? [...]
>>>
>>
>> This signature doesn't make sense, see above: Shifting a pointer by an
>> arbitrary amount of bytes will probably change the type of what you're
>> pointing to. If you shift by units of the underlying type, well, that's
>> peekElemOff.
>>
>>
>>> [19:35:31] <chessai_> carter: i am glad we agree on the smell
>>>
>>
>> I don't have the full chat log, but I think I don't even agree on the
>> smell, at least not at the places I've seen... :-)
>>
>>
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
12345