Proposal: Move primitive-Data.Primitive.Addr API into base

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
87 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
Before we bike shed these details, let’s evaluate / document the pieces of base that should also be moved to using Addr.  Though that module where

1) What exported types / data definitions / functions will have changes ?

2) what internal apis need to change?

as a lower bound estimate of work surface area, i grepped for "Ptr ()"  and "Addr#" in a fresh recursive clone of ghc at commit 503514b94f8dc7bd9eab5392206649aee45f140b,
limiting my search to .hs and .hsc files

THERES A LOT of uses of Ptr (), and its siblings StablePtr () and RemotePtr () ,   The latter two dont have a simple Addr# underneath, 

heres the list of files that match that search   rg 'Ptr \(\)' --type haskell -c

ghc/GHCi/Leak.hs:1
utils/hsc2hs/Main.hs:1
utils/runghc/Main.hs:1
compiler/ghci/GHCi.hs:4
compiler/ghci/ByteCodeLink.hs:3
compiler/ghci/ByteCodeGen.hs:4
compiler/ghci/ByteCodeTypes.hs:1
compiler/ghci/ByteCodeAsm.hs:2
compiler/main/HscTypes.hs:1
libraries/ghci/GHCi/ObjLink.hs:3
libraries/ghci/GHCi/StaticPtrTable.hs:1
libraries/ghci/GHCi/ResolvedBCO.hs:1
libraries/ghci/GHCi/RemoteTypes.hs:2
libraries/ghci/GHCi/Run.hs:2
libraries/ghci/GHCi/Message.hs:5
libraries/base/GHC/Stable.hs:2
libraries/base/GHC/StaticPtr.hs:1
libraries/base/GHC/Generics.hs:2
libraries/base/GHC/TopHandler.hs:1
libraries/base/GHC/Windows.hs:1
libraries/base/tests/dynamic002.hs:2
utils/haddock/haddock-api/src/Haddock.hs:1
libraries/array/Data/Array/IO.hs:2
libraries/base/Foreign/Marshal/Pool.hs:1
libraries/bytestring/Data/ByteString/Builder/Extra.hs:1
libraries/base/GHC/IO/Handle/Text.hs:1
testsuite/tests/rts/T7160.hs:1
testsuite/tests/rts/overflow3.hs:1
testsuite/tests/rts/overflow2.hs:1
testsuite/tests/rts/T5250.hs:1
testsuite/tests/rts/T10296b.hs:2
testsuite/tests/th/T3319.hs:1
testsuite/tests/th/TH_foreignInterruptible.hs:1
testsuite/tests/th/TH_foreignCallingConventions.hs:1
libraries/Cabal/cabal-install/Distribution/Client/Win32SelfUpgrade.hs:1
libraries/Cabal/Cabal/Distribution/Simple/InstallDirs.hs:2
testsuite/tests/llvm/should_compile/T5486.hs:2
testsuite/tests/codeGen/should_run/cgrun070.hs:5
testsuite/tests/codeGen/should_run/cgrun026.hs:3
testsuite/tests/primops/should_run/T4442.hs:1
testsuite/tests/ffi/should_fail/T7243.hs:1
testsuite/tests/ffi/should_fail/T3066.hs:1
testsuite/tests/ffi/should_compile/cc009.hs:1
testsuite/tests/ffi/should_run/ffi021.hs:1
testsuite/tests/typecheck/should_compile/T7408.hs:1
libraries/Cabal/cabal-install/Distribution/Client/Compat/ExecutablePath.hs:1
libraries/Cabal/Cabal/Distribution/Simple/Build/PathsModule.hs:1
libraries/time/lib/Data/Time/Clock/Internal/CTimeval.hs:1
testsuite/tests/rename/should_compile/T3103/Foreign/Ptr.hs:1

likewise the locations where Addr# comes up are as follows (some of these perhaps should be an Addr? )

$ rg 'Addr#' --type haskell -c
utils/genprimopcode/Main.hs:2
ghc/GHCi/Leak.hs:2
compiler/cmm/CmmMachOp.hs:1
compiler/cmm/CmmUtils.hs:1
compiler/specialise/Rules.hs:2
compiler/basicTypes/MkId.hs:2
compiler/basicTypes/Literal.hs:6
compiler/basicTypes/Unique.hs:2
compiler/utils/StringBuffer.hs:1
compiler/utils/FastString.hs:4
compiler/utils/Encoding.hs:9
compiler/codeGen/StgCmmPrim.hs:3
compiler/coreSyn/MkCore.hs:5
compiler/coreSyn/CoreLint.hs:2
compiler/coreSyn/CoreSyn.hs:4
compiler/coreSyn/CorePrep.hs:1
compiler/stranal/WwLib.hs:1
compiler/simplCore/CSE.hs:1
compiler/simplCore/Simplify.hs:2
compiler/deSugar/DsExpr.hs:1
compiler/typecheck/TcGenDeriv.hs:5
compiler/prelude/ForeignCall.hs:1
compiler/prelude/PrelNames.hs:1
compiler/prelude/TysPrim.hs:1
compiler/ghci/ByteCodeGen.hs:8
libraries/bytestring/tests/revcomp.hs:1
libraries/bytestring/Data/ByteString.hs:5
libraries/ghc-heap/tests/heap_all.hs:1
libraries/ghc-prim/GHC/CString.hs:17
libraries/ghc-prim/GHC/Classes.hs:1
libraries/ghc-prim/GHC/Types.hs:3
libraries/ghci/GHCi/BinaryArray.hs:1
libraries/ghci/GHCi/CreateBCO.hs:5
libraries/base/codepages/MakeTable.hs:1
libraries/base/GHC/Ptr.hs:8
libraries/base/GHC/Generics.hs:5
libraries/base/GHC/Storable.hs:32
libraries/base/GHC/ForeignPtr.hs:6
libraries/base/Foreign/Ptr.hs:2
libraries/bytestring/Data/ByteString/Unsafe.hs:6
libraries/bytestring/Data/ByteString/Internal.hs:7
libraries/ghc-heap/GHC/Exts/Heap.hs:1
libraries/text/Data/Text/Show.hs:2
libraries/text/Data/Text/Lazy.hs:2
libraries/containers/Data/IntSet/Internal.hs:4
libraries/ghc-compact/GHC/Compact/Serialized.hs:11
libraries/base/GHC/IO/Exception.hs:1
libraries/base/Data/Typeable/Internal.hs:3
libraries/base/Control/Exception/Base.hs:1
utils/haddock/html-test/src/Ticket112.hs:1
testsuite/tests/rts/T8242.hs:19
testsuite/tests/ghci.debugger/HappyTest.hs:3
testsuite/tests/arityanal/Main.hs:3
testsuite/tests/th/TH_StringPrimL.hs:2
testsuite/tests/th/T1830_3a.hs:2
libraries/template-haskell/Language/Haskell/TH/Syntax.hs:1
libraries/bytestring/Data/ByteString/Short/Internal.hs:16
libraries/integer-gmp/src/GHC/Integer/Type.hs:11
libraries/base/GHC/IO/Encoding/CodePage.hs:3
libraries/Cabal/Cabal/Distribution/Parsec/Lexer.hs:9
testsuite/tests/generics/GEq/GEq1A.hs:1
testsuite/tests/generics/GEq/GEq1.hs:2
testsuite/tests/codeGen/should_compile/cg008.hs:1
testsuite/tests/codeGen/should_run/cgrun070.hs:4
testsuite/tests/codeGen/should_run/T9577.hs:1
testsuite/tests/codeGen/should_run/cgrun066.hs:1
testsuite/tests/codeGen/should_run/cgrun026.hs:2
testsuite/tests/codeGen/should_run/T2080.hs:2
testsuite/tests/codeGen/should_run/T9577_A.hs:1
testsuite/tests/primops/should_run/T4442.hs:4
testsuite/tests/simplCore/should_compile/T13367.hs:1
testsuite/tests/simplCore/should_compile/T13155.hs:2
testsuite/tests/simplCore/should_compile/T14779a.hs:1
testsuite/tests/simplCore/should_compile/T14779b.hs:2
testsuite/tests/simplCore/should_compile/spec001.hs:1
testsuite/tests/lib/integer/integerGmpInternals.hs:9
testsuite/tests/perf/compiler/T14683.hs:3
testsuite/tests/ghci/should_run/T2589.hs:1
libraries/integer-gmp/src/GHC/Integer/GMP/Internals.hs:7
libraries/text/Data/Text/Internal/Fusion/Common.hs:3
libraries/base/GHC/IO/Encoding/CodePage/Table.hs:1



On Fri, Oct 26, 2018 at 11:01 AM Daniel Cartwright <[hidden email]> wrote:
Also, where in base should Addr go?

On Fri, Oct 26, 2018 at 10:15 AM Daniel Cartwright <[hidden email]> wrote:
I would, yeah. Should the next version of `primitive` just re-export `Addr` from base in `Data.Primitive.Addr`?

On Fri, Oct 26, 2018 at 10:14 AM Carter Schonwald <[hidden email]> wrote:
Daniel or David, would either of you be interested/willing to do the legwork on this patch wise (subject to sanity checking whats the impact surface area?)

On Fri, Oct 26, 2018 at 12:04 AM Carter Schonwald <[hidden email]> wrote:
Cool. What apis in base should move to use Addr from fake pointer ? Cause just adding it in isolation seems lame! 

I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) what apis would be changed?

2) what internal things would be changed albeit not Api visible 

3). Address arithmetic for Addr is very different from addres arithmetic on larger than word8 size storable values, is there anything which would move representations where the change to byte indexed would change the calculations ?


If we can layout what the impact / scope of changes needed for these and other considerations, I guess I’d be all for it. Assuming there’s the corresponding interest in executing changes. (I just have this worry in my head that it might be one of those icebergs.. like the split base effort )



On Thu, Oct 25, 2018 at 11:33 PM David Feuer <[hidden email]> wrote:
On Thu, Oct 25, 2018, 11:28 PM Carter Schonwald <[hidden email]> wrote:
Pretending we’re talking about this with storable as our example semantics : 

Ptr Void 
Corresponds to a memory address you don’t want to read or write to.

This is not so easy to think about (there's not much utility in such a type). Perhaps Ptr Void is a reasonable type for pointers known to be null or outside the address space, but I'm not sure.


Ptr () corresponds to a memory location that’s pretty boring to read / write to

Yes! In other words, it's *not* what people should be using for a pointer to an arbitrary non-Haskell address. That's precisely what Addr is for.


On Thu, Oct 25, 2018 at 11:22 PM Carter Schonwald <[hidden email]> wrote:
... when is a valid pointer not going to point at byte addressable memory on memory architectures ghc can support or target? 

On Thu, Oct 25, 2018 at 8:27 PM Daniel Cartwright <[hidden email]> wrote:
Now, one could argue that `Ptr ()` isn't a lie, it sort of reads like C's void pointer. But surely something like `Ptr Word8` is a lie, when it is not actually a Ptr to Word8 values.

On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright <[hidden email]> wrote:
yes, only the type and its instances should be moved as far as i'm aware.

Also, it's more than just base.

in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO ()`, this Ptr () is also a lie

These are just off the top of my head, there are more


On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald <[hidden email]> wrote:
hrmm, what are the pieces of base that are using Ptr when they really should be using Addr? This would help me understand what would be made better in base :) 

On Thu, Oct 25, 2018 at 6:19 PM David Feuer <[hidden email]> wrote:
We shouldn't really need to move anything into base except Addr and its base instances.

On Oct 25, 2018 5:59 PM, "Carter Schonwald" <[hidden email]> wrote:
Indeed.  The monad transformer instances for primmonad need to live in primmonad OR transformers to avoid orphans. 

Either way, unless transformers moves into base (unlikely), no way anything using prim monad will. 

On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin <[hidden email]> wrote:
I like the idea of moving the type Addr into base. But we cannot move the entire module since it has functions that talk about PrimMonad, and we definitely don't want to move that into base.

On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright <[hidden email]> wrote:
Motivation: There are a lot of places in base where 'Ptr a' is used in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. The problem lies in the fact that many of these uses of 'Ptr a' are lying; the 'a' value is meaningless. Authors of functions therein have used things like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they say they mean - they're just Addr. There are probably other motivations for this that I can't think of off the top of my head right now.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
granted most of the Addr# sites dont matter

a related issue is what operations WOULD be added for Addr? 

maybe just memcopy friends plus address arithmetic, plus a mapper from/to Ptr a?

does it simply become a byte size step sibling to Ptr a that put in Foreign.Ptr?

On Fri, Oct 26, 2018 at 12:18 PM Carter Schonwald <[hidden email]> wrote:
Before we bike shed these details, let’s evaluate / document the pieces of base that should also be moved to using Addr.  Though that module where

1) What exported types / data definitions / functions will have changes ?

2) what internal apis need to change?

as a lower bound estimate of work surface area, i grepped for "Ptr ()"  and "Addr#" in a fresh recursive clone of ghc at commit 503514b94f8dc7bd9eab5392206649aee45f140b,
limiting my search to .hs and .hsc files

THERES A LOT of uses of Ptr (), and its siblings StablePtr () and RemotePtr () ,   The latter two dont have a simple Addr# underneath, 

heres the list of files that match that search   rg 'Ptr \(\)' --type haskell -c

ghc/GHCi/Leak.hs:1
utils/hsc2hs/Main.hs:1
utils/runghc/Main.hs:1
compiler/ghci/GHCi.hs:4
compiler/ghci/ByteCodeLink.hs:3
compiler/ghci/ByteCodeGen.hs:4
compiler/ghci/ByteCodeTypes.hs:1
compiler/ghci/ByteCodeAsm.hs:2
compiler/main/HscTypes.hs:1
libraries/ghci/GHCi/ObjLink.hs:3
libraries/ghci/GHCi/StaticPtrTable.hs:1
libraries/ghci/GHCi/ResolvedBCO.hs:1
libraries/ghci/GHCi/RemoteTypes.hs:2
libraries/ghci/GHCi/Run.hs:2
libraries/ghci/GHCi/Message.hs:5
libraries/base/GHC/Stable.hs:2
libraries/base/GHC/StaticPtr.hs:1
libraries/base/GHC/Generics.hs:2
libraries/base/GHC/TopHandler.hs:1
libraries/base/GHC/Windows.hs:1
libraries/base/tests/dynamic002.hs:2
utils/haddock/haddock-api/src/Haddock.hs:1
libraries/array/Data/Array/IO.hs:2
libraries/base/Foreign/Marshal/Pool.hs:1
libraries/bytestring/Data/ByteString/Builder/Extra.hs:1
libraries/base/GHC/IO/Handle/Text.hs:1
testsuite/tests/rts/T7160.hs:1
testsuite/tests/rts/overflow3.hs:1
testsuite/tests/rts/overflow2.hs:1
testsuite/tests/rts/T5250.hs:1
testsuite/tests/rts/T10296b.hs:2
testsuite/tests/th/T3319.hs:1
testsuite/tests/th/TH_foreignInterruptible.hs:1
testsuite/tests/th/TH_foreignCallingConventions.hs:1
libraries/Cabal/cabal-install/Distribution/Client/Win32SelfUpgrade.hs:1
libraries/Cabal/Cabal/Distribution/Simple/InstallDirs.hs:2
testsuite/tests/llvm/should_compile/T5486.hs:2
testsuite/tests/codeGen/should_run/cgrun070.hs:5
testsuite/tests/codeGen/should_run/cgrun026.hs:3
testsuite/tests/primops/should_run/T4442.hs:1
testsuite/tests/ffi/should_fail/T7243.hs:1
testsuite/tests/ffi/should_fail/T3066.hs:1
testsuite/tests/ffi/should_compile/cc009.hs:1
testsuite/tests/ffi/should_run/ffi021.hs:1
testsuite/tests/typecheck/should_compile/T7408.hs:1
libraries/Cabal/cabal-install/Distribution/Client/Compat/ExecutablePath.hs:1
libraries/Cabal/Cabal/Distribution/Simple/Build/PathsModule.hs:1
libraries/time/lib/Data/Time/Clock/Internal/CTimeval.hs:1
testsuite/tests/rename/should_compile/T3103/Foreign/Ptr.hs:1

likewise the locations where Addr# comes up are as follows (some of these perhaps should be an Addr? )

$ rg 'Addr#' --type haskell -c
utils/genprimopcode/Main.hs:2
ghc/GHCi/Leak.hs:2
compiler/cmm/CmmMachOp.hs:1
compiler/cmm/CmmUtils.hs:1
compiler/specialise/Rules.hs:2
compiler/basicTypes/MkId.hs:2
compiler/basicTypes/Literal.hs:6
compiler/basicTypes/Unique.hs:2
compiler/utils/StringBuffer.hs:1
compiler/utils/FastString.hs:4
compiler/utils/Encoding.hs:9
compiler/codeGen/StgCmmPrim.hs:3
compiler/coreSyn/MkCore.hs:5
compiler/coreSyn/CoreLint.hs:2
compiler/coreSyn/CoreSyn.hs:4
compiler/coreSyn/CorePrep.hs:1
compiler/stranal/WwLib.hs:1
compiler/simplCore/CSE.hs:1
compiler/simplCore/Simplify.hs:2
compiler/deSugar/DsExpr.hs:1
compiler/typecheck/TcGenDeriv.hs:5
compiler/prelude/ForeignCall.hs:1
compiler/prelude/PrelNames.hs:1
compiler/prelude/TysPrim.hs:1
compiler/ghci/ByteCodeGen.hs:8
libraries/bytestring/tests/revcomp.hs:1
libraries/bytestring/Data/ByteString.hs:5
libraries/ghc-heap/tests/heap_all.hs:1
libraries/ghc-prim/GHC/CString.hs:17
libraries/ghc-prim/GHC/Classes.hs:1
libraries/ghc-prim/GHC/Types.hs:3
libraries/ghci/GHCi/BinaryArray.hs:1
libraries/ghci/GHCi/CreateBCO.hs:5
libraries/base/codepages/MakeTable.hs:1
libraries/base/GHC/Ptr.hs:8
libraries/base/GHC/Generics.hs:5
libraries/base/GHC/Storable.hs:32
libraries/base/GHC/ForeignPtr.hs:6
libraries/base/Foreign/Ptr.hs:2
libraries/bytestring/Data/ByteString/Unsafe.hs:6
libraries/bytestring/Data/ByteString/Internal.hs:7
libraries/ghc-heap/GHC/Exts/Heap.hs:1
libraries/text/Data/Text/Show.hs:2
libraries/text/Data/Text/Lazy.hs:2
libraries/containers/Data/IntSet/Internal.hs:4
libraries/ghc-compact/GHC/Compact/Serialized.hs:11
libraries/base/GHC/IO/Exception.hs:1
libraries/base/Data/Typeable/Internal.hs:3
libraries/base/Control/Exception/Base.hs:1
utils/haddock/html-test/src/Ticket112.hs:1
testsuite/tests/rts/T8242.hs:19
testsuite/tests/ghci.debugger/HappyTest.hs:3
testsuite/tests/arityanal/Main.hs:3
testsuite/tests/th/TH_StringPrimL.hs:2
testsuite/tests/th/T1830_3a.hs:2
libraries/template-haskell/Language/Haskell/TH/Syntax.hs:1
libraries/bytestring/Data/ByteString/Short/Internal.hs:16
libraries/integer-gmp/src/GHC/Integer/Type.hs:11
libraries/base/GHC/IO/Encoding/CodePage.hs:3
libraries/Cabal/Cabal/Distribution/Parsec/Lexer.hs:9
testsuite/tests/generics/GEq/GEq1A.hs:1
testsuite/tests/generics/GEq/GEq1.hs:2
testsuite/tests/codeGen/should_compile/cg008.hs:1
testsuite/tests/codeGen/should_run/cgrun070.hs:4
testsuite/tests/codeGen/should_run/T9577.hs:1
testsuite/tests/codeGen/should_run/cgrun066.hs:1
testsuite/tests/codeGen/should_run/cgrun026.hs:2
testsuite/tests/codeGen/should_run/T2080.hs:2
testsuite/tests/codeGen/should_run/T9577_A.hs:1
testsuite/tests/primops/should_run/T4442.hs:4
testsuite/tests/simplCore/should_compile/T13367.hs:1
testsuite/tests/simplCore/should_compile/T13155.hs:2
testsuite/tests/simplCore/should_compile/T14779a.hs:1
testsuite/tests/simplCore/should_compile/T14779b.hs:2
testsuite/tests/simplCore/should_compile/spec001.hs:1
testsuite/tests/lib/integer/integerGmpInternals.hs:9
testsuite/tests/perf/compiler/T14683.hs:3
testsuite/tests/ghci/should_run/T2589.hs:1
libraries/integer-gmp/src/GHC/Integer/GMP/Internals.hs:7
libraries/text/Data/Text/Internal/Fusion/Common.hs:3
libraries/base/GHC/IO/Encoding/CodePage/Table.hs:1



On Fri, Oct 26, 2018 at 11:01 AM Daniel Cartwright <[hidden email]> wrote:
Also, where in base should Addr go?

On Fri, Oct 26, 2018 at 10:15 AM Daniel Cartwright <[hidden email]> wrote:
I would, yeah. Should the next version of `primitive` just re-export `Addr` from base in `Data.Primitive.Addr`?

On Fri, Oct 26, 2018 at 10:14 AM Carter Schonwald <[hidden email]> wrote:
Daniel or David, would either of you be interested/willing to do the legwork on this patch wise (subject to sanity checking whats the impact surface area?)

On Fri, Oct 26, 2018 at 12:04 AM Carter Schonwald <[hidden email]> wrote:
Cool. What apis in base should move to use Addr from fake pointer ? Cause just adding it in isolation seems lame! 

I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) what apis would be changed?

2) what internal things would be changed albeit not Api visible 

3). Address arithmetic for Addr is very different from addres arithmetic on larger than word8 size storable values, is there anything which would move representations where the change to byte indexed would change the calculations ?


If we can layout what the impact / scope of changes needed for these and other considerations, I guess I’d be all for it. Assuming there’s the corresponding interest in executing changes. (I just have this worry in my head that it might be one of those icebergs.. like the split base effort )



On Thu, Oct 25, 2018 at 11:33 PM David Feuer <[hidden email]> wrote:
On Thu, Oct 25, 2018, 11:28 PM Carter Schonwald <[hidden email]> wrote:
Pretending we’re talking about this with storable as our example semantics : 

Ptr Void 
Corresponds to a memory address you don’t want to read or write to.

This is not so easy to think about (there's not much utility in such a type). Perhaps Ptr Void is a reasonable type for pointers known to be null or outside the address space, but I'm not sure.


Ptr () corresponds to a memory location that’s pretty boring to read / write to

Yes! In other words, it's *not* what people should be using for a pointer to an arbitrary non-Haskell address. That's precisely what Addr is for.


On Thu, Oct 25, 2018 at 11:22 PM Carter Schonwald <[hidden email]> wrote:
... when is a valid pointer not going to point at byte addressable memory on memory architectures ghc can support or target? 

On Thu, Oct 25, 2018 at 8:27 PM Daniel Cartwright <[hidden email]> wrote:
Now, one could argue that `Ptr ()` isn't a lie, it sort of reads like C's void pointer. But surely something like `Ptr Word8` is a lie, when it is not actually a Ptr to Word8 values.

On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright <[hidden email]> wrote:
yes, only the type and its instances should be moved as far as i'm aware.

Also, it's more than just base.

in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO ()`, this Ptr () is also a lie

These are just off the top of my head, there are more


On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald <[hidden email]> wrote:
hrmm, what are the pieces of base that are using Ptr when they really should be using Addr? This would help me understand what would be made better in base :) 

On Thu, Oct 25, 2018 at 6:19 PM David Feuer <[hidden email]> wrote:
We shouldn't really need to move anything into base except Addr and its base instances.

On Oct 25, 2018 5:59 PM, "Carter Schonwald" <[hidden email]> wrote:
Indeed.  The monad transformer instances for primmonad need to live in primmonad OR transformers to avoid orphans. 

Either way, unless transformers moves into base (unlikely), no way anything using prim monad will. 

On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin <[hidden email]> wrote:
I like the idea of moving the type Addr into base. But we cannot move the entire module since it has functions that talk about PrimMonad, and we definitely don't want to move that into base.

On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright <[hidden email]> wrote:
Motivation: There are a lot of places in base where 'Ptr a' is used in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. The problem lies in the fact that many of these uses of 'Ptr a' are lying; the 'a' value is meaningless. Authors of functions therein have used things like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they say they mean - they're just Addr. There are probably other motivations for this that I can't think of off the top of my head right now.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
On the flip side we could just add data Addr ... to foreign .ptr or a child / sibling module like System.Mem.Address and defer the associated cleanup

I kinda like the idea of System.Mem.Address, 

i put a strawman change set on phab https://phabricator.haskell.org/D5268

On Fri, Oct 26, 2018 at 12:23 PM Carter Schonwald <[hidden email]> wrote:
granted most of the Addr# sites dont matter

a related issue is what operations WOULD be added for Addr? 

maybe just memcopy friends plus address arithmetic, plus a mapper from/to Ptr a?

does it simply become a byte size step sibling to Ptr a that put in Foreign.Ptr?

On Fri, Oct 26, 2018 at 12:18 PM Carter Schonwald <[hidden email]> wrote:
Before we bike shed these details, let’s evaluate / document the pieces of base that should also be moved to using Addr.  Though that module where

1) What exported types / data definitions / functions will have changes ?

2) what internal apis need to change?

as a lower bound estimate of work surface area, i grepped for "Ptr ()"  and "Addr#" in a fresh recursive clone of ghc at commit 503514b94f8dc7bd9eab5392206649aee45f140b,
limiting my search to .hs and .hsc files

THERES A LOT of uses of Ptr (), and its siblings StablePtr () and RemotePtr () ,   The latter two dont have a simple Addr# underneath, 

heres the list of files that match that search   rg 'Ptr \(\)' --type haskell -c

ghc/GHCi/Leak.hs:1
utils/hsc2hs/Main.hs:1
utils/runghc/Main.hs:1
compiler/ghci/GHCi.hs:4
compiler/ghci/ByteCodeLink.hs:3
compiler/ghci/ByteCodeGen.hs:4
compiler/ghci/ByteCodeTypes.hs:1
compiler/ghci/ByteCodeAsm.hs:2
compiler/main/HscTypes.hs:1
libraries/ghci/GHCi/ObjLink.hs:3
libraries/ghci/GHCi/StaticPtrTable.hs:1
libraries/ghci/GHCi/ResolvedBCO.hs:1
libraries/ghci/GHCi/RemoteTypes.hs:2
libraries/ghci/GHCi/Run.hs:2
libraries/ghci/GHCi/Message.hs:5
libraries/base/GHC/Stable.hs:2
libraries/base/GHC/StaticPtr.hs:1
libraries/base/GHC/Generics.hs:2
libraries/base/GHC/TopHandler.hs:1
libraries/base/GHC/Windows.hs:1
libraries/base/tests/dynamic002.hs:2
utils/haddock/haddock-api/src/Haddock.hs:1
libraries/array/Data/Array/IO.hs:2
libraries/base/Foreign/Marshal/Pool.hs:1
libraries/bytestring/Data/ByteString/Builder/Extra.hs:1
libraries/base/GHC/IO/Handle/Text.hs:1
testsuite/tests/rts/T7160.hs:1
testsuite/tests/rts/overflow3.hs:1
testsuite/tests/rts/overflow2.hs:1
testsuite/tests/rts/T5250.hs:1
testsuite/tests/rts/T10296b.hs:2
testsuite/tests/th/T3319.hs:1
testsuite/tests/th/TH_foreignInterruptible.hs:1
testsuite/tests/th/TH_foreignCallingConventions.hs:1
libraries/Cabal/cabal-install/Distribution/Client/Win32SelfUpgrade.hs:1
libraries/Cabal/Cabal/Distribution/Simple/InstallDirs.hs:2
testsuite/tests/llvm/should_compile/T5486.hs:2
testsuite/tests/codeGen/should_run/cgrun070.hs:5
testsuite/tests/codeGen/should_run/cgrun026.hs:3
testsuite/tests/primops/should_run/T4442.hs:1
testsuite/tests/ffi/should_fail/T7243.hs:1
testsuite/tests/ffi/should_fail/T3066.hs:1
testsuite/tests/ffi/should_compile/cc009.hs:1
testsuite/tests/ffi/should_run/ffi021.hs:1
testsuite/tests/typecheck/should_compile/T7408.hs:1
libraries/Cabal/cabal-install/Distribution/Client/Compat/ExecutablePath.hs:1
libraries/Cabal/Cabal/Distribution/Simple/Build/PathsModule.hs:1
libraries/time/lib/Data/Time/Clock/Internal/CTimeval.hs:1
testsuite/tests/rename/should_compile/T3103/Foreign/Ptr.hs:1

likewise the locations where Addr# comes up are as follows (some of these perhaps should be an Addr? )

$ rg 'Addr#' --type haskell -c
utils/genprimopcode/Main.hs:2
ghc/GHCi/Leak.hs:2
compiler/cmm/CmmMachOp.hs:1
compiler/cmm/CmmUtils.hs:1
compiler/specialise/Rules.hs:2
compiler/basicTypes/MkId.hs:2
compiler/basicTypes/Literal.hs:6
compiler/basicTypes/Unique.hs:2
compiler/utils/StringBuffer.hs:1
compiler/utils/FastString.hs:4
compiler/utils/Encoding.hs:9
compiler/codeGen/StgCmmPrim.hs:3
compiler/coreSyn/MkCore.hs:5
compiler/coreSyn/CoreLint.hs:2
compiler/coreSyn/CoreSyn.hs:4
compiler/coreSyn/CorePrep.hs:1
compiler/stranal/WwLib.hs:1
compiler/simplCore/CSE.hs:1
compiler/simplCore/Simplify.hs:2
compiler/deSugar/DsExpr.hs:1
compiler/typecheck/TcGenDeriv.hs:5
compiler/prelude/ForeignCall.hs:1
compiler/prelude/PrelNames.hs:1
compiler/prelude/TysPrim.hs:1
compiler/ghci/ByteCodeGen.hs:8
libraries/bytestring/tests/revcomp.hs:1
libraries/bytestring/Data/ByteString.hs:5
libraries/ghc-heap/tests/heap_all.hs:1
libraries/ghc-prim/GHC/CString.hs:17
libraries/ghc-prim/GHC/Classes.hs:1
libraries/ghc-prim/GHC/Types.hs:3
libraries/ghci/GHCi/BinaryArray.hs:1
libraries/ghci/GHCi/CreateBCO.hs:5
libraries/base/codepages/MakeTable.hs:1
libraries/base/GHC/Ptr.hs:8
libraries/base/GHC/Generics.hs:5
libraries/base/GHC/Storable.hs:32
libraries/base/GHC/ForeignPtr.hs:6
libraries/base/Foreign/Ptr.hs:2
libraries/bytestring/Data/ByteString/Unsafe.hs:6
libraries/bytestring/Data/ByteString/Internal.hs:7
libraries/ghc-heap/GHC/Exts/Heap.hs:1
libraries/text/Data/Text/Show.hs:2
libraries/text/Data/Text/Lazy.hs:2
libraries/containers/Data/IntSet/Internal.hs:4
libraries/ghc-compact/GHC/Compact/Serialized.hs:11
libraries/base/GHC/IO/Exception.hs:1
libraries/base/Data/Typeable/Internal.hs:3
libraries/base/Control/Exception/Base.hs:1
utils/haddock/html-test/src/Ticket112.hs:1
testsuite/tests/rts/T8242.hs:19
testsuite/tests/ghci.debugger/HappyTest.hs:3
testsuite/tests/arityanal/Main.hs:3
testsuite/tests/th/TH_StringPrimL.hs:2
testsuite/tests/th/T1830_3a.hs:2
libraries/template-haskell/Language/Haskell/TH/Syntax.hs:1
libraries/bytestring/Data/ByteString/Short/Internal.hs:16
libraries/integer-gmp/src/GHC/Integer/Type.hs:11
libraries/base/GHC/IO/Encoding/CodePage.hs:3
libraries/Cabal/Cabal/Distribution/Parsec/Lexer.hs:9
testsuite/tests/generics/GEq/GEq1A.hs:1
testsuite/tests/generics/GEq/GEq1.hs:2
testsuite/tests/codeGen/should_compile/cg008.hs:1
testsuite/tests/codeGen/should_run/cgrun070.hs:4
testsuite/tests/codeGen/should_run/T9577.hs:1
testsuite/tests/codeGen/should_run/cgrun066.hs:1
testsuite/tests/codeGen/should_run/cgrun026.hs:2
testsuite/tests/codeGen/should_run/T2080.hs:2
testsuite/tests/codeGen/should_run/T9577_A.hs:1
testsuite/tests/primops/should_run/T4442.hs:4
testsuite/tests/simplCore/should_compile/T13367.hs:1
testsuite/tests/simplCore/should_compile/T13155.hs:2
testsuite/tests/simplCore/should_compile/T14779a.hs:1
testsuite/tests/simplCore/should_compile/T14779b.hs:2
testsuite/tests/simplCore/should_compile/spec001.hs:1
testsuite/tests/lib/integer/integerGmpInternals.hs:9
testsuite/tests/perf/compiler/T14683.hs:3
testsuite/tests/ghci/should_run/T2589.hs:1
libraries/integer-gmp/src/GHC/Integer/GMP/Internals.hs:7
libraries/text/Data/Text/Internal/Fusion/Common.hs:3
libraries/base/GHC/IO/Encoding/CodePage/Table.hs:1



On Fri, Oct 26, 2018 at 11:01 AM Daniel Cartwright <[hidden email]> wrote:
Also, where in base should Addr go?

On Fri, Oct 26, 2018 at 10:15 AM Daniel Cartwright <[hidden email]> wrote:
I would, yeah. Should the next version of `primitive` just re-export `Addr` from base in `Data.Primitive.Addr`?

On Fri, Oct 26, 2018 at 10:14 AM Carter Schonwald <[hidden email]> wrote:
Daniel or David, would either of you be interested/willing to do the legwork on this patch wise (subject to sanity checking whats the impact surface area?)

On Fri, Oct 26, 2018 at 12:04 AM Carter Schonwald <[hidden email]> wrote:
Cool. What apis in base should move to use Addr from fake pointer ? Cause just adding it in isolation seems lame! 

I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) what apis would be changed?

2) what internal things would be changed albeit not Api visible 

3). Address arithmetic for Addr is very different from addres arithmetic on larger than word8 size storable values, is there anything which would move representations where the change to byte indexed would change the calculations ?


If we can layout what the impact / scope of changes needed for these and other considerations, I guess I’d be all for it. Assuming there’s the corresponding interest in executing changes. (I just have this worry in my head that it might be one of those icebergs.. like the split base effort )



On Thu, Oct 25, 2018 at 11:33 PM David Feuer <[hidden email]> wrote:
On Thu, Oct 25, 2018, 11:28 PM Carter Schonwald <[hidden email]> wrote:
Pretending we’re talking about this with storable as our example semantics : 

Ptr Void 
Corresponds to a memory address you don’t want to read or write to.

This is not so easy to think about (there's not much utility in such a type). Perhaps Ptr Void is a reasonable type for pointers known to be null or outside the address space, but I'm not sure.


Ptr () corresponds to a memory location that’s pretty boring to read / write to

Yes! In other words, it's *not* what people should be using for a pointer to an arbitrary non-Haskell address. That's precisely what Addr is for.


On Thu, Oct 25, 2018 at 11:22 PM Carter Schonwald <[hidden email]> wrote:
... when is a valid pointer not going to point at byte addressable memory on memory architectures ghc can support or target? 

On Thu, Oct 25, 2018 at 8:27 PM Daniel Cartwright <[hidden email]> wrote:
Now, one could argue that `Ptr ()` isn't a lie, it sort of reads like C's void pointer. But surely something like `Ptr Word8` is a lie, when it is not actually a Ptr to Word8 values.

On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright <[hidden email]> wrote:
yes, only the type and its instances should be moved as far as i'm aware.

Also, it's more than just base.

in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO ()`, this Ptr () is also a lie

These are just off the top of my head, there are more


On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald <[hidden email]> wrote:
hrmm, what are the pieces of base that are using Ptr when they really should be using Addr? This would help me understand what would be made better in base :) 

On Thu, Oct 25, 2018 at 6:19 PM David Feuer <[hidden email]> wrote:
We shouldn't really need to move anything into base except Addr and its base instances.

On Oct 25, 2018 5:59 PM, "Carter Schonwald" <[hidden email]> wrote:
Indeed.  The monad transformer instances for primmonad need to live in primmonad OR transformers to avoid orphans. 

Either way, unless transformers moves into base (unlikely), no way anything using prim monad will. 

On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin <[hidden email]> wrote:
I like the idea of moving the type Addr into base. But we cannot move the entire module since it has functions that talk about PrimMonad, and we definitely don't want to move that into base.

On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright <[hidden email]> wrote:
Motivation: There are a lot of places in base where 'Ptr a' is used in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. The problem lies in the fact that many of these uses of 'Ptr a' are lying; the 'a' value is meaningless. Authors of functions therein have used things like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they say they mean - they're just Addr. There are probably other motivations for this that I can't think of off the top of my head right now.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald

On Fri, Oct 26, 2018 at 1:08 PM Carter Schonwald <[hidden email]> wrote:
On the flip side we could just add data Addr ... to foreign .ptr or a child / sibling module like System.Mem.Address and defer the associated cleanup

I kinda like the idea of System.Mem.Address, 

i put a strawman change set on phab https://phabricator.haskell.org/D5268

On Fri, Oct 26, 2018 at 12:23 PM Carter Schonwald <[hidden email]> wrote:
granted most of the Addr# sites dont matter

a related issue is what operations WOULD be added for Addr? 

maybe just memcopy friends plus address arithmetic, plus a mapper from/to Ptr a?

does it simply become a byte size step sibling to Ptr a that put in Foreign.Ptr?

On Fri, Oct 26, 2018 at 12:18 PM Carter Schonwald <[hidden email]> wrote:
Before we bike shed these details, let’s evaluate / document the pieces of base that should also be moved to using Addr.  Though that module where

1) What exported types / data definitions / functions will have changes ?

2) what internal apis need to change?

as a lower bound estimate of work surface area, i grepped for "Ptr ()"  and "Addr#" in a fresh recursive clone of ghc at commit 503514b94f8dc7bd9eab5392206649aee45f140b,
limiting my search to .hs and .hsc files

THERES A LOT of uses of Ptr (), and its siblings StablePtr () and RemotePtr () ,   The latter two dont have a simple Addr# underneath, 

heres the list of files that match that search   rg 'Ptr \(\)' --type haskell -c

ghc/GHCi/Leak.hs:1
utils/hsc2hs/Main.hs:1
utils/runghc/Main.hs:1
compiler/ghci/GHCi.hs:4
compiler/ghci/ByteCodeLink.hs:3
compiler/ghci/ByteCodeGen.hs:4
compiler/ghci/ByteCodeTypes.hs:1
compiler/ghci/ByteCodeAsm.hs:2
compiler/main/HscTypes.hs:1
libraries/ghci/GHCi/ObjLink.hs:3
libraries/ghci/GHCi/StaticPtrTable.hs:1
libraries/ghci/GHCi/ResolvedBCO.hs:1
libraries/ghci/GHCi/RemoteTypes.hs:2
libraries/ghci/GHCi/Run.hs:2
libraries/ghci/GHCi/Message.hs:5
libraries/base/GHC/Stable.hs:2
libraries/base/GHC/StaticPtr.hs:1
libraries/base/GHC/Generics.hs:2
libraries/base/GHC/TopHandler.hs:1
libraries/base/GHC/Windows.hs:1
libraries/base/tests/dynamic002.hs:2
utils/haddock/haddock-api/src/Haddock.hs:1
libraries/array/Data/Array/IO.hs:2
libraries/base/Foreign/Marshal/Pool.hs:1
libraries/bytestring/Data/ByteString/Builder/Extra.hs:1
libraries/base/GHC/IO/Handle/Text.hs:1
testsuite/tests/rts/T7160.hs:1
testsuite/tests/rts/overflow3.hs:1
testsuite/tests/rts/overflow2.hs:1
testsuite/tests/rts/T5250.hs:1
testsuite/tests/rts/T10296b.hs:2
testsuite/tests/th/T3319.hs:1
testsuite/tests/th/TH_foreignInterruptible.hs:1
testsuite/tests/th/TH_foreignCallingConventions.hs:1
libraries/Cabal/cabal-install/Distribution/Client/Win32SelfUpgrade.hs:1
libraries/Cabal/Cabal/Distribution/Simple/InstallDirs.hs:2
testsuite/tests/llvm/should_compile/T5486.hs:2
testsuite/tests/codeGen/should_run/cgrun070.hs:5
testsuite/tests/codeGen/should_run/cgrun026.hs:3
testsuite/tests/primops/should_run/T4442.hs:1
testsuite/tests/ffi/should_fail/T7243.hs:1
testsuite/tests/ffi/should_fail/T3066.hs:1
testsuite/tests/ffi/should_compile/cc009.hs:1
testsuite/tests/ffi/should_run/ffi021.hs:1
testsuite/tests/typecheck/should_compile/T7408.hs:1
libraries/Cabal/cabal-install/Distribution/Client/Compat/ExecutablePath.hs:1
libraries/Cabal/Cabal/Distribution/Simple/Build/PathsModule.hs:1
libraries/time/lib/Data/Time/Clock/Internal/CTimeval.hs:1
testsuite/tests/rename/should_compile/T3103/Foreign/Ptr.hs:1

likewise the locations where Addr# comes up are as follows (some of these perhaps should be an Addr? )

$ rg 'Addr#' --type haskell -c
utils/genprimopcode/Main.hs:2
ghc/GHCi/Leak.hs:2
compiler/cmm/CmmMachOp.hs:1
compiler/cmm/CmmUtils.hs:1
compiler/specialise/Rules.hs:2
compiler/basicTypes/MkId.hs:2
compiler/basicTypes/Literal.hs:6
compiler/basicTypes/Unique.hs:2
compiler/utils/StringBuffer.hs:1
compiler/utils/FastString.hs:4
compiler/utils/Encoding.hs:9
compiler/codeGen/StgCmmPrim.hs:3
compiler/coreSyn/MkCore.hs:5
compiler/coreSyn/CoreLint.hs:2
compiler/coreSyn/CoreSyn.hs:4
compiler/coreSyn/CorePrep.hs:1
compiler/stranal/WwLib.hs:1
compiler/simplCore/CSE.hs:1
compiler/simplCore/Simplify.hs:2
compiler/deSugar/DsExpr.hs:1
compiler/typecheck/TcGenDeriv.hs:5
compiler/prelude/ForeignCall.hs:1
compiler/prelude/PrelNames.hs:1
compiler/prelude/TysPrim.hs:1
compiler/ghci/ByteCodeGen.hs:8
libraries/bytestring/tests/revcomp.hs:1
libraries/bytestring/Data/ByteString.hs:5
libraries/ghc-heap/tests/heap_all.hs:1
libraries/ghc-prim/GHC/CString.hs:17
libraries/ghc-prim/GHC/Classes.hs:1
libraries/ghc-prim/GHC/Types.hs:3
libraries/ghci/GHCi/BinaryArray.hs:1
libraries/ghci/GHCi/CreateBCO.hs:5
libraries/base/codepages/MakeTable.hs:1
libraries/base/GHC/Ptr.hs:8
libraries/base/GHC/Generics.hs:5
libraries/base/GHC/Storable.hs:32
libraries/base/GHC/ForeignPtr.hs:6
libraries/base/Foreign/Ptr.hs:2
libraries/bytestring/Data/ByteString/Unsafe.hs:6
libraries/bytestring/Data/ByteString/Internal.hs:7
libraries/ghc-heap/GHC/Exts/Heap.hs:1
libraries/text/Data/Text/Show.hs:2
libraries/text/Data/Text/Lazy.hs:2
libraries/containers/Data/IntSet/Internal.hs:4
libraries/ghc-compact/GHC/Compact/Serialized.hs:11
libraries/base/GHC/IO/Exception.hs:1
libraries/base/Data/Typeable/Internal.hs:3
libraries/base/Control/Exception/Base.hs:1
utils/haddock/html-test/src/Ticket112.hs:1
testsuite/tests/rts/T8242.hs:19
testsuite/tests/ghci.debugger/HappyTest.hs:3
testsuite/tests/arityanal/Main.hs:3
testsuite/tests/th/TH_StringPrimL.hs:2
testsuite/tests/th/T1830_3a.hs:2
libraries/template-haskell/Language/Haskell/TH/Syntax.hs:1
libraries/bytestring/Data/ByteString/Short/Internal.hs:16
libraries/integer-gmp/src/GHC/Integer/Type.hs:11
libraries/base/GHC/IO/Encoding/CodePage.hs:3
libraries/Cabal/Cabal/Distribution/Parsec/Lexer.hs:9
testsuite/tests/generics/GEq/GEq1A.hs:1
testsuite/tests/generics/GEq/GEq1.hs:2
testsuite/tests/codeGen/should_compile/cg008.hs:1
testsuite/tests/codeGen/should_run/cgrun070.hs:4
testsuite/tests/codeGen/should_run/T9577.hs:1
testsuite/tests/codeGen/should_run/cgrun066.hs:1
testsuite/tests/codeGen/should_run/cgrun026.hs:2
testsuite/tests/codeGen/should_run/T2080.hs:2
testsuite/tests/codeGen/should_run/T9577_A.hs:1
testsuite/tests/primops/should_run/T4442.hs:4
testsuite/tests/simplCore/should_compile/T13367.hs:1
testsuite/tests/simplCore/should_compile/T13155.hs:2
testsuite/tests/simplCore/should_compile/T14779a.hs:1
testsuite/tests/simplCore/should_compile/T14779b.hs:2
testsuite/tests/simplCore/should_compile/spec001.hs:1
testsuite/tests/lib/integer/integerGmpInternals.hs:9
testsuite/tests/perf/compiler/T14683.hs:3
testsuite/tests/ghci/should_run/T2589.hs:1
libraries/integer-gmp/src/GHC/Integer/GMP/Internals.hs:7
libraries/text/Data/Text/Internal/Fusion/Common.hs:3
libraries/base/GHC/IO/Encoding/CodePage/Table.hs:1



On Fri, Oct 26, 2018 at 11:01 AM Daniel Cartwright <[hidden email]> wrote:
Also, where in base should Addr go?

On Fri, Oct 26, 2018 at 10:15 AM Daniel Cartwright <[hidden email]> wrote:
I would, yeah. Should the next version of `primitive` just re-export `Addr` from base in `Data.Primitive.Addr`?

On Fri, Oct 26, 2018 at 10:14 AM Carter Schonwald <[hidden email]> wrote:
Daniel or David, would either of you be interested/willing to do the legwork on this patch wise (subject to sanity checking whats the impact surface area?)

On Fri, Oct 26, 2018 at 12:04 AM Carter Schonwald <[hidden email]> wrote:
Cool. What apis in base should move to use Addr from fake pointer ? Cause just adding it in isolation seems lame! 

I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) what apis would be changed?

2) what internal things would be changed albeit not Api visible 

3). Address arithmetic for Addr is very different from addres arithmetic on larger than word8 size storable values, is there anything which would move representations where the change to byte indexed would change the calculations ?


If we can layout what the impact / scope of changes needed for these and other considerations, I guess I’d be all for it. Assuming there’s the corresponding interest in executing changes. (I just have this worry in my head that it might be one of those icebergs.. like the split base effort )



On Thu, Oct 25, 2018 at 11:33 PM David Feuer <[hidden email]> wrote:
On Thu, Oct 25, 2018, 11:28 PM Carter Schonwald <[hidden email]> wrote:
Pretending we’re talking about this with storable as our example semantics : 

Ptr Void 
Corresponds to a memory address you don’t want to read or write to.

This is not so easy to think about (there's not much utility in such a type). Perhaps Ptr Void is a reasonable type for pointers known to be null or outside the address space, but I'm not sure.


Ptr () corresponds to a memory location that’s pretty boring to read / write to

Yes! In other words, it's *not* what people should be using for a pointer to an arbitrary non-Haskell address. That's precisely what Addr is for.


On Thu, Oct 25, 2018 at 11:22 PM Carter Schonwald <[hidden email]> wrote:
... when is a valid pointer not going to point at byte addressable memory on memory architectures ghc can support or target? 

On Thu, Oct 25, 2018 at 8:27 PM Daniel Cartwright <[hidden email]> wrote:
Now, one could argue that `Ptr ()` isn't a lie, it sort of reads like C's void pointer. But surely something like `Ptr Word8` is a lie, when it is not actually a Ptr to Word8 values.

On Thu, Oct 25, 2018 at 8:11 PM Daniel Cartwright <[hidden email]> wrote:
yes, only the type and its instances should be moved as far as i'm aware.

Also, it's more than just base.

in GHC.Stats, the foreign import "getRTSStats" has `Ptr () -> IO ()`, this Ptr () is also a lie

These are just off the top of my head, there are more


On Thu, Oct 25, 2018 at 6:46 PM Carter Schonwald <[hidden email]> wrote:
hrmm, what are the pieces of base that are using Ptr when they really should be using Addr? This would help me understand what would be made better in base :) 

On Thu, Oct 25, 2018 at 6:19 PM David Feuer <[hidden email]> wrote:
We shouldn't really need to move anything into base except Addr and its base instances.

On Oct 25, 2018 5:59 PM, "Carter Schonwald" <[hidden email]> wrote:
Indeed.  The monad transformer instances for primmonad need to live in primmonad OR transformers to avoid orphans. 

Either way, unless transformers moves into base (unlikely), no way anything using prim monad will. 

On Thu, Oct 25, 2018 at 3:34 PM Andrew Martin <[hidden email]> wrote:
I like the idea of moving the type Addr into base. But we cannot move the entire module since it has functions that talk about PrimMonad, and we definitely don't want to move that into base.

On Thu, Oct 25, 2018 at 11:25 AM Daniel Cartwright <[hidden email]> wrote:
Motivation: There are a lot of places in base where 'Ptr a' is used in place of 'Addr', because in base there is no 'Addr', only 'Addr#'. The problem lies in the fact that many of these uses of 'Ptr a' are lying; the 'a' value is meaningless. Authors of functions therein have used things like 'Ptr ()', 'Ptr Word8', 'Ptr a', but these types do not mean what they say they mean - they're just Addr. There are probably other motivations for this that I can't think of off the top of my head right now.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Sven Panne-2
In reply to this post by Henning Thielemann
Am Fr., 26. Okt. 2018 um 11:15 Uhr schrieb Henning Thielemann <[hidden email]>:
[...] There are the Storable methods

peekByteOff :: Ptr b -> Int -> IO a
pokeByteOff :: Ptr b -> Int -> a -> IO ()

They ignore the target types of their pointers.

... and for a good reason: This was intentionally specified this way to avoid the castPtr-Kung-Fu which would arise if the pointer arguments were e.g. "Ptr ()". Remember: You are on a totally unsafe level here, anyway, so adding tons of casts would not really help here much.
 
[...] And if we are touching Storable class, [...]

I really hope that Storable remains *totally* untouched, at least in the sense that it is only changed in a 100% backwards-compatible away. Otherwise basically the whole Haskell ecosystem with its tons of native bindings will explode... There is nothing wrong with a new class, but this one is effectively "untouchable".

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Sven Panne-2
In reply to this post by Carter Schonwald
Am Fr., 26. Okt. 2018 um 05:28 Uhr schrieb Carter Schonwald <[hidden email]>:
[...] Ptr Void 
Corresponds to a memory address you don’t want to read or write to.  

Hmmm, I haven't seen that much in the wild... 

Ptr () corresponds to a memory location that’s pretty boring to read / write to 

The intention in lots of APIs is quite different: A "Ptr ()" is a pointer to some memory of unknown/externally specified format, *and* the consumer of the API should be forced to insert explicit casts, for whatever reason. This can be a good idea, but it can be *extremely* annoying, too, it depends. If you don't want to force casts upon your API users, you use "Ptr a".

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Sven Panne-2
In reply to this post by Carter Schonwald
Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald <[hidden email]>:
[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess? 

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If 

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of  using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive, 

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 
1) i dont see it being super useful within base/ghc apis, as they exist today
2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :) 



On Fri, Oct 26, 2018 at 1:26 PM Sven Panne <[hidden email]> wrote:
Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald <[hidden email]>:
[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

chessai .
there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'. 

'super useful' was never the goal - only correctness and convenience.

Yeah, i agree that Storable should not be touched.

On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald <[hidden email]> wrote:
your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess? 

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If 

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of  using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive, 

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 
1) i dont see it being super useful within base/ghc apis, as they exist today
2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :) 



On Fri, Oct 26, 2018 at 1:26 PM Sven Panne <[hidden email]> wrote:
Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald <[hidden email]>:
[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald

What api? It’s not obvious, could you spell out what you have in mind. Relative to the phab straw man? 

On Fri, Oct 26, 2018 at 1:42 PM Daniel Cartwright <[hidden email]> wrote:
there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'. 

'super useful' was never the goal - only correctness and convenience.

Yeah, i agree that Storable should not be touched.

On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald <[hidden email]> wrote:
your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess? 

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If 

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of  using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive, 

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 
1) i dont see it being super useful within base/ghc apis, as they exist today
2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :) 



On Fri, Oct 26, 2018 at 1:26 PM Sven Panne <[hidden email]> wrote:
Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald <[hidden email]>:
[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.
_______________________________________________

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
anyways, if you want to do a cleaned up sibling of https://phabricator.haskell.org/D5268, i vote yes

On Fri, Oct 26, 2018 at 1:55 PM Carter Schonwald <[hidden email]> wrote:

What api? It’s not obvious, could you spell out what you have in mind. Relative to the phab straw man? 

On Fri, Oct 26, 2018 at 1:42 PM Daniel Cartwright <[hidden email]> wrote:
there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'. 

'super useful' was never the goal - only correctness and convenience.

Yeah, i agree that Storable should not be touched.

On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald <[hidden email]> wrote:
your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess? 

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If 

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of  using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive, 

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 
1) i dont see it being super useful within base/ghc apis, as they exist today
2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :) 



On Fri, Oct 26, 2018 at 1:26 PM Sven Panne <[hidden email]> wrote:
Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald <[hidden email]>:
[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.
_______________________________________________

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

chessai .
In reply to this post by Carter Schonwald
I'm only saying that 'Addr', its base-scoped instances, and the API of 'Data.Primitive.Addr' that is base-scoped should be added to base. Eventually any non-user-facing functions that use 'Ptr LyingType' should be switched from using 'Ptr' to 'Addr'. There's nothing more to my proposal as far as action items.

As far as consequences, users may want to have access to 'Addr' without needing 'primitive'. 'primitive' shouldn't really see any API change, as it will just re-export whatever gets implemented in base for 'Addr'.

On Fri, Oct 26, 2018 at 1:55 PM Carter Schonwald <[hidden email]> wrote:

What api? It’s not obvious, could you spell out what you have in mind. Relative to the phab straw man? 

On Fri, Oct 26, 2018 at 1:42 PM Daniel Cartwright <[hidden email]> wrote:
there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'. 

'super useful' was never the goal - only correctness and convenience.

Yeah, i agree that Storable should not be touched.

On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald <[hidden email]> wrote:
your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess? 

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If 

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of  using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive, 

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 
1) i dont see it being super useful within base/ghc apis, as they exist today
2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :) 



On Fri, Oct 26, 2018 at 1:26 PM Sven Panne <[hidden email]> wrote:
Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald <[hidden email]>:
[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.
_______________________________________________

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

David Feuer
In reply to this post by Carter Schonwald
The arithmetic parts are pretty obvious (indeed, it might be nice to add some more arithmetic primops to treat Addr# as a uintptr_t). The rest are much less obvious; it may make more sense to convert an Addr to a Ptr before performing any dereferencing operations.

On Fri, Oct 26, 2018, 1:55 PM Carter Schonwald <[hidden email]> wrote:

What api? It’s not obvious, could you spell out what you have in mind. Relative to the phab straw man? 

On Fri, Oct 26, 2018 at 1:42 PM Daniel Cartwright <[hidden email]> wrote:
there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'. 

'super useful' was never the goal - only correctness and convenience.

Yeah, i agree that Storable should not be touched.

On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald <[hidden email]> wrote:
your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess? 

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If 

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of  using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive, 

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 
1) i dont see it being super useful within base/ghc apis, as they exist today
2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :) 



On Fri, Oct 26, 2018 at 1:26 PM Sven Panne <[hidden email]> wrote:
Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald <[hidden email]>:
[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.
_______________________________________________
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
yeah, lets do that, so roughly very little impact but easy opt in.

 Is the design  code strawman  in https://phabricator.haskell.org/D5268 acceptable?

[hidden email]  could you help me dig out where in ghc / base / ghc-prims we expose the memmove/memcopy/memset intrinsics? I can only seem to find the c ffi call ones, 
but I know ghc itself can compile that more smartly? i started but then i got lost, seems like everywhere in base its just a bespoked c ffi binding introduced locally ..



On Fri, Oct 26, 2018 at 2:02 PM David Feuer <[hidden email]> wrote:
The arithmetic parts are pretty obvious (indeed, it might be nice to add some more arithmetic primops to treat Addr# as a uintptr_t). The rest are much less obvious; it may make more sense to convert an Addr to a Ptr before performing any dereferencing operations.

On Fri, Oct 26, 2018, 1:55 PM Carter Schonwald <[hidden email]> wrote:

What api? It’s not obvious, could you spell out what you have in mind. Relative to the phab straw man? 

On Fri, Oct 26, 2018 at 1:42 PM Daniel Cartwright <[hidden email]> wrote:
there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'. 

'super useful' was never the goal - only correctness and convenience.

Yeah, i agree that Storable should not be touched.

On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald <[hidden email]> wrote:
your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess? 

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If 

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of  using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive, 

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 
1) i dont see it being super useful within base/ghc apis, as they exist today
2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :) 



On Fri, Oct 26, 2018 at 1:26 PM Sven Panne <[hidden email]> wrote:
Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald <[hidden email]>:
[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.
_______________________________________________
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Carter Schonwald
hrmm, it looks like those aren't exposed directly to the HS layer because of the necessary compiler time static alignment requirements for that in code gen, 


On Fri, Oct 26, 2018 at 2:13 PM Carter Schonwald <[hidden email]> wrote:
yeah, lets do that, so roughly very little impact but easy opt in.

 Is the design  code strawman  in https://phabricator.haskell.org/D5268 acceptable?

[hidden email]  could you help me dig out where in ghc / base / ghc-prims we expose the memmove/memcopy/memset intrinsics? I can only seem to find the c ffi call ones, 
but I know ghc itself can compile that more smartly? i started but then i got lost, seems like everywhere in base its just a bespoked c ffi binding introduced locally ..



On Fri, Oct 26, 2018 at 2:02 PM David Feuer <[hidden email]> wrote:
The arithmetic parts are pretty obvious (indeed, it might be nice to add some more arithmetic primops to treat Addr# as a uintptr_t). The rest are much less obvious; it may make more sense to convert an Addr to a Ptr before performing any dereferencing operations.

On Fri, Oct 26, 2018, 1:55 PM Carter Schonwald <[hidden email]> wrote:

What api? It’s not obvious, could you spell out what you have in mind. Relative to the phab straw man? 

On Fri, Oct 26, 2018 at 1:42 PM Daniel Cartwright <[hidden email]> wrote:
there was never any hint at helping primitive, i'm not sure where you got that from. also, the primary motivation was to avoid using 'Ptr a', where 'a' is a lie. not only that, but base seems a natural home for 'Addr', and people can avoid incurring a dep on primitive when they just need 'Addr'. 

'super useful' was never the goal - only correctness and convenience.

Yeah, i agree that Storable should not be touched.

On Fri, Oct 26, 2018 at 1:36 PM Carter Schonwald <[hidden email]> wrote:
your perspective here is a good one

1) i absolutely agree, nothing should touch storable, whatesoever,

2) the baseline proposal is to add an Addr to Base (essentially Ptr ()), which would be a crippled sibling of how its exposed in Data.Primitive I guess? 

3) as a litmus for "what would this acomplish/support", i was asking "how would this get used/help base be nicer "? If 

anyways: i did a strawman of what Addr ripped out of Data.Primitive.Addr and into base would look like, and it doesn't look especially compelling / nicer than Ptr shenanigans. Especially since to be useful in base it would have to have a bunch of IO / ST specific operations that have the option perhaps of  using Storable to read /write at locations. At which point I still need to have the same API surface area again in Primitive, 

see https://phabricator.haskell.org/D5268 for the bare bones stuff (doesn't type check)

the api i could expose there in base does not reduce the implementation surface area needed for Primitve...

i'm open to being convinced otherwise, but with Sven's perspective weighed in, 
1) i dont see it being super useful within base/ghc apis, as they exist today
2) it doesn't reduce implementation surface area burden in the Primitive package.

so at this point im' weakly against the addition. more work, no clear reward for me :) 



On Fri, Oct 26, 2018 at 1:26 PM Sven Panne <[hidden email]> wrote:
Am Fr., 26. Okt. 2018 um 06:05 Uhr schrieb Carter Schonwald <[hidden email]>:
[...] I guess I’m just trying to say “what would the impact on base, if every fake Ptr was moved to be Addr?” Because it’s not something which should be done by halves. 

1) [...]

2) [...]

3). [...]

The most important question is missing from this list: What are the benefits of touching such a crucial part of our API ecosystem? There should better be extremely good reasons, otherwise lots of annoying work and incompatibility is generated for nothing. Reading through this thread, I haven't seen a good motivation for touching Storable/Ptr and friends. Perhaps I have misunderstood what the actual proposal is, initially I thought it is just replacing various "Ptr foo" with Addr only within GHC itself. That's fine, but the scope seems to have broadened.

Just a historical remark: Addr# is a GHCism, and so was Addr. At the time the FFI came up, Ptr was intended to be the portable type. So yes, "Ptr a" and "Ptr ()" are basically just Addr (modulo casts), but intentionally so.
_______________________________________________
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Sven Panne-2
In reply to this post by chessai .
Am Fr., 26. Okt. 2018 um 20:00 Uhr schrieb Daniel Cartwright <[hidden email]>:
[...] Eventually any non-user-facing functions that use 'Ptr LyingType' should be switched from using 'Ptr' to 'Addr'. [...]

Again my question is: Why? When someone sees a "Ptr a" or "Ptr ()", it's blindingly obvious that this is equivalent to Addr, i.e. you need some additional information what that thing is pointing to or you don't care about what's at that location. This is not "lying", quite the opposite. 

You can turn the whole proposal 180 degrees around: Let's nuke "Addr" in favor of "Ptr a"/"Ptr ()"! :-) That would even get rid of a whole type, something the current proposal doesn't achieve. Addr is strictly less powerful than Ptr, so why keep the former?

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

chessai .
I am referring to the situations when someone uses 'Ptr a', but the Ptr does not point to anything of type 'a'. If I write 'Ptr Word8', but I am pointing to a Char, then that is not true.

On Fri, Oct 26, 2018 at 4:08 PM Sven Panne <[hidden email]> wrote:
Am Fr., 26. Okt. 2018 um 20:00 Uhr schrieb Daniel Cartwright <[hidden email]>:
[...] Eventually any non-user-facing functions that use 'Ptr LyingType' should be switched from using 'Ptr' to 'Addr'. [...]

Again my question is: Why? When someone sees a "Ptr a" or "Ptr ()", it's blindingly obvious that this is equivalent to Addr, i.e. you need some additional information what that thing is pointing to or you don't care about what's at that location. This is not "lying", quite the opposite. 

You can turn the whole proposal 180 degrees around: Let's nuke "Addr" in favor of "Ptr a"/"Ptr ()"! :-) That would even get rid of a whole type, something the current proposal doesn't achieve. Addr is strictly less powerful than Ptr, so why keep the former?

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Sven Panne-2
Am Fr., 26. Okt. 2018 um 22:16 Uhr schrieb Daniel Cartwright <[hidden email]>:
I am referring to the situations when someone uses 'Ptr a', but the Ptr does not point to anything of type 'a'. If I write 'Ptr Word8', but I am pointing to a Char, then that is not true.

It might not be true, but you might be forced to do such things to use some ugly C library. The Ptr type carries some valuable information, namely: What does my C counterpart expect? It might not be 100% true on the Haskell side, but you are in "unsafe land", anyway, and at some level you *have* to be able to use some white lie when necessary. Note that I'm not saying that this is nice, but I haven't heard of a better alternative yet.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Andrew Martin
The better alternative suggested in this thread is to use Addr instead. That way, you don't have to lie about the type of the serialized data that the pointer is pointing to.

On Fri, Oct 26, 2018 at 4:26 PM Sven Panne <[hidden email]> wrote:
Am Fr., 26. Okt. 2018 um 22:16 Uhr schrieb Daniel Cartwright <[hidden email]>:
I am referring to the situations when someone uses 'Ptr a', but the Ptr does not point to anything of type 'a'. If I write 'Ptr Word8', but I am pointing to a Char, then that is not true.

It might not be true, but you might be forced to do such things to use some ugly C library. The Ptr type carries some valuable information, namely: What does my C counterpart expect? It might not be 100% true on the Haskell side, but you are in "unsafe land", anyway, and at some level you *have* to be able to use some white lie when necessary. Note that I'm not saying that this is nice, but I haven't heard of a better alternative yet.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


--
-Andrew Thaddeus Martin

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move primitive-Data.Primitive.Addr API into base

Sven Panne-2
Am Fr., 26. Okt. 2018 um 22:31 Uhr schrieb Andrew Martin <[hidden email]>:
The better alternative suggested in this thread is to use Addr instead. That way, you don't have to lie about the type of the serialized data that the pointer is pointing to.

Even if I'm repeating myself: That is not lying, quite the opposite. If you e.g. use a C library with suboptimal typing (i.e. more often than not), you make the impedance matching explicit. How would you achieve that with Addr?

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
12345