How does the RTS pin ByteArray# objects during FFI calls?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How does the RTS pin ByteArray# objects during FFI calls?

Reiner Pope
Hi Haskell-Cafe,

I understand from https://mail.haskell.org/pipermail/haskell-cafe/2014-June/114763.html that it is safe to pass ByteArray# objects to FFI calls, using the UnliftedFFITypes language extension. As I understand it, the implicit guarantee is that the RTS will pin the ByteArray#'s address in memory for the duration of the FFI call -- even if the ByteArray# wasn't allocated pinned.

I'm curious: how does the RTS achieve this "retroactive" pinning? The documentation at https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/Storage/GC/Pinned suggests that all ByteArrays are allocated pinned to account for the possibility that they will in future be passed to the FFI. Is this really the case? This seems like it forces an unreasonably slow allocator on small (say, ~10-byte) ByteArray allocations where the GC's usual bump-pointer allocator might otherwise be preferable.

Regards,
Reiner

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: How does the RTS pin ByteArray# objects during FFI calls?

winter
You’d better use `UnliftedFFITypes` with unsafe FFI calls, since unsafe FFI calls act like fat prim-ops.

GC can happen during safe FFI calls, safe FFI calls are running on other OS threads rather than GHC scheduling threads.

Note: older version of GHCi used to implemented unsafe FFI using safe FFI, there’s an issue about this here: https://ghc.haskell.org/trac/ghc/ticket/8281


On 3 Jan 2018, at 10:51 AM, Reiner Pope <[hidden email]> wrote:

Hi Haskell-Cafe,

I understand from https://mail.haskell.org/pipermail/haskell-cafe/2014-June/114763.html that it is safe to pass ByteArray# objects to FFI calls, using the UnliftedFFITypes language extension. As I understand it, the implicit guarantee is that the RTS will pin the ByteArray#'s address in memory for the duration of the FFI call -- even if the ByteArray# wasn't allocated pinned.

I'm curious: how does the RTS achieve this "retroactive" pinning? The documentation at https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/Storage/GC/Pinned suggests that all ByteArrays are allocated pinned to account for the possibility that they will in future be passed to the FFI. Is this really the case? This seems like it forces an unreasonably slow allocator on small (say, ~10-byte) ByteArray allocations where the GC's usual bump-pointer allocator might otherwise be preferable.

Regards,
Reiner
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: How does the RTS pin ByteArray# objects during FFI calls?

Reiner Pope
I see, so you're saying that there is no pinning of ByteArray# objects during FFI calls, and the only property that guarantees correctness of these FFI calls is that no GC happens during the FFI call. Furthermore, that guarantee is only available for FFI calls tagged 'unsafe'. Good to know!

The bug you linked you linked is scary in how much code is buggy in this regard. Thanks for the heads-up!

Reiner

On Tue, Jan 2, 2018 at 7:31 PM winter <[hidden email]> wrote:
You’d better use `UnliftedFFITypes` with unsafe FFI calls, since unsafe FFI calls act like fat prim-ops.

GC can happen during safe FFI calls, safe FFI calls are running on other OS threads rather than GHC scheduling threads.

Note: older version of GHCi used to implemented unsafe FFI using safe FFI, there’s an issue about this here: https://ghc.haskell.org/trac/ghc/ticket/8281


On 3 Jan 2018, at 10:51 AM, Reiner Pope <[hidden email]> wrote:

Hi Haskell-Cafe,

I understand from https://mail.haskell.org/pipermail/haskell-cafe/2014-June/114763.html that it is safe to pass ByteArray# objects to FFI calls, using the UnliftedFFITypes language extension. As I understand it, the implicit guarantee is that the RTS will pin the ByteArray#'s address in memory for the duration of the FFI call -- even if the ByteArray# wasn't allocated pinned.

I'm curious: how does the RTS achieve this "retroactive" pinning? The documentation at https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/Storage/GC/Pinned suggests that all ByteArrays are allocated pinned to account for the possibility that they will in future be passed to the FFI. Is this really the case? This seems like it forces an unreasonably slow allocator on small (say, ~10-byte) ByteArray allocations where the GC's usual bump-pointer allocator might otherwise be preferable.

Regards,
Reiner
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.