Safe FFI and Blocking IO

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Safe FFI and Blocking IO

Andrew Martin
According to the FFI chapter [1] in the GHC manual, the safe FFI is useful when you need to call a C function that can call back into haskell code. I had always assumed that the scheduler could somehow interrupt safe FFI calls, but the manual does not indicate this, and in some recent testing I did in the posix library [2], I found that scheduling interrupts definitely do not happen. With the non-threaded runtime, the following test always hangs:

    testSocketsD :: IO ()
    testSocketsD = do
      (a,b) <- demand =<< S.socketPair P.unix P.datagram P.defaultProtocol
      _ <- forkIO $ do
        bytesSent <- demand =<< S.sendByteArray b sample 0 5 mempty
        when (bytesSent /= 5) (fail "testSocketsD: bytesSent was wrong")
      actual <- demand =<< S.receiveByteArray a 5 mempty
      actual @=? sample

    sample :: ByteArray
    sample = E.fromList [1,2,3,4,5]

    demand :: Either Errno a -> IO a
    demand = either (\e -> ioError (errnoToIOError "test" e Nothing Nothing)) pure

In the above example, sendByteArray and receiveByteArray are safe FFI wrappers around send and recv. It is necessary to use threadWaitRead and threadWaitWrite before these calls to predictably get the correct behavior.

This brings to my question. In issue #34 on the github library for the unix package [3], there is a discussion about whether to use the safe or unsafe FFI for various POSIX system calls. On the issue there is strong consensus that the safe FFI calls lead to better performance.

Simon Marlow writes [4] that "Unsafe foreign imports which can block for unpredictable amounts of time cause performance problems that only emerge when scaling to multiple cores, because they delay the GC sync. This is a really annoying problem if it happens to you, because it's almost impossible to diagnose, and if it happens due to an unsafe call in a library then it's also really hard to fix."

And Gregory Collins adds that "If the call would ever block (and that includes most filesystem functions) that means you want 'safe'."

There's something I'm definitely missing. My experience is that safe FFI calls do not help with blocking IO (again, I've been using the non-threaded runtime, but I doubt this makes a difference), that they only help with C functions that call back into haskell. However, a lot of other people seem to have a difference experience.


--
-Andrew Thaddeus Martin

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Safe FFI and Blocking IO

Andrew Martin
Sorry. I just found the answer to this in the manual:

"When you call a foreign imported function that is annotated as safe (the default), and the program was linked using -threaded, then the call will run concurrently with other running Haskell threads. If the program was linked without -threaded, then the other Haskell threads will be blocked until the call returns."

"This means that if you need to make a foreign call to a function that takes a long time or blocks indefinitely, then you should mark it safe and use -threaded. Some library functions make such calls internally; their documentation should indicate when this is the case."



On Tue, Dec 4, 2018 at 8:23 AM Andrew Martin <[hidden email]> wrote:
According to the FFI chapter [1] in the GHC manual, the safe FFI is useful when you need to call a C function that can call back into haskell code. I had always assumed that the scheduler could somehow interrupt safe FFI calls, but the manual does not indicate this, and in some recent testing I did in the posix library [2], I found that scheduling interrupts definitely do not happen. With the non-threaded runtime, the following test always hangs:

    testSocketsD :: IO ()
    testSocketsD = do
      (a,b) <- demand =<< S.socketPair P.unix P.datagram P.defaultProtocol
      _ <- forkIO $ do
        bytesSent <- demand =<< S.sendByteArray b sample 0 5 mempty
        when (bytesSent /= 5) (fail "testSocketsD: bytesSent was wrong")
      actual <- demand =<< S.receiveByteArray a 5 mempty
      actual @=? sample

    sample :: ByteArray
    sample = E.fromList [1,2,3,4,5]

    demand :: Either Errno a -> IO a
    demand = either (\e -> ioError (errnoToIOError "test" e Nothing Nothing)) pure

In the above example, sendByteArray and receiveByteArray are safe FFI wrappers around send and recv. It is necessary to use threadWaitRead and threadWaitWrite before these calls to predictably get the correct behavior.

This brings to my question. In issue #34 on the github library for the unix package [3], there is a discussion about whether to use the safe or unsafe FFI for various POSIX system calls. On the issue there is strong consensus that the safe FFI calls lead to better performance.

Simon Marlow writes [4] that "Unsafe foreign imports which can block for unpredictable amounts of time cause performance problems that only emerge when scaling to multiple cores, because they delay the GC sync. This is a really annoying problem if it happens to you, because it's almost impossible to diagnose, and if it happens due to an unsafe call in a library then it's also really hard to fix."

And Gregory Collins adds that "If the call would ever block (and that includes most filesystem functions) that means you want 'safe'."

There's something I'm definitely missing. My experience is that safe FFI calls do not help with blocking IO (again, I've been using the non-threaded runtime, but I doubt this makes a difference), that they only help with C functions that call back into haskell. However, a lot of other people seem to have a difference experience.


--
-Andrew Thaddeus Martin


--
-Andrew Thaddeus Martin

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Safe FFI and Blocking IO

Carter Schonwald
yup! (this is also kinda related to how the IO manager only runs on -threaded built applications)

I actually do the following pattern in some libraries i've written: bind both unsafe and safe versions of a functions, and when work input is below some size that i think will be less than ~ 1-10 microseconds, i do an unsafe call, otherwise i do a safe call! (unsafe calls block the GC, which is bad in say a server app, as you can well guess)

the most recent and tiny example of this is a tiny sha3 implementation (still need to tweak it, i think i left another 4-6x performance on the table https://hackage.haskell.org/package/SecureHash-SHA3)

On Tue, Dec 4, 2018 at 8:26 AM Andrew Martin <[hidden email]> wrote:
Sorry. I just found the answer to this in the manual:

"When you call a foreign imported function that is annotated as safe (the default), and the program was linked using -threaded, then the call will run concurrently with other running Haskell threads. If the program was linked without -threaded, then the other Haskell threads will be blocked until the call returns."

"This means that if you need to make a foreign call to a function that takes a long time or blocks indefinitely, then you should mark it safe and use -threaded. Some library functions make such calls internally; their documentation should indicate when this is the case."



On Tue, Dec 4, 2018 at 8:23 AM Andrew Martin <[hidden email]> wrote:
According to the FFI chapter [1] in the GHC manual, the safe FFI is useful when you need to call a C function that can call back into haskell code. I had always assumed that the scheduler could somehow interrupt safe FFI calls, but the manual does not indicate this, and in some recent testing I did in the posix library [2], I found that scheduling interrupts definitely do not happen. With the non-threaded runtime, the following test always hangs:

    testSocketsD :: IO ()
    testSocketsD = do
      (a,b) <- demand =<< S.socketPair P.unix P.datagram P.defaultProtocol
      _ <- forkIO $ do
        bytesSent <- demand =<< S.sendByteArray b sample 0 5 mempty
        when (bytesSent /= 5) (fail "testSocketsD: bytesSent was wrong")
      actual <- demand =<< S.receiveByteArray a 5 mempty
      actual @=? sample

    sample :: ByteArray
    sample = E.fromList [1,2,3,4,5]

    demand :: Either Errno a -> IO a
    demand = either (\e -> ioError (errnoToIOError "test" e Nothing Nothing)) pure

In the above example, sendByteArray and receiveByteArray are safe FFI wrappers around send and recv. It is necessary to use threadWaitRead and threadWaitWrite before these calls to predictably get the correct behavior.

This brings to my question. In issue #34 on the github library for the unix package [3], there is a discussion about whether to use the safe or unsafe FFI for various POSIX system calls. On the issue there is strong consensus that the safe FFI calls lead to better performance.

Simon Marlow writes [4] that "Unsafe foreign imports which can block for unpredictable amounts of time cause performance problems that only emerge when scaling to multiple cores, because they delay the GC sync. This is a really annoying problem if it happens to you, because it's almost impossible to diagnose, and if it happens due to an unsafe call in a library then it's also really hard to fix."

And Gregory Collins adds that "If the call would ever block (and that includes most filesystem functions) that means you want 'safe'."

There's something I'm definitely missing. My experience is that safe FFI calls do not help with blocking IO (again, I've been using the non-threaded runtime, but I doubt this makes a difference), that they only help with C functions that call back into haskell. However, a lot of other people seem to have a difference experience.


--
-Andrew Thaddeus Martin


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Safe FFI and Blocking IO

Andrew Martin
What's your heuristic for deciding between safe and interruptible? I find that every time something takes long enough to warrant using the safe FFI, I also want to be able to kill it from a separate thread.

On Tue, Dec 4, 2018 at 9:58 AM Carter Schonwald <[hidden email]> wrote:
yup! (this is also kinda related to how the IO manager only runs on -threaded built applications)

I actually do the following pattern in some libraries i've written: bind both unsafe and safe versions of a functions, and when work input is below some size that i think will be less than ~ 1-10 microseconds, i do an unsafe call, otherwise i do a safe call! (unsafe calls block the GC, which is bad in say a server app, as you can well guess)

the most recent and tiny example of this is a tiny sha3 implementation (still need to tweak it, i think i left another 4-6x performance on the table https://hackage.haskell.org/package/SecureHash-SHA3)

On Tue, Dec 4, 2018 at 8:26 AM Andrew Martin <[hidden email]> wrote:
Sorry. I just found the answer to this in the manual:

"When you call a foreign imported function that is annotated as safe (the default), and the program was linked using -threaded, then the call will run concurrently with other running Haskell threads. If the program was linked without -threaded, then the other Haskell threads will be blocked until the call returns."

"This means that if you need to make a foreign call to a function that takes a long time or blocks indefinitely, then you should mark it safe and use -threaded. Some library functions make such calls internally; their documentation should indicate when this is the case."



On Tue, Dec 4, 2018 at 8:23 AM Andrew Martin <[hidden email]> wrote:
According to the FFI chapter [1] in the GHC manual, the safe FFI is useful when you need to call a C function that can call back into haskell code. I had always assumed that the scheduler could somehow interrupt safe FFI calls, but the manual does not indicate this, and in some recent testing I did in the posix library [2], I found that scheduling interrupts definitely do not happen. With the non-threaded runtime, the following test always hangs:

    testSocketsD :: IO ()
    testSocketsD = do
      (a,b) <- demand =<< S.socketPair P.unix P.datagram P.defaultProtocol
      _ <- forkIO $ do
        bytesSent <- demand =<< S.sendByteArray b sample 0 5 mempty
        when (bytesSent /= 5) (fail "testSocketsD: bytesSent was wrong")
      actual <- demand =<< S.receiveByteArray a 5 mempty
      actual @=? sample

    sample :: ByteArray
    sample = E.fromList [1,2,3,4,5]

    demand :: Either Errno a -> IO a
    demand = either (\e -> ioError (errnoToIOError "test" e Nothing Nothing)) pure

In the above example, sendByteArray and receiveByteArray are safe FFI wrappers around send and recv. It is necessary to use threadWaitRead and threadWaitWrite before these calls to predictably get the correct behavior.

This brings to my question. In issue #34 on the github library for the unix package [3], there is a discussion about whether to use the safe or unsafe FFI for various POSIX system calls. On the issue there is strong consensus that the safe FFI calls lead to better performance.

Simon Marlow writes [4] that "Unsafe foreign imports which can block for unpredictable amounts of time cause performance problems that only emerge when scaling to multiple cores, because they delay the GC sync. This is a really annoying problem if it happens to you, because it's almost impossible to diagnose, and if it happens due to an unsafe call in a library then it's also really hard to fix."

And Gregory Collins adds that "If the call would ever block (and that includes most filesystem functions) that means you want 'safe'."

There's something I'm definitely missing. My experience is that safe FFI calls do not help with blocking IO (again, I've been using the non-threaded runtime, but I doubt this makes a difference), that they only help with C functions that call back into haskell. However, a lot of other people seem to have a difference experience.


--
-Andrew Thaddeus Martin


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


--
-Andrew Thaddeus Martin

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Safe FFI and Blocking IO

Carter Schonwald
i only do unsafe vs safe ffi, most things i do i make sure that a c call running to completion (and associated system calls) dont prevent other work from happening. Interrupts can require resource cleanup. what sort of examples do you have in mind?

you *could* do interruptable in lieu of safe as long as you have some way to cleanup memory allocations i guess?

to quote the documentation
"

interruptible behaves exactly as safe, except that when a throwTo is directed at a thread in an interruptible foreign call, an OS-specific mechanism will be used to attempt to cause the foreign call to return:

Unix systems
The thread making the foreign call is sent a SIGPIPE signal using pthread_kill(). This is usually enough to cause a blocking system call to return with EINTR (GHC by default installs an empty signal handler for SIGPIPE, to override the default behaviour which is to terminate the process immediately).
Windows systems
[Vista and later only] The RTS calls the Win32 function CancelSynchronousIo, which will cause a blocking I/O operation to return with the error ERROR_OPERATION_ABORTED.

"

i dont know if the cffi overheads differ when doing an interruptable call, and you really do have to think carefully about state cleanup... I like avoiding needing to do state cleanup personally.

if you mean to ask about SAFE vs UNSAFE, its mostly about often the code i'm binding has VERY predictable complexity / runtime behavior as a function of input size, and i choose an input size threshold thats more than ~ 1 microsecond and less than 10 microseconds.  smaller stuff gets unsafe and larger gets safe. 

Last time i measure stuff years ago, safe ffi calls had ~ 200 nanosecond overhead on the applicable laptop, so as long as youre not doing scribbles to unpinned memory, you should always always use the SAFE ffi for operations that will likely take >= 1-10 microseconds ALWAYS. (theres lots of math C code on hackage which does unsafe FFI and yet the input sizes of interest will likely take several second to compute, a nasty combo in a server/networked env).

tl;dr any network api / file system api, pretty safe to do safe api calls always, afaik few to none of those are sub microsecond, heck just a memory read from ram is 5-10 microseconds right?


On Tue, Dec 4, 2018 at 1:08 PM Andrew Martin <[hidden email]> wrote:
What's your heuristic for deciding between safe and interruptible? I find that every time something takes long enough to warrant using the safe FFI, I also want to be able to kill it from a separate thread.

On Tue, Dec 4, 2018 at 9:58 AM Carter Schonwald <[hidden email]> wrote:
yup! (this is also kinda related to how the IO manager only runs on -threaded built applications)

I actually do the following pattern in some libraries i've written: bind both unsafe and safe versions of a functions, and when work input is below some size that i think will be less than ~ 1-10 microseconds, i do an unsafe call, otherwise i do a safe call! (unsafe calls block the GC, which is bad in say a server app, as you can well guess)

the most recent and tiny example of this is a tiny sha3 implementation (still need to tweak it, i think i left another 4-6x performance on the table https://hackage.haskell.org/package/SecureHash-SHA3)

On Tue, Dec 4, 2018 at 8:26 AM Andrew Martin <[hidden email]> wrote:
Sorry. I just found the answer to this in the manual:

"When you call a foreign imported function that is annotated as safe (the default), and the program was linked using -threaded, then the call will run concurrently with other running Haskell threads. If the program was linked without -threaded, then the other Haskell threads will be blocked until the call returns."

"This means that if you need to make a foreign call to a function that takes a long time or blocks indefinitely, then you should mark it safe and use -threaded. Some library functions make such calls internally; their documentation should indicate when this is the case."



On Tue, Dec 4, 2018 at 8:23 AM Andrew Martin <[hidden email]> wrote:
According to the FFI chapter [1] in the GHC manual, the safe FFI is useful when you need to call a C function that can call back into haskell code. I had always assumed that the scheduler could somehow interrupt safe FFI calls, but the manual does not indicate this, and in some recent testing I did in the posix library [2], I found that scheduling interrupts definitely do not happen. With the non-threaded runtime, the following test always hangs:

    testSocketsD :: IO ()
    testSocketsD = do
      (a,b) <- demand =<< S.socketPair P.unix P.datagram P.defaultProtocol
      _ <- forkIO $ do
        bytesSent <- demand =<< S.sendByteArray b sample 0 5 mempty
        when (bytesSent /= 5) (fail "testSocketsD: bytesSent was wrong")
      actual <- demand =<< S.receiveByteArray a 5 mempty
      actual @=? sample

    sample :: ByteArray
    sample = E.fromList [1,2,3,4,5]

    demand :: Either Errno a -> IO a
    demand = either (\e -> ioError (errnoToIOError "test" e Nothing Nothing)) pure

In the above example, sendByteArray and receiveByteArray are safe FFI wrappers around send and recv. It is necessary to use threadWaitRead and threadWaitWrite before these calls to predictably get the correct behavior.

This brings to my question. In issue #34 on the github library for the unix package [3], there is a discussion about whether to use the safe or unsafe FFI for various POSIX system calls. On the issue there is strong consensus that the safe FFI calls lead to better performance.

Simon Marlow writes [4] that "Unsafe foreign imports which can block for unpredictable amounts of time cause performance problems that only emerge when scaling to multiple cores, because they delay the GC sync. This is a really annoying problem if it happens to you, because it's almost impossible to diagnose, and if it happens due to an unsafe call in a library then it's also really hard to fix."

And Gregory Collins adds that "If the call would ever block (and that includes most filesystem functions) that means you want 'safe'."

There's something I'm definitely missing. My experience is that safe FFI calls do not help with blocking IO (again, I've been using the non-threaded runtime, but I doubt this makes a difference), that they only help with C functions that call back into haskell. However, a lot of other people seem to have a difference experience.


--
-Andrew Thaddeus Martin


--
-Andrew Thaddeus Martin
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


--
-Andrew Thaddeus Martin

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Safe FFI and Blocking IO

Niklas Hambüchen
Hello Andrew,

I have done some work on interruptibility and accidental blocking in GHC and Unix over the last 2 years with FP Complete for our clients.

Summarising from what was already written/linked, the key things to understand are:

* `safe` calls run in a separate OS thread in -threaded, so they protect from any blocking.
* The separate threads spawned by `safe` calls do not count to the +RTS -N limit.
* `unsafe` calls block the entire capability, always (e.g. 1 out of 4 +RTS -N4 threads).
* There is only one way to interrupt running system calls on Unix: Sending the thread that does them a signal. The syscalls then return an error and `errno = EINTR`. Many (but not all) syscalls can be interrupted that way.
* `interruptible` is thus implemented by sending a signal to the thread that does the syscall.
* That happens in particular when you send an exception via `throwTo` to a Haskell thread that's blocked in a foreign call (for example, `timeout` uses `throwTo`).
* You can only use `interruptible` on FFI code that is written on purpose to return back to Haskell when EINTR is encountered, so that Haskell can then raise the exception. If the code doesn't do that, but instead just retries the syscall in C, then there's no point in using `interruptible`, as it won't have any effect.

Important for non-threaded is:

* In non`-threaded`, behaviour varies a lot across platforms.
* On Linux it really has only a single thread. Some things happen to be more interruptible on Linux because the timer signal wakes up all kinds of syscalls regularly, so most things work like `interruptible` is implemented on Linux.
* On e.g. OSX, non-threaded actually uses threads, namely 2: One for the timer signal, and one for the Haskell stuff.
* These differences make it very difficult to expect similar behaviour from the non-threaded runtime across platforms.
* I have an open proposal + half-done implementation to make non-threaded on Linux work like it does on OSX to unify these things. https://phabricator.haskell.org/D42#128355

The key rules are:

* Do not use `unsafe` on anything that can block on non-CPU-bound tasks, ever. It will massively limit the ability to use multiple cores.
* Use `unsafe` only for CPU-bound activities.
* For all other things, `interruptible` is the best of all, but as mentioned above, the called code must be designed do return EINTR all the way up to Haskell.
* Where this is not the case and you thus can't use `interruptible`, use `safe`.

The `unix` package unfortunately uses `unsafe` calls in many places where it really shouldn't, such as `stat()` (see the ticket you linked).
I think this is very bad and we must fix it.
For some of my own tools (like a parallel file-copy tool designed to work well on network file systems), I use a fork of the package where everything uses `safe`.

For many details on these topics, check out the tickets I filed / worked on:
 
* https://ghc.haskell.org/trac/ghc/ticket/8684 - hWaitForInput cannot be interrupted by async exceptions on unix
* https://ghc.haskell.org/trac/ghc/ticket/13497 - GHC does not use select()/poll() correctly on non-Linux platforms
* https://ghc.haskell.org/trac/ghc/ticket/15153 - GHC uses O_NONBLOCK on regular files, which has no effect, and blocks the runtime

Also happy to answer any questions!

Niklas
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries