Locking of threads in one OS thread

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Locking of threads in one OS thread

Nikolay Amiantov
Hello Cafe,

I'm using FFI to interact with a library which calls, when fail, leave
the reason in some kind of "errno"-like variable which is retrived via
another call. AFAIU, this is not thread-safe in Haskell even if
thread-local storage is used inside the library, because Haskell uses
its own thread management and the Haskell thread in the same OS thread
might be switched between the actual call and the retrival of errno
value. This should be somehow handled already in Haskell (errno is
widely used with syscalls in Linux, for example), but the source of
Foreign.C.Error suggests that this is not handled in any way at all.
For example, throwErrnoIf is implemented as such:

throwErrno loc  = do
    errno <- getErrno
    ioError (errnoToIOError loc errno Nothing Nothing)

throwErrnoIf pred loc f  = do
    res <- f
    if pred res then throwErrno loc else return res

So, the question is: how is it ensured that this block is "atomic" in
sense that at most one Haskell thread computes this whole function at
every moment in every OS thread?

Thanks for any explanations! (And sorry if my English is poor)

Nikolay.
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Locking of threads in one OS thread

Carter Schonwald
several ideas
a) you can use forkOS to run the ffi call and the the subsequent errno value checking in a pinned OS thread so you can guarantee you can access the thread local storage
b) you could have a CallManager that keeps a MVar or queue or something, along with a handle on the forkOS thread id, and "intiialize" the forkOS thread with a teeny worker that read from an input MVar and writes to a result MVar (thus guarantee only one call a time)
c) something that uses these ideas together.

theres a lot of strategies you can use. but i hope this gives you a sketch (i can spell it out more later this week if need be)


On Tue, Aug 19, 2014 at 5:36 PM, Nikolay Amiantov <[hidden email]> wrote:
Hello Cafe,

I'm using FFI to interact with a library which calls, when fail, leave
the reason in some kind of "errno"-like variable which is retrived via
another call. AFAIU, this is not thread-safe in Haskell even if
thread-local storage is used inside the library, because Haskell uses
its own thread management and the Haskell thread in the same OS thread
might be switched between the actual call and the retrival of errno
value. This should be somehow handled already in Haskell (errno is
widely used with syscalls in Linux, for example), but the source of
Foreign.C.Error suggests that this is not handled in any way at all.
For example, throwErrnoIf is implemented as such:

throwErrno loc  = do
    errno <- getErrno
    ioError (errnoToIOError loc errno Nothing Nothing)

throwErrnoIf pred loc f  = do
    res <- f
    if pred res then throwErrno loc else return res

So, the question is: how is it ensured that this block is "atomic" in
sense that at most one Haskell thread computes this whole function at
every moment in every OS thread?

Thanks for any explanations! (And sorry if my English is poor)

Nikolay.
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Locking of threads in one OS thread

Nikolay Amiantov
Thanks for the answer! I've thought about (c) already, and (a) and (b)
are interesting, too -- maybe (a) is my answer.

I'm very interested in how Haskell handles this already -- for
example, in "network" or "unix" package sources I haven't found any
handling of this (see [1], [2] for example) and it looks to me that
these packages are prone to this race condition -- am I right?

If this is not a known error, then my thoughts is that to add a way to
fix this without losing much speed we could add new prims -- something
like "maskPreemption#", "unmaskPreemption#" and "getPreemptionState#"
(using "mask" implementation as an example) and new "maskPreemption"
function in Control.Concurrent, for example. Prims can use
thread-local flag without any locking that runtime checks before
thread preemption -- this should be nearly unnoticeable operation in
terms of performance.

This is all just an idea of how I would implement this, and I don't
have knowledge of RTS internals, so for someone competent this may be
utter nonsense -- I'm sorry for this.

[1]: https://hackage.haskell.org/package/network-2.6.0.1/docs/src/Network-Socket-Internal.html
[2]: http://hackage.haskell.org/package/unix-2.7.0.1/docs/src/System-Posix-Error.html
С уважением,
Николай Амиантов.


On Wed, Aug 20, 2014 at 5:02 AM, Carter Schonwald
<[hidden email]> wrote:

> several ideas
> a) you can use forkOS to run the ffi call and the the subsequent errno value
> checking in a pinned OS thread so you can guarantee you can access the
> thread local storage
> b) you could have a CallManager that keeps a MVar or queue or something,
> along with a handle on the forkOS thread id, and "intiialize" the forkOS
> thread with a teeny worker that read from an input MVar and writes to a
> result MVar (thus guarantee only one call a time)
> c) something that uses these ideas together.
>
> theres a lot of strategies you can use. but i hope this gives you a sketch
> (i can spell it out more later this week if need be)
>
>
> On Tue, Aug 19, 2014 at 5:36 PM, Nikolay Amiantov <[hidden email]>
> wrote:
>>
>> Hello Cafe,
>>
>> I'm using FFI to interact with a library which calls, when fail, leave
>> the reason in some kind of "errno"-like variable which is retrived via
>> another call. AFAIU, this is not thread-safe in Haskell even if
>> thread-local storage is used inside the library, because Haskell uses
>> its own thread management and the Haskell thread in the same OS thread
>> might be switched between the actual call and the retrival of errno
>> value. This should be somehow handled already in Haskell (errno is
>> widely used with syscalls in Linux, for example), but the source of
>> Foreign.C.Error suggests that this is not handled in any way at all.
>> For example, throwErrnoIf is implemented as such:
>>
>> throwErrno loc  = do
>>     errno <- getErrno
>>     ioError (errnoToIOError loc errno Nothing Nothing)
>>
>> throwErrnoIf pred loc f  = do
>>     res <- f
>>     if pred res then throwErrno loc else return res
>>
>> So, the question is: how is it ensured that this block is "atomic" in
>> sense that at most one Haskell thread computes this whole function at
>> every moment in every OS thread?
>>
>> Thanks for any explanations! (And sorry if my English is poor)
>>
>> Nikolay.
>> _______________________________________________
>> Haskell-Cafe mailing list
>> [hidden email]
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Locking of threads in one OS thread

Alexander Kjeldaas
In reply to this post by Nikolay Amiantov



On Tue, Aug 19, 2014 at 11:36 PM, Nikolay Amiantov <[hidden email]> wrote:
Hello Cafe,

I'm using FFI to interact with a library which calls, when fail, leave
the reason in some kind of "errno"-like variable which is retrived via
another call. AFAIU, this is not thread-safe in Haskell even if
thread-local storage is used inside the library, because Haskell uses
its own thread management and the Haskell thread in the same OS thread
might be switched between the actual call and the retrival of errno
value.

Good question!
 
This should be somehow handled already in Haskell (errno is
widely used with syscalls in Linux, for example), but the source of
Foreign.C.Error suggests that this is not handled in any way at all.
For example, throwErrnoIf is implemented as such:

throwErrno loc  = do
    errno <- getErrno
    ioError (errnoToIOError loc errno Nothing Nothing)

throwErrnoIf pred loc f  = do
    res <- f
    if pred res then throwErrno loc else return res

So, the question is: how is it ensured that this block is "atomic" in
sense that at most one Haskell thread computes this whole function at
every moment in every OS thread?


I do not know the RTS very well, but I think this might be unsafe.  If the thread is bound, then getErrno is guaranteed to be executed on the same thread, but otherwise no such guarantee is given.

However, reading http://blog.ezyang.com/2013/01/the-ghc-scheduler/ it might be unlikely.

If GC happens, "Threads are put in front (pushOnRunQueue) if: ... In the threaded runtime, if a thread was interrupted because another Capability needed to do a stop-the-world GC (see commit 6d18141d8);"

However, the same post indicates that you can force this behavior using signals.

"Threads are put in back (appendToRunQueue) in the case of pre-emption, or if it’s new; particularly, if: ...A thread was pre-empted via the context switch flag (e.g. incoming message from another thread, the timer fired, the thread cooperatively yielded, etc; see also [8] on how this interacts with heap overflows);"

Reading errno directly after the FFI call can eliminate heap overflows, but the async exception and timer issues still seem possible.

I would also like to see a good explanation of this.

Alexander

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Locking of threads in one OS thread

Nikolay Amiantov

On Wed, Aug 20, 2014 at 4:36 PM, Alexander Kjeldaas <[hidden email]> wrote:
Reading errno directly after the FFI call can eliminate heap overflows, but the async exception and timer issues still seem possible.

I have played around a bit and I can't understand what's going on with this anymore. Check out this example: https://gist.github.com/abbradar/76dafcee1807c9c5ac4d. Compile it with "ghc test_c.c test.hs". I used "mask_" here to check if it can fix the problem.

There will be multiple discrepancies seen between written and read values because of threads preemption if my_errno is used. However, if you use "errno" (change #define for that) instead, everything seems good.

Anyway, it looks like getErrno and friends rely on this magical behaviour of errno -- if some other library which uses global error state like "my_errno" in example is used (I remember SDL doing that, and I have also done it myself), there should be problems without some way to temporary disable threads preemption, which I haven't found. Hence -- should I maybe post a bug report about this?

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Locking of threads in one OS thread

Sylvain HENRY
AFAIK "errno" is handled explicitly in the RTS: https://ghc.haskell.org/trac/ghc/browser/ghc/rts/Schedule.c#L484

-Sylvain


2014-08-26 16:22 GMT+02:00 Nikolay Amiantov <[hidden email]>:

On Wed, Aug 20, 2014 at 4:36 PM, Alexander Kjeldaas <[hidden email]> wrote:
Reading errno directly after the FFI call can eliminate heap overflows, but the async exception and timer issues still seem possible.

I have played around a bit and I can't understand what's going on with this anymore. Check out this example: https://gist.github.com/abbradar/76dafcee1807c9c5ac4d. Compile it with "ghc test_c.c test.hs". I used "mask_" here to check if it can fix the problem.

There will be multiple discrepancies seen between written and read values because of threads preemption if my_errno is used. However, if you use "errno" (change #define for that) instead, everything seems good.

Anyway, it looks like getErrno and friends rely on this magical behaviour of errno -- if some other library which uses global error state like "my_errno" in example is used (I remember SDL doing that, and I have also done it myself), there should be problems without some way to temporary disable threads preemption, which I haven't found. Hence -- should I maybe post a bug report about this?

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe



_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Fwd: Locking of threads in one OS thread

Nikolay Amiantov
On Tue, Aug 26, 2014 at 6:59 PM, Sylvain Henry <[hidden email]> wrote:
AFAIK "errno" is handled explicitly in the RTS: https://ghc.haskell.org/trac/ghc/browser/ghc/rts/Schedule.c#L484

-Sylvain
 
Oh, thanks for pointing this out! Now this behaviour makes sense.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe