[GHC] #13167: GC and weak reference finalizers and exceptions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[GHC] #13167: GC and weak reference finalizers and exceptions

GHC - devs mailing list
#13167: GC and weak reference finalizers and exceptions
-------------------------------------+-------------------------------------
           Reporter:  Yuras          |             Owner:
               Type:  bug            |            Status:  new
           Priority:  normal         |         Milestone:
          Component:  Compiler       |           Version:  8.0.1
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 When GC runs a number of finalizers in a row, and the first of them throws
 an exception, then other finalizers are ignored. The relevant piece of
 code is
 [https://github.com/ghc/ghc/blob/fb4092642f057f258d07cd6979925f4e2579eda6/libraries/base/GHC/Weak.hs#L144
 here].

 The following program reproduces the issue:

 {{{
 import Data.IORef
 import Control.Monad
 import Control.Exception
 import System.Mem

 main :: IO ()
 main = do
   run
   run
   run
   run
   performMajorGC
   performMajorGC

 run :: IO ()
 run = do
   ref <- newIORef ()
   void $ mkWeakIORef ref $ do
     putStr "."
     throwIO $ ErrorCall "failed"
 }}}

 I expect it to output "....", but I get only "."

 The issue makes it unsafe to rely on finalizer for resource cleanup
 because unrelated finalizer (e.g. from some other library) may prevent
 your finalizer from running.

 Actually I was sure the issue is known, but today I tried to find a
 reference to it, and failed.

 If it is by design, then it should be documented somewhere.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13167>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler
_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #13167: GC and weak reference finalizers and exceptions

GHC - devs mailing list
#13167: GC and weak reference finalizers and exceptions
-------------------------------------+-------------------------------------
        Reporter:  Yuras             |                Owner:
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by simonpj):

 * cc: simonmar (added)


Comment:

 Adding `simonmar` who is king of finalisers.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13167#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler
_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #13167: GC and weak reference finalizers and exceptions

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#13167: GC and weak reference finalizers and exceptions
-------------------------------------+-------------------------------------
        Reporter:  Yuras             |                Owner:
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by simonmar):

 Yes, this is technically a bug.  We could have `runFinalizerBatch` catch
 and discard exceptions (or print them to stdout; it's really a bug if a
 finalizer throws an exception).

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13167#comment:2>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler
_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #13167: GC and weak reference finalizers and exceptions

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#13167: GC and weak reference finalizers and exceptions
-------------------------------------+-------------------------------------
        Reporter:  Yuras             |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by andrewthad):

 I've open a PR that adds docs explaining this behavior.
 https://github.com/ghc/ghc/pull/51

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13167#comment:3>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #13167: GC and weak reference finalizers and exceptions

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#13167: GC and weak reference finalizers and exceptions
-------------------------------------+-------------------------------------
        Reporter:  Yuras             |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by Phyx-):

 * cc: Phyx- (added)


Comment:

 Replying to [comment:2 simonmar]:
 > Yes, this is technically a bug.  We could have `runFinalizerBatch` catch
 and discard exceptions (or print them to stdout; it's really a bug if a
 finalizer throws an exception).

 Any suggestions on how? the I/O manager seems to rely on using finalizers
 to flush buffers, however for some reason my new I/O manager's finalizers
 never get called. I assume because something else threw an exception.

 I've modified `runFinalizerBatch` to handle exceptions (unless I
 misunderstood the code)

 {{{
 runFinalizerBatch :: Int -> Array# (State# RealWorld -> State# RealWorld)
                   -> IO ()
 runFinalizerBatch (I# n) arr =
    let  go m  = IO $ \s ->
                   case m of
                   0# -> (# s, () #)
                   _  -> let !m' = m -# 1# in
                         case indexArray# arr m' of { (# io #) ->
                         case catch# (\p -> (# io p, () #)) (\_ s'' -> (#
 s'', () #)) s of          {
                          (# s', _ #) -> unIO (go m') s'
                         }}
    in
         go n
 }}}

 With `mio` the output varies between `..` and `....`, so not very
 deterministic. with `winio` none of it ever runs...

 Any ideas what else might be making these handlers not run at all? or how
 I can debug this? for `winio` I can imagine one of the finalizers
 deadlocking, an ffi call maybe. But the mio output is strange.

 If we're going to rely on them for managing handles and buffers they would
 ideally be a bit more robust...

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13167#comment:4>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #13167: GC and weak reference finalizers and exceptions

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#13167: GC and weak reference finalizers and exceptions
-------------------------------------+-------------------------------------
        Reporter:  Yuras             |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by simonmar):

 The example program in the description doesn't wait for the finalizer
 threads to complete, so it might exit while there are still finalizers
 alive. This is why you don't see deterministic output. Probably adding a
 `threadDelay 1000000` at the end would be enough.

 A deadlock sounds like a plausible explanation for the behaviour you're
 seeing.

 The fix to `runFinalizerBatch` looks good - we should probably do that
 anyway (I'm sure you could construct a test case to demonstrate the
 problem that it fixes).

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13167#comment:5>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #13167: GC and weak reference finalizers and exceptions

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#13167: GC and weak reference finalizers and exceptions
-------------------------------------+-------------------------------------
        Reporter:  Yuras             |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by Phyx-):

 ah indeed, `threadDelay` worked!

 Ok I'll whip up a testcase and patch.

 I'll see if I can't add some debug code to figure out which finalizer is
 deadlocking.

 Thanks!

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13167#comment:6>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #13167: GC and weak reference finalizers and exceptions

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#13167: GC and weak reference finalizers and exceptions
-------------------------------------+-------------------------------------
        Reporter:  Yuras             |                Owner:  (none)
            Type:  bug               |               Status:  patch
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4693
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by Phyx-):

 * status:  new => patch
 * differential:   => Phab:D4693


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13167#comment:7>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #13167: GC and weak reference finalizers and exceptions

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#13167: GC and weak reference finalizers and exceptions
-------------------------------------+-------------------------------------
        Reporter:  Yuras             |                Owner:  (none)
            Type:  bug               |               Status:  patch
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4693
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by Ben Gamari <ben@…>):

 In [changeset:"01bb17fd4dc6d92cf08632bbb62656428db6e7fa/ghc"
 01bb17fd/ghc]:
 {{{
 #!CommitTicketReference repository="ghc"
 revision="01bb17fd4dc6d92cf08632bbb62656428db6e7fa"
 Make finalizers more reliable.

 Ignore any errors thrown by finalizers when running them.

 This prevents a faulty finalizer from stopping the rest being called.

 Test Plan: ./validate, new test T13167

 Reviewers: hvr, bgamari, simonmar

 Reviewed By: bgamari, simonmar

 Subscribers: rwbarton, thomie, carter

 GHC Trac Issues: #13167

 Differential Revision: https://phabricator.haskell.org/D4693
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13167#comment:8>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #13167: GC and weak reference finalizers and exceptions

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#13167: GC and weak reference finalizers and exceptions
-------------------------------------+-------------------------------------
        Reporter:  Yuras             |                Owner:  (none)
            Type:  bug               |               Status:  closed
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.0.1
      Resolution:  fixed             |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4693
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by Phyx-):

 * status:  patch => closed
 * resolution:   => fixed
 * milestone:   => 8.6.1


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13167#comment:9>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets