[GHC] #15038: Memory Corruption (strange closure type)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
45 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
           Reporter:  andrewthad     |             Owner:  (none)
               Type:  bug            |            Status:  new
           Priority:  normal         |         Milestone:  8.6.1
          Component:  Compiler       |           Version:  8.4.1
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 I've been encountering corruption of memory in a library of mine that uses
 some of GHC's more recent features (levity polymorphism and unboxed sums).
 The library is `packed`, and it's source code can be found on my github.
 But, more relevant to this issue is the minimal reproducible example I've
 put together here: https://github.com/andrewthad/corrupted-memory-example

 It's still a bit larger than I'd like it to be, and if no one has any
 insights into this, I can keep whittling it down to make it more minimal.
 Here's how to run it:

 {{{
 git clone https://github.com/andrewthad/corrupted-memory-example
 cd corrupted-memory-example
 cabal new-build --enable-tests test
 ./dist-newstyle/build/corrupted-memory-example-0.1/build/test/test
 }}}

 This consistently crashes with:

 {{{
 test: internal error: evacuate(static): strange closure type 16648
     (GHC version 8.4.1 for x86_64_unknown_linux)
     Please report this as a GHC bug:
 http://www.haskell.org/ghc/reportabug
 }}}

 Additional information: This issue can be reproduced using GHC 8.2.2, GHC
 8.4.1, and GHC HEAD. This project cannot be built with GHC 8.0.2 and
 earlier since it uses unboxed sums. Building this project with either
 `stack` or `cabal new-build` and then running the test causes the same
 crash. However, building it with plain old GHC and then running it does
 not. This is really weird, since there aren't any GHC options in the cabal
 file that should affect this. In the original library, I have a bunch of
 tests that use the parsers `byte` and `any` in a variety of situations,
 and none of them every trigger this crash except for the ones where I
 stick a `Parser` inside of a `Trie`. This project does a lot of manual
 passing of state tokens, but I believe all of these uses to be correct
 (mostly because of the extensive tests in the original repo, but I've also
 spent a lot of time just looking at the code to figure out if this was
 something on my end).

 Let me know if the example is too big. I can spend some more time
 shrinking it further, but I wanted to go ahead and get it up here.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by osa1):

 * cc: osa1 (added)


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by simonpj):

 Thanks for making a repro case.  Yes, shrinking it is incredibly valuable.

 Did you compile everything with `-dcore-lint`?  Always worth doing that.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:2>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by simonpj):

 * failure:  None/Unknown => Runtime crash


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:3>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by andrewthad):

 I’ll work on shrinking it further. And yes, I forgot to mention that core
 lint comes back fine and (on ghc Head) stg lint comes back fine as well.
 I’ve tried to consolidate the modules, but merging them all into a single
 module causes the problem to disappear.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:4>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by andrewthad):

 I have gotten it a little smaller. I've removed some functions from
 `Packed.Bytes.Parser`, and there are also no more levity polymorphic
 functions. Reminder about weird things:

 1. Problems only happen when using `cabal` or `stack` to build, not when
 using ghc on its own.
 2. Collapsing all modules into a single module (which affects inlining)
 makes the crash disappear.
 3. This happens with `-O0` and `-O2`.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:5>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by andrewthad):

 Here's my rudimentary analysis of the error message.

 {{{
 internal error: evacuate(static): strange closure type 16648
 }}}

 If this were caused by user code, I think it could only be the result of
 an out-of-bounds call to `writeArray`, `writeByteArray`, `readArray`, or
 `indexArray`. An out of bounds function that reads from a byte array would
 not cause this. It would simply give the user a garbage value. Well, I
 guess if it were far enough out-of-bounds, the operating system would kill
 the process, but that doesn't happen here. The function `writeByteArray`
 is used a few times, but all of its uses look safe to me.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:6>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by osa1):

 I managed to reproduce this without stack or cabal. Steps:

 - Install dependencies: `cabal install containers split ghc-prim
 primitive`
 - Build the executable: `ghc-stage2 common/Data/Trie/Naive.hs
 src/Packed/Bytes.hs src/Packed/Bytes/Parser.hs src/Packed/Bytes/Small.hs
 src/Packed/Bytes/Stream/ST.hs src/Packed/Bytes/Window.hs test/Main.hs
 test/Parser.hs  -fforce-recomp -o main`
 - Run `./main`

 It works fine with `-O0`, fails with `-O1` and `-O2`, but with different
 "strange closure" types. `-O1` prints 16661, `-O2` prints 16665. Even more
 interestingly I get a different number if I also add `-debug`: 16621.

 The object with "strange closure type" is a static one, and does not
 reside in heap. On my system it's at 0x967900.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:7>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by andrewthad):

 Thanks osa1 for reproducing this with just ghc and for the analysis. I'll
 try to poke around over the next few days to see if I can minimize failing
 case further.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:8>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bgamari):

 Any update on this, andrewthad?

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:9>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by andrewthad):

 I just spent some time making it a little more minimal. It's down to about
 450 lines of code at this point, which is still pretty high. I've also
 consolidated the modules somewhat. I'll keep trying to get this smaller
 over the next few days.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:10>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by andrewthad):

 My additional changes have been pushed to the github repo. Also, here's my
 two cents about what I think is going on. The thing that causes the
 problem appears to be:

 {{{
 snmptrapdNaive :: Naive.Trie (Parser Word)
 snmptrapdNaive = Naive.fromStringList
   [ ("STRING: ", P.any >>= \_ -> return 5)
   ]
 }}}

 If I use my parser language to simply build up a parser, and then I run
 the parser, everything works fine. However, sticking it inside of some
 other data structure seems like it causes problems (I'm not sure if it
 matters that the value is a CAF). Also, my parser type is a newtype
 wrapper around a function that uses unboxed sums. I'm also not sure if
 that matters.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:11>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by osa1):

 Thanks for simplifying the code.

 I made an attempt at removing unboxed sums (and had to remove some unboxed
 tuples on the way) [https://github.com/andrewthad/corrupted-memory-
 example/pull/1 here], and that made the bug disappear. So this may be
 related with unboxed tuples/sums.

 > If I use my parser language to simply build up a parser, and then I run
 the parser

 Can you elaborate on this? What do you mean by building your parser
 language? Can you share the code?

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:12>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by andrewthad):

 The full original code is the `packed` repository (along with its test
 suite): https://github.com/andrewthad/packed.

 The minimal example repo I put up is just `packed` but with everything
 that wasn't needed to make the error happen stripped out. Here is an
 example of a parser that I build in the original code that doesn't cause
 errors:
 https://github.com/andrewthad/packed/blob/master/test/Parser.hs#L112-L122.
 None of the parsers from the original test suite ever cause crashes
 excepted for the trie-based one where a `Parser` is nested inside of a
 `Trie`, which causes a crash whenever the GC is run. Let me know if
 there's a way that I can clarify this further.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:13>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by osa1):

 I spent some more time debugging this. Basically at one point the GC
 overwrites an info table, messing up the type field.

 {{{
 $ gdb ./Main -nh
 Reading symbols from ./Main...done.
 (gdb) start
 Temporary breakpoint 1 at 0x40d29e
 Starting program: /home/omer/haskell/corrupted-memory-example/Main
 [Thread debugging using libthread_db enabled]
 Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

 Temporary breakpoint 1, 0x000000000040d29e in main ()
 }}}

 Now I can see that type of an info table is `CONSTR_2_0`:

 {{{
 (gdb) print get_itbl(0x9815f8)->type
 $1 = 4
 }}}

 Now continue until it crashes:

 {{{
 (gdb) c
 Continuing.
 Main: internal error: evacuate(static): strange closure type 16580
     (GHC version 8.5.20180425 for x86_64_unknown_linux)
     Please report this as a GHC bug:
 http://www.haskell.org/ghc/reportabug

 Program received signal SIGABRT, Aborted.
 0x00007ffff6e90428 in __GI_raise (sig=sig@entry=6) at
 ../sysdeps/unix/sysv/linux/raise.c:54
 54      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
 }}}

 and at this point the same info table has wrong type field:

 {{{
 (gdb) print get_itbl(0x9815f8)->type
 $2 = 16580
 }}}

 Adding a watchpoint to this location causes gdb to leak memory so I
 couldn't do it from the start. I realized that this happens when evacuate
 is called 143851nd time so I run the program until that call, then enable
 the watchpoint, and continue. First hit is

 {{{
 Watchpoint 3: get_itbl(0x9815f8)->type

 Old value = 4
 New value = 0
 evacuate_static_object (link_field=0x9815f8, q=0x9815f0) at
 rts/sm/Evac.c:347
 347             gct->static_objects = (StgClosure *)new_list_head;

 (gdb) bt
 #0  evacuate_static_object (link_field=0x9815f8, q=0x9815f0) at
 rts/sm/Evac.c:347
 #1  0x00000000008e79e0 in evacuate (p=0x981608) at rts/sm/Evac.c:546
 #2  0x0000000000910c29 in scavenge_static () at rts/sm/Scav.c:1831
 #3  0x00000000009111f8 in scavenge_loop () at rts/sm/Scav.c:2185
 #4  0x00000000008e34bb in scavenge_until_all_done () at rts/sm/GC.c:1092
 #5  0x00000000008e2118 in GarbageCollect (collect_gen=1,
 do_heap_census=false, gc_type=0, cap=0xad4080 <MainCapability>,
 idle_cap=0x0) at rts/sm/GC.c:418
 #6  0x00000000008d4dfb in scheduleDoGC (pcap=0x7fffffffd630,
 task=0xaebd00, force_major=false) at rts/Schedule.c:1799
 #7  0x00000000008d4347 in schedule (initialCapability=0xad4080
 <MainCapability>, task=0xaebd00) at rts/Schedule.c:545
 #8  0x00000000008d57a1 in scheduleWaitThread (tso=0x4200105388, ret=0x0,
 pcap=0x7fffffffd720) at rts/Schedule.c:2533
 #9  0x00000000008d7997 in rts_evalLazyIO (cap=0x7fffffffd720, p=0x981710,
 ret=0x0) at rts/RtsAPI.c:530
 #10 0x00000000008d808a in hs_main (argc=1, argv=0x7fffffffd918,
 main_closure=0x981710, rts_config=...) at rts/RtsMain.c:72
 #11 0x000000000040d39a in main ()
 }}}

 Here it gets overwritten to `0`. When I continue it gets overwritten one
 more time:

 {{{
 (gdb) c
 Continuing.

 Watchpoint 3: get_itbl(0x9815f8)->type

 Old value = 0
 New value = 16580
 scavenge_static () at rts/sm/Scav.c:1789
 1789        gct->scavenged_static_objects = flagged_p;

 (gdb) bt
 #0  scavenge_static () at rts/sm/Scav.c:1789
 #1  0x00000000009111f8 in scavenge_loop () at rts/sm/Scav.c:2185
 #2  0x00000000008e34bb in scavenge_until_all_done () at rts/sm/GC.c:1092
 #3  0x00000000008e2118 in GarbageCollect (collect_gen=1,
 do_heap_census=false, gc_type=0, cap=0xad4080 <MainCapability>,
 idle_cap=0x0) at rts/sm/GC.c:418
 #4  0x00000000008d4dfb in scheduleDoGC (pcap=0x7fffffffd630,
 task=0xaebd00, force_major=false) at rts/Schedule.c:1799
 #5  0x00000000008d4347 in schedule (initialCapability=0xad4080
 <MainCapability>, task=0xaebd00) at rts/Schedule.c:545
 #6  0x00000000008d57a1 in scheduleWaitThread (tso=0x4200105388, ret=0x0,
 pcap=0x7fffffffd720) at rts/Schedule.c:2533
 #7  0x00000000008d7997 in rts_evalLazyIO (cap=0x7fffffffd720, p=0x981710,
 ret=0x0) at rts/RtsAPI.c:530
 #8  0x00000000008d808a in hs_main (argc=1, argv=0x7fffffffd918,
 main_closure=0x981710, rts_config=...) at rts/RtsMain.c:72
 #9  0x000000000040d39a in main ()
 }}}

 The value `16580` is what we see in the error message.

 I don't know if this is a problem with GC, maybe it's something wrong with
 the code gen.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:14>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  highest           |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by osa1):

 * cc: simonpj, bgamari (added)
 * priority:  normal => highest


Comment:

 Finally found the problem. In summary; in unarise we introduce
 `absentError` calls for unboxed sum slots that are not supposed to be used
 and `absentError` id (`aBSENT_ERROR_ID`) is marked as CAFFY. Now suppose
 we had a top-level definition that is not CAFFY but uses unboxed sums.
 `TidyPgm` thinks that it's not CAFFY. But in unarise we make it CAFFY.
 This causes mismatch between the CAF info recorded in the binder id for
 this top-level definition and the actual RHS. The code that generates
 closure layout doesn't add a static_link field, but the code that
 generates the info table generates a SRT and SRT bitmap.

 Some potential solutions:

 - Why is `absentError` marked as CAF? I think error ids are conservatively
 marked as `MayHaveCafRefs` but perhaps it's fine to not mark `absentError`
 as `MayHaveCafRefs`. (I implemented this and it fixed the bug)

 - Update CAFFY-ness of ids in unarise if we introduce `absentError`
 (haven't tried this yet)

 - Something else?

 Any ideas? Ping @simonpj @bgamari

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:15>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  highest           |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bgamari):

 Good sleuthing!

 However, I'm not sure about marking `absentError` as non-CAFFY. Afterall,
 judging by its RHS it looks as though it may carry a reference to at least
 one CAF (namely `unpackCString# "Oops!  Entered absent arg"#). Have you
 looked at the STG that is produced for this?

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:16>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  highest           |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by osa1):

 > STG that is produced for this?

 Do you mean STG for `absentError` or the function with wrong CAFFY-ness
 information? I haven't checked `absentError` but this is the STG for the
 function with wrong CAFFYness information:

 {{{
 lvl3_rctu
   :: forall s.
      Packed.Bytes.Parser.Maybe# (Packed.Bytes.Parser.Leftovers# s)
      -> GHC.Prim.State# s
      -> (# GHC.Prim.State# s,
            Packed.Bytes.Parser.Result# s GHC.Types.Word #)
 [GblId,
  Arity=2,
  Caf=NoCafRefs,
  Str=<S,1*U><S,U>,
  Unf=OtherCon []] =
     [] \r [us_gcwc us_gcwd us_gcwe us_gcwf us_gcwg void_0E]
         case us_gcwc of tag_gcwh {
           __DEFAULT ->
               (#,,,,,,#) [1#
                           Control.Exception.Base.absentError
                           Control.Exception.Base.absentError
                           0##
                           0##
                           1#
                           Control.Exception.Base.absentError];
           2# ->
               case ># [us_gcwg 0#] of {
                 __DEFAULT ->
                     case Packed.Bytes.Parser.nextNonEmpty us_gcwe
 GHC.Prim.void# of {
                       (#,,,,#) us_gcwi us_gcwj us_gcwk us_gcwl us_gcwm ->
                           case us_gcwi of tag_gcwn {
                             __DEFAULT ->
                                 (#,,,,,,#) [1#
 Control.Exception.Base.absentError
 Control.Exception.Base.absentError
                                             0##
                                             0##
                                             1#
 Control.Exception.Base.absentError];
                             2# ->
                                 case -# [us_gcwm 1#] of sat_scuk {
                                   __DEFAULT ->
                                       case +# [us_gcwl 1#] of sat_scuj {
                                         __DEFAULT ->
                                             (#,,,,,,#) [2#
                                                         us_gcwj
                                                         us_gcwk
                                                         sat_scuj
                                                         sat_scuk
                                                         2#
                                                         a1_rcbR];
                                       };
                                 };
                           };
                     };
                 1# ->
                     case -# [us_gcwg 1#] of sat_scur {
                       __DEFAULT ->
                           case +# [us_gcwf 1#] of sat_scuq {
                             __DEFAULT ->
                                 (#,,,,,,#) [2# us_gcwd us_gcwe sat_scuq
 sat_scur 2# a1_rcbR];
                           };
                     };
               };
         };
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:17>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  highest           |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bgamari):

 The STG for `absentError`; we want to know specifically whether it is
 necessary to mark it as CAFFY.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:18>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15038: Memory Corruption (strange closure type)

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15038: Memory Corruption (strange closure type)
-------------------------------------+-------------------------------------
        Reporter:  andrewthad        |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  highest           |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.4.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  Runtime crash     |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by osa1):

 STG for `absentError`:

 {{{
 lvl_r3wv :: GHC.Prim.Addr#
 [GblId, Caf=NoCafRefs, Unf=OtherCon []] =
     "Oops!  Entered absent arg "#;

 Control.Exception.Base.absentError :: forall a. GHC.Prim.Addr# -> a
 [GblId, Arity=1, Str=<B,U>x, Unf=OtherCon []] =
     [] \r [s_s3BA]
         let {
           sat_s3BB [Occ=Once] :: [GHC.Types.Char]
           [LclId] =
               [s_s3BA] \u [] GHC.CString.unpackCStringUtf8# s_s3BA;
         } in
           case
               GHC.CString.unpackAppendCString# lvl_r3wv sat_s3BB
           of
           sat_s3BC
           { __DEFAULT -> GHC.Err.errorWithoutStackTrace sat_s3BC;
           };
 }}}

 Apparently this is CAFFY (othewise we'd see `Caf=NoCafRefs`).

 I also realized that unarise uses `absentError` wrong: `absentError`
 expects an `Addr#` argument (for a string that represents the "absent"
 argument) but unarise just uses `absentError` without an argument. So
 there's also a type error.

 Applying `absentError` to an argument in unarise would mean let bindings
 (heap allocation) so perhaps we should use something else for unused
 pointer location in unboxed sums. (IIRC when we first used `absentError`
 it wasn't taking an argument)

 If the id we'll use instead of `absentError` is not CAFFY then this
 problem will disappear. However I wonder if we could somehow implement a
 check that ensures we won't make things more caffy after `TidyPgm` in the
 future.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15038#comment:19>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
123