[GHC] #15225: `-fno-state-hack` produces incorrect results in nofib

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[GHC] #15225: `-fno-state-hack` produces incorrect results in nofib

GHC - devs mailing list
#15225: `-fno-state-hack` produces incorrect results in nofib
-------------------------------------+-------------------------------------
           Reporter:  tdammers       |             Owner:  (none)
               Type:  bug            |            Status:  new
           Priority:  normal         |         Milestone:  8.6.1
          Component:  Compiler       |           Version:  8.5
           Keywords:                 |  Operating System:  Linux
       Architecture:  x86_64         |   Type of failure:  Incorrect result
  (amd64)                            |  at runtime
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:  #7411
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 While investigating #7411, I found that nofib fails with incorrect output
 from the `fasta` test.

 A vanilla `prof` build behaves normally; however, with the following
 modifications, `nofib` fails:

 - Compile GHC from scratch with:
   {{{
     GhcStage2HcOpts    = -O -fno-state-hack
     GhcLibHcOpts       = -O -fno-state-hack
   }}}
 - Run nofib with `EXTRA_HC_OPTS=-fno-state-hack`

 Doing this, the nofib test output is:
 {{{
 ------------------------------------------------------------------------
 == make all --no-print-directory;
  in /home/tobias/well-typed/devel/ghc/HEAD/nofib/shootout/fasta
 ------------------------------------------------------------------------
 HC = /home/tobias/well-typed/devel/ghc/HEAD/inplace/bin/ghc-stage2
 HC_OPTS = -O2 -Rghc-timing -H32m -hisuf hi -fno-state-hack -O2 -rtsopts
 -O2 -XBangPatterns -XOverloadedStrings -package bytestring
 RUNTEST_OPTS = -ghc-timing
 ==nofib== fasta: time to compile Main follows...
 /home/tobias/well-typed/devel/ghc/HEAD/inplace/bin/ghc-stage2 -O2 -Rghc-
 timing -H32m -hisuf hi -fno-state-hack -O2 -rtsopts -O2 -XBangPatterns
 -XOverloadedStrings -package bytestring -c Main.hs -o Main.o

 Main.hs:30:1: warning: [-Wtabs]
     Tab character found here, and in 13 further locations.
     Please use spaces instead.
    |
 30 |         let !k = min modulus (floor (fromIntegral modulus *
 (p::Float) + 1))
    | ^^^^^^^^
 <<ghc: 401958512 bytes, 72 GCs, 10720054/28796976 avg/max bytes residency
 (6 samples), 61M in use, 0.001 INIT (0.001 elapsed), 0.342 MUT (0.420
 elapsed), 0.240 GC (0.238 elapsed) :ghc>>
 ==nofib== fasta: size of Main.o follows...
    text    data     bss     dec     hex filename
    7245    2624       0    9869    268d Main.o
 ==nofib== fasta: time to link fasta follows...
 <<ghc: 49626024 bytes, 45 GCs, 1356153/2672728 avg/max bytes residency (5
 samples), 8M in use, 0.001 INIT (0.001 elapsed), 0.025 MUT (0.468
 elapsed), 0.060 GC (0.060 elapsed) :ghc>>
 ==nofib== fasta: size of fasta follows...
    text    data     bss     dec     hex filename
  708996  127712   16232  852940   d03cc fasta
 ==nofib== fasta: time to run fasta follows...
 ../../runstdtest/runstdtest ./fasta  -o1 fasta.stdout -o1 fasta.stdout
 -ghc-timing     2500000;   ../../runstdtest/runstdtest ./fasta  -o1
 fasta.stdout -o1 fasta.stdout  -ghc-timing     2500000;
 ../../runstdtest/runstdtest ./fasta  -o1 fasta.stdout -o1 fasta.stdout
 -ghc-timing     2500000;   ../../runstdtest/runstdtest ./fasta  -o1
 fasta.stdout -o1 fasta.stdout  -ghc-timing     2500000;
 ../../runstdtest/runstdtest ./fasta  -o1 fasta.stdout -o1 fasta.stdout
 -ghc-timing     2500000;
 0.43user 0.02system 0:00.45elapsed 99%CPU (0avgtext+0avgdata
 4824maxresident)k
 0inputs+49656outputs (0major+608minor)pagefaults 0swaps
 ././fasta 2500000 < /dev/null
 expected stdout not matched by reality
 --- fasta.stdout        2018-06-02 03:00:43.887025433 +0200
 +++ /tmp/runtest4673.1  2018-06-02 03:09:19.651755697 +0200
 @@ -83337,333335 +83337,333335 @@
  cttBtatcatatgctaKggNcataaaSatgtaaaDcDRtBggDtctttataattcBgtcg
 -tactDtDagcctatttSVHtHttKtgtHMaSattgWaHKHttttagacatWatgtRgaaa
 -NtactMcSMtYtcMgRtacttctWBacgaaatatagScDtttgaagacacatagtVgYgt
 -cattHWtMMWcStgttaggKtSgaYaaccWStcgBttgcgaMttBYatcWtgacaYcaga
 -gtaBDtRacttttcWatMttDBcatWtatcttactaBgaYtcttgttttttttYaaScYa
 -HgtgttNtSatcMtcVaaaStccRcctDaataataStcYtRDSaMtDttgttSagtRRca
 #### ~3.1 million similar lines follow ####
 -taatataagctgcgccaggggatttttccagatcatctggcctgtgtatatgttcaaatc
 +gacgaatatattagttatagtttactatccaaataaattaagcgaatcgaaataaactgt
 +gacgaatatattagttatagtttactatccaaataaattaagcgaatcgaaataaactgt
 +gacgaatatattagttatagtttactatccaaataaattaagcgaatcgaaataaactgt
 #### ~208k identical lines follow
 +gacgaatatattagttatagtttactatccaaataaattaagcgaatcgaaataaactgt
  taatagccgagagaaattac
 ../../mk/target.mk:101: recipe for target 'runtests' failed
 make[2]: *** [runtests] Error 1
 Failed making all in fasta: 1
 ../mk/ghc-recurse.mk:65: recipe for target 'all' failed
 make[1]: *** [all] Error 1
 Failed making all in shootout: 1
 mk/ghc-recurse.mk:65: recipe for target 'all' failed
 make: *** [all] Error 1
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15225>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15225: `-fno-state-hack` produces incorrect results in nofib

GHC - devs mailing list
#15225: `-fno-state-hack` produces incorrect results in nofib
-------------------------------------+-------------------------------------
        Reporter:  tdammers          |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.5
      Resolution:                    |             Keywords:
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Incorrect result  |  (amd64)
  at runtime                         |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #7411             |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by alpmestan):

 * cc: alpmestan (added)


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15225#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15225: `-fno-state-hack` produces incorrect results in nofib

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15225: `-fno-state-hack` produces incorrect results in nofib
-------------------------------------+-------------------------------------
        Reporter:  tdammers          |                Owner:  tdammers
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.5
      Resolution:                    |             Keywords:
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Incorrect result  |  (amd64)
  at runtime                         |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #7411             |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by tdammers):

 * owner:  (none) => tdammers


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15225#comment:2>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15225: `-fno-state-hack` produces incorrect results in nofib

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15225: `-fno-state-hack` produces incorrect results in nofib
-------------------------------------+-------------------------------------
        Reporter:  tdammers          |                Owner:  tdammers
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.5
      Resolution:                    |             Keywords:
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Incorrect result  |  (amd64)
  at runtime                         |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #7411             |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by tdammers):

 Running only nofib with `-fno-state-hack`, but with GHC and libraries
 compiled normally, the problem does not appear.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15225#comment:3>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15225: `-fno-state-hack` produces incorrect results in nofib

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15225: `-fno-state-hack` produces incorrect results in nofib
-------------------------------------+-------------------------------------
        Reporter:  tdammers          |                Owner:  tdammers
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.5
      Resolution:                    |             Keywords:
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Incorrect result  |  (amd64)
  at runtime                         |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #7411             |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by tdammers):

 Two more test runs:

 - GHC with state hack, libraries without state hack, nofib without state
 hack triggers problem
 - GHC with state hack, libraries without state hack, nofib with state hack
 does not trigger problem

 So apparently, in order to trigger the problem, both nofib and libraries
 must be compiled with `-fno-state-hack`.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15225#comment:4>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15225: `-fno-state-hack` produces incorrect results in nofib

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15225: `-fno-state-hack` produces incorrect results in nofib
-------------------------------------+-------------------------------------
        Reporter:  tdammers          |                Owner:  tdammers
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.5
      Resolution:                    |             Keywords:
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Incorrect result  |  (amd64)
  at runtime                         |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #7411             |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by tdammers):

 So here's a hint. The documentation to `unsafeUseAsCString` mentions this:

 > After calling this function the `CString` shares the underlying byte
 > buffer with the original `ByteString`. Thus **modifying the
 > `CString`**, either in C, or **using poke**, will cause the contents
 > of the `ByteString` to change, **breaking referential transparency**.
 > Other `ByteStrings` created by sharing (such as those produced via
 > 'take' or 'drop') will also reflect these changes.  Modifying the
 > `CString` will break referential transparency. To avoid this, use
 > `useAsCString`, which makes a copy of the original `ByteString`.

 (emphasis mine).

 And then in the `fasta/Main.hs` source code, we see this:

 {{{#!hs
   unsafeUseAsCString line $ \ptr -> do
     --- snip ---
     plusPtr ptr i `poke` unsafeIndex lookupTable newseed
     --- snip ---
 }}}

 In other words, `fasta` does exactly what the documentation says not to
 do.

 Which would suggest that the state hack itself isn't really to blame, it
 just causes different sharing behavior.

 I'll run a few tests to get more certainty here.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15225#comment:5>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15225: `-fno-state-hack` produces incorrect results in nofib

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15225: `-fno-state-hack` produces incorrect results in nofib
-------------------------------------+-------------------------------------
        Reporter:  tdammers          |                Owner:  tdammers
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.5
      Resolution:                    |             Keywords:
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Incorrect result  |  (amd64)
  at runtime                         |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #7411             |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bgamari):

 Yikes! Indeed that does look quite suspicious since `line` is also used
 elsewhere in this block. Sounds like this is the culprit.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15225#comment:6>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15225: `-fno-state-hack` produces incorrect results in nofib

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15225: `-fno-state-hack` produces incorrect results in nofib
-------------------------------------+-------------------------------------
        Reporter:  tdammers          |                Owner:  tdammers
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.5
      Resolution:                    |             Keywords:
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Incorrect result  |  (amd64)
  at runtime                         |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #7411             |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by tdammers):

 I quickly and naively tried just using `useAsCString` instead of
 `unsafeUseAsCString`, but of course that doesn't quite work - probably
 because the code relies on actually manipulating the bytestring in place,
 whereas `useAsCString` creates a copy and manipulates that. So the output
 no longer matches that of the C implementation, but the `-fno-state-hack`
 flag no longer causes the output to differ.

 So I went a step further, and rewrote the relevant part in more idiomatic
 Haskell, using `ByteString.unfoldrN` instead of manipulating C strings in-
 place:

 {{{#!diff
 commit dc2753f4e10fec79c02ed293b72573f1aeaa2271
 Author: Tobias Dammers <[hidden email]>
 Date:   Fri Jun 15 11:27:36 2018 +0200

     Rewrite fasta in more idiomatic Haskell

 diff --git a/shootout/fasta/Main.hs b/shootout/fasta/Main.hs
 index 4bd0849..795a470 100644
 --- a/shootout/fasta/Main.hs
 +++ b/shootout/fasta/Main.hs
 @@ -11,7 +11,9 @@ import Foreign.Ptr
  import Foreign.Storable
  import System.Environment
  import qualified Data.ByteString.Char8 as B
 +import qualified Data.ByteString as BS
  import qualified Data.ByteString.Lazy.Char8 as L
 +import Data.Word (Word8)

  main = do
      n <- getArgs >>= readIO.head
 @@ -27,21 +29,30 @@ make name n0 tbl seed0 = do
    B.putStrLn name
    let modulus = 139968
        fill ((c,p):cps) j =
 -       let !k = min modulus (floor (fromIntegral modulus * (p::Float) +
 1))
 -       in B.replicate (k - j) c : fill cps k
 +        let !k = min modulus (floor (fromIntegral modulus * (p::Float) +
 1))
 +        in B.replicate (k - j) c : fill cps k
        fill _ _ = []
        lookupTable = B.concat $ fill (scanl1 (\(_,p) (c,q) -> (c,p+q))
 tbl) 0
 -      line = B.replicate 60 '\0'
 -  unsafeUseAsCString line $ \ptr -> do
 -    let make' n !i seed
 -           | n > (0::Int) = do
 -               let newseed = rem (seed * 3877 + 29573) modulus
 -               plusPtr ptr i `poke` unsafeIndex lookupTable newseed
 -               if i+1 >= 60
 -                   then puts line 60 >> make' (n-1) 0 newseed
 -                   else make' (n-1) (i+1) newseed
 -           | otherwise = when (i > 0) (puts line i) >> return seed
 -    make' n0 0 seed0
 +
 +  let next :: Int -> Maybe (Word8, Int)
 +      next seed =
 +        let newseed = rem (seed * 3877 + 29573) modulus
 +            val = unsafeIndex lookupTable newseed
 +        in Just (val, newseed)
 +
 +  let make' :: Int -> Int -> IO Int
 +      make' n seed = do
 +        if n <= (0 :: Int) then
 +          return seed
 +        else if n >= 60 then do
 +          let (line, Just newseed) = BS.unfoldrN 60 next seed
 +          puts line 60
 +          make' (n-60) newseed
 +        else do
 +          let (line, Just newseed) = BS.unfoldrN n next seed
 +          puts line n
 +          return newseed
 +  make' n0 seed0

  alu =
 "GGCCGGGCGCGGTGGCTCACGCCTGTAATCCCAGCACTTTGGGAGGCCGAGGCGGGCGGATCACCTGAGG\
 \TCAGGAGTTCGAGACCAGCCTGGCCAACATGGTGAAACCCCGTCTCTACTAAAAATACAAAAATTAGCCGGG\
 }}}

 The patched version produces the correct output regardless of the state
 hack; however, I haven't done a full profiling run yet to see how it
 affects performance.

 However, considering that the unidiomatic version is essentially wrong,
 and, well, unidiomatic, I'm not so sure whether it told us anything
 meaningful in the first place.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15225#comment:7>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #15225: `-fno-state-hack` produces incorrect results in nofib

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#15225: `-fno-state-hack` produces incorrect results in nofib
-------------------------------------+-------------------------------------
        Reporter:  tdammers          |                Owner:  tdammers
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Compiler          |              Version:  8.5
      Resolution:                    |             Keywords:
Operating System:  Linux             |         Architecture:  x86_64
 Type of failure:  Incorrect result  |  (amd64)
  at runtime                         |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #7411             |  Differential Rev(s):  Phab:D4868
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by tdammers):

 * differential:   => Phab:D4868


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15225#comment:8>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets