T3294 variies in whether it fails or not

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

T3294 variies in whether it fails or not

Simon Peyton Jones
Ian

Perf/compiler/T3294 has very different max_bytes_used when BINDIST=YES than without. See below.  

I don't know why.  I don't know whether it matters.  Maybe it's just the moment that GC strikes.  But it's uncomfortable to have such an erratic test failure.

Simon

simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$ make TEST=T3294
python2 ../../../driver/runtests.py  -e ghc_compiler_always_flags="'-fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts '" -e ghc_debugged=False -e ghc_with_native_codegen=1 -e ghc_with_vanilla=1 -e ghc_with_dynamic=1 -e ghc_with_profiling=0 -e ghc_with_threaded_rts=1 -e ghc_with_dynamic_rts=1 -e ghc_with_interpreter=1 -e ghc_unregisterised=0 -e ghc_dynamic_by_default=False -e ghc_dynamic=True -e ghc_with_smp=1 -e ghc_with_llvm=1 -e windows=False -e darwin=False -e in_tree_compiler=True -e clean_only=False --rootdir=. --config=../../../config/ghc -e 'config.confdir="../../../config"' -e 'config.compiler="/5playpen/simonpj/HEAD-1/inplace/bin/ghc-stage2"' -e 'config.ghc_pkg="/5playpen/simonpj/HEAD-1/inplace/bin/ghc-pkg"' -e 'config.hp2ps="/5playpen/simonpj/HEAD-1/inplace/bin/hp2ps"' -e 'config.hpc="/5playpen/simonpj/HEAD-1/inplace/bin/hpc"' -e 'config.gs="gs"' -e 'config.platform="x86_64-unknown-linux"' -e 'config.os="linux"' -e 'config.arch="x86_64"' -e 'config.wordsize="64"' -e 'default_testopts.cleanup=""' -e 'config.timeout=int() or config.timeout' -e 'config.timeout_prog="../../../timeout/install-inplace/bin/timeout"' -e 'config.exeext=""' -e 'config.top="/5playpen/simonpj/HEAD-1/testsuite"'  \
                 --only=T3294 \
                 \
                 \
                 \
                 \
                 \
               
Timeout is 300
Found 1 .T files...
Beginning test run at Tue May 28 09:21:29 BST 2013
====> Scanning ./all.T
=====> T3294(normal) 2 of 14 [0, 0, 0]
cd . && '/5playpen/simonpj/HEAD-1/inplace/bin/ghc-stage2' -fforce-recomp -dno-debug-output -no-user-package-db -rtsopts -fno-ghci-history -c T3294.hs   +RTS -V0 -tT3294.comp.stats --machine-readable -RTS  >T3294.comp.stderr 2>&1

OVERALL SUMMARY for test run started at Tue May 28 09:21:29 BST 2013
      14 total tests, which gave rise to
      47 test cases, of which
      46 were skipped

       0 had missing libraries
       1 expected passes
       0 expected failures

       0 caused framework failures
       0 unexpected passes
       0 unexpected failures

simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$ cat T3294.comp.stats
/5playpen/simonpj/HEAD-1/inplace/lib/bin/ghc-stage2 -B/5playpen/simonpj/HEAD-1/inplace/lib -fforce-recomp -dno-debug-output -no-user-package-db -rtsopts -fno-ghci-history -c T3294.hs +RTS -V0 -tT3294.comp.stats --machine-readable
 [("bytes allocated", "2721056936")
 ,("num_GCs", "540")
 ,("average_bytes_used", "15190408")
 ,("max_bytes_used", "36726792")
 ,("num_byte_usage_samples", "15")
 ,("peak_megabytes_allocated", "105")
 ,("init_cpu_seconds", "0.00")
 ,("init_wall_seconds", "0.00")
 ,("mutator_cpu_seconds", "2.64")
 ,("mutator_wall_seconds", "3.25")
 ,("GC_cpu_seconds", "2.03")
 ,("GC_wall_seconds", "2.03")
 ]
simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$ make TEST=T3294 BINDIST=YES
python2 ../../../driver/runtests.py  -e ghc_compiler_always_flags="'-fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts '" -e ghc_debugged=False -e ghc_with_native_codegen=1 -e ghc_with_vanilla=1 -e ghc_with_dynamic=1 -e ghc_with_profiling=0 -e ghc_with_threaded_rts=1 -e ghc_with_dynamic_rts=1 -e ghc_with_interpreter=1 -e ghc_unregisterised=0 -e ghc_dynamic_by_default=False -e ghc_dynamic=True -e ghc_with_smp=1 -e ghc_with_llvm=1 -e windows=False -e darwin=False -e in_tree_compiler=True -e clean_only=False --rootdir=. --config=../../../config/ghc -e 'config.confdir="../../../config"' -e 'config.compiler="/5playpen/simonpj/HEAD-1/bindisttest/install   dir/bin/ghc"' -e 'config.ghc_pkg="/5playpen/simonpj/HEAD-1/bindisttest/install   dir/bin/ghc-pkg"' -e 'config.hp2ps="/5playpen/simonpj/HEAD-1/bindisttest/install   dir/bin/hp2ps"' -e 'config.hpc="/5playpen/simonpj/HEAD-1/bindisttest/install   dir/bin/hpc"' -e 'config.gs="gs"' -e 'config.platform="x86_64-unknown-linux"' -e 'config.os="linux"' -e 'config.arch="x86_64"' -e 'config.wordsize="64"' -e 'default_testopts.cleanup=""' -e 'config.timeout=int() or config.timeout' -e 'config.timeout_prog="../../../timeout/install-inplace/bin/timeout"' -e 'config.exeext=""' -e 'config.top="/5playpen/simonpj/HEAD-1/testsuite"'  \
                 --only=T3294 \
                 \
                 \
                 \
                 \
                 \
               
Timeout is 300
Found 1 .T files...
Beginning test run at Tue May 28 09:21:49 BST 2013
====> Scanning ./all.T
=====> T3294(normal) 2 of 14 [0, 0, 0]
cd . && '/5playpen/simonpj/HEAD-1/bindisttest/install   dir/bin/ghc' -fforce-recomp -dno-debug-output -no-user-package-db -rtsopts -fno-ghci-history -c T3294.hs   +RTS -V0 -tT3294.comp.stats --machine-readable -RTS  >T3294.comp.stderr 2>&1
max_bytes_used value is too high:
    Expected    max_bytes_used: 36904752 +/-15%
    Lower bound max_bytes_used: 31369039
    Upper bound max_bytes_used: 42440465
    Actual      max_bytes_used: 43818472
*** unexpected failure for T3294(normal)

Unexpected results from:
TEST="T3294"

OVERALL SUMMARY for test run started at Tue May 28 09:21:49 BST 2013
      14 total tests, which gave rise to
      47 test cases, of which
      46 were skipped

       0 had missing libraries
       0 expected passes
       0 expected failures

       0 caused framework failures
       0 unexpected passes
       1 unexpected failures

Unexpected failures:
   .  T3294 [stat not good enough] (normal)

simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$ cat T3294.comp.stats
/5playpen/simonpj/HEAD-1/bindisttest/install   dir/lib/ghc-7.7/bin/ghc -B/5playpen/simonpj/HEAD-1/bindisttest/install   dir/lib/ghc-7.7 -fforce-recomp -dno-debug-output -no-user-package-db -rtsopts -fno-ghci-history -c T3294.hs +RTS -V0 -tT3294.comp.stats --machine-readable
 [("bytes allocated", "2724158504")
 ,("num_GCs", "525")
 ,("average_bytes_used", "16042228")
 ,("max_bytes_used", "43818472")
 ,("num_byte_usage_samples", "15")
 ,("peak_megabytes_allocated", "116")
 ,("init_cpu_seconds", "0.00")
 ,("init_wall_seconds", "0.00")
 ,("mutator_cpu_seconds", "2.77")
 ,("mutator_wall_seconds", "3.38")
 ,("GC_cpu_seconds", "2.38")
 ,("GC_wall_seconds", "2.38")
 ]
simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$


Reply | Threaded
Open this post in threaded view
|

T3294 variies in whether it fails or not

Ian Lynagh-2
On Tue, May 28, 2013 at 08:24:34AM +0000, Simon Peyton-Jones wrote:

> Ian
>
> Perf/compiler/T3294 has very different max_bytes_used when BINDIST=YES than without. See below.  
>
> I don't know why.  I don't know whether it matters.  Maybe it's just the moment that GC strikes.  But it's uncomfortable to have such an erratic test failure.
>
> Simon
>
> simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$ make TEST=T3294
>  ,("max_bytes_used", "36726792")
>
> simonpj at cam-05-unx:~/5builds/HEAD-1/testsuite/tests/perf/compiler$ make TEST=T3294 BINDIST=YES
>  ,("max_bytes_used", "43818472")

Hmm, my first (inplace) run was 41241960, but since then I'm reliably
getting 38586080 for inplace, and 38904040 for BINDIST=YES.

I'm not sure why they are different at all; perhaps it's just due to the
size of the package.conf or something.


--
Ian Lynagh, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/