Unreliability of the build system

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
40 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Geoffrey Mainland
On 06/24/2013 01:16 PM, Ian Lynagh wrote:

> On Mon, Jun 24, 2013 at 12:37:17PM +0100, Geoffrey Mainland wrote:
>>
>> It looks like "/home/gmainlan/ghc/ghc-working-build/bindisttest/install
>> dir/lib/ghc-7.7.20130624/package.conf.d" gets *younger*, leading to the
>> recache error. Rather odd... Any ideas why this is happening?
>
> That's expected: When we install xhtml, we add it to the DB, and the
> cache gets updated.
>
> It looks like the problem is this:
>
> Timestamp 2013-06-24 11:02:14.669097 UTC for
[...]/package.conf.d/package.cache

> Timestamp 2013-06-24 11:02:14.669197 UTC for [...]/package.conf.d (NEWER
>
> I would guess that we assume that when we write package.cache, the
> directory gets updated with the same timestamp; but actually, both use
> the timestamp at which they really happen. This is of course much more
> likely to be a problem if your filesystem has sub-second precision.
>
> If that's it, then it should be possible to fix it by explicitly setting
> the modification time of the directory to match the cache when we write
> the cache in ghc-pkg.

I'm using ZFS, so I suspect that this is indeed the issue.

Thanks,
Geoff



Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Geoffrey Mainland
In reply to this post by Jan Stolarek
On 06/24/2013 01:03 PM, Jan Stolarek wrote:
>> The same tests consistently fail for me.
> Even if you run them separately with 'make TEST=failingTest'?
>
>> I only get the errors when running the testsuite with BINDIST=YES, as
>> expected.
> I'm running validation without BINDIST=YES at the moment, will run the
second validation later.

I believe validate uses the binary distribution unless you specify
--fast, e.g., when you specify no arguments to validate.

> Where do you put BINDIST=YES setting?
>
> Jan

One recipe for reliable failure:

cd testsuite && make TEST=T1372 BINDIST=YES

Geoff



Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Jan Stolarek
> One recipe for reliable failure:
>
> cd testsuite && make TEST=T1372 BINDIST=YES
Not reliable I'm afraid - it passes here. Unless I do something wrong.

Jan


[killy at xerxes : /dane/uczelnia/projekty/ghc-validate] cd testsuite && make TEST=T1372 BINDIST=YES
make -C ./tests all
make[1]: Wej?cie do katalogu `/dane/uczelnia/projekty/ghc-validate/testsuite/tests'
python2 ../driver/runtests.py  -e
ghc_compiler_always_flags="'-fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts '" -e
ghc_debugged=False -e ghc_with_native_codegen=1 -e ghc_with_vanilla=1 -e ghc_with_dynamic=1 -e
ghc_with_profiling=0 -e ghc_with_threaded_rts=1 -e ghc_with_dynamic_rts=1 -e
ghc_with_interpreter=1 -e ghc_unregisterised=0 -e ghc_dynamic_by_default=False -e
ghc_dynamic=True -e ghc_with_smp=1 -e ghc_with_llvm=1 -e windows=False -e darwin=False -e
in_tree_compiler=True -e
clean_only=False --rootdir=. --config=../config/ghc -e 'config.confdir="../config"' -e 'config.compiler="/dane/uczelnia/projekty/ghc-validate/bindisttest/install  
dir/bin/ghc"' -e 'config.ghc_pkg="/dane/uczelnia/projekty/ghc-validate/bindisttest/install  
dir/bin/ghc-pkg"' -e 'config.hp2ps="/dane/uczelnia/projekty/ghc-validate/bindisttest/install  
dir/bin/hp2ps"' -e 'config.hpc="/dane/uczelnia/projekty/ghc-validate/bindisttest/install  
dir/bin/hpc"' -e 'config.gs="gs"' -e 'config.platform="x86_64-unknown-linux"' -e 'config.os="linux"' -e 'config.arch="x86_64"' -e 'config.wordsize="64"' -e 'default_testopts.cleanup=""' -e 'config.timeout=int()
or
config.timeout' -e 'config.timeout_prog="../timeout/install-inplace/bin/timeout"' -e 'config.exeext=""' -e 'config.top="/dane/uczelnia/projekty/ghc-validate/testsuite"'   --rootdir=../../libraries/ghc-prim/tests  --rootdir=../../libraries/array/tests  --rootdir=../../libraries/containers/tests  --rootdir=../../libraries/hpc/tests  --rootdir=../../libraries/binary/tests  --rootdir=../../libraries/haskeline/tests  --rootdir=../../libraries/vector/tests  --rootdir=../../libraries/bytestring/tests  --rootdir=../../libraries/random/tests  --rootdir=../../libraries/old-time/tests  --rootdir=../../libraries/pretty/tests  --rootdir=../../libraries/process/tests  --rootdir=../../libraries/directory/tests  --rootdir=../../libraries/filepath/tests  --rootdir=../../libraries/stm/tests  --rootdir=../../libraries/unix/tests  --rootdir=../../libraries/template-haskell/tests  --rootdir=../../libraries/parallel/tests  --rootdir=../../libraries/base/tests
\
         --only=T1372 \
         \
         \
         \
         \
         \

Timeout is 300
Found 243 .T files...
Beginning test run at pon, 24 cze 2013, 15:50:17 CEST
====> Scanning ./cps/all.T
====> Scanning ./layout/all.T
====> Scanning ./annotations/should_run/all.T
====> Scanning ./annotations/should_compile/all.T
====> Scanning ./annotations/should_fail/all.T
====> Scanning ./lib/integer/all.T
====> Scanning ./numeric/should_run/all.T
====> Scanning ./th/TH_import_loop/TH_import_loop.T
====> Scanning ./th/TH_spliceViewPat/test.T
====> Scanning ./th/all.T
====> Scanning ./th/T2014/all.T
====> Scanning ./arrows/should_run/all.T
====> Scanning ./arrows/should_compile/all.T
====> Scanning ./arrows/should_fail/all.T
====> Scanning ./polykinds/all.T
====> Scanning ./ghci.debugger/scripts/all.T
====> Scanning ./ghci.debugger/scripts/break023/all.T
====> Scanning ./ghci.debugger/scripts/break022/all.T
====> Scanning ./profiling/should_run/all.T
====> Scanning ./profiling/should_compile/all.T
====> Scanning ./profiling/should_fail/all.T
====> Scanning ./codeGen/should_gen_asm/all.T
====> Scanning ./codeGen/should_run/all.T
====> Scanning ./codeGen/should_compile/all.T
====> Scanning ./deriving/should_run/all.T
====> Scanning ./deriving/should_compile/all.T
====> Scanning ./deriving/should_fail/all.T
====> Scanning ./boxy/all.T
====> Scanning ./dynlibs/all.T
====> Scanning ./plugins/all.T
====> Scanning ./perf/compiler/all.T
====> Scanning ./perf/space_leaks/all.T
====> Scanning ./perf/should_run/all.T
====> Scanning ./perf/haddock/all.T
====> Scanning ./mdo/should_run/all.T
====> Scanning ./mdo/should_compile/all.T
====> Scanning ./mdo/should_fail/all.T
====> Scanning ./generics/GFunctor/test.T
====> Scanning ./generics/all.T
====> Scanning ./generics/Uniplate/test.T
====> Scanning ./generics/GShow/test.T
====> Scanning ./generics/GEq/test.T
====> Scanning ./generics/GMap/test.T
====> Scanning ./ghc-api/T4891/all.T
====> Scanning ./ghc-api/all.T
====> Scanning ./ghc-api/T7478/all.T
====> Scanning ./ghc-api/dynCompileExpr/all.T
====> Scanning ./ghc-api/apirecomp001/all.T
====> Scanning ./concurrent/prog002/all.T
====> Scanning ./concurrent/prog003/all.T
====> Scanning ./concurrent/T2317/all.T
====> Scanning ./concurrent/prog001/all.T
====> Scanning ./concurrent/should_run/all.T
====> Scanning ./safeHaskell/check/pkg01/all.T
====> Scanning ./safeHaskell/check/all.T
====> Scanning ./safeHaskell/unsafeLibs/all.T
====> Scanning ./safeHaskell/flags/all.T
====> Scanning ./safeHaskell/ghci/all.T
====> Scanning ./safeHaskell/safeInfered/all.T
====> Scanning ./safeHaskell/safeLanguage/all.T
====> Scanning ./llvm/should_compile/all.T
====> Scanning ./dph/enumfromto/dph-enumfromto.T
====> Scanning ./dph/sumnats/dph-sumnats.T
====> Scanning ./dph/nbody/dph-nbody.T
====> Scanning ./dph/quickhull/dph-quickhull.T
====> Scanning ./dph/diophantine/dph-diophantine.T
====> Scanning ./dph/classes/dph-classes.T
====> Scanning ./dph/words/dph-words.T
====> Scanning ./dph/smvm/dph-smvm.T
====> Scanning ./dph/dotp/dph-dotp.T
====> Scanning ./dph/modules/dph-modules.T
====> Scanning ./dph/primespj/dph-primespj.T
====> Scanning ./programs/record_upd/test.T
====> Scanning ./programs/jq_readsPrec/test.T
====> Scanning ./programs/andre_monad/test.T
====> Scanning ./programs/launchbury/test.T
====> Scanning ./programs/barton-mangler-bug/test.T
====> Scanning ./programs/fast2haskell/test.T
====> Scanning ./programs/jules_xref/test.T
====> Scanning ./programs/seward-space-leak/test.T
====> Scanning ./programs/jules_xref2/test.T
====> Scanning ./programs/thurston-modular-arith/test.T
====> Scanning ./programs/joao-circular/test.T
====> Scanning ./programs/fun_insts/test.T
====> Scanning ./programs/hs-boot/all.T
====> Scanning ./programs/life_space_leak/test.T
====> Scanning ./programs/sanders_array/test.T
====> Scanning ./programs/okeefe_neural/test.T
====> Scanning ./programs/Queens/test.T
====> Scanning ./programs/lennart_range/test.T
====> Scanning ./programs/jtod_circint/test.T
====> Scanning ./programs/10queens/test.T
====> Scanning ./programs/jl_defaults/test.T
====> Scanning ./programs/maessen-hashtab/test.T
====> Scanning ./programs/strict_anns/test.T
====> Scanning ./programs/lex/test.T
====> Scanning ./programs/cvh_unboxing/test.T
====> Scanning ./programs/galois_raytrace/test.T
====> Scanning ./programs/north_array/test.T
====> Scanning ./programs/andy_cherry/test.T
====> Scanning ./programs/rittri/test.T
====> Scanning ./programs/cholewo-eval/test.T
====> Scanning ./parser/unicode/all.T
====> Scanning ./parser/prog001/test.T
====> Scanning ./parser/should_run/all.T
====> Scanning ./parser/should_compile/all.T
====> Scanning ./parser/should_compile/T7476/all.T
====> Scanning ./parser/should_fail/all.T
====> Scanning ./typecheck/prog002/test.T
====> Scanning ./typecheck/testeq1/test.T
====> Scanning ./typecheck/prog001/test.T
====> Scanning ./typecheck/bug1465/all.T
====> Scanning ./typecheck/should_run/all.T
====> Scanning ./typecheck/should_compile/all.T
====> Scanning ./typecheck/should_fail/all.T
====> Scanning ./cpranal/should_compile/all.T
====> Scanning ./overloadedlists/should_run/all.T
====> Scanning ./overloadedlists/should_fail/all.T
====> Scanning ./ffi/should_run/all.T
====> Scanning ./ffi/should_compile/all.T
====> Scanning ./ffi/should_fail/all.T
====> Scanning ./cabal/all.T
====> Scanning ./cabal/cabal01/all.T
====> Scanning ./cabal/cabal03/all.T
====> Scanning ./cabal/cabal04/all.T
====> Scanning ./cabal/pkg02/all.T
====> Scanning ./indexed-types/should_run/all.T
====> Scanning ./indexed-types/should_compile/all.T
====> Scanning ./indexed-types/should_fail/all.T
====> Scanning ./rebindable/all.T
====> Scanning ./rename/prog002/test.T
====> Scanning ./rename/prog003/test.T
====> Scanning ./rename/prog004/test.T
====> Scanning ./rename/prog001/test.T
====> Scanning ./rename/prog006/all.T
====> Scanning ./rename/should_compile/all.T
====> Scanning ./rename/should_compile/T3103/test.T
====> Scanning ./rename/should_fail/all.T
====> Scanning ./rename/prog005/test.T
====> Scanning ./rts/all.T
====> Scanning ./rts/T5644/all.T
====> Scanning ./simplCore/prog002/test.T
====> Scanning ./simplCore/prog001/test.T
====> Scanning ./simplCore/should_run/all.T
====> Scanning ./simplCore/should_compile/all.T
====> Scanning ./ghc-e/should_run/all.T
====> Scanning ./module/base01/all.T
====> Scanning ./module/all.T
====> Scanning ./module/mod175/all.T
====> Scanning ./hsc2hs/all.T
====> Scanning ./quasiquotation/qq006/test.T
====> Scanning ./quasiquotation/qq007/test.T
====> Scanning ./quasiquotation/qq008/test.T
====> Scanning ./quasiquotation/all.T
====> Scanning ./quasiquotation/qq005/test.T
====> Scanning ./quasiquotation/qq002/test.T
====> Scanning ./quasiquotation/qq001/test.T
====> Scanning ./quasiquotation/T4491/test.T
====> Scanning ./quasiquotation/qq004/test.T
====> Scanning ./quasiquotation/qq003/test.T
====> Scanning ./gadt/all.T
====> Scanning ./ghci/prog002/prog002.T
====> Scanning ./ghci/prog003/prog003.T
====> Scanning ./ghci/prog007/prog007.T
====> Scanning ./ghci/prog009/ghci.prog009.T
====> Scanning ./ghci/prog008/prog008.T
====> Scanning ./ghci/prog004/prog004.T
====> Scanning ./ghci/prog001/prog001.T
====> Scanning ./ghci/prog006/prog006.T
====> Scanning ./ghci/linking/all.T
====> Scanning ./ghci/prog011/prog011.T
====> Scanning ./ghci/should_run/all.T
====> Scanning ./ghci/prog012/all.T
====> Scanning ./ghci/prog005/prog005.T
====> Scanning ./ghci/scripts/all.T
====> Scanning ./deSugar/should_run/all.T
====> Scanning ./deSugar/should_compile/all.T
====> Scanning ./array/should_run/all.T
====> Scanning ./runghc/all.T
====> Scanning ./esc/all.T
====> Scanning ./haddock/should_fail_flag_haddock/all.T
====> Scanning ./haddock/should_compile_noflag_nohaddock/all.T
====> Scanning ./haddock/should_compile_flag_haddock/all.T
====> Scanning ./haddock/should_compile_flag_nohaddock/all.T
====> Scanning ./haddock/haddock_examples/test.T
====> Scanning ./haddock/should_compile_noflag_haddock/all.T
====> Scanning ./stranal/should_run/all.T
====> Scanning ./stranal/should_compile/all.T
====> Scanning ./driver/T3007/all.T
====> Scanning ./driver/dynamic_flags_001/all.T
====> Scanning ./driver/bug1677/all.T
====> Scanning ./driver/recomp003/all.T
====> Scanning ./driver/recomp008/all.T
====> Scanning ./driver/all.T
====> Scanning ./driver/T7373/all.T
====> Scanning ./driver/recomp001/all.T
====> Scanning ./driver/recomp005/all.T
====> Scanning ./driver/recomp009/all.T
====> Scanning ./driver/recomp011/all.T
====> Scanning ./driver/T1959/test.T
====> Scanning ./driver/conflicting_flags/test.T
====> Scanning ./driver/T437/all.T
====> Scanning ./driver/recomp012/all.T
====> Scanning ./driver/recomp006/all.T
====> Scanning ./driver/T1372/all.T
====> Scanning ./driver/dynamic_flags_002/all.T
====> Scanning ./driver/dynamicToo/all.T
====> Scanning ./driver/dynamicToo/dynamicToo002/test.T
====> Scanning ./driver/dynamicToo/dynamicToo004/test.T
====> Scanning ./driver/dynamicToo/dynamicToo001/test.T
====> Scanning ./driver/objc/all.T
====> Scanning ./driver/T5147/all.T
====> Scanning ./driver/recomp010/all.T
====> Scanning ./driver/recomp007/all.T
====> Scanning ./driver/recomp004/all.T
====> Scanning ./driver/recomp002/all.T
====> Scanning ./ext-core/all.T
====> Scanning ../../libraries/array/tests/all.T
====> Scanning ../../libraries/hpc/tests/function2/test.T
====> Scanning ../../libraries/hpc/tests/function/test.T
====> Scanning ../../libraries/hpc/tests/ghc_ghci/test.T
====> Scanning ../../libraries/hpc/tests/raytrace/tixs/test.T
====> Scanning ../../libraries/hpc/tests/raytrace/test.T
====> Scanning ../../libraries/hpc/tests/simple/tixs/test.T
====> Scanning ../../libraries/hpc/tests/simple/test.T
====> Scanning ../../libraries/hpc/tests/fork/test.T
====> Scanning ../../libraries/random/tests/all.T
====> Scanning ../../libraries/old-time/tests/all.T
====> Scanning ../../libraries/pretty/tests/all.T
====> Scanning ../../libraries/process/tests/all.T
====> Scanning ../../libraries/directory/tests/all.T
====> Scanning ../../libraries/filepath/tests/all.T
====> Scanning ../../libraries/stm/tests/all.T
====> Scanning ../../libraries/unix/tests/libposix/all.T
====> Scanning ../../libraries/unix/tests/all.T
====> Scanning ../../libraries/template-haskell/tests/all.T
====> Scanning ../../libraries/parallel/tests/all.T
====> Scanning ../../libraries/base/tests/all.T
====> Scanning ../../libraries/base/tests/Numeric/all.T
====> Scanning ../../libraries/base/tests/Concurrent/all.T
====> Scanning ../../libraries/base/tests/Text.Printf/all.T
====> Scanning ../../libraries/base/tests/IO/all.T
====> Scanning ../../libraries/base/tests/System/all.T
=====> T1372(normal) 3378 of 3699 [0, 0, 0]
cd ./driver/T1372 && $MAKE -s --no-print-directory T1372    </dev/null >T1372.run.stdout
2>T1372.run.stderr

OVERALL SUMMARY for test run started at pon, 24 cze 2013, 15:50:17 CEST
    3699 total tests, which gave rise to
   14706 test cases, of which
   14705 were skipped

       0 had missing libraries
       1 expected passes
       0 expected failures

       0 caused framework failures
       0 unexpected passes
       0 unexpected failures

make[1]: Opuszczenie katalogu `/dane/uczelnia/projekty/ghc-validate/testsuite/tests'


Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Geoffrey Mainland
I thought it was obvious that I only meant that it was reliable for me :)

Regardless, it seems pretty clear that the issue is file system
timestamp granularity. What file system are you running? ext4?

Is there any method by which you can consistently reproduce the recache
error on your system?

Geoff

On 06/24/2013 02:55 PM, Jan Stolarek wrote:

>> One recipe for reliable failure:
>>
>> cd testsuite && make TEST=T1372 BINDIST=YES
> Not reliable I'm afraid - it passes here. Unless I do something wrong.
>
> Jan
>
>
> [killy at xerxes : /dane/uczelnia/projekty/ghc-validate] cd testsuite && make TEST=T1372 BINDIST=YES
> make -C ./tests all
> make[1]: Wej?cie do katalogu `/dane/uczelnia/projekty/ghc-validate/testsuite/tests'
> python2 ../driver/runtests.py  -e
> ghc_compiler_always_flags="'-fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts '" -e
> ghc_debugged=False -e ghc_with_native_codegen=1 -e ghc_with_vanilla=1 -e ghc_with_dynamic=1 -e
> ghc_with_profiling=0 -e ghc_with_threaded_rts=1 -e ghc_with_dynamic_rts=1 -e
> ghc_with_interpreter=1 -e ghc_unregisterised=0 -e ghc_dynamic_by_default=False -e
> ghc_dynamic=True -e ghc_with_smp=1 -e ghc_with_llvm=1 -e windows=False -e darwin=False -e
> in_tree_compiler=True -e
> clean_only=False --rootdir=. --config=../config/ghc -e 'config.confdir="../config"' -e 'config.compiler="/dane/uczelnia/projekty/ghc-validate/bindisttest/install  
> dir/bin/ghc"' -e 'config.ghc_pkg="/dane/uczelnia/projekty/ghc-validate/bindisttest/install  
> dir/bin/ghc-pkg"' -e 'config.hp2ps="/dane/uczelnia/projekty/ghc-validate/bindisttest/install  
> dir/bin/hp2ps"' -e 'config.hpc="/dane/uczelnia/projekty/ghc-validate/bindisttest/install  
> dir/bin/hpc"' -e 'config.gs="gs"' -e 'config.platform="x86_64-unknown-linux"' -e 'config.os="linux"' -e 'config.arch="x86_64"' -e 'config.wordsize="64"' -e 'default_testopts.cleanup=""' -e 'config.timeout=int()
> or
> config.timeout' -e 'config.timeout_prog="../timeout/install-inplace/bin/timeout"' -e 'config.exeext=""' -e 'config.top="/dane/uczelnia/projekty/ghc-validate/testsuite"'   --rootdir=../../libraries/ghc-prim/tests  --rootdir=../../libraries/array/tests  --rootdir=../../libraries/containers/tests  --rootdir=../../libraries/hpc/tests  --rootdir=../../libraries/binary/tests  --rootdir=../../libraries/haskeline/tests  --rootdir=../../libraries/vector/tests  --rootdir=../../libraries/bytestring/tests  --rootdir=../../libraries/random/tests  --rootdir=../../libraries/old-time/tests  --rootdir=../../libraries/pretty/tests  --rootdir=../../libraries/process/tests  --rootdir=../../libraries/directory/tests  --rootdir=../../libraries/filepath/tests  --rootdir=../../libraries/stm/tests  --rootdir=../../libraries/unix/tests  --rootdir=../../libraries/template-haskell/tests  --rootdir=../../libraries/parallel/tests  --rootdir=../../libraries/base/tests
> \
>          --only=T1372 \
>          \
>          \
>          \
>          \
>          \
>
> Timeout is 300
> Found 243 .T files...
> Beginning test run at pon, 24 cze 2013, 15:50:17 CEST
> ====> Scanning ./cps/all.T
> ====> Scanning ./layout/all.T
> ====> Scanning ./annotations/should_run/all.T
> ====> Scanning ./annotations/should_compile/all.T
> ====> Scanning ./annotations/should_fail/all.T
> ====> Scanning ./lib/integer/all.T
> ====> Scanning ./numeric/should_run/all.T
> ====> Scanning ./th/TH_import_loop/TH_import_loop.T
> ====> Scanning ./th/TH_spliceViewPat/test.T
> ====> Scanning ./th/all.T
> ====> Scanning ./th/T2014/all.T
> ====> Scanning ./arrows/should_run/all.T
> ====> Scanning ./arrows/should_compile/all.T
> ====> Scanning ./arrows/should_fail/all.T
> ====> Scanning ./polykinds/all.T
> ====> Scanning ./ghci.debugger/scripts/all.T
> ====> Scanning ./ghci.debugger/scripts/break023/all.T
> ====> Scanning ./ghci.debugger/scripts/break022/all.T
> ====> Scanning ./profiling/should_run/all.T
> ====> Scanning ./profiling/should_compile/all.T
> ====> Scanning ./profiling/should_fail/all.T
> ====> Scanning ./codeGen/should_gen_asm/all.T
> ====> Scanning ./codeGen/should_run/all.T
> ====> Scanning ./codeGen/should_compile/all.T
> ====> Scanning ./deriving/should_run/all.T
> ====> Scanning ./deriving/should_compile/all.T
> ====> Scanning ./deriving/should_fail/all.T
> ====> Scanning ./boxy/all.T
> ====> Scanning ./dynlibs/all.T
> ====> Scanning ./plugins/all.T
> ====> Scanning ./perf/compiler/all.T
> ====> Scanning ./perf/space_leaks/all.T
> ====> Scanning ./perf/should_run/all.T
> ====> Scanning ./perf/haddock/all.T
> ====> Scanning ./mdo/should_run/all.T
> ====> Scanning ./mdo/should_compile/all.T
> ====> Scanning ./mdo/should_fail/all.T
> ====> Scanning ./generics/GFunctor/test.T
> ====> Scanning ./generics/all.T
> ====> Scanning ./generics/Uniplate/test.T
> ====> Scanning ./generics/GShow/test.T
> ====> Scanning ./generics/GEq/test.T
> ====> Scanning ./generics/GMap/test.T
> ====> Scanning ./ghc-api/T4891/all.T
> ====> Scanning ./ghc-api/all.T
> ====> Scanning ./ghc-api/T7478/all.T
> ====> Scanning ./ghc-api/dynCompileExpr/all.T
> ====> Scanning ./ghc-api/apirecomp001/all.T
> ====> Scanning ./concurrent/prog002/all.T
> ====> Scanning ./concurrent/prog003/all.T
> ====> Scanning ./concurrent/T2317/all.T
> ====> Scanning ./concurrent/prog001/all.T
> ====> Scanning ./concurrent/should_run/all.T
> ====> Scanning ./safeHaskell/check/pkg01/all.T
> ====> Scanning ./safeHaskell/check/all.T
> ====> Scanning ./safeHaskell/unsafeLibs/all.T
> ====> Scanning ./safeHaskell/flags/all.T
> ====> Scanning ./safeHaskell/ghci/all.T
> ====> Scanning ./safeHaskell/safeInfered/all.T
> ====> Scanning ./safeHaskell/safeLanguage/all.T
> ====> Scanning ./llvm/should_compile/all.T
> ====> Scanning ./dph/enumfromto/dph-enumfromto.T
> ====> Scanning ./dph/sumnats/dph-sumnats.T
> ====> Scanning ./dph/nbody/dph-nbody.T
> ====> Scanning ./dph/quickhull/dph-quickhull.T
> ====> Scanning ./dph/diophantine/dph-diophantine.T
> ====> Scanning ./dph/classes/dph-classes.T
> ====> Scanning ./dph/words/dph-words.T
> ====> Scanning ./dph/smvm/dph-smvm.T
> ====> Scanning ./dph/dotp/dph-dotp.T
> ====> Scanning ./dph/modules/dph-modules.T
> ====> Scanning ./dph/primespj/dph-primespj.T
> ====> Scanning ./programs/record_upd/test.T
> ====> Scanning ./programs/jq_readsPrec/test.T
> ====> Scanning ./programs/andre_monad/test.T
> ====> Scanning ./programs/launchbury/test.T
> ====> Scanning ./programs/barton-mangler-bug/test.T
> ====> Scanning ./programs/fast2haskell/test.T
> ====> Scanning ./programs/jules_xref/test.T
> ====> Scanning ./programs/seward-space-leak/test.T
> ====> Scanning ./programs/jules_xref2/test.T
> ====> Scanning ./programs/thurston-modular-arith/test.T
> ====> Scanning ./programs/joao-circular/test.T
> ====> Scanning ./programs/fun_insts/test.T
> ====> Scanning ./programs/hs-boot/all.T
> ====> Scanning ./programs/life_space_leak/test.T
> ====> Scanning ./programs/sanders_array/test.T
> ====> Scanning ./programs/okeefe_neural/test.T
> ====> Scanning ./programs/Queens/test.T
> ====> Scanning ./programs/lennart_range/test.T
> ====> Scanning ./programs/jtod_circint/test.T
> ====> Scanning ./programs/10queens/test.T
> ====> Scanning ./programs/jl_defaults/test.T
> ====> Scanning ./programs/maessen-hashtab/test.T
> ====> Scanning ./programs/strict_anns/test.T
> ====> Scanning ./programs/lex/test.T
> ====> Scanning ./programs/cvh_unboxing/test.T
> ====> Scanning ./programs/galois_raytrace/test.T
> ====> Scanning ./programs/north_array/test.T
> ====> Scanning ./programs/andy_cherry/test.T
> ====> Scanning ./programs/rittri/test.T
> ====> Scanning ./programs/cholewo-eval/test.T
> ====> Scanning ./parser/unicode/all.T
> ====> Scanning ./parser/prog001/test.T
> ====> Scanning ./parser/should_run/all.T
> ====> Scanning ./parser/should_compile/all.T
> ====> Scanning ./parser/should_compile/T7476/all.T
> ====> Scanning ./parser/should_fail/all.T
> ====> Scanning ./typecheck/prog002/test.T
> ====> Scanning ./typecheck/testeq1/test.T
> ====> Scanning ./typecheck/prog001/test.T
> ====> Scanning ./typecheck/bug1465/all.T
> ====> Scanning ./typecheck/should_run/all.T
> ====> Scanning ./typecheck/should_compile/all.T
> ====> Scanning ./typecheck/should_fail/all.T
> ====> Scanning ./cpranal/should_compile/all.T
> ====> Scanning ./overloadedlists/should_run/all.T
> ====> Scanning ./overloadedlists/should_fail/all.T
> ====> Scanning ./ffi/should_run/all.T
> ====> Scanning ./ffi/should_compile/all.T
> ====> Scanning ./ffi/should_fail/all.T
> ====> Scanning ./cabal/all.T
> ====> Scanning ./cabal/cabal01/all.T
> ====> Scanning ./cabal/cabal03/all.T
> ====> Scanning ./cabal/cabal04/all.T
> ====> Scanning ./cabal/pkg02/all.T
> ====> Scanning ./indexed-types/should_run/all.T
> ====> Scanning ./indexed-types/should_compile/all.T
> ====> Scanning ./indexed-types/should_fail/all.T
> ====> Scanning ./rebindable/all.T
> ====> Scanning ./rename/prog002/test.T
> ====> Scanning ./rename/prog003/test.T
> ====> Scanning ./rename/prog004/test.T
> ====> Scanning ./rename/prog001/test.T
> ====> Scanning ./rename/prog006/all.T
> ====> Scanning ./rename/should_compile/all.T
> ====> Scanning ./rename/should_compile/T3103/test.T
> ====> Scanning ./rename/should_fail/all.T
> ====> Scanning ./rename/prog005/test.T
> ====> Scanning ./rts/all.T
> ====> Scanning ./rts/T5644/all.T
> ====> Scanning ./simplCore/prog002/test.T
> ====> Scanning ./simplCore/prog001/test.T
> ====> Scanning ./simplCore/should_run/all.T
> ====> Scanning ./simplCore/should_compile/all.T
> ====> Scanning ./ghc-e/should_run/all.T
> ====> Scanning ./module/base01/all.T
> ====> Scanning ./module/all.T
> ====> Scanning ./module/mod175/all.T
> ====> Scanning ./hsc2hs/all.T
> ====> Scanning ./quasiquotation/qq006/test.T
> ====> Scanning ./quasiquotation/qq007/test.T
> ====> Scanning ./quasiquotation/qq008/test.T
> ====> Scanning ./quasiquotation/all.T
> ====> Scanning ./quasiquotation/qq005/test.T
> ====> Scanning ./quasiquotation/qq002/test.T
> ====> Scanning ./quasiquotation/qq001/test.T
> ====> Scanning ./quasiquotation/T4491/test.T
> ====> Scanning ./quasiquotation/qq004/test.T
> ====> Scanning ./quasiquotation/qq003/test.T
> ====> Scanning ./gadt/all.T
> ====> Scanning ./ghci/prog002/prog002.T
> ====> Scanning ./ghci/prog003/prog003.T
> ====> Scanning ./ghci/prog007/prog007.T
> ====> Scanning ./ghci/prog009/ghci.prog009.T
> ====> Scanning ./ghci/prog008/prog008.T
> ====> Scanning ./ghci/prog004/prog004.T
> ====> Scanning ./ghci/prog001/prog001.T
> ====> Scanning ./ghci/prog006/prog006.T
> ====> Scanning ./ghci/linking/all.T
> ====> Scanning ./ghci/prog011/prog011.T
> ====> Scanning ./ghci/should_run/all.T
> ====> Scanning ./ghci/prog012/all.T
> ====> Scanning ./ghci/prog005/prog005.T
> ====> Scanning ./ghci/scripts/all.T
> ====> Scanning ./deSugar/should_run/all.T
> ====> Scanning ./deSugar/should_compile/all.T
> ====> Scanning ./array/should_run/all.T
> ====> Scanning ./runghc/all.T
> ====> Scanning ./esc/all.T
> ====> Scanning ./haddock/should_fail_flag_haddock/all.T
> ====> Scanning ./haddock/should_compile_noflag_nohaddock/all.T
> ====> Scanning ./haddock/should_compile_flag_haddock/all.T
> ====> Scanning ./haddock/should_compile_flag_nohaddock/all.T
> ====> Scanning ./haddock/haddock_examples/test.T
> ====> Scanning ./haddock/should_compile_noflag_haddock/all.T
> ====> Scanning ./stranal/should_run/all.T
> ====> Scanning ./stranal/should_compile/all.T
> ====> Scanning ./driver/T3007/all.T
> ====> Scanning ./driver/dynamic_flags_001/all.T
> ====> Scanning ./driver/bug1677/all.T
> ====> Scanning ./driver/recomp003/all.T
> ====> Scanning ./driver/recomp008/all.T
> ====> Scanning ./driver/all.T
> ====> Scanning ./driver/T7373/all.T
> ====> Scanning ./driver/recomp001/all.T
> ====> Scanning ./driver/recomp005/all.T
> ====> Scanning ./driver/recomp009/all.T
> ====> Scanning ./driver/recomp011/all.T
> ====> Scanning ./driver/T1959/test.T
> ====> Scanning ./driver/conflicting_flags/test.T
> ====> Scanning ./driver/T437/all.T
> ====> Scanning ./driver/recomp012/all.T
> ====> Scanning ./driver/recomp006/all.T
> ====> Scanning ./driver/T1372/all.T
> ====> Scanning ./driver/dynamic_flags_002/all.T
> ====> Scanning ./driver/dynamicToo/all.T
> ====> Scanning ./driver/dynamicToo/dynamicToo002/test.T
> ====> Scanning ./driver/dynamicToo/dynamicToo004/test.T
> ====> Scanning ./driver/dynamicToo/dynamicToo001/test.T
> ====> Scanning ./driver/objc/all.T
> ====> Scanning ./driver/T5147/all.T
> ====> Scanning ./driver/recomp010/all.T
> ====> Scanning ./driver/recomp007/all.T
> ====> Scanning ./driver/recomp004/all.T
> ====> Scanning ./driver/recomp002/all.T
> ====> Scanning ./ext-core/all.T
> ====> Scanning ../../libraries/array/tests/all.T
> ====> Scanning ../../libraries/hpc/tests/function2/test.T
> ====> Scanning ../../libraries/hpc/tests/function/test.T
> ====> Scanning ../../libraries/hpc/tests/ghc_ghci/test.T
> ====> Scanning ../../libraries/hpc/tests/raytrace/tixs/test.T
> ====> Scanning ../../libraries/hpc/tests/raytrace/test.T
> ====> Scanning ../../libraries/hpc/tests/simple/tixs/test.T
> ====> Scanning ../../libraries/hpc/tests/simple/test.T
> ====> Scanning ../../libraries/hpc/tests/fork/test.T
> ====> Scanning ../../libraries/random/tests/all.T
> ====> Scanning ../../libraries/old-time/tests/all.T
> ====> Scanning ../../libraries/pretty/tests/all.T
> ====> Scanning ../../libraries/process/tests/all.T
> ====> Scanning ../../libraries/directory/tests/all.T
> ====> Scanning ../../libraries/filepath/tests/all.T
> ====> Scanning ../../libraries/stm/tests/all.T
> ====> Scanning ../../libraries/unix/tests/libposix/all.T
> ====> Scanning ../../libraries/unix/tests/all.T
> ====> Scanning ../../libraries/template-haskell/tests/all.T
> ====> Scanning ../../libraries/parallel/tests/all.T
> ====> Scanning ../../libraries/base/tests/all.T
> ====> Scanning ../../libraries/base/tests/Numeric/all.T
> ====> Scanning ../../libraries/base/tests/Concurrent/all.T
> ====> Scanning ../../libraries/base/tests/Text.Printf/all.T
> ====> Scanning ../../libraries/base/tests/IO/all.T
> ====> Scanning ../../libraries/base/tests/System/all.T
> =====> T1372(normal) 3378 of 3699 [0, 0, 0]
> cd ./driver/T1372 && $MAKE -s --no-print-directory T1372    </dev/null >T1372.run.stdout
> 2>T1372.run.stderr
>
> OVERALL SUMMARY for test run started at pon, 24 cze 2013, 15:50:17 CEST
>     3699 total tests, which gave rise to
>    14706 test cases, of which
>    14705 were skipped
>
>        0 had missing libraries
>        1 expected passes
>        0 expected failures
>
>        0 caused framework failures
>        0 unexpected passes
>        0 unexpected failures
>
> make[1]: Opuszczenie katalogu `/dane/uczelnia/projekty/ghc-validate/testsuite/tests'




Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Jan Stolarek
> I thought it was obvious that I only meant that it was reliable for me :)
"Reliable for me" is a bit of oxymoron in the context of validating correctness of software :) I
guess that's why I titled this thread "Unreliability of the build system".

I'm running ext4.

> Is there any method by which you can consistently reproduce the recache
> error on your system?
Not that I'm aware of.

Jan


Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Geoffrey Mainland
On 06/24/2013 03:47 PM, Jan Stolarek wrote:
>> I thought it was obvious that I only meant that it was reliable for me :)
> "Reliable for me" is a bit of oxymoron in the context of validating
correctness of software :) I
> guess that's why I titled this thread "Unreliability of the build system".

My only claim was that I have a set of steps that can reliably reproduce
an error on my system, not that these steps will reliably reproduce it
on an arbitrary system, e.g., yours. Still, being able to reliably
reproduce an error, even if only on one system, is *very* useful.

> I'm running ext4.
>
>> Is there any method by which you can consistently reproduce the recache
>> error on your system?
> Not that I'm aware of.

So, to be clear, you see recache errors sporadically and for different
sets of tests across different runs, and you are running validate with
no extra arguments, single-threaded, on a Linux x86_64 system with an
ext4 file system?

Geoff



Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Jan Stolarek
> My only claim was that I have a set of steps that can reliably reproduce
> an error on my system, not that these steps will reliably reproduce it
> on an arbitrary system, e.g., yours. Still, being able to reliably
> reproduce an error, even if only on one system, is *very* useful.
 just misunderstood your use of the word "reliable". Certainly having this level of
reproducibility is very helpful in debugging, even if it's only on one machine.

> So, to be clear, you see recache errors sporadically and for different
> sets of tests across different runs, and you are running validate with
> no extra arguments, single-threaded, on a Linux x86_64 system with an
> ext4 file system?
Yes, except I would say "very often" instead of "sporadically". I'm not sure about
single-threaded. I run validate without any parameters - doesn't it use more then one thread in
this case?

Jan


Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Geoffrey Mainland
On 06/24/2013 09:21 PM, Jan Stolarek wrote:
>> My only claim was that I have a set of steps that can reliably reproduce
>> an error on my system, not that these steps will reliably reproduce it
>> on an arbitrary system, e.g., yours. Still, being able to reliably
>> reproduce an error, even if only on one system, is *very* useful.
>  just misunderstood your use of the word "reliable". Certainly having
this level of
> reproducibility is very helpful in debugging, even if it's only on one
machine.
>
>> So, to be clear, you see recache errors sporadically and for different
>> sets of tests across different runs, and you are running validate with
>> no extra arguments, single-threaded, on a Linux x86_64 system with an
>> ext4 file system?
> Yes, except I would say "very often" instead of "sporadically". I'm
not sure about
> single-threaded. I run validate without any parameters - doesn't it
use more then one thread in
> this case?

It looks like running validate without any parameters will use 2
threads.

I've pushed a fix (I hope) for this issue. Could you check and see if
it's fixed for you now?

Geoff




Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Jan Stolarek
I just ran validation (once without any parameters and once with --normal) and got no failures,
except for some tests that always fail. Looks like the problem may be solved.

Janek

Dnia wtorek, 25 czerwca 2013, Geoffrey Mainland napisa?:

> On 06/24/2013 09:21 PM, Jan Stolarek wrote:
> >> My only claim was that I have a set of steps that can reliably reproduce
> >> an error on my system, not that these steps will reliably reproduce it
> >> on an arbitrary system, e.g., yours. Still, being able to reliably
> >> reproduce an error, even if only on one system, is *very* useful.
> >
> >  just misunderstood your use of the word "reliable". Certainly having
>
> this level of
>
> > reproducibility is very helpful in debugging, even if it's only on one
>
> machine.
>
> >> So, to be clear, you see recache errors sporadically and for different
> >> sets of tests across different runs, and you are running validate with
> >> no extra arguments, single-threaded, on a Linux x86_64 system with an
> >> ext4 file system?
> >
> > Yes, except I would say "very often" instead of "sporadically". I'm
>
> not sure about
>
> > single-threaded. I run validate without any parameters - doesn't it
>
> use more then one thread in
>
> > this case?
>
> It looks like running validate without any parameters will use 2
> threads.
>
> I've pushed a fix (I hope) for this issue. Could you check and see if
> it's fixed for you now?
>
> Geoff




Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Mateusz Kowalczyk
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 25/06/13 15:43, Jan Stolarek wrote:

> I just ran validation (once without any parameters and once with
> --normal) and got no failures, except for some tests that always
> fail. Looks like the problem may be solved.
>
> Janek
>
> Dnia wtorek, 25 czerwca 2013, Geoffrey Mainland napisa?:
>> On 06/24/2013 09:21 PM, Jan Stolarek wrote:
>>>> My only claim was that I have a set of steps that can
>>>> reliably reproduce an error on my system, not that these
>>>> steps will reliably reproduce it on an arbitrary system,
>>>> e.g., yours. Still, being able to reliably reproduce an
>>>> error, even if only on one system, is *very* useful.
>>>
>>> just misunderstood your use of the word "reliable". Certainly
>>> having
>>
>> this level of
>>
>>> reproducibility is very helpful in debugging, even if it's only
>>> on one
>>
>> machine.
>>
>>>> So, to be clear, you see recache errors sporadically and for
>>>> different sets of tests across different runs, and you are
>>>> running validate with no extra arguments, single-threaded, on
>>>> a Linux x86_64 system with an ext4 file system?
>>>
>>> Yes, except I would say "very often" instead of "sporadically".
>>> I'm
>>
>> not sure about
>>
>>> single-threaded. I run validate without any parameters -
>>> doesn't it
>>
>> use more then one thread in
>>
>>> this case?
>>
>> It looks like running validate without any parameters will use 2
>> threads.
>>
>> I've pushed a fix (I hope) for this issue. Could you check and
>> see if it's fixed for you now?
>>
>> Geoff
>
>
>
> _______________________________________________ ghc-devs mailing
> list ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
Which tests are these? I ran `./validate' yesterday on a stock (that
is, I haven't made any changes to the source) and had about 5-7 tests
fail and wasn't sure if this was a known issue or something I should
be reporting.

- --
Mateusz K.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iQIcBAEBAgAGBQJRya5EAAoJEM1mucMq2pqXxhQQAKbdvG0n+UzMb7UZEOOlIDhi
+gjIWm6Pz3nJTAHPmK4u/oDJD2iwsso/AMuuIPj641jUIc+2BC3QslODebFRXMEg
Dh0PC7DlqaHsGh+wsjpvksdaVL5oZ4hijXKuKpo3hYOJlMZmXBghethv55tOknzU
3spoL+Tt+iwQaG7zxxAey7Su4cMi4qNtFdTMV8aBjbGgTzrCQWqSmj4AYmwqLWP+
Vuqk2WfhC8KDezhn2wNFdqcyUqlqssVIjykTCHJQaAj3M+G6+Otx0iQi0KraRNYx
r64KW65z1Yx7wzNr2TJoCQCfAy/iLKqgh0xsuWkCB/Nrgi2run5EE/Z1rn0eb24q
HskL7anITbgjupjpKINlx2gkmx5n+0fhQSZixMzoZvTw4G7HPtuswz5er1x0XT3c
21kX2/sxWu8aq4UprkIl79Omn19YH/lq39GW4A+Gh7YbIye71uJPLl916CP3qZlb
8lPFluJlaZ5yRY75CzEB0J2T9F6QcIobWjOxp4kJCF5BaJnOJpPNm3XVhHvuZR3f
iLshECwf+Cb0WEzJhjgsaaWVU2von505JmNHw99Uijz2AHXjjpD7LYZprXJiuqBw
1O+l7TiFRWtiC44OCjipoCZfwPmKnQ4U5TQXn6LQ9C98GMiZAhwpwbu+vs3pImYa
+sCwFqiIpL2tFbxmbX/u
=RPpV
-----END PGP SIGNATURE-----


Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Geoffrey Mainland
In reply to this post by Jan Stolarek
Great, thanks for checking!

On 06/25/2013 03:43 PM, Jan Stolarek wrote:

> I just ran validation (once without any parameters and once with --normal) and got no failures,
> except for some tests that always fail. Looks like the problem may be solved.
>
> Janek
>
> Dnia wtorek, 25 czerwca 2013, Geoffrey Mainland napisa?:
>> On 06/24/2013 09:21 PM, Jan Stolarek wrote:
>>>> My only claim was that I have a set of steps that can reliably reproduce
>>>> an error on my system, not that these steps will reliably reproduce it
>>>> on an arbitrary system, e.g., yours. Still, being able to reliably
>>>> reproduce an error, even if only on one system, is *very* useful.
>>>  just misunderstood your use of the word "reliable". Certainly having
>> this level of
>>
>>> reproducibility is very helpful in debugging, even if it's only on one
>> machine.
>>
>>>> So, to be clear, you see recache errors sporadically and for different
>>>> sets of tests across different runs, and you are running validate with
>>>> no extra arguments, single-threaded, on a Linux x86_64 system with an
>>>> ext4 file system?
>>> Yes, except I would say "very often" instead of "sporadically". I'm
>> not sure about
>>
>>> single-threaded. I run validate without any parameters - doesn't it
>> use more then one thread in
>>
>>> this case?
>> It looks like running validate without any parameters will use 2
>> threads.
>>
>> I've pushed a fix (I hope) for this issue. Could you check and see if
>> it's fixed for you now?
>>
>> Geoff


Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Jan Stolarek
In reply to this post by Mateusz Kowalczyk
T149 has been failing for a very long time (a few weeks at least). I also had process007 fail.

Janek

Dnia wtorek, 25 czerwca 2013, Mateusz Kowalczyk napisa?:

> On 25/06/13 15:43, Jan Stolarek wrote:
> > I just ran validation (once without any parameters and once with
> > --normal) and got no failures, except for some tests that always
> > fail. Looks like the problem may be solved.
> >
> > Janek
> >
> > Dnia wtorek, 25 czerwca 2013, Geoffrey Mainland napisa?:
> >> On 06/24/2013 09:21 PM, Jan Stolarek wrote:
> >>>> My only claim was that I have a set of steps that can
> >>>> reliably reproduce an error on my system, not that these
> >>>> steps will reliably reproduce it on an arbitrary system,
> >>>> e.g., yours. Still, being able to reliably reproduce an
> >>>> error, even if only on one system, is *very* useful.
> >>>
> >>> just misunderstood your use of the word "reliable". Certainly
> >>> having
> >>
> >> this level of
> >>
> >>> reproducibility is very helpful in debugging, even if it's only
> >>> on one
> >>
> >> machine.
> >>
> >>>> So, to be clear, you see recache errors sporadically and for
> >>>> different sets of tests across different runs, and you are
> >>>> running validate with no extra arguments, single-threaded, on
> >>>> a Linux x86_64 system with an ext4 file system?
> >>>
> >>> Yes, except I would say "very often" instead of "sporadically".
> >>> I'm
> >>
> >> not sure about
> >>
> >>> single-threaded. I run validate without any parameters -
> >>> doesn't it
> >>
> >> use more then one thread in
> >>
> >>> this case?
> >>
> >> It looks like running validate without any parameters will use 2
> >> threads.
> >>
> >> I've pushed a fix (I hope) for this issue. Could you check and
> >> see if it's fixed for you now?
> >>
> >> Geoff
> >
> > _______________________________________________ ghc-devs mailing
> > list ghc-devs at haskell.org
> > http://www.haskell.org/mailman/listinfo/ghc-devs
>
> Which tests are these? I ran `./validate' yesterday on a stock (that
> is, I haven't made any changes to the source) and had about 5-7 tests
> fail and wasn't sure if this was a known issue or something I should
> be reporting.




Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Mateusz Kowalczyk
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 25/06/13 21:06, Jan Stolarek wrote:

> T149 has been failing for a very long time (a few weeks at least).
> I also had process007 fail.
>
> Janek
>
> Dnia wtorek, 25 czerwca 2013, Mateusz Kowalczyk napisa?:
>> On 25/06/13 15:43, Jan Stolarek wrote:
>>> I just ran validation (once without any parameters and once
>>> with --normal) and got no failures, except for some tests that
>>> always fail. Looks like the problem may be solved.
>>>
>>> Janek
>>>
>> Which tests are these? I ran `./validate' yesterday on a stock
>> (that is, I haven't made any changes to the source) and had about
>> 5-7 tests fail and wasn't sure if this was a known issue or
>> something I should be reporting.
>
>

Thanks, I seem to get quite a few more failing. Seeing the nature of
this thread, I think it's not totally out of order to post the summary
here, considering I had a tree without any personal changes.
Surprisingly (?), no T149 or process007.

I can post up the full log (1.3M) and/or start a new thread if anyone
from the dev team is interested in taking a look.


OVERALL SUMMARY for test run started at Mon 24 Jun 13:35:28 BST 2013
    3675 total tests, which gave rise to
   14517 test cases, of which
   11178 were skipped

      28 had missing libraries
    3237 expected passes
      58 expected failures

       0 caused framework failures
       1 unexpected passes
      15 unexpected failures

Unexpected passes:
   perf/should_run  T149 (normal)

Unexpected failures:
   ../../libraries/process/tests  process007 [bad stdout] (normal)
   ffi/should_run                 T1288_ghci [bad exit code] (ghci)
   ghci.debugger/scripts          dynbrk009 [bad exit code] (ghci)
   ghci.debugger/scripts          print020 [bad exit code] (ghci)
   ghci/linking                   ghcilink002 [bad exit code] (normal)
   ghci/linking                   ghcilink005 [bad exit code] (normal)
   perf/compiler                  T1969 [stat not good enough] (normal)
   perf/compiler                  T3064 [stat not good enough] (normal)
   perf/compiler                  T3294 [stat not good enough] (normal)
   perf/haddock                   haddock.Cabal [stat not good enough]
(normal)
   perf/haddock                   haddock.base [stat not good enough]
(normal)
   perf/haddock                   haddock.compiler [stat too good]
(normal)
   perf/should_run                T7797 [stat too good] (normal)
   perf/should_run                T7954 [stat not good enough] (normal)
   perf/should_run                lazy-bs-alloc [stat too good] (normal)



- --
Mateusz K.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iQIcBAEBAgAGBQJRyfoMAAoJEM1mucMq2pqXCSMP/ilPBFHAilICXhzmzoiNajik
+3dlcrtnPOkJNR+ES2iqpxeEI4QFq4wjlCLKs1bqmWlViOoRk03PAYeEFpQzHuR/
MUAG63ChgEn8DhzUHk73Bpq7jGYauNqQD8BAiDtn+9PyRXHkZpQjzqDh8C9nlqMa
aMRDbL3S1NLrXcIMqtLSuU4a60BmzXolFYNfY1uJoH+rtxBmr42J0uEVXPwUFrvS
fu+kqe38rEiQ04XIxHkcIhxuOwa9TLFOla6AwI2cBQBx+67NEXuXR7Tx9uB+CPon
b0Z4DGV1fc2+1XUEZ0XfFXrSXDX+HtVtnGhAE9UlXcPZvJbofLh8dc5zuxuytz6P
OXhHVrh02zx1TMMFTKKo5jEjaPFJOwii0VQCYP+nsHaj1KXeQxpQNa6puHVd3aZr
3UVTTOzFVd1sIF/36V4JoG/XfFae/WfyjNPFZso8mfzXsDhZRJ33dJV4qatV3V4b
mbofJIcU9KWdulFdxWNzYZtcJ1yIRPnxIvQTv77j6XLd3ctwMOmg4tU5wc6Dsy41
/X5/9e9/5EBdecmmAqaAZr+FZTtXVPw5FWjUpajnPH6sDbVynfJNB4DMOTgd51EA
SQkUa6UbnGHs+s2MV/sENN3u2m88NJlFd5S1sgkT8C4tvgswXJv1kR4XlGY5DEF8
nLaqesxc5L48WtlfC2dy
=kZRZ
-----END PGP SIGNATURE-----


Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Simon Peyton Jones
In reply to this post by Jan Stolarek
T149 is a CSE test.  I think the CSE is now happening, but it's fragile.    It's fine that it happens, so perhaps we should record it as ok, and look again if it starts failing.

Simon

|  -----Original Message-----
|  From: ghc-devs-bounces at haskell.org [mailto:ghc-devs-bounces at haskell.org] On
|  Behalf Of Jan Stolarek
|  Sent: 25 June 2013 21:06
|  To: ghc-devs at haskell.org
|  Subject: Re: Unreliability of the build system
|  
|  T149 has been failing for a very long time (a few weeks at least). I also had
|  process007 fail.
|  
|  Janek
|  
|  Dnia wtorek, 25 czerwca 2013, Mateusz Kowalczyk napisa?:
|  > On 25/06/13 15:43, Jan Stolarek wrote:
|  > > I just ran validation (once without any parameters and once with
|  > > --normal) and got no failures, except for some tests that always
|  > > fail. Looks like the problem may be solved.
|  > >
|  > > Janek
|  > >
|  > > Dnia wtorek, 25 czerwca 2013, Geoffrey Mainland napisa?:
|  > >> On 06/24/2013 09:21 PM, Jan Stolarek wrote:
|  > >>>> My only claim was that I have a set of steps that can
|  > >>>> reliably reproduce an error on my system, not that these
|  > >>>> steps will reliably reproduce it on an arbitrary system,
|  > >>>> e.g., yours. Still, being able to reliably reproduce an
|  > >>>> error, even if only on one system, is *very* useful.
|  > >>>
|  > >>> just misunderstood your use of the word "reliable". Certainly
|  > >>> having
|  > >>
|  > >> this level of
|  > >>
|  > >>> reproducibility is very helpful in debugging, even if it's only
|  > >>> on one
|  > >>
|  > >> machine.
|  > >>
|  > >>>> So, to be clear, you see recache errors sporadically and for
|  > >>>> different sets of tests across different runs, and you are
|  > >>>> running validate with no extra arguments, single-threaded, on
|  > >>>> a Linux x86_64 system with an ext4 file system?
|  > >>>
|  > >>> Yes, except I would say "very often" instead of "sporadically".
|  > >>> I'm
|  > >>
|  > >> not sure about
|  > >>
|  > >>> single-threaded. I run validate without any parameters -
|  > >>> doesn't it
|  > >>
|  > >> use more then one thread in
|  > >>
|  > >>> this case?
|  > >>
|  > >> It looks like running validate without any parameters will use 2
|  > >> threads.
|  > >>
|  > >> I've pushed a fix (I hope) for this issue. Could you check and
|  > >> see if it's fixed for you now?
|  > >>
|  > >> Geoff
|  > >
|  > > _______________________________________________ ghc-devs mailing
|  > > list ghc-devs at haskell.org
|  > > http://www.haskell.org/mailman/listinfo/ghc-devs
|  >
|  > Which tests are these? I ran `./validate' yesterday on a stock (that
|  > is, I haven't made any changes to the source) and had about 5-7 tests
|  > fail and wasn't sure if this was a known issue or something I should
|  > be reporting.
|  
|  
|  
|  _______________________________________________
|  ghc-devs mailing list
|  ghc-devs at haskell.org
|  http://www.haskell.org/mailman/listinfo/ghc-devs

Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Jan Stolarek
In reply to this post by Mateusz Kowalczyk
> Surprisingly (?), no T149 or process007.
Yes, you do have these tests failing:

> Unexpected passes:
>    perf/should_run  T149 (normal)
>
> Unexpected failures:
>    ../../libraries/process/tests  process007 [bad stdout] (normal)

I learned not to worry about performance tests - they seem to be quite fragile. As for the others
I always verify why they failed by rerunning them separately (cd testsuite/tests && make
TEST=T149).

Janek


Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Simon Marlow-7
On 26/06/13 07:55, Jan Stolarek wrote:

>> Surprisingly (?), no T149 or process007.
> Yes, you do have these tests failing:
>
>> Unexpected passes:
>>     perf/should_run  T149 (normal)
>>
>> Unexpected failures:
>>     ../../libraries/process/tests  process007 [bad stdout] (normal)
>
> I learned not to worry about performance tests - they seem to be quite fragile.

This makes me sad.  You should shout when performance tests fail.

Cheers,
        Simon



Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Simon Marlow-7
In reply to this post by Geoffrey Mainland
Geoff - is it possible that this problem might be caused by time skew
between NFS servers?  Are any parts of the build, or files accessed by
it, on NFS?

Cheers,
        Simon

On 24/06/13 23:44, Geoffrey Mainland wrote:

> On 06/24/2013 09:21 PM, Jan Stolarek wrote:
>>> My only claim was that I have a set of steps that can reliably reproduce
>>> an error on my system, not that these steps will reliably reproduce it
>>> on an arbitrary system, e.g., yours. Still, being able to reliably
>>> reproduce an error, even if only on one system, is *very* useful.
>>   just misunderstood your use of the word "reliable". Certainly having
> this level of
>> reproducibility is very helpful in debugging, even if it's only on one
> machine.
>>
>>> So, to be clear, you see recache errors sporadically and for different
>>> sets of tests across different runs, and you are running validate with
>>> no extra arguments, single-threaded, on a Linux x86_64 system with an
>>> ext4 file system?
>> Yes, except I would say "very often" instead of "sporadically". I'm
> not sure about
>> single-threaded. I run validate without any parameters - doesn't it
> use more then one thread in
>> this case?
>
> It looks like running validate without any parameters will use 2
> threads.
>
> I've pushed a fix (I hope) for this issue. Could you check and see if
> it's fixed for you now?
>
> Geoff
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>



Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Geoffrey Mainland
No, it's all done on a local (ZFS) file system (I don't even have any
NFS volumes mounted), although I *am* using a separate build tree.

Geoff

On 06/28/2013 10:30 AM, Simon Marlow wrote:

> Geoff - is it possible that this problem might be caused by time skew
> between NFS servers?  Are any parts of the build, or files accessed by
> it, on NFS?
>
> Cheers,
>     Simon
>
> On 24/06/13 23:44, Geoffrey Mainland wrote:
>> On 06/24/2013 09:21 PM, Jan Stolarek wrote:
>>>> My only claim was that I have a set of steps that can reliably
>>>> reproduce
>>>> an error on my system, not that these steps will reliably reproduce it
>>>> on an arbitrary system, e.g., yours. Still, being able to reliably
>>>> reproduce an error, even if only on one system, is *very* useful.
>>>   just misunderstood your use of the word "reliable". Certainly having
>> this level of
>>> reproducibility is very helpful in debugging, even if it's only on one
>> machine.
>>>
>>>> So, to be clear, you see recache errors sporadically and for different
>>>> sets of tests across different runs, and you are running validate with
>>>> no extra arguments, single-threaded, on a Linux x86_64 system with an
>>>> ext4 file system?
>>> Yes, except I would say "very often" instead of "sporadically". I'm
>> not sure about
>>> single-threaded. I run validate without any parameters - doesn't it
>> use more then one thread in
>>> this case?
>>
>> It looks like running validate without any parameters will use 2
>> threads.
>>
>> I've pushed a fix (I hope) for this issue. Could you check and see if
>> it's fixed for you now?
>>
>> Geoff
>>
>>
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>
>



Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Jan Stolarek
In reply to this post by Simon Marlow-7
> This makes me sad.  You should shout when performance tests fail.
Even if they sometimes fail and sometimes not?

Janek


Reply | Threaded
Open this post in threaded view
|

Unreliability of the build system

Simon Marlow-7
On 28/06/13 11:08, Jan Stolarek wrote:
>> This makes me sad.  You should shout when performance tests fail.
> Even if they sometimes fail and sometimes not?

Absolutely.  Better still, submit a patch to fix it.

Cheers,
        Simon




12