Observation on Hadrian's relative performance re current buildsystem

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Observation on Hadrian's relative performance re current buildsystem

Herbert Valerio Riedel-3
Hello GHC devs,

I took the opportunity to give Hadrian a test-run to see whether it
could live up to the big promise of delivering a "more scalable, faster"
system than the current GNU Make based system. Unfortunately, my
preliminary results don't back this claim, and actually make Hadrian
appear to be significantly slower.

----

Here's the summary of the results:

                         | Hadrian   | GNU Make  |
+------------------------+-----------+-----------+
| Compiling `hadrian`    | 5m25s     | 0         |
| (one-time setup)       |           |           |
+------------------------+-----------+-----------+
| build "all" at -j8     | 38m       | 33m       |
+------------------------+-----------+-----------+
| no-op build at -j8     | 10.977s   | 3.258s    |
+------------------------+-----------+-----------+
| "clean"                | 21s       | 51s       |
+------------------------+-----------+-----------+


So, Hadrian is ~5 minutes than GNU Make (or even ~10 minutes if
you also count the one-time setup cost).

And what I personally consider a bit annoying is that it's ~3 times
slower detecting; i.e. you have to wait 11s for Hadrian to detect
there's nothing to be done which compared to GNU Make (which currently
needlessly re-runs Sphinx; so it could be even faster!) is very
noticeable to me.

There's a silver-lining though, deleting files is the part which is a
lot more costly in the GNU Make system currently since artifacts are
spread over several (scroll to the end of this email) subfolders
there. Whereas Hadrian did something we should have done for the GNU
Make system as well (and probably would have done sooner or later anyway
in order to support the srcdir!=buildir scheme that people are used to
from GNU Autotools projects); Hadrian places build-artifacts into a few
top-level folders, and so cleaning up is trivial and requires to unlink
only a few folders from the filesystem.

----

At the very least, I'd expect Hadrian to be as fast as the GNU Make
system (and ideally beat it, not the least as this was besides
maintainability its big promise), but so far it doesn't seem to deliver
that promise for me.

It could easily be that I'm comparing apples to oranges here or that
I've otherwise overlooked something, so let me describe how I came to
this conclusion:

I tried this on an reasonably idle Linux workstation with an Intel(R)
Core(TM) i7-3770 CPU @ 3.40GHz CPU, and with 32GiB RAM (i.e. the
filesystem content was well-cached into memory; NB: a ghc source tree +
compiled artifacts takes up about 4GiB on the filesystem).

I've started from a fresh Git clone, i.e.

  git clone --recursive git://git.haskell.org/ghc.git

followed by

  ./boot
  ./configure

At this point, we're at the common point from which both the Hadrian and
the GNU Make build-system would start diverging:

For the Hadrian build-system, we need to pay for a one-time setup, since
we need to build the `hadrian` executable (which requires to build the
in-tree lib:Cabal as an in-place library):


  $ time ./hadrian/build.sh --help
   
  ...
   
  real  5m25.992s
  user  6m19.196s
  sys   0m6.079s


I'm not too worried about this part, as there's a few tricks by which we
could likely bring that down to about 2 minutes or so, and we mostly pay
this setup-cost, when lib:Cabal and/or `hadrian` changes and requires to
be recompiled.


Now, after having made sure that the `hadrian` executable is fresh, I
started the actual build:

    $ time ./hadrian/build.sh -j8
     
    ...
     
    shakeArgsWith                        0.000s    0%                          
    Function shake                       0.178s    0%                          
    Database read                        0.000s    0%                          
    With database                        0.000s    0%                          
    Running rules                     2338.398s   99%  =========================
    Pool finished (5261 threads, 8 max)  0.002s    0%                          
    Lint checking                        0.111s    0%                          
    Total                             2338.690s  100%                          
    Build completed in 38:59m
     
    real 38m59.626s
    user 219m7.421s
    sys  11m7.584s


then I immediately re-issued the same command to test how long it takes
to perform a no-op build:


    $ time ./hadrian/build.sh -j8
     
    Up to date
    Up to date
    shakeArgsWith                        0.000s    0%                          
    Function shake                       0.183s    1%                          
    Database read                        0.144s    1%                          
    With database                        0.241s    2%                          
    Running rules                        9.379s   93%  =========================
    Pool finished (4165 threads, 8 max)  0.004s    0%                          
    Lint checking                        0.101s    1%                          
    Total                               10.051s  100%                          
    Build completed in 0:11m
     
     
    real 0m10.977s
    user 0m19.399s
    sys  0m2.443s


Same without -j8:

    $ time ./hadrian/build.sh
    Up to date
    Up to date
    shakeArgsWith                     0.000s    0%                          
    Function shake                    0.175s    2%                          
    Database read                     0.123s    1%                          
    With database                     0.197s    2%                          
    Running rules                     7.710s   92%  =========================
    Pool finished (1 threads, 1 max)  0.003s    0%                          
    Lint checking                     0.084s    1%                          
    Total                             8.293s  100%                          
    Build completed in 0:09m
     
     
    real 0m9.196s
    user 0m8.656s
    sys 0m0.724s


And finally clean it again:

    $ sync
    $ time ./hadrian/build.sh clean
    Up to date
    Up to date
    | Remove directory _build/stage0
    | Remove directory _build/stage1
    | Remove directory _build/stage2
    | Remove directory _build/stage3
    | Remove directory inplace/bin
    | Remove directory inplace/lib
    | Remove directory sdistprep
    | Remove Hadrian files...
    | Remove directory _build/generated
    | Done.
    shakeArgsWith                     0.000s    0%                          
    Function shake                    0.175s    0%                          
    Database read                     0.114s    0%                          
    With database                     0.205s    0%                          
    Running rules                    20.037s   97%  ========================
    Pool finished (1 threads, 1 max)  0.002s    0%                          
    Lint checking                     0.006s    0%                          
    Total                            20.540s  100%                          
    Build completed in 0:21m
     
     
    real 0m21.426s
    user 0m1.415s
    sys 0m1.045s



----

Running the full build via our rusty GNU Make system:

    $ time make V=0 -j8
    real 33m30.801s
    user 157m49.520s
    sys  6m49.289s

A no-op build:

    $ time make V=0 -j8
    ===--- building phase 0
    make --no-print-directory -f ghc.mk phase=0 phase_0_builds
    make[1]: Nothing to be done for 'phase_0_builds'.
    ===--- building phase 1
    make --no-print-directory -f ghc.mk phase=1 phase_1_builds
    make[1]: Nothing to be done for 'phase_1_builds'.
    ===--- building final phase
    make --no-print-directory -f ghc.mk phase=final all
    make -C utils/haddock/doc html SPHINX_BUILD=/usr/bin/sphinx-build
    /usr/bin/sphinx-build -b html . .build-html
    Running Sphinx v1.3.6
    loading translations [en]... done
    loading pickled environment... done
    building [mo]: targets for 0 po files that are out of date
    building [html]: targets for 0 source files that are out of date
    updating environment: 0 added, 0 changed, 0 removed
    looking for now-outdated files... none found
    no targets are out of date.
    build succeeded.
    cp -R utils/haddock/doc/.build-html utils/haddock/doc/haddock
     
    real 0m3.258s
    user 0m3.134s
    sys 0m0.283s

And finally `make clean`:

    $ sync
    $ time make clean
    make --no-print-directory -f ghc.mk clean CLEANING=YES
    "rm" -rf inplace/bin inplace/lib  
    "rm" -rf utils/touchy/dist  
    "rm" -rf inplace/lib/bin/touchy  
    "rm" -rf utils/unlit/dist  
    "rm" -rf inplace/lib/bin/unlit  
    "rm" -rf utils/unlit/dist-install  
    "rm" -rf utils/hp2ps/dist  
    "rm" -rf inplace/bin/hp2ps  
    "rm" -rf inplace/lib/bin/hp2ps  
    "rm" -rf utils/hp2ps/dist-install  
    "rm" -rf driver/split/dist inplace/lib/bin/ghc-split  
    "rm" -rf utils/genprimopcode/dist  
    "rm" -rf inplace/bin/genprimopcode  
    "rm" -rf  libffi/build libffi/stamp.ffi.static-shared.configure libffi/stamp.ffi.static-shared.build libffi/stamp.ffi.static-shared.install libffi/dist-install  
    "rm" -rf utils/deriveConstants/dist  
    "rm" -rf inplace/bin/deriveConstants  
    "rm" -rf  includes/ghcautoconf.h includes/ghcplatform.h includes/ghcversion.h  
    "rm" -rf rts/dist  
    "rm" -rf bindisttest/"install   dir"  bindisttest/HelloWorld bindisttest/HelloWorld.o bindisttest/HelloWorld.hi bindisttest/output  
    "rm" -rf utils/genapply/dist  
    "rm" -rf inplace/bin/genapply  
    "rm" -rf  libraries/integer-gmp/include/ghc-gmp.h libraries/integer-gmp/gmp/config.mk libraries/integer-gmp/gmp/libgmp.a libraries/integer-gmp/gmp/gmp.h libraries/integer-gmp/gmp/gmpbuild libraries/integer-gmp/gmp/gmp-6.1.2  
    "rm" -rf libraries/integer-gmp/gmp/objs  
    "rm" -rf libraries/integer-gmp/gmp/gmpbuild  
    "rm" -rf utils/haddock/dist  
    "rm" -rf inplace/bin/haddock  
    "rm" -rf inplace/lib/bin/haddock  
    "rm" -rf compiler/stage1  
    "rm" -rf compiler/stage2  
    "rm" -rf compiler/stage3  
    "rm" -rf utils/hsc2hs/dist  
    "rm" -rf inplace/bin/hsc2hs  
    "rm" -rf inplace/lib/bin/hsc2hs  
    "rm" -rf utils/hsc2hs/dist-install  
    "rm" -rf utils/ghc-pkg/dist  
    "rm" -rf inplace/bin/ghc-pkg  
    "rm" -rf inplace/lib/bin/ghc-pkg  
    "rm" -rf utils/ghc-pkg/dist-install  
    "rm" -rf utils/ghctags/dist-install  
    "rm" -rf inplace/bin/ghctags  
    "rm" -rf utils/check-api-annotations/dist-install  
    "rm" -rf inplace/bin/check-api-annotations  
    "rm" -rf utils/check-ppr/dist-install  
    "rm" -rf inplace/bin/check-ppr  
    "rm" -rf  utils/ghc-cabal/dist bootstrapping  
    "rm" -rf utils/ghc-cabal/dist-install  
    "rm" -rf utils/hpc/dist-install  
    "rm" -rf inplace/bin/hpc  
    "rm" -rf inplace/lib/bin/hpc  
    "rm" -rf utils/runghc/dist-install  
    "rm" -rf inplace/bin/runghc  
    "rm" -rf inplace/lib/bin/runghc  
    "rm" -rf ghc/stage1  
    "rm" -rf inplace/bin/ghc-stage1  
    "rm" -rf inplace/lib/bin/ghc-stage1  
    "rm" -rf ghc/stage2  
    "rm" -rf inplace/bin/ghc-stage2  
    "rm" -rf inplace/lib/bin/ghc-stage2  
    "rm" -rf ghc/stage3  
    "rm" -rf docs/users_guide/.doctrees-html/ docs/users_guide/.doctrees-pdf/ docs/users_guide/build-html/ docs/users_guide/build-pdf/ docs/users_guide/users_guide.pdf  
    "rm" -rf docs/users_guide/.doctrees-man/ docs/users_guide/build-man/  
    "rm" -rf utils/count_lines/dist inplace/bin/count_lines  
    "rm" -rf utils/compare_sizes/dist-install  
    "rm" -rf iserv/stage2  
    "rm" -rf inplace/lib/bin/ghc-iserv  
    "rm" -rf iserv/stage2_p  
    "rm" -rf inplace/lib/bin/ghc-iserv-prof  
    "rm" -rf iserv/stage2_dyn  
    "rm" -rf inplace/lib/bin/ghc-iserv-dyn  
    "rm" -f libraries/integer-gmp/include/HsIntegerGmp.h libraries/base/include/EventConfig.h mk/config.mk.old mk/project.mk.old compiler/ghc.cabal.old includes/GHCConstants.h includes/DerivedConstants.h includes/ghcautoconf.h includes/ghcplatform.h includes/ghcversion.h utils/ghc-pkg/Version.hs compiler/prelude/primops.txt  
    "rm" -rf includes/dist-derivedconstants  
    "rm" -rf inplace/bin  
    "rm" -rf inplace/lib  
    "rm" -rf libraries/bootstrapping.conf  
    "rm" -f mk/are-validating.mk  
    "rm" -rf libraries/ghc-boot-th/dist-install  
    "rm" -rf libraries/ghc-boot/dist-install  
    "rm" -rf libraries/ghci/dist-install  
    "rm" -rf libraries/base/dist-install  
    "rm" -rf libraries/ghc-prim/dist-install  
    "rm" -rf libraries/integer-gmp/dist-install  
    "rm" -rf libraries/integer-simple/dist-install  
    "rm" -rf libraries/template-haskell/dist-install  
    "rm" -rf libraries/array/dist-install  
    "rm" -rf libraries/binary/dist-install  
    "rm" -rf libraries/bytestring/dist-install  
    "rm" -rf libraries/Cabal/Cabal/dist-install  
    "rm" -rf libraries/ghc-compact/dist-install  
    "rm" -rf libraries/containers/dist-install  
    "rm" -rf libraries/deepseq/dist-install  
    "rm" -rf libraries/directory/dist-install  
    "rm" -rf libraries/filepath/dist-install  
    "rm" -rf libraries/haskeline/dist-install  
    "rm" -rf libraries/hpc/dist-install  
    "rm" -rf libraries/mtl/dist-install  
    "rm" -rf libraries/parsec/dist-install  
    "rm" -rf libraries/pretty/dist-install  
    "rm" -rf libraries/process/dist-install  
    "rm" -rf libraries/terminfo/dist-install  
    "rm" -rf libraries/text/dist-install  
    "rm" -rf libraries/time/dist-install  
    "rm" -rf libraries/transformers/dist-install  
    "rm" -rf libraries/unix/dist-install  
    "rm" -rf libraries/Win32/dist-install  
    "rm" -rf libraries/xhtml/dist-install  
    "rm" -rf libraries/parallel/dist-install  
    "rm" -rf libraries/stm/dist-install  
    "rm" -rf libraries/random/dist-install  
    "rm" -rf libraries/primitive/dist-install  
    "rm" -rf libraries/vector/dist-install  
    "rm" -rf libraries/dph/dph-base/dist-install  
    "rm" -rf libraries/dph/dph-prim-interface/dist-install  
    "rm" -rf libraries/dph/dph-prim-seq/dist-install  
    "rm" -rf libraries/dph/dph-prim-par/dist-install  
    "rm" -rf libraries/dph/dph-lifted-base/dist-install  
    "rm" -rf libraries/dph/dph-lifted-boxed/dist-install  
    "rm" -rf libraries/dph/dph-lifted-copy/dist-install  
    "rm" -rf libraries/dph/dph-lifted-vseg/dist-install  
    "rm" -rf libraries/ghc-boot-th/dist-boot  
    "rm" -rf libraries/ghc-boot/dist-boot  
    "rm" -rf libraries/ghci/dist-boot  
    "rm" -rf libraries/base/dist-boot  
    "rm" -rf libraries/ghc-prim/dist-boot  
    "rm" -rf libraries/integer-gmp/dist-boot  
    "rm" -rf libraries/integer-simple/dist-boot  
    "rm" -rf libraries/template-haskell/dist-boot  
    "rm" -rf libraries/array/dist-boot  
    "rm" -rf libraries/binary/dist-boot  
    "rm" -rf libraries/bytestring/dist-boot  
    "rm" -rf libraries/Cabal/Cabal/dist-boot  
    "rm" -rf libraries/ghc-compact/dist-boot  
    "rm" -rf libraries/containers/dist-boot  
    "rm" -rf libraries/deepseq/dist-boot  
    "rm" -rf libraries/directory/dist-boot  
    "rm" -rf libraries/filepath/dist-boot  
    "rm" -rf libraries/haskeline/dist-boot  
    "rm" -rf libraries/hpc/dist-boot  
    "rm" -rf libraries/mtl/dist-boot  
    "rm" -rf libraries/parsec/dist-boot  
    "rm" -rf libraries/pretty/dist-boot  
    "rm" -rf libraries/process/dist-boot  
    "rm" -rf libraries/terminfo/dist-boot  
    "rm" -rf libraries/text/dist-boot  
    "rm" -rf libraries/time/dist-boot  
    "rm" -rf libraries/transformers/dist-boot  
    "rm" -rf libraries/unix/dist-boot  
    "rm" -rf libraries/Win32/dist-boot  
    "rm" -rf libraries/xhtml/dist-boot  
    "rm" -rf libraries/parallel/dist-boot  
    "rm" -rf libraries/stm/dist-boot  
    "rm" -rf libraries/random/dist-boot  
    "rm" -rf libraries/primitive/dist-boot  
    "rm" -rf libraries/vector/dist-boot  
    "rm" -rf libraries/dph/dph-base/dist-boot  
    "rm" -rf libraries/dph/dph-prim-interface/dist-boot  
    "rm" -rf libraries/dph/dph-prim-seq/dist-boot  
    "rm" -rf libraries/dph/dph-prim-par/dist-boot  
    "rm" -rf libraries/dph/dph-lifted-base/dist-boot  
    "rm" -rf libraries/dph/dph-lifted-boxed/dist-boot  
    "rm" -rf libraries/dph/dph-lifted-copy/dist-boot  
    "rm" -rf libraries/dph/dph-lifted-vseg/dist-boot  
    "rm" -rf  libraries/ghc-boot-th/dist  libraries/ghc-boot/dist  libraries/ghci/dist  libraries/base/dist  libraries/ghc-prim/dist  libraries/integer-gmp/dist  libraries/integer-simple/dist  libraries/template-haskell/dist  libraries/array/dist  libraries/binary/dist  libraries/bytestring/dist  libraries/Cabal/Cabal/dist  libraries/ghc-compact/dist  libraries/containers/dist  libraries/deepseq/dist  libraries/directory/dist  libraries/filepath/dist  libraries/haskeline/dist  libraries/hpc/dist  libraries/mtl/dist  libraries/parsec/dist  libraries/pretty/dist  libraries/process/dist  libraries/terminfo/dist  libraries/text/dist  libraries/time/dist  libraries/transformers/dist  libraries/unix/dist  libraries/Win32/dist  libraries/xhtml/dist  libraries/parallel/dist  libraries/stm/dist  libraries/random/dist  libraries/primitive/dist  libraries/vector/dist  libraries/dph/dph-base/dist  libraries/dph/dph-prim-interface/dist  libraries/dph/dph-prim-seq/dist  libraries/dph/dph-prim-par/dist  libraries/dph/dph-lifted-base/dist  libraries/dph/dph-lifted-boxed/dist  libraries/dph/dph-lifted-copy/dist  libraries/dph/dph-lifted-vseg/dist  
    "rm" -f libraries/base/base.buildinfo libraries/integer-gmp/integer-gmp.buildinfo libraries/terminfo/terminfo.buildinfo libraries/unix/unix.buildinfo  
    "rm" -f  libraries/ghc-boot-th/config.log  libraries/ghc-boot/config.log  libraries/ghci/config.log  libraries/base/config.log  libraries/ghc-prim/config.log  libraries/integer-gmp/config.log  libraries/integer-simple/config.log  libraries/template-haskell/config.log  libraries/array/config.log  libraries/binary/config.log  libraries/bytestring/config.log  libraries/Cabal/Cabal/config.log  libraries/ghc-compact/config.log  libraries/containers/config.log  libraries/deepseq/config.log  libraries/directory/config.log  libraries/filepath/config.log  libraries/haskeline/config.log  libraries/hpc/config.log  libraries/mtl/config.log  libraries/parsec/config.log  libraries/pretty/config.log  libraries/process/config.log  libraries/terminfo/config.log  libraries/text/config.log  libraries/time/config.log  libraries/transformers/config.log  libraries/unix/config.log  libraries/Win32/config.log  libraries/xhtml/config.log  libraries/parallel/config.log  libraries/stm/config.log  libraries/random/config.log  libraries/primitive/config.log  libraries/vector/config.log  libraries/dph/dph-base/config.log  libraries/dph/dph-prim-interface/config.log  libraries/dph/dph-prim-seq/config.log  libraries/dph/dph-prim-par/config.log  libraries/dph/dph-lifted-base/config.log  libraries/dph/dph-lifted-boxed/config.log  libraries/dph/dph-lifted-copy/config.log  libraries/dph/dph-lifted-vseg/config.log  
    "rm" -f  libraries/ghc-boot-th/config.status  libraries/ghc-boot/config.status  libraries/ghci/config.status  libraries/base/config.status  libraries/ghc-prim/config.status  libraries/integer-gmp/config.status  libraries/integer-simple/config.status  libraries/template-haskell/config.status  libraries/array/config.status  libraries/binary/config.status  libraries/bytestring/config.status  libraries/Cabal/Cabal/config.status  libraries/ghc-compact/config.status  libraries/containers/config.status  libraries/deepseq/config.status  libraries/directory/config.status  libraries/filepath/config.status  libraries/haskeline/config.status  libraries/hpc/config.status  libraries/mtl/config.status  libraries/parsec/config.status  libraries/pretty/config.status  libraries/process/config.status  libraries/terminfo/config.status  libraries/text/config.status  libraries/time/config.status  libraries/transformers/config.status  libraries/unix/config.status  libraries/Win32/config.status  libraries/xhtml/config.status  libraries/parallel/config.status  libraries/stm/config.status  libraries/random/config.status  libraries/primitive/config.status  libraries/vector/config.status  libraries/dph/dph-base/config.status  libraries/dph/dph-prim-interface/config.status  libraries/dph/dph-prim-seq/config.status  libraries/dph/dph-prim-par/config.status  libraries/dph/dph-lifted-base/config.status  libraries/dph/dph-lifted-boxed/config.status  libraries/dph/dph-lifted-copy/config.status  libraries/dph/dph-lifted-vseg/config.status  
    "rm" -f libraries/base/include/HsBaseConfig.h libraries/process/include/HsProcessConfig.h libraries/unix/include/HsUnixConfig.h  
    "rm" -rf libraries/dist-haddock  
    "rm" -rf bindistprep/  
    test ! -d testsuite || make -C testsuite clean
    make[1]: Entering directory '/stuff3/work/GHC2/ghc/testsuite'
    make -C ./timeout clean
    make[2]: Entering directory '/stuff3/work/GHC2/ghc/testsuite/timeout'
    test ! -f Setup || ./Setup clean
    rm -f -rf install-inplace
    rm -f -f calibrate.out
    rm -f -f Setup Setup.exe Setup.hi Setup.o
    make[2]: Leaving directory '/stuff3/work/GHC2/ghc/testsuite/timeout'
    rm -f -f mk/*.o
    rm -f -f mk/*.hi
    rm -f -f mk/ghcconfig*.mk
    rm -f -f mk/ghc-config mk/ghc-config.exe
    rm -f -f driver/*.pyc
    make[1]: Leaving directory '/stuff3/work/GHC2/ghc/testsuite'
     
    real 0m50.990s
    user 0m0.496s
    sys 0m1.582s



--
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: Observation on Hadrian's relative performance re current buildsystem

GHC - devs mailing list
Urk! I expected Hadrian to be faster because it has more accurate dependencies.

Simon

|  -----Original Message-----
|  From: ghc-devs [mailto:[hidden email]] On Behalf Of
|  Herbert Valerio Riedel
|  Sent: 17 November 2017 13:08
|  To: [hidden email]
|  Subject: Observation on Hadrian's relative performance re current
|  buildsystem
|  
|  Hello GHC devs,
|  
|  I took the opportunity to give Hadrian a test-run to see whether it
|  could live up to the big promise of delivering a "more scalable,
|  faster"
|  system than the current GNU Make based system. Unfortunately, my
|  preliminary results don't back this claim, and actually make Hadrian
|  appear to be significantly slower.
|  
|  ----
|  
|  Here's the summary of the results:
|  
|                           | Hadrian   | GNU Make  |
|  +------------------------+-----------+-----------+
|  | Compiling `hadrian`    | 5m25s     | 0         |
|  | (one-time setup)       |           |           |
|  +------------------------+-----------+-----------+
|  | build "all" at -j8     | 38m       | 33m       |
|  +------------------------+-----------+-----------+
|  | no-op build at -j8     | 10.977s   | 3.258s    |
|  +------------------------+-----------+-----------+
|  | "clean"                | 21s       | 51s       |
|  +------------------------+-----------+-----------+
|  
|  
|  So, Hadrian is ~5 minutes than GNU Make (or even ~10 minutes if you
|  also count the one-time setup cost).
|  
|  And what I personally consider a bit annoying is that it's ~3 times
|  slower detecting; i.e. you have to wait 11s for Hadrian to detect
|  there's nothing to be done which compared to GNU Make (which currently
|  needlessly re-runs Sphinx; so it could be even faster!) is very
|  noticeable to me.
|  
|  There's a silver-lining though, deleting files is the part which is a
|  lot more costly in the GNU Make system currently since artifacts are
|  spread over several (scroll to the end of this email) subfolders
|  there. Whereas Hadrian did something we should have done for the GNU
|  Make system as well (and probably would have done sooner or later
|  anyway in order to support the srcdir!=buildir scheme that people are
|  used to from GNU Autotools projects); Hadrian places build-artifacts
|  into a few top-level folders, and so cleaning up is trivial and
|  requires to unlink only a few folders from the filesystem.
|  
|  ----
|  
|  At the very least, I'd expect Hadrian to be as fast as the GNU Make
|  system (and ideally beat it, not the least as this was besides
|  maintainability its big promise), but so far it doesn't seem to
|  deliver that promise for me.
|  
|  It could easily be that I'm comparing apples to oranges here or that
|  I've otherwise overlooked something, so let me describe how I came to
|  this conclusion:
|  
|  I tried this on an reasonably idle Linux workstation with an Intel(R)
|  Core(TM) i7-3770 CPU @ 3.40GHz CPU, and with 32GiB RAM (i.e. the
|  filesystem content was well-cached into memory; NB: a ghc source tree
|  + compiled artifacts takes up about 4GiB on the filesystem).
|  
|  I've started from a fresh Git clone, i.e.
|  
|    git clone --recursive git://git.haskell.org/ghc.git
|  
|  followed by
|  
|    ./boot
|    ./configure
|  
|  At this point, we're at the common point from which both the Hadrian
|  and the GNU Make build-system would start diverging:
|  
|  For the Hadrian build-system, we need to pay for a one-time setup,
|  since we need to build the `hadrian` executable (which requires to
|  build the in-tree lib:Cabal as an in-place library):
|  
|  
|    $ time ./hadrian/build.sh --help
|  
|    ...
|  
|    real  5m25.992s
|    user  6m19.196s
|    sys   0m6.079s
|  
|  
|  I'm not too worried about this part, as there's a few tricks by which
|  we could likely bring that down to about 2 minutes or so, and we
|  mostly pay this setup-cost, when lib:Cabal and/or `hadrian` changes
|  and requires to be recompiled.
|  
|  
|  Now, after having made sure that the `hadrian` executable is fresh, I
|  started the actual build:
|  
|      $ time ./hadrian/build.sh -j8
|  
|      ...
|  
|      shakeArgsWith                        0.000s    0%
|      Function shake                       0.178s    0%
|      Database read                        0.000s    0%
|      With database                        0.000s    0%
|      Running rules                     2338.398s   99%
|  =========================
|      Pool finished (5261 threads, 8 max)  0.002s    0%
|      Lint checking                        0.111s    0%
|      Total                             2338.690s  100%
|      Build completed in 38:59m
|  
|      real 38m59.626s
|      user 219m7.421s
|      sys  11m7.584s
|  
|  
|  then I immediately re-issued the same command to test how long it
|  takes to perform a no-op build:
|  
|  
|      $ time ./hadrian/build.sh -j8
|  
|      Up to date
|      Up to date
|      shakeArgsWith                        0.000s    0%
|      Function shake                       0.183s    1%
|      Database read                        0.144s    1%
|      With database                        0.241s    2%
|      Running rules                        9.379s   93%
|  =========================
|      Pool finished (4165 threads, 8 max)  0.004s    0%
|      Lint checking                        0.101s    1%
|      Total                               10.051s  100%
|      Build completed in 0:11m
|  
|  
|      real 0m10.977s
|      user 0m19.399s
|      sys  0m2.443s
|  
|  
|  Same without -j8:
|  
|      $ time ./hadrian/build.sh
|      Up to date
|      Up to date
|      shakeArgsWith                     0.000s    0%
|      Function shake                    0.175s    2%
|      Database read                     0.123s    1%
|      With database                     0.197s    2%
|      Running rules                     7.710s   92%
|  =========================
|      Pool finished (1 threads, 1 max)  0.003s    0%
|      Lint checking                     0.084s    1%
|      Total                             8.293s  100%
|      Build completed in 0:09m
|  
|  
|      real 0m9.196s
|      user 0m8.656s
|      sys 0m0.724s
|  
|  
|  And finally clean it again:
|  
|      $ sync
|      $ time ./hadrian/build.sh clean
|      Up to date
|      Up to date
|      | Remove directory _build/stage0
|      | Remove directory _build/stage1
|      | Remove directory _build/stage2
|      | Remove directory _build/stage3
|      | Remove directory inplace/bin
|      | Remove directory inplace/lib
|      | Remove directory sdistprep
|      | Remove Hadrian files...
|      | Remove directory _build/generated
|      | Done.
|      shakeArgsWith                     0.000s    0%
|      Function shake                    0.175s    0%
|      Database read                     0.114s    0%
|      With database                     0.205s    0%
|      Running rules                    20.037s   97%
|  ========================
|      Pool finished (1 threads, 1 max)  0.002s    0%
|      Lint checking                     0.006s    0%
|      Total                            20.540s  100%
|      Build completed in 0:21m
|  
|  
|      real 0m21.426s
|      user 0m1.415s
|      sys 0m1.045s
|  
|  
|  
|  ----
|  
|  Running the full build via our rusty GNU Make system:
|  
|      $ time make V=0 -j8
|      real 33m30.801s
|      user 157m49.520s
|      sys  6m49.289s
|  
|  A no-op build:
|  
|      $ time make V=0 -j8
|      ===--- building phase 0
|      make --no-print-directory -f ghc.mk phase=0 phase_0_builds
|      make[1]: Nothing to be done for 'phase_0_builds'.
|      ===--- building phase 1
|      make --no-print-directory -f ghc.mk phase=1 phase_1_builds
|      make[1]: Nothing to be done for 'phase_1_builds'.
|      ===--- building final phase
|      make --no-print-directory -f ghc.mk phase=final all
|      make -C utils/haddock/doc html SPHINX_BUILD=/usr/bin/sphinx-build
|      /usr/bin/sphinx-build -b html . .build-html
|      Running Sphinx v1.3.6
|      loading translations [en]... done
|      loading pickled environment... done
|      building [mo]: targets for 0 po files that are out of date
|      building [html]: targets for 0 source files that are out of date
|      updating environment: 0 added, 0 changed, 0 removed
|      looking for now-outdated files... none found
|      no targets are out of date.
|      build succeeded.
|      cp -R utils/haddock/doc/.build-html utils/haddock/doc/haddock
|  
|      real 0m3.258s
|      user 0m3.134s
|      sys 0m0.283s
|  
|  And finally `make clean`:
|  
|      $ sync
|      $ time make clean
|      make --no-print-directory -f ghc.mk clean CLEANING=YES
|      "rm" -rf inplace/bin inplace/lib
|      "rm" -rf utils/touchy/dist
|      "rm" -rf inplace/lib/bin/touchy
|      "rm" -rf utils/unlit/dist
|      "rm" -rf inplace/lib/bin/unlit
|      "rm" -rf utils/unlit/dist-install
|      "rm" -rf utils/hp2ps/dist
|      "rm" -rf inplace/bin/hp2ps
|      "rm" -rf inplace/lib/bin/hp2ps
|      "rm" -rf utils/hp2ps/dist-install
|      "rm" -rf driver/split/dist inplace/lib/bin/ghc-split
|      "rm" -rf utils/genprimopcode/dist
|      "rm" -rf inplace/bin/genprimopcode
|      "rm" -rf  libffi/build libffi/stamp.ffi.static-shared.configure
|  libffi/stamp.ffi.static-shared.build libffi/stamp.ffi.static-
|  shared.install libffi/dist-install
|      "rm" -rf utils/deriveConstants/dist
|      "rm" -rf inplace/bin/deriveConstants
|      "rm" -rf  includes/ghcautoconf.h includes/ghcplatform.h
|  includes/ghcversion.h
|      "rm" -rf rts/dist
|      "rm" -rf bindisttest/"install   dir"  bindisttest/HelloWorld
|  bindisttest/HelloWorld.o bindisttest/HelloWorld.hi bindisttest/output
|      "rm" -rf utils/genapply/dist
|      "rm" -rf inplace/bin/genapply
|      "rm" -rf  libraries/integer-gmp/include/ghc-gmp.h
|  libraries/integer-gmp/gmp/config.mk libraries/integer-gmp/gmp/libgmp.a
|  libraries/integer-gmp/gmp/gmp.h libraries/integer-gmp/gmp/gmpbuild
|  libraries/integer-gmp/gmp/gmp-6.1.2
|      "rm" -rf libraries/integer-gmp/gmp/objs
|      "rm" -rf libraries/integer-gmp/gmp/gmpbuild
|      "rm" -rf utils/haddock/dist
|      "rm" -rf inplace/bin/haddock
|      "rm" -rf inplace/lib/bin/haddock
|      "rm" -rf compiler/stage1
|      "rm" -rf compiler/stage2
|      "rm" -rf compiler/stage3
|      "rm" -rf utils/hsc2hs/dist
|      "rm" -rf inplace/bin/hsc2hs
|      "rm" -rf inplace/lib/bin/hsc2hs
|      "rm" -rf utils/hsc2hs/dist-install
|      "rm" -rf utils/ghc-pkg/dist
|      "rm" -rf inplace/bin/ghc-pkg
|      "rm" -rf inplace/lib/bin/ghc-pkg
|      "rm" -rf utils/ghc-pkg/dist-install
|      "rm" -rf utils/ghctags/dist-install
|      "rm" -rf inplace/bin/ghctags
|      "rm" -rf utils/check-api-annotations/dist-install
|      "rm" -rf inplace/bin/check-api-annotations
|      "rm" -rf utils/check-ppr/dist-install
|      "rm" -rf inplace/bin/check-ppr
|      "rm" -rf  utils/ghc-cabal/dist bootstrapping
|      "rm" -rf utils/ghc-cabal/dist-install
|      "rm" -rf utils/hpc/dist-install
|      "rm" -rf inplace/bin/hpc
|      "rm" -rf inplace/lib/bin/hpc
|      "rm" -rf utils/runghc/dist-install
|      "rm" -rf inplace/bin/runghc
|      "rm" -rf inplace/lib/bin/runghc
|      "rm" -rf ghc/stage1
|      "rm" -rf inplace/bin/ghc-stage1
|      "rm" -rf inplace/lib/bin/ghc-stage1
|      "rm" -rf ghc/stage2
|      "rm" -rf inplace/bin/ghc-stage2
|      "rm" -rf inplace/lib/bin/ghc-stage2
|      "rm" -rf ghc/stage3
|      "rm" -rf docs/users_guide/.doctrees-html/
|  docs/users_guide/.doctrees-pdf/ docs/users_guide/build-html/
|  docs/users_guide/build-pdf/ docs/users_guide/users_guide.pdf
|      "rm" -rf docs/users_guide/.doctrees-man/ docs/users_guide/build-
|  man/
|      "rm" -rf utils/count_lines/dist inplace/bin/count_lines
|      "rm" -rf utils/compare_sizes/dist-install
|      "rm" -rf iserv/stage2
|      "rm" -rf inplace/lib/bin/ghc-iserv
|      "rm" -rf iserv/stage2_p
|      "rm" -rf inplace/lib/bin/ghc-iserv-prof
|      "rm" -rf iserv/stage2_dyn
|      "rm" -rf inplace/lib/bin/ghc-iserv-dyn
|      "rm" -f libraries/integer-gmp/include/HsIntegerGmp.h
|  libraries/base/include/EventConfig.h mk/config.mk.old
|  mk/project.mk.old compiler/ghc.cabal.old includes/GHCConstants.h
|  includes/DerivedConstants.h includes/ghcautoconf.h
|  includes/ghcplatform.h includes/ghcversion.h utils/ghc-pkg/Version.hs
|  compiler/prelude/primops.txt
|      "rm" -rf includes/dist-derivedconstants
|      "rm" -rf inplace/bin
|      "rm" -rf inplace/lib
|      "rm" -rf libraries/bootstrapping.conf
|      "rm" -f mk/are-validating.mk
|      "rm" -rf libraries/ghc-boot-th/dist-install
|      "rm" -rf libraries/ghc-boot/dist-install
|      "rm" -rf libraries/ghci/dist-install
|      "rm" -rf libraries/base/dist-install
|      "rm" -rf libraries/ghc-prim/dist-install
|      "rm" -rf libraries/integer-gmp/dist-install
|      "rm" -rf libraries/integer-simple/dist-install
|      "rm" -rf libraries/template-haskell/dist-install
|      "rm" -rf libraries/array/dist-install
|      "rm" -rf libraries/binary/dist-install
|      "rm" -rf libraries/bytestring/dist-install
|      "rm" -rf libraries/Cabal/Cabal/dist-install
|      "rm" -rf libraries/ghc-compact/dist-install
|      "rm" -rf libraries/containers/dist-install
|      "rm" -rf libraries/deepseq/dist-install
|      "rm" -rf libraries/directory/dist-install
|      "rm" -rf libraries/filepath/dist-install
|      "rm" -rf libraries/haskeline/dist-install
|      "rm" -rf libraries/hpc/dist-install
|      "rm" -rf libraries/mtl/dist-install
|      "rm" -rf libraries/parsec/dist-install
|      "rm" -rf libraries/pretty/dist-install
|      "rm" -rf libraries/process/dist-install
|      "rm" -rf libraries/terminfo/dist-install
|      "rm" -rf libraries/text/dist-install
|      "rm" -rf libraries/time/dist-install
|      "rm" -rf libraries/transformers/dist-install
|      "rm" -rf libraries/unix/dist-install
|      "rm" -rf libraries/Win32/dist-install
|      "rm" -rf libraries/xhtml/dist-install
|      "rm" -rf libraries/parallel/dist-install
|      "rm" -rf libraries/stm/dist-install
|      "rm" -rf libraries/random/dist-install
|      "rm" -rf libraries/primitive/dist-install
|      "rm" -rf libraries/vector/dist-install
|      "rm" -rf libraries/dph/dph-base/dist-install
|      "rm" -rf libraries/dph/dph-prim-interface/dist-install
|      "rm" -rf libraries/dph/dph-prim-seq/dist-install
|      "rm" -rf libraries/dph/dph-prim-par/dist-install
|      "rm" -rf libraries/dph/dph-lifted-base/dist-install
|      "rm" -rf libraries/dph/dph-lifted-boxed/dist-install
|      "rm" -rf libraries/dph/dph-lifted-copy/dist-install
|      "rm" -rf libraries/dph/dph-lifted-vseg/dist-install
|      "rm" -rf libraries/ghc-boot-th/dist-boot
|      "rm" -rf libraries/ghc-boot/dist-boot
|      "rm" -rf libraries/ghci/dist-boot
|      "rm" -rf libraries/base/dist-boot
|      "rm" -rf libraries/ghc-prim/dist-boot
|      "rm" -rf libraries/integer-gmp/dist-boot
|      "rm" -rf libraries/integer-simple/dist-boot
|      "rm" -rf libraries/template-haskell/dist-boot
|      "rm" -rf libraries/array/dist-boot
|      "rm" -rf libraries/binary/dist-boot
|      "rm" -rf libraries/bytestring/dist-boot
|      "rm" -rf libraries/Cabal/Cabal/dist-boot
|      "rm" -rf libraries/ghc-compact/dist-boot
|      "rm" -rf libraries/containers/dist-boot
|      "rm" -rf libraries/deepseq/dist-boot
|      "rm" -rf libraries/directory/dist-boot
|      "rm" -rf libraries/filepath/dist-boot
|      "rm" -rf libraries/haskeline/dist-boot
|      "rm" -rf libraries/hpc/dist-boot
|      "rm" -rf libraries/mtl/dist-boot
|      "rm" -rf libraries/parsec/dist-boot
|      "rm" -rf libraries/pretty/dist-boot
|      "rm" -rf libraries/process/dist-boot
|      "rm" -rf libraries/terminfo/dist-boot
|      "rm" -rf libraries/text/dist-boot
|      "rm" -rf libraries/time/dist-boot
|      "rm" -rf libraries/transformers/dist-boot
|      "rm" -rf libraries/unix/dist-boot
|      "rm" -rf libraries/Win32/dist-boot
|      "rm" -rf libraries/xhtml/dist-boot
|      "rm" -rf libraries/parallel/dist-boot
|      "rm" -rf libraries/stm/dist-boot
|      "rm" -rf libraries/random/dist-boot
|      "rm" -rf libraries/primitive/dist-boot
|      "rm" -rf libraries/vector/dist-boot
|      "rm" -rf libraries/dph/dph-base/dist-boot
|      "rm" -rf libraries/dph/dph-prim-interface/dist-boot
|      "rm" -rf libraries/dph/dph-prim-seq/dist-boot
|      "rm" -rf libraries/dph/dph-prim-par/dist-boot
|      "rm" -rf libraries/dph/dph-lifted-base/dist-boot
|      "rm" -rf libraries/dph/dph-lifted-boxed/dist-boot
|      "rm" -rf libraries/dph/dph-lifted-copy/dist-boot
|      "rm" -rf libraries/dph/dph-lifted-vseg/dist-boot
|      "rm" -rf  libraries/ghc-boot-th/dist  libraries/ghc-boot/dist
|  libraries/ghci/dist  libraries/base/dist  libraries/ghc-prim/dist
|  libraries/integer-gmp/dist  libraries/integer-simple/dist
|  libraries/template-haskell/dist  libraries/array/dist
|  libraries/binary/dist  libraries/bytestring/dist
|  libraries/Cabal/Cabal/dist  libraries/ghc-compact/dist
|  libraries/containers/dist  libraries/deepseq/dist
|  libraries/directory/dist  libraries/filepath/dist
|  libraries/haskeline/dist  libraries/hpc/dist  libraries/mtl/dist
|  libraries/parsec/dist  libraries/pretty/dist  libraries/process/dist
|  libraries/terminfo/dist  libraries/text/dist  libraries/time/dist
|  libraries/transformers/dist  libraries/unix/dist  libraries/Win32/dist
|  libraries/xhtml/dist  libraries/parallel/dist  libraries/stm/dist
|  libraries/random/dist  libraries/primitive/dist  libraries/vector/dist
|  libraries/dph/dph-base/dist  libraries/dph/dph-prim-interface/dist
|  libraries/dph/dph-prim-seq/dist  libraries/dph/dph-prim-par/dist
|  libraries/dph/dph-lifted-base/dist  libraries/dph/dph-lifted-
|  boxed/dist  libraries/dph/dph-lifted-copy/dist  libraries/dph/dph-
|  lifted-vseg/dist
|      "rm" -f libraries/base/base.buildinfo libraries/integer-
|  gmp/integer-gmp.buildinfo libraries/terminfo/terminfo.buildinfo
|  libraries/unix/unix.buildinfo
|      "rm" -f  libraries/ghc-boot-th/config.log  libraries/ghc-
|  boot/config.log  libraries/ghci/config.log  libraries/base/config.log
|  libraries/ghc-prim/config.log  libraries/integer-gmp/config.log
|  libraries/integer-simple/config.log  libraries/template-
|  haskell/config.log  libraries/array/config.log
|  libraries/binary/config.log  libraries/bytestring/config.log
|  libraries/Cabal/Cabal/config.log  libraries/ghc-compact/config.log
|  libraries/containers/config.log  libraries/deepseq/config.log
|  libraries/directory/config.log  libraries/filepath/config.log
|  libraries/haskeline/config.log  libraries/hpc/config.log
|  libraries/mtl/config.log  libraries/parsec/config.log
|  libraries/pretty/config.log  libraries/process/config.log
|  libraries/terminfo/config.log  libraries/text/config.log
|  libraries/time/config.log  libraries/transformers/config.log
|  libraries/unix/config.log  libraries/Win32/config.log
|  libraries/xhtml/config.log  libraries/parallel/config.log
|  libraries/stm/config.log  libraries/random/config.log
|  libraries/primitive/config.log  libraries/vector/config.log
|  libraries/dph/dph-base/config.log  libraries/dph/dph-prim-
|  interface/config.log  libraries/dph/dph-prim-seq/config.log
|  libraries/dph/dph-prim-par/config.log  libraries/dph/dph-lifted-
|  base/config.log  libraries/dph/dph-lifted-boxed/config.log
|  libraries/dph/dph-lifted-copy/config.log  libraries/dph/dph-lifted-
|  vseg/config.log
|      "rm" -f  libraries/ghc-boot-th/config.status  libraries/ghc-
|  boot/config.status  libraries/ghci/config.status
|  libraries/base/config.status  libraries/ghc-prim/config.status
|  libraries/integer-gmp/config.status  libraries/integer-
|  simple/config.status  libraries/template-haskell/config.status
|  libraries/array/config.status  libraries/binary/config.status
|  libraries/bytestring/config.status
|  libraries/Cabal/Cabal/config.status  libraries/ghc-
|  compact/config.status  libraries/containers/config.status
|  libraries/deepseq/config.status  libraries/directory/config.status
|  libraries/filepath/config.status  libraries/haskeline/config.status
|  libraries/hpc/config.status  libraries/mtl/config.status
|  libraries/parsec/config.status  libraries/pretty/config.status
|  libraries/process/config.status  libraries/terminfo/config.status
|  libraries/text/config.status  libraries/time/config.status
|  libraries/transformers/config.status  libraries/unix/config.status
|  libraries/Win32/config.status  libraries/xhtml/config.status
|  libraries/parallel/config.status  libraries/stm/config.status
|  libraries/random/config.status  libraries/primitive/config.status
|  libraries/vector/config.status  libraries/dph/dph-base/config.status
|  libraries/dph/dph-prim-interface/config.status  libraries/dph/dph-
|  prim-seq/config.status  libraries/dph/dph-prim-par/config.status
|  libraries/dph/dph-lifted-base/config.status  libraries/dph/dph-lifted-
|  boxed/config.status  libraries/dph/dph-lifted-copy/config.status
|  libraries/dph/dph-lifted-vseg/config.status
|      "rm" -f libraries/base/include/HsBaseConfig.h
|  libraries/process/include/HsProcessConfig.h
|  libraries/unix/include/HsUnixConfig.h
|      "rm" -rf libraries/dist-haddock
|      "rm" -rf bindistprep/
|      test ! -d testsuite || make -C testsuite clean
|      make[1]: Entering directory '/stuff3/work/GHC2/ghc/testsuite'
|      make -C ./timeout clean
|      make[2]: Entering directory
|  '/stuff3/work/GHC2/ghc/testsuite/timeout'
|      test ! -f Setup || ./Setup clean
|      rm -f -rf install-inplace
|      rm -f -f calibrate.out
|      rm -f -f Setup Setup.exe Setup.hi Setup.o
|      make[2]: Leaving directory
|  '/stuff3/work/GHC2/ghc/testsuite/timeout'
|      rm -f -f mk/*.o
|      rm -f -f mk/*.hi
|      rm -f -f mk/ghcconfig*.mk
|      rm -f -f mk/ghc-config mk/ghc-config.exe
|      rm -f -f driver/*.pyc
|      make[1]: Leaving directory '/stuff3/work/GHC2/ghc/testsuite'
|  
|      real 0m50.990s
|      user 0m0.496s
|      sys 0m1.582s
|  
|  
|  
|  --
|  _______________________________________________
|  ghc-devs mailing list
|  [hidden email]
|  https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h
|  askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cec692486529544de389e08d5
|  2dbc7432%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6364652097984492
|  46&sdata=I1TY6c2l59jou87OVjqScDvct%2FaygyM4HRuSirCSp5w%3D&reserved=0
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: Observation on Hadrian's relative performance re current buildsystem

Ben Gamari-2
Simon Peyton Jones via ghc-devs <[hidden email]> writes:

> Urk! I expected Hadrian to be faster because it has more accurate dependencies.
>
While this is just speculation, this might actually be one of the
reasons why the no-op case is slower: in make's case the dependency
graph is mostly static whereas in Hadrian the build system needs to
discover dependencies dynamically.

Either way, thanks for this characterization, Herbert!

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Observation on Hadrian's relative performance re current buildsystem

Malcolm Wallace-2

On 17 Nov 2017, at 18:46, Ben Gamari wrote:

> Simon Peyton Jones via ghc-devs <[hidden email]> writes:
>
>> Urk! I expected Hadrian to be faster because it has more accurate dependencies.
>>
> While this is just speculation, this might actually be one of the
> reasons why the no-op case is slower: in make's case the dependency
> graph is mostly static whereas in Hadrian the build system needs to
> discover dependencies dynamically.
>
> Either way, thanks for this characterization, Herbert!


But surely the timing for a full build from scratch is not the most important thing to compare?  In my work environment, full builds are extremely rare; the common case is an incremental build after pulling changes from upstream.  Is this something you can measure?

Regards,
    Malcolm

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Observation on Hadrian's relative performance re current buildsystem

Herbert Valerio Riedel-3
Hi Malcolm,

On 2017-11-18 at 11:09:28 +0000, Malcolm Wallace wrote:

[...]

> But surely the timing for a full build from scratch is not the most
> important thing to compare?  In my work environment, full builds are
> extremely rare; the common case is an incremental build after pulling
> changes from upstream.  Is this something you can measure?

If Hadrian is more exact about the dependency tracking, I'd expect
something close to a full rebuild when I `git pull` from GHC HEAD, since
the Git commit and the snapshot versioning we infer from that,
pervasively transcends most most artifacts of GHC, and in general you
should boot & configure everytime you `git pull` unless you're sure it
won't matter.

You'd have to suppress/mask this logic if you want to avoid full
rebuilds.

Also I'm not sure how Hadrian tracks itself as a dependency (NB: The GNU
Make system doesn't); When I used Shake myself, I remember that
meta-depending on the rules per se wasn't a trivial thing to do; and the
simplest way was to introduce very coarse (either manual or by hashing)
global versioning over all rules, which would invalidate the full
build. But it's been some time since I did that, so I may be wrong here.

However, what I think would be a more relevant benchmark matching the
usual GHC developer workflow, would be to see how well Hadrian manages
to minimize the work needed to rebuild GHC after editing a source file
in GHC's source-tree without changing the Git commit. As that's what
matters most to me when I'm actively working on a GHC patch.

I'll try to measure/compare this with the next GHC patch I hack on.
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Observation on Hadrian's relative performance re current buildsystem

Ben Gamari-3
Herbert Valerio Riedel <[hidden email]> writes:

> Hi Malcolm,
>
> On 2017-11-18 at 11:09:28 +0000, Malcolm Wallace wrote:
>
> [...]
>
>> But surely the timing for a full build from scratch is not the most
>> important thing to compare?  In my work environment, full builds are
>> extremely rare; the common case is an incremental build after pulling
>> changes from upstream.  Is this something you can measure?
>
> If Hadrian is more exact about the dependency tracking, I'd expect
> something close to a full rebuild when I `git pull` from GHC HEAD, since
> the Git commit and the snapshot versioning we infer from that,
> pervasively transcends most most artifacts of GHC, and in general you
> should boot & configure everytime you `git pull` unless you're sure it
> won't matter.
>
To be honest, I'm hoping that we can make this less necessary in the
future. Relatively few changes really require a re-`configure`. In my
experience most changes that current demand reconfigure are in .in
files. In these cases I often just hack the changes in to the generated
files myself. In principle there is no reason why Hadrian couldn't do
this as well. I opened #416 to track this some time ago.

Cheers,

- Ben



_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment