Compilation time

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Compilation time

Deest, Gaël
Hi all,

As you are probably well aware, GHC performance has been a growing concern over the last few years. Many Haskell programmers complain that build time has significantly increased over the last few releases. However, to our knowledge, there isn't much data available to substantiate this claim and the severity of these problems is not well known.

That's why we would like to bring some anecdotal evidence to your attention that seems to indicate really major performance regressions. We stumbled upon the CI of the data-reify package, which is built against all GHC releases since 2010 :

https://travis-ci.org/ku-fpg/data-reify

tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC 8.2. The 8.2 release alone seems to have increased compilation time by almost 2 minutes, with the current development branch bringing only minor performance improvements.

Of course, this single data point is not sufficient to establish how severe and widespread these problems are. More data could probably be gathered from other packages. However, it certainly matches our (subjective) experience and we felt important to report it to you.

Regards,

--
Gaël Deest
Tweag I/O

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Compilation time

Robin Palotai
Note that Travis CI time is a sum of all operations, including fetching/saving caches etc.

Opening the build log and looking for the compilation might be more revealing. For example, 7.10 seems ~10 seconds, 8.2.1 seems ~50 seconds.

There are two issues I can see with this, that should be addressed by benchmarking:

1) The Travis CI build hosts might show varying performance across reruns due to shared machines.

2) Tuning the RTS params when invoking GHC as a compiler can yield significant benefits, at least on larger compilations (such as -A128m).

It would probably be doable to fetch a set of packages from *ackage and build with various compilers under controlled circumstances?

Robin

2017-07-04 9:16 GMT+02:00 Deest, Gaël <[hidden email]>:
Hi all,

As you are probably well aware, GHC performance has been a growing concern over the last few years. Many Haskell programmers complain that build time has significantly increased over the last few releases. However, to our knowledge, there isn't much data available to substantiate this claim and the severity of these problems is not well known.

That's why we would like to bring some anecdotal evidence to your attention that seems to indicate really major performance regressions. We stumbled upon the CI of the data-reify package, which is built against all GHC releases since 2010 :

https://travis-ci.org/ku-fpg/data-reify

tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC 8.2. The 8.2 release alone seems to have increased compilation time by almost 2 minutes, with the current development branch bringing only minor performance improvements.

Of course, this single data point is not sufficient to establish how severe and widespread these problems are. More data could probably be gathered from other packages. However, it certainly matches our (subjective) experience and we felt important to report it to you.

Regards,

--
Gaël Deest
Tweag I/O

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs



_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Compilation time

Matthew Pickering
In reply to this post by Deest, Gaël
I tried the package with 8.2.1 and 8.0.2 and both build in less than 5
seconds with quite a large amount of variability. As Robin points out,
there are varying factors which can account for longer build times on
Travis.

On Tue, Jul 4, 2017 at 8:16 AM, Deest, Gaël <[hidden email]> wrote:

> Hi all,
>
> As you are probably well aware, GHC performance has been a growing concern
> over the last few years. Many Haskell programmers complain that build time
> has significantly increased over the last few releases. However, to our
> knowledge, there isn't much data available to substantiate this claim and
> the severity of these problems is not well known.
>
> That's why we would like to bring some anecdotal evidence to your attention
> that seems to indicate really major performance regressions. We stumbled
> upon the CI of the data-reify package, which is built against all GHC
> releases since 2010 :
>
> https://travis-ci.org/ku-fpg/data-reify
>
> tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC
> 8.2. The 8.2 release alone seems to have increased compilation time by
> almost 2 minutes, with the current development branch bringing only minor
> performance improvements.
>
> Of course, this single data point is not sufficient to establish how severe
> and widespread these problems are. More data could probably be gathered from
> other packages. However, it certainly matches our (subjective) experience
> and we felt important to report it to you.
>
> Regards,
>
> --
> Gaël Deest
> Tweag I/O
>
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: Compilation time

GHC - devs mailing list
In reply to this post by Deest, Gaël

Thanks Gael.  

 

As you may know, we’ve focused a lot on compile time in the last year or so, and 8.4 is a lot faster.  I think there are still plenty of bad cases, but things are better.

 

Our compile-time perf page is here: https://ghc.haskell.org/trac/ghc/wiki/Performance/Compiler

 

It links to a list of open perf tickets: <a href="https://ghc.haskell.org/trac/ghc/query?status=!closed&amp;failure=Compile-time&#43;performance&#43;bug"> https://ghc.haskell.org/trac/ghc/query?status=!closed&failure=Compile-time+performance+bug

 

What helps above all are reproducible test cases.  Package X compiles much more slowly with 8.2 than with 7.10, say.   You did exactly that, thank you – can you open a ticket explaining exactly how to reproduce it, and what measurements you took.  (Matthew’s numbers seemed two orders of magnitude different; I wonder why?)

 

Another thing that is really helpful is people who characterise what the problem is:

  • Is GHC generating a lot more code?   -dshow-passes can reveal this.
  • Is it GHC at all, or is it link-times, or some other part of the tool chain?
  • If it is GHC, and the output of compilation is no bigger than before, where is the time going?  -dshow-passes gives a rough idea of that too.
  • Is there a particular characteristic of the program that seems to trigger the poor behaviour?  Can you accentuate that characteristic to make it behave wildly badly?

 

Everyone: GHC is your compiler!  We need your help.

 

Simon

 

 

From: ghc-devs [mailto:[hidden email]] On Behalf Of Deest, Gaël
Sent: 04 July 2017 08:17
To: [hidden email]
Cc: Manuel Chakravarty <[hidden email]>
Subject: Compilation time

 

Hi all,

As you are probably well aware, GHC performance has been a growing concern over the last few years. Many Haskell programmers complain that build time has significantly increased over the last few releases. However, to our knowledge, there isn't much data available to substantiate this claim and the severity of these problems is not well known.

That's why we would like to bring some anecdotal evidence to your attention that seems to indicate really major performance regressions. We stumbled upon the CI of the data-reify package, which is built against all GHC releases since 2010 :

https://travis-ci.org/ku-fpg/data-reify

tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC 8.2. The 8.2 release alone seems to have increased compilation time by almost 2 minutes, with the current development branch bringing only minor performance improvements.

Of course, this single data point is not sufficient to establish how severe and widespread these problems are. More data could probably be gathered from other packages. However, it certainly matches our (subjective) experience and we felt important to report it to you.

Regards,

 

--

Gaël Deest

Tweag I/O


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Compilation time

Ryan Scott
In reply to this post by Deest, Gaël
Another things to note about 8.2 build times is that a bulk of the time is probably being spent during the linking phase. This is because GHC 8.2 has switched over to split-sections by default, but unfortunately, old versions of the BFD linker take a long time to link code that uses split-sections (see the discussion at [1]). There is ongoing work (which should hopefully land before the final 8.2 release) to use gold instead of BFD as the default linker when available, which will reduce linking times dramatically.

Ryan S.
-----

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Compilation time

Ben Gamari-2
Ryan Scott <[hidden email]> writes:

> Another things to note about 8.2 build times is that a bulk of the time is
> probably being spent during the linking phase. This is because GHC 8.2 has
> switched over to split-sections by default, but unfortunately, old versions
> of the BFD linker take a long time to link code that uses split-sections
> (see the discussion at [1]). There is ongoing work (which should hopefully
> land before the final 8.2 release) to use gold instead of BFD as the
> default linker when available, which will reduce linking times dramatically.
>
Indeed it has already landed. See #13541.

In short, ./configure will now choose to use ld.gold or ld.lld if
available (although this can be disabled using the --disable-ld-override
configure flag).

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Compilation time

Evan Laforge
On Fri, Jul 7, 2017 at 9:28 AM, Ben Gamari <[hidden email]> wrote:

> Ryan Scott <[hidden email]> writes:
>
>> Another things to note about 8.2 build times is that a bulk of the time is
>> probably being spent during the linking phase. This is because GHC 8.2 has
>> switched over to split-sections by default, but unfortunately, old versions
>> of the BFD linker take a long time to link code that uses split-sections
>> (see the discussion at [1]). There is ongoing work (which should hopefully
>> land before the final 8.2 release) to use gold instead of BFD as the
>> default linker when available, which will reduce linking times dramatically.
>>
> Indeed it has already landed. See #13541.
>
> In short, ./configure will now choose to use ld.gold or ld.lld if
> available (although this can be disabled using the --disable-ld-override
> configure flag).

Just from curiosity, does this apply to OS X?  Of course, gold is
ELF-only, so it surely doesn't apply, but there's still lld.  OS X
uses clang to compile so I thought it might already use lld, but the
'ld -v' output looks different, and lld.llvm.org implies it's a
replacement for OS X ld, so maybe not.  But it doesn't look like GNU
ld either, so maybe it's not affected by the BFD bug?

I'll try 8.2 on OS X and see if the link time changes.
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Compilation time

Ben Gamari-2
Evan Laforge <[hidden email]> writes:

> On Fri, Jul 7, 2017 at 9:28 AM, Ben Gamari <[hidden email]> wrote:
>>
>> In short, ./configure will now choose to use ld.gold or ld.lld if
>> available (although this can be disabled using the --disable-ld-override
>> configure flag).
>
> Just from curiosity, does this apply to OS X?  Of course, gold is
> ELF-only, so it surely doesn't apply, but there's still lld.  OS X
> uses clang to compile so I thought it might already use lld, but the
> 'ld -v' output looks different, and lld.llvm.org implies it's a
> replacement for OS X ld, so maybe not.  But it doesn't look like GNU
> ld either, so maybe it's not affected by the BFD bug?
>
> I'll try 8.2 on OS X and see if the link time changes.
8.2 will prefer both gold and lld over bfd ld. However two conditions
must hold for these to be used,

 * The ld.lld/ld.gold executable must be in $PATH (or explicitly named
   by passing the LD variable to configure)

 * $CC must understand the `-fuse-ld={gold,lld}` option. For (IMHO quite
   silly) political reasons, gcc doesn't support `-fuse-ld=lld`. Debian
   happens to patch gcc to add support but I don't know how common this
   is in other distributions.

Unfortunately, some earlier `gcc` versions didn't fail if given a
`-fuse-ld` option that they didn't understand. Sadly we have no reliable
way to detect this, so in this case we may end up passing a `-fuse-ld`
option that gcc simply ignores.

In the case of OS X we use Apple's own home-grown linker. I'm not sure
whether/how OS X's gcc wrapper treats `-fuse-ld` (beyond that it doesn't
error if the flag is given). I also don't know whether lld is currently a
capable replacement for OS X ld.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Compilation time

Moritz Angermann-2
For those of us who are on macOS,

you usually do not have any gcc, it’s all clang behind the curtains. E.g.
see

$ /usr/bin/gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

ld, is apples ld64 (see https://opensource.apple.com/source/ld64/)

$ /usr/bin/ld -v
@(#)PROGRAM:ld  PROJECT:ld64-278.4
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em (tvOS)
LTO support using: LLVM version 8.1.0, (clang-802.0.42)
TAPI support using: Apple TAPI version 1.33.11

if you end up having clang+llvm-4.0.0-x86_64-apple-darwin/bin or similar
in your path, you WILL have `ld.lld`, but that one is an ELF linker,
which on a MachO platform won’t be very useful (and will therefore fail
to link MachO objects).

Llvm’s `lld` will provide you with the following semi helpful note:
$ lld
lld is a generic driver.
Invoke ld.lld (Unix), ld (Mac) or lld-link (Windows) instead.

however, my copy of clang+llvm, does only ship ld.lld, and lld-link, but
not `ld`.

Thus, if you end up having the llvm tools (and ld.lld) in your PATH, you
will need to set `--disable-ld-override` during configure on macOS, or your
ghc build will fail, because you end up trying to link MachO object files
with an ELF only linker.

Cheers,
 Moritz

> On Jul 8, 2017, at 7:33 AM, Ben Gamari <[hidden email]> wrote:
>
> Evan Laforge <[hidden email]> writes:
>
>> On Fri, Jul 7, 2017 at 9:28 AM, Ben Gamari <[hidden email]> wrote:
>>>
>>> In short, ./configure will now choose to use ld.gold or ld.lld if
>>> available (although this can be disabled using the --disable-ld-override
>>> configure flag).
>>
>> Just from curiosity, does this apply to OS X?  Of course, gold is
>> ELF-only, so it surely doesn't apply, but there's still lld.  OS X
>> uses clang to compile so I thought it might already use lld, but the
>> 'ld -v' output looks different, and lld.llvm.org implies it's a
>> replacement for OS X ld, so maybe not.  But it doesn't look like GNU
>> ld either, so maybe it's not affected by the BFD bug?
>>
>> I'll try 8.2 on OS X and see if the link time changes.
>
> 8.2 will prefer both gold and lld over bfd ld. However two conditions
> must hold for these to be used,
>
> * The ld.lld/ld.gold executable must be in $PATH (or explicitly named
>   by passing the LD variable to configure)
>
> * $CC must understand the `-fuse-ld={gold,lld}` option. For (IMHO quite
>   silly) political reasons, gcc doesn't support `-fuse-ld=lld`. Debian
>   happens to patch gcc to add support but I don't know how common this
>   is in other distributions.
>
> Unfortunately, some earlier `gcc` versions didn't fail if given a
> `-fuse-ld` option that they didn't understand. Sadly we have no reliable
> way to detect this, so in this case we may end up passing a `-fuse-ld`
> option that gcc simply ignores.
>
> In the case of OS X we use Apple's own home-grown linker. I'm not sure
> whether/how OS X's gcc wrapper treats `-fuse-ld` (beyond that it doesn't
> error if the flag is given). I also don't know whether lld is currently a
> capable replacement for OS X ld.
>
> Cheers,
>
> - Ben
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Compilation time

Ben Gamari-2
Moritz Angermann <[hidden email]> writes:

> For those of us who are on macOS,

...

> Thus, if you end up having the llvm tools (and ld.lld) in your PATH, you
> will need to set `--disable-ld-override` during configure on macOS, or your
> ghc build will fail, because you end up trying to link MachO object files
> with an ELF only linker.
>
Yikes, this is quite bad. We'll need to teach autoconf to recognize this
before the release. Thanks for bringing this up!

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Compilation time

Karel Gardas
In reply to this post by Ben Gamari-2
On 07/ 8/17 01:33 AM, Ben Gamari wrote:

> 8.2 will prefer both gold and lld over bfd ld. However two conditions
> must hold for these to be used,
>
>   * The ld.lld/ld.gold executable must be in $PATH (or explicitly named
>     by passing the LD variable to configure)
>
>   * $CC must understand the `-fuse-ld={gold,lld}` option. For (IMHO quite
>     silly) political reasons, gcc doesn't support `-fuse-ld=lld`. Debian
>     happens to patch gcc to add support but I don't know how common this
>     is in other distributions.
>
> Unfortunately, some earlier `gcc` versions didn't fail if given a
> `-fuse-ld` option that they didn't understand. Sadly we have no reliable
> way to detect this, so in this case we may end up passing a `-fuse-ld`
> option that gcc simply ignores.
I've run into this issue too, but it looks like the issue is not in gcc,
but in ghc's aclocal.m4 (one '$' missing). Attached patch solves this on
OpenBSD 6.1-current where HEAD fails building on linker error. It passes
build-id option to the gcc's linker (bfd ld) which does not support it
as configure detects lld is presented on this system too.

Sorry for not being able to push that through usual arc.

Thanks,
Karel


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

aclocal.m4.diff (1K) Download Attachment