[commit: testsuite] master: Update some perf results for master (4afbacc)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[commit: testsuite] master: Update some perf results for master (4afbacc)

Simon Peyton Jones
Wait. Are you really increasing "bytes allocated" from 97M to 237M?  That's a big change, and in an important number.  Do you know why?

Simon

| -----Original Message-----
| From: ghc-commits [mailto:ghc-commits-bounces at haskell.org] On Behalf Of
| git at git.haskell.org
| Sent: 28 November 2013 12:36
| To: ghc-commits at haskell.org
| Subject: [commit: testsuite] master: Update some perf results for
| master (4afbacc)
|
| Repository : ssh://git at git.haskell.org/testsuite
|
| On branch  : master
| Link       :
| http://ghc.haskell.org/trac/ghc/changeset/4afbacc299d843eb632df1f71eee2
| c91746c85a8/testsuite
|
| >---------------------------------------------------------------
|
| commit 4afbacc299d843eb632df1f71eee2c91746c85a8
| Author: Joachim Breitner <mail at joachim-breitner.de>
| Date:   Thu Nov 28 12:36:15 2013 +0000
|
|     Update some perf results for master
|
|
| >---------------------------------------------------------------
|
| 4afbacc299d843eb632df1f71eee2c91746c85a8
|  tests/perf/compiler/all.T    |   11 ++++++-----
|  tests/perf/space_leaks/all.T |    2 +-
|  2 files changed, 7 insertions(+), 6 deletions(-)
|
| diff --git a/tests/perf/compiler/all.T b/tests/perf/compiler/all.T
| index a580bf9..759b5d0 100644
| --- a/tests/perf/compiler/all.T
| +++ b/tests/perf/compiler/all.T
| @@ -344,10 +344,11 @@ test('T5837',
|  test('T6048',
|       [ only_ways(['optasm']),
|        compiler_stats_num_field('bytes allocated',
| -          [(wordsize(32), 48887164, 10),
| -            # prev:       38000000 (x86/Linux)
| -            # 2012-10-08: 48887164 (x86/Linux)
| -           (wordsize(64), 97247032, 10)])
| -             # 18/09/2012 97247032 amd64/Linux
| +          [(wordsize(32),   48887164, 10),
| +            # prev:         38000000 (x86/Linux)
| +            # 2012-10-08:   48887164 (x86/Linux)
| +           (wordsize(64),  237077056, 10)])
| +             # 18/09/2012   97247032 amd64/Linux
| +     # 2013-11-26: 237077056 amd64/Linux
|        ],
|        compile,[''])
| diff --git a/tests/perf/space_leaks/all.T
| b/tests/perf/space_leaks/all.T
| index a1fd641..ac60c8f 100644
| --- a/tests/perf/space_leaks/all.T
| +++ b/tests/perf/space_leaks/all.T
| @@ -6,7 +6,7 @@ test('space_leak_001',
|       #           5 (x86/Linux)
|       [stats_num_field('peak_megabytes_allocated', (4, 1)),
|        stats_num_field('max_bytes_used',
| -          [(wordsize(64), 440000, 10),
| +          [(wordsize(64), 440000, 15),
|                          # 440224 (amd64/Linux)
|                          # 417016 (x86/OS X)
|                          # 415672 (x86/Windows)
|
| _______________________________________________
| ghc-commits mailing list
| ghc-commits at haskell.org
| http://www.haskell.org/mailman/listinfo/ghc-commits

Reply | Threaded
Open this post in threaded view
|

[commit: testsuite] master: Update some perf results for master (4afbacc)

Joachim Breitner-2
Hi,

Am Donnerstag, den 28.11.2013, 12:42 +0000 schrieb Simon Peyton-Jones:
> Wait. Are you really increasing "bytes allocated" from 97M to 237M?
> That's a big change, and in an important number.  Do you know why?

no; but this is master, not my branch, and it seems to be a test not run
by validate (which calls "make -C testsuite fast"), so I have to assume
it could be anything in the recent past.

If I get access to a fast machine to build stuff, I can try to bisect
it.

But that why we need a proper CI setup, which should then probably run
"./validate.sh --slow"! The test suite is only half as useful as it
could be if one first has to clean it up before one can reliably
investigate the effect of one own?s changes.

Greetings,
Joachim


--
Joachim ?nomeata? Breitner
  mail at joachim-breitner.de ? http://www.joachim-breitner.de/
  Jabber: nomeata at joachim-breitner.de  ? GPG-Key: 0x4743206C
  Debian Developer: nomeata at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131128/0807fbbd/attachment.sig>

Reply | Threaded
Open this post in threaded view
|

[commit: testsuite] master: Update some perf results for master (4afbacc)

Simon Marlow-7
In reply to this post by Simon Peyton Jones
I now get:

update the test so that GHC doesn't regress again)
     Expected    bytes allocated: 237077056 +/-10%
     Lower bound bytes allocated: 213369350
     Upper bound bytes allocated: 260784762
     Actual      bytes allocated: 101150736

Joachim: if you're building with DEBUG, that will adversely affect the
performance tests, so you probably shouldn't run those.

Cheers,
Simon

On 28/11/13 12:42, Simon Peyton-Jones wrote:

> Wait. Are you really increasing "bytes allocated" from 97M to 237M?  That's a big change, and in an important number.  Do you know why?
>
> Simon
>
> | -----Original Message-----
> | From: ghc-commits [mailto:ghc-commits-bounces at haskell.org] On Behalf Of
> | git at git.haskell.org
> | Sent: 28 November 2013 12:36
> | To: ghc-commits at haskell.org
> | Subject: [commit: testsuite] master: Update some perf results for
> | master (4afbacc)
> |
> | Repository : ssh://git at git.haskell.org/testsuite
> |
> | On branch  : master
> | Link       :
> | http://ghc.haskell.org/trac/ghc/changeset/4afbacc299d843eb632df1f71eee2
> | c91746c85a8/testsuite
> |
> | >---------------------------------------------------------------
> |
> | commit 4afbacc299d843eb632df1f71eee2c91746c85a8
> | Author: Joachim Breitner <mail at joachim-breitner.de>
> | Date:   Thu Nov 28 12:36:15 2013 +0000
> |
> |     Update some perf results for master
> |
> |
> | >---------------------------------------------------------------
> |
> | 4afbacc299d843eb632df1f71eee2c91746c85a8
> |  tests/perf/compiler/all.T    |   11 ++++++-----
> |  tests/perf/space_leaks/all.T |    2 +-
> |  2 files changed, 7 insertions(+), 6 deletions(-)
> |
> | diff --git a/tests/perf/compiler/all.T b/tests/perf/compiler/all.T
> | index a580bf9..759b5d0 100644
> | --- a/tests/perf/compiler/all.T
> | +++ b/tests/perf/compiler/all.T
> | @@ -344,10 +344,11 @@ test('T5837',
> |  test('T6048',
> |       [ only_ways(['optasm']),
> |        compiler_stats_num_field('bytes allocated',
> | -          [(wordsize(32), 48887164, 10),
> | -            # prev:       38000000 (x86/Linux)
> | -            # 2012-10-08: 48887164 (x86/Linux)
> | -           (wordsize(64), 97247032, 10)])
> | -             # 18/09/2012 97247032 amd64/Linux
> | +          [(wordsize(32),   48887164, 10),
> | +            # prev:         38000000 (x86/Linux)
> | +            # 2012-10-08:   48887164 (x86/Linux)
> | +           (wordsize(64),  237077056, 10)])
> | +             # 18/09/2012   97247032 amd64/Linux
> | +     # 2013-11-26: 237077056 amd64/Linux
> |        ],
> |        compile,[''])
> | diff --git a/tests/perf/space_leaks/all.T
> | b/tests/perf/space_leaks/all.T
> | index a1fd641..ac60c8f 100644
> | --- a/tests/perf/space_leaks/all.T
> | +++ b/tests/perf/space_leaks/all.T
> | @@ -6,7 +6,7 @@ test('space_leak_001',
> |       #           5 (x86/Linux)
> |       [stats_num_field('peak_megabytes_allocated', (4, 1)),
> |        stats_num_field('max_bytes_used',
> | -          [(wordsize(64), 440000, 10),
> | +          [(wordsize(64), 440000, 15),
> |                          # 440224 (amd64/Linux)
> |                          # 417016 (x86/OS X)
> |                          # 415672 (x86/Windows)
> |
> | _______________________________________________
> | ghc-commits mailing list
> | ghc-commits at haskell.org
> | http://www.haskell.org/mailman/listinfo/ghc-commits
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>


Reply | Threaded
Open this post in threaded view
|

[commit: testsuite] master: Update some perf results for master (4afbacc)

Simon Marlow-7
In reply to this post by Joachim Breitner-2
On 28/11/13 13:20, Joachim Breitner wrote:
> Hi,
>
> Am Donnerstag, den 28.11.2013, 12:42 +0000 schrieb Simon Peyton-Jones:
>> Wait. Are you really increasing "bytes allocated" from 97M to 237M?
>> That's a big change, and in an important number.  Do you know why?
>
> no; but this is master, not my branch, and it seems to be a test not run
> by validate (which calls "make -C testsuite fast"), so I have to assume
> it could be anything in the recent past.

But T6048 *is* run by validate.

Cheers,
Simon


Reply | Threaded
Open this post in threaded view
|

[commit: testsuite] master: Update some perf results for master (4afbacc)

Joachim Breitner-2
In reply to this post by Simon Marlow-7
Hi,

Am Donnerstag, den 28.11.2013, 13:36 +0000 schrieb Simon Marlow:
> update the test so that GHC doesn't regress again)
>      Expected    bytes allocated: 237077056 +/-10%
>      Lower bound bytes allocated: 213369350
>      Upper bound bytes allocated: 260784762
>      Actual      bytes allocated: 101150736
>
> Joachim: if you're building with DEBUG, that will adversely affect the
> performance tests, so you probably shouldn't run those.

ah, ok, sorry, that?s it. My bad.

With these you mean perf/compiler, but probably not all perf tests,
right? Could we mark those tests as with
        when(compiler_debugged(),skip())
then? At the risk of repeating my self, and especially with a ever
growing number of contributors to GHC, especially new ones, the test
should should give reproducible and correct results.


Greetings,
Joachim

--
Joachim ?nomeata? Breitner
  mail at joachim-breitner.de ? http://www.joachim-breitner.de/
  Jabber: nomeata at joachim-breitner.de  ? GPG-Key: 0x4743206C
  Debian Developer: nomeata at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131128/85380a7c/attachment.sig>

Reply | Threaded
Open this post in threaded view
|

[commit: testsuite] master: Update some perf results for master (4afbacc)

Simon Marlow-7
On 28/11/13 13:48, Joachim Breitner wrote:

> Hi,
>
> Am Donnerstag, den 28.11.2013, 13:36 +0000 schrieb Simon Marlow:
>> update the test so that GHC doesn't regress again)
>>       Expected    bytes allocated: 237077056 +/-10%
>>       Lower bound bytes allocated: 213369350
>>       Upper bound bytes allocated: 260784762
>>       Actual      bytes allocated: 101150736
>>
>> Joachim: if you're building with DEBUG, that will adversely affect the
>> performance tests, so you probably shouldn't run those.
>
> ah, ok, sorry, that?s it. My bad.
>
> With these you mean perf/compiler, but probably not all perf tests,
> right? Could we mark those tests as with
>          when(compiler_debugged(),skip())
> then?

Yes, that would be good.

> At the risk of repeating my self, and especially with a ever
> growing number of contributors to GHC, especially new ones, the test
> should should give reproducible and correct results.

Of course!  If you find cases where that's not true, please go ahead and
fix them.

Cheers,
Simon


Reply | Threaded
Open this post in threaded view
|

[commit: testsuite] master: Update some perf results for master (4afbacc)

Joachim Breitner-2
Hi,

Am Donnerstag, den 28.11.2013, 14:11 +0000 schrieb Simon Marlow:
> > With these you mean perf/compiler, but probably not all perf tests,
> > right? Could we mark those tests as with
> >          when(compiler_debugged(),skip())
> > then?
>
> Yes, that would be good.

Done in the test runner for all tests with compiler_stats_num_field,
and mentioned on
https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Adding
in the documentation for compiler_stats_num_field.

Greetings,
Joachim

--
Joachim ?nomeata? Breitner
  mail at joachim-breitner.de ? http://www.joachim-breitner.de/
  Jabber: nomeata at joachim-breitner.de  ? GPG-Key: 0x4743206C
  Debian Developer: nomeata at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131128/29c8d7ce/attachment.sig>