Salam Muhaimin,

So to answer my own question, the current practice seems to be to just

eyeball

> Geometric Mean -0.0% -0.0% -0.3% -0.1% +0.1%

and if the numbers are within historical epsilons of 0, that means no

change.

For a moment, I thought this was some erratically behaving VM.

The compile times are across-the-board lower, nice!

-- Kim-Ee

On Sat, Sep 28, 2013 at 4:39 AM, Muhaimin Ahsan <leroux at fezrev.com> wrote:

> Kim-Ee,

>

> The updated fib-analyse report from a few hours ago is posted here:

>

https://gist.github.com/leroux/6725810#file-headvordnub-analysis-L2988> Comment (

http://ghc.haskell.org/trac/ghc/ticket/8173#comment:9)

>

> Sorry for the misunderstanding.

>

> Muhaimin

>

> On Sep 27, 2013, at 3:56 PM, Kim-Ee Yeoh <ky3 at atamo.com> wrote:

>

>

> On Fri, Sep 27, 2013 at 8:14 PM, GHC <ghc-devs at haskell.org> wrote:

>

>> A 5% improvement in compile time is remarkable, if it's true. Great! But

>> I'm always worried about the noise in compile times measured in seconds.

>>

>

> Does anyone else think the noise in runtimes is alarming considering that

> the following is the fib-analysis of /binary-identical/ programs?

>

> > Min -0.1% -0.0% -25.4% -32.2% -1.3%

> > Max +0.1% +0.0% +19.0% +22.2% +10.0%

>

> Shouldn't we find an explanation for this before believing the compile

> time numbers? What would cause these wide swings on the benchmarking

> machine?

>

> p.s. For the record: Should do more rigorous statistical testing instead

> of naive percentages, yes?

>

> -- Kim-Ee

> _______________________________________________

> ghc-tickets mailing list

> ghc-tickets at haskell.org

>

http://www.haskell.org/mailman/listinfo/ghc-tickets>

>

>

> _______________________________________________

> ghc-devs mailing list

> ghc-devs at haskell.org

>

http://www.haskell.org/mailman/listinfo/ghc-devs>

>

-------------- next part --------------

An HTML attachment was scrubbed...

URL: <

http://www.haskell.org/pipermail/ghc-devs/attachments/20130928/ad70d704/attachment.htm>