nofib output difference

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

nofib output difference

Joachim Breitner-2
Hi,

currently, perf.haskell.org has stopped because recent commits fail to
build with:

expected stdout not matched by reality
--- fft2.stdout7 2018-11-28 12:00:24.126015465 -0500
+++ /tmp/runtest6392.1 2018-11-28 12:59:59.329423367 -0500
@@ -1,3 +1,3 @@
-result1 = 2.59635799135966e-12
-result2 = 2.59635799135966e-12
-result3 = 4.8279900966008427e-8
+result1 = 2.649891777856828e-12
+result2 = 2.649891777856828e-12
+result3 = 4.830900479646516e-8
make[2]: *** [../../mk/target.mk:102: runtests] Error 1

do others observe this as well? This is worrying, because there has
been no change to nofib since it last worked.

Can we add a check of the functional correctness of the nofib programs
to the CI somehow?

Cheers,
Joachim

--
Joachim Breitner
  [hidden email]
  http://www.joachim-breitner.de/


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: nofib output difference

Ben Gamari-3
Joachim Breitner <[hidden email]> writes:

> Hi,
>
> currently, perf.haskell.org has stopped because recent commits fail to
> build with:
>
> expected stdout not matched by reality
> --- fft2.stdout7 2018-11-28 12:00:24.126015465 -0500
> +++ /tmp/runtest6392.1 2018-11-28 12:59:59.329423367 -0500
> @@ -1,3 +1,3 @@
> -result1 = 2.59635799135966e-12
> -result2 = 2.59635799135966e-12
> -result3 = 4.8279900966008427e-8
> +result1 = 2.649891777856828e-12
> +result2 = 2.649891777856828e-12
> +result3 = 4.830900479646516e-8
> make[2]: *** [../../mk/target.mk:102: runtests] Error 1
>
> do others observe this as well? This is worrying, because there has
> been no change to nofib since it last worked.
>
> Can we add a check of the functional correctness of the nofib programs
> to the CI somehow?
>
Yikes, this is indeed somewhat worrying. I see the same result as you
for result1 and result2 but a slightly different number for result3.

Looking at this particular test it appears to be quite numerically
unstable. I count over ten distinct sets of .stdout files in
spectral/fft2, some of which match what the results I observe quite
closely.

Overall we should indeed make sure these get tested in CI.

Do you know how long ago this change occurred?

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: nofib output difference

Ben Gamari-2
In reply to this post by Joachim Breitner-2
Joachim Breitner <[hidden email]> writes:

> Hi,
>
> currently, perf.haskell.org has stopped because recent commits fail to
> build with:
>
I have opened #15972 to track this.

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: nofib output difference

Joachim Breitner-2
In reply to this post by Ben Gamari-3
Hi,

Am Donnerstag, den 29.11.2018, 10:13 -0500 schrieb Ben Gamari:

> Yikes, this is indeed somewhat worrying. I see the same result as you
> for result1 and result2 but a slightly different number for result3.
>
> Looking at this particular test it appears to be quite numerically
> unstable. I count over ten distinct sets of .stdout files in
> spectral/fft2, some of which match what the results I observe quite
> closely.
>
> Overall we should indeed make sure these get tested in CI.
>
> Do you know how long ago this change occurred?
Not easily, there were other breakages on perf.haskell.org that
probably shadowed this.

Cheers,
Joachim

--
Joachim Breitner
  [hidden email]
  http://www.joachim-breitner.de/


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: nofib output difference

Joachim Breitner-2
Hi,

now it fail building nofib with a different error:

==nofib== compress: time to compile Uncompress follows...
/home/nomeata/logs/ghc-tmp-REV/inplace/bin/ghc-stage2 -O2 -Wno-tabs -Rghc-timing -H32m -hisuf hi -rtsopts -c Uncompress.hs -o Uncompress.o

Decode.hs:25:1: error:
Main.hs:16:1: error:

Encode.hs:9:1: error:Uncompress.hs:15:1: error:
    Could not find module ‘Defaults’
    Perhaps you meant TcDefaults (needs flag -package-key ghc-8.7)
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
25 | import Defaults
   | ^^^^^^^^^^^^^^^

Decode.hs:26:1: error:
    Could not find module ‘Defaults’
    Perhaps you meant TcDefaults (needs flag -package-key ghc-8.7)
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
16 | import Defaults
   | ^^^^^^^^^^^^^^^

Main.hs:17:1: error:
    Could not find module ‘Defaults’
    Perhaps you meant TcDefaults (needs flag -package-key ghc-8.7)
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
  |
9 | import Defaults
  | ^^^^^^^^^^^^^^^

Encode.hs:10:1: error:
    Could not find module ‘Defaults’
    Perhaps you meant TcDefaults (needs flag -package-key ghc-8.7)
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
15 | import Defaults
   | ^^^^^^^^^^^^^^^

Uncompress.hs:16:1: error:
    Could not find module ‘BinConv’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
26 | import BinConv
   | ^^^^^^^^^^^^^^

    Could not find module ‘BinConv’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
17 | import BinConv    -- binary conversion routines
   | ^^^^^^^^^^^^^^

Main.hs:18:1: error:
    Could not find module ‘PTTrees’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
10 | import PTTrees
   | ^^^^^^^^^^^^^^

    Could not find module ‘BinConv’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
16 | import BinConv    -- binary conversion routines
   | ^^^^^^^^^^^^^^

Uncompress.hs:17:1: error:
    Could not find module ‘Encode’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
18 | import Encode     -- coding routine
   | ^^^^^^^^^^^^^

    Could not find module ‘Decode’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
17 | import Decode     -- decoding routines
   | ^^^^^^^^^^^^^

--
Joachim “nomeata” Breitner
  [hidden email]
  https://www.joachim-breitner.de/


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: nofib output difference

Andreas Klebinger
In reply to this post by Joachim Breitner-2
I fear that one is my fault.

This should fix it: https://phabricator.haskell.org/D5426

I did not want to add a large binary file to the repo so instead
compress generated the out file during boot.
Which used to work well on my box.

However make boot also creates the dependency files that guarantee the
proper build order. Turns out sometimes
we end up trying to build the compress executable before building the
dependency file, so the build fails.

I've just gave in to storing the output file in the repo now.


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs