testsuite failures in DEBUG

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

testsuite failures in DEBUG

Richard Eisenberg-4
Hi devs,

It seems our CI infrastructure has become too good. My workflow normally involves working with a DEBUG compiler, and then occasionally running the whole testsuite. I don't validate from scratch mid-patch, but it's good to know if I'm failing tests. With HEAD, I'm getting quite a few testsuite failures, and they all seem to be coming from ASSERTs, and they're all problems that I'm pretty sure I didn't cause. My full list of failures (I'm on Mac) is below, but after looking into 5-6 of them and finding only stuff that isn't my fault, I've given up.

So: would it be possible to have our CI infrastructure validate DEBUG mode as well? Then it would be easy to spot where these failures are from.

   backpack/cabal/bkpcabal02/bkpcabal02.run          bkpcabal02 [bad stderr] (normal)
   backpack/cabal/bkpcabal04/bkpcabal04.run          bkpcabal04 [bad stderr] (normal)
   backpack/cabal/bkpcabal03/bkpcabal03.run          bkpcabal03 [bad stderr] (normal)
   backpack/cabal/bkpcabal05/bkpcabal05.run          bkpcabal05 [bad stderr] (normal)
   backpack/cabal/T14304/T14304.run                  T14304 [bad stderr] (normal)
   backpack/cabal/bkpcabal06/bkpcabal06.run          bkpcabal06 [bad stderr] (normal)
   backpack/cabal/bkpcabal07/bkpcabal07.run          bkpcabal07 [bad stderr] (normal)
   backpack/cabal/bkpcabal01/bkpcabal01.run          bkpcabal01 [bad stderr] (normal)
   cabal/cabal04/cabal04.run                         cabal04 [bad stderr] (normal)
   cabal/T12733/T12733.run                           T12733 [bad stderr] (normal)
   cabal/cabal01/cabal01.run                         cabal01 [bad stderr] (normal)
   cabal/cabal03/cabal03.run                         cabal03 [bad stderr] (normal)
   cabal/cabal09/cabal09.run                         cabal09 [bad stderr] (normal)
   cabal/cabal08/cabal08.run                         cabal08 [bad stderr] (normal)
   cabal/cabal05/cabal05.run                         cabal05 [bad stderr] (normal)
   cabal/cabal06/cabal06.run                         cabal06 [bad stderr] (normal)
   dependent/should_compile/T12442.run               T12442 [exit code non-0] (normal)
   dependent/should_compile/T12176.run               T12176 [exit code non-0] (normal)
   driver/T3007/T3007.run                            T3007 [bad stderr] (normal)
   gadt/T12087.run                                   T12087 [stderr mismatch] (normal)
   ghci/scripts/ghci063.run                          ghci063 [bad stderr] (ghci)
   indexed-types/should_fail/T13877.run              T13877 [stderr mismatch] (normal)
   parser/should_fail/NumericUnderscoresFail0.run    NumericUnderscoresFail0 [stderr mismatch] (normal)
   parser/should_fail/NumericUnderscoresFail1.run    NumericUnderscoresFail1 [stderr mismatch] (normal)
   patsyn/should_compile/T13350/T13350.run           T13350 [bad stderr] (normal)
   pmcheck/should_compile/T11195.run                 T11195 [exit code non-0] (normal)
   polykinds/T14174a.run                             T14174a [exit code non-0] (normal)
   printer/T13050p.run                               T13050p [bad exit code] (normal)
   safeHaskell/check/pkg01/safePkg01.run             safePkg01 [bad stderr] (normal)
   simplCore/should_compile/T13025.run               T13025 [bad stderr] (normal)
   simplCore/should_compile/T13410.run               T13410 [exit code non-0] (normal)
   typecheck/T13168/T13168.run                       T13168 [bad stderr] (normal)
   typecheck/bug1465/bug1465.run                     bug1465 [bad stderr] (normal)
   typecheck/should_compile/holes2.run               holes2 [exit code non-0] (normal)
   typecheck/should_compile/valid_substitutions.run  valid_substitutions [exit code non-0] (normal)
   typecheck/should_compile/holes3.run               holes3 [stderr mismatch] (normal)
   typecheck/should_compile/holes.run                holes [stderr mismatch] (normal)
   typecheck/should_compile/T13050.run               T13050 [exit code non-0] (normal)
   typecheck/should_compile/T13822.run               T13822 [exit code non-0] (normal)
   typecheck/should_compile/T14590.run               T14590 [exit code non-0] (normal)
   typecheck/should_compile/T13032.run               T13032 [stderr mismatch] (normal)
   typecheck/should_fail/T7175.run                   T7175 [stderr mismatch] (normal)

Thanks!
Richard

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: testsuite failures in DEBUG

Joachim Breitner-2
Hi,

Am Freitag, den 26.01.2018, 22:53 -0500 schrieb Richard Eisenberg:
> So: would it be possible to have our CI infrastructure validate DEBUG mode as well? Then it would be easy to spot where these failures are from.

unhelpful comment: Travis used to validate with DEBUG, but it no longer
runs the test suite for build time reason. So plausibly we could add a
-DEBUG variant to CircleCI?

Joachim
--
Joachim Breitner
  [hidden email]
  http://www.joachim-breitner.de/

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: testsuite failures in DEBUG

Joachim Breitner-2
Hi,

Am Freitag, den 26.01.2018, 23:23 -0500 schrieb Joachim Breitner:
> Am Freitag, den 26.01.2018, 22:53 -0500 schrieb Richard Eisenberg:
> > So: would it be possible to have our CI infrastructure validate DEBUG mode as well? Then it would be easy to spot where these failures are from.
>
> unhelpful comment: Travis used to validate with DEBUG, but it no longer
> runs the test suite for build time reason. So plausibly we could add a
> -DEBUG variant to CircleCI?

I now also stumble about the inability to properly test a build built
with -DDEBUG.

So let me rephrase: Could we please add a -DDEBUG variant to CircleCI?

(And then fix the failing tests, or at least mark them as known-to-be-
failing on a debugged GHC)

Cheers,
Joachim

--
Joachim Breitner
  [hidden email]
  http://www.joachim-breitner.de/

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: testsuite failures in DEBUG

Ben Gamari-2
Joachim Breitner <[hidden email]> writes:

> Hi,
>
> Am Freitag, den 26.01.2018, 23:23 -0500 schrieb Joachim Breitner:
>> Am Freitag, den 26.01.2018, 22:53 -0500 schrieb Richard Eisenberg:
>> > So: would it be possible to have our CI infrastructure validate
>> > DEBUG mode as well? Then it would be easy to spot where these
>> > failures are from.
>>
>> unhelpful comment: Travis used to validate with DEBUG, but it no longer
>> runs the test suite for build time reason. So plausibly we could add a
>> -DEBUG variant to CircleCI?
>
> I now also stumble about the inability to properly test a build built
> with -DDEBUG.
>
> So let me rephrase: Could we please add a -DDEBUG variant to CircleCI?
>
See D4411.

> (And then fix the failing tests, or at least mark them as known-to-be-
> failing on a debugged GHC)

I'll add this to my list.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment