T13615 fragile?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

T13615 fragile?

Richard Eisenberg-5
Hi devs,

https://gitlab.haskell.org/rae/ghc/-/jobs/173504 tells me that T13615 failed -- but only in DEBUG mode. My patch is nowhere near this code. Is this a new fragile test?

The number of fragile tests seems to be growing quickly. And once a test is labeled fragile, we're essentially no longer tracking the test, if I understand correctly. So that's a bit unfortunate.

Thanks,
Richard

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: T13615 fragile?

Alp Mestanogullari-2

For what it's worth, I found two recent tickets documenting failures for that very same test: #16897 and #17269.

On 09/10/2019 10:45, Richard Eisenberg wrote:
Hi devs,

https://gitlab.haskell.org/rae/ghc/-/jobs/173504 tells me that T13615 failed -- but only in DEBUG mode. My patch is nowhere near this code. Is this a new fragile test?

The number of fragile tests seems to be growing quickly. And once a test is labeled fragile, we're essentially no longer tracking the test, if I understand correctly. So that's a bit unfortunate.

Thanks,
Richard

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-- 
Alp Mestanogullari, Haskell Consultant
Well-Typed LLP, https://www.well-typed.com/

Registered in England and Wales, OC335890
118 Wymering Mansions, Wymering Road, London, W9 2NF, England

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: T13615 fragile?

Ben Gamari-3
In reply to this post by Richard Eisenberg-5
Unfortunately T13615 is indeed known to be fragile. This is tracked as #17269, ss you discovered. This is especially scary given that the test tests some rather subtle RTS logic. It's been on my list of things to investigate but unfortunately I've not been able to reproduce the failure locally so progress has been slow.

Given that it seems to fail fairly reliably on CI in some cases we probably ought to mark it as fragile.

Note that it's not quite true that fragile test outcomes are no longer tracked. We maintain a database of test outcomes from CI runs which includes both passing and failing ribs of fragile tests. I've written about this infrastructure here [1].

Cheers,

- Ben


[1] https://www.haskell.org/ghc/blog/20190708-testsuite-work.html




On October 9, 2019 4:45:57 AM EDT, Richard Eisenberg <[hidden email]> wrote:
Hi devs,

https://gitlab.haskell.org/rae/ghc/-/jobs/173504 tells me that T13615 failed -- but only in DEBUG mode. My patch is nowhere near this code. Is this a new fragile test?

The number of fragile tests seems to be growing quickly. And once a test is labeled fragile, we're essentially no longer tracking the test, if I understand correctly. So that's a bit unfortunate.

Thanks,
Richard

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs