questions about validating in the presence of known failures

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

questions about validating in the presence of known failures

Nicolas Frisby
[You can generalize this email from OS X to other architectures/operating
systems.]

Question 1: Is a list of "unexpecteds expected on OS X" maintained
somewhere? The comments on #5757 seemed like a start, but they're 1.5 years
old.

The Trac "Custom Query" for OS X isn't helpful for my actual question of
"can I ignore these wrt to validating my patch?".

A wiki page collating tickets for any "known unexpecteds" for OS X (and
other OSs) would be helpful.

Question 2: Can we add another bit to unexpected results marking them as
known/unknown?

That would yield an immediacy for the validation workflow in the cases
where we have long-standing known bugs, eg on various platforms. I think
#5757 is on the right path, but marking a test that should pass as an
expected failure seems disappointing. Is there a traditional workflow for
handling this?

Question 3: What's the status on the build bot farm?

Thanks for your time, everyone.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130502/499ce2cc/attachment.htm>

Reply | Threaded
Open this post in threaded view
|

questions about validating in the presence of known failures

Ian Lynagh-2
On Thu, May 02, 2013 at 10:23:26AM -0500, Nicolas Frisby wrote:
>
> Question 2: Can we add another bit to unexpected results marking them as
> known/unknown?

Unexpected results are all unknown. If they're known then they're
expected results.

I've just written this on how to deal with validate failures not caused
by local patches:

http://hackage.haskell.org/trac/ghc/wiki/TestingPatches#ValidatehasfailingtestswithoutanylocalpatcheswhatdoIdo

> Question 3: What's the status on the build bot farm?

I hope to do some work on this soon, e.g. add a way for build results to
be uploaded to the server by the builders. However, I've been spending
my time on more pressing matters recently.


Thanks
Ian



Reply | Threaded
Open this post in threaded view
|

questions about validating in the presence of known failures

Nicolas Frisby
Your amendment to the wiki page is very helpful. In particular, it totally
supplants my suggestion about "known unexpecteds".

Thanks.


On Thu, May 2, 2013 at 11:25 AM, Ian Lynagh <ian at well-typed.com> wrote:

> On Thu, May 02, 2013 at 10:23:26AM -0500, Nicolas Frisby wrote:
> >
> > Question 2: Can we add another bit to unexpected results marking them as
> > known/unknown?
>
> Unexpected results are all unknown. If they're known then they're
> expected results.
>
> I've just written this on how to deal with validate failures not caused
> by local patches:
>
>
> http://hackage.haskell.org/trac/ghc/wiki/TestingPatches#ValidatehasfailingtestswithoutanylocalpatcheswhatdoIdo
>
> > Question 3: What's the status on the build bot farm?
>
> I hope to do some work on this soon, e.g. add a way for build results to
> be uploaded to the server by the builders. However, I've been spending
> my time on more pressing matters recently.
>
>
> Thanks
> Ian
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130502/dc697499/attachment.htm>