Validation Failures on aarch64

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Validation Failures on aarch64

Travis Whitaker
Hello GHC devs,

When attempting to validate a patch on aarch64, it seems there are a large number of validation failures:

SUMMARY for test run started at Mon Apr  8 07:19:05 2019 UTC
 0:15:35 spent to go through
    6890 total tests, which gave rise to
   17169 test cases, of which
   10018 were skipped
      41 had missing libraries
    3151 expected passes
     150 expected failures
      11 caused framework failures
       0 caused framework warnings
       0 unexpected passes
    3798 unexpected failures
       0 unexpected stat failures

The failures seem consistent on recent-ish master, specifically the neighborhood of 6113d0d4540af7853c71e9f42a41c3b0bab386fd. Is this to be expected?

Thanks,

Travis Whitaker

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Validation Failures on aarch64

Matthew Pickering
What kinds of tests are failing? Have they always failed?

On Mon, Apr 8, 2019 at 8:55 PM Travis Whitaker <[hidden email]> wrote:

>
> Hello GHC devs,
>
> When attempting to validate a patch on aarch64, it seems there are a large number of validation failures:
>
> SUMMARY for test run started at Mon Apr  8 07:19:05 2019 UTC
>  0:15:35 spent to go through
>     6890 total tests, which gave rise to
>    17169 test cases, of which
>    10018 were skipped
>       41 had missing libraries
>     3151 expected passes
>      150 expected failures
>       11 caused framework failures
>        0 caused framework warnings
>        0 unexpected passes
>     3798 unexpected failures
>        0 unexpected stat failures
>
> The failures seem consistent on recent-ish master, specifically the neighborhood of 6113d0d4540af7853c71e9f42a41c3b0bab386fd. Is this to be expected?
>
> Thanks,
>
> Travis Whitaker
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Validation Failures on aarch64

Phyx
In reply to this post by Travis Whitaker
Just a wild guess, but considering the time it's taken to run through the testsuite you're running it on a reasonable AArch64 machine.
You may be hitting the weak memory ordering bugs https://gitlab.haskell.org/ghc/ghc/issues/15449 , though I haven't followed the ticket very closely...

On Mon, Apr 8, 2019 at 8:55 PM Travis Whitaker <[hidden email]> wrote:
Hello GHC devs,

When attempting to validate a patch on aarch64, it seems there are a large number of validation failures:

SUMMARY for test run started at Mon Apr  8 07:19:05 2019 UTC
 0:15:35 spent to go through
    6890 total tests, which gave rise to
   17169 test cases, of which
   10018 were skipped
      41 had missing libraries
    3151 expected passes
     150 expected failures
      11 caused framework failures
       0 caused framework warnings
       0 unexpected passes
    3798 unexpected failures
       0 unexpected stat failures

The failures seem consistent on recent-ish master, specifically the neighborhood of 6113d0d4540af7853c71e9f42a41c3b0bab386fd. Is this to be expected?

Thanks,

Travis Whitaker
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Validation Failures on aarch64

Travis Whitaker
The patch I’m validating is actually to fix those memory barrier issues.

It turns out these failures were all due to GHC complaining about the LLVM version on stderr; sorry for rubber ducking the list!

On Mon, Apr 8, 2019 at 1:26 PM Phyx <[hidden email]> wrote:
Just a wild guess, but considering the time it's taken to run through the testsuite you're running it on a reasonable AArch64 machine.
You may be hitting the weak memory ordering bugs https://gitlab.haskell.org/ghc/ghc/issues/15449 , though I haven't followed the ticket very closely...

On Mon, Apr 8, 2019 at 8:55 PM Travis Whitaker <[hidden email]> wrote:
Hello GHC devs,

When attempting to validate a patch on aarch64, it seems there are a large number of validation failures:

SUMMARY for test run started at Mon Apr  8 07:19:05 2019 UTC
 0:15:35 spent to go through
    6890 total tests, which gave rise to
   17169 test cases, of which
   10018 were skipped
      41 had missing libraries
    3151 expected passes
     150 expected failures
      11 caused framework failures
       0 caused framework warnings
       0 unexpected passes
    3798 unexpected failures
       0 unexpected stat failures

The failures seem consistent on recent-ish master, specifically the neighborhood of 6113d0d4540af7853c71e9f42a41c3b0bab386fd. Is this to be expected?

Thanks,

Travis Whitaker
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Validation Failures on aarch64

Ben Gamari-2
In reply to this post by Travis Whitaker
Indeed this is much worse than it used to be and quite unexpected. Last I knee there were only 100 or so failures, mostly external-interpeter related. Any investigation you could do would be greatly appreciated. I would be happy to offer access to hardware if this helps.

Cheers,

- Ben


On April 8, 2019 3:55:18 PM EDT, Travis Whitaker <[hidden email]> wrote:
Hello GHC devs,

When attempting to validate a patch on aarch64, it seems there are a large number of validation failures:

SUMMARY for test run started at Mon Apr  8 07:19:05 2019 UTC
 0:15:35 spent to go through
    6890 total tests, which gave rise to
   17169 test cases, of which
   10018 were skipped
      41 had missing libraries
    3151 expected passes
     150 expected failures
      11 caused framework failures
       0 caused framework warnings
       0 unexpected passes
    3798 unexpected failures
       0 unexpected stat failures

The failures seem consistent on recent-ish master, specifically the neighborhood of 6113d0d4540af7853c71e9f42a41c3b0bab386fd. Is this to be expected?

Thanks,

Travis Whitaker

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Validation Failures on aarch64

Ben Gamari-2
Ben Gamari <[hidden email]> writes:

> Indeed this is much worse than it used to be and quite unexpected.
> Last I knee there were only 100 or so failures, mostly
> external-interpeter related. Any investigation you could do would be
> greatly appreciated. I would be happy to offer access to hardware if
> this helps.
>
Whoops. It looks like I was a bit late to the party here. Thanks for
pinning down the issue, Travis.

That being said, it seems that our CI job is failing due to missing
bzip2. Fixed in
https://gitlab.haskell.org/ghc/ci-images/merge_requests/10.

It would be great if we could fix/mark-as-broken enough tests that we
can remove allow_failure from the aarch64 job.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment