Any ways to test a GHC build against large set of packages (including test suites)?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Any ways to test a GHC build against large set of packages (including test suites)?

Ömer Sinan Ağacan
Hi,

I'd like to test some GHC builds + some compile and runtime flag combinations
against a large set of packages by building them and running test suites. For
this I need

- A set of packages that are known to work with latest GHC
- A way to build them and run their test suites (if I could specify compile and
  runtime flags that'd be even better)

I think stackage can serve as (1) but I don't know how to do (2). Can anyone
point me to the right direction? I vaguely remember some nix-based solution for
this that was being discussed on the IRC channel, but can't recall any details.

Thanks,

Ömer
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Ben Gamari-2
Ömer Sinan Ağacan <[hidden email]> writes:

> Hi,
>
> I'd like to test some GHC builds + some compile and runtime flag combinations
> against a large set of packages by building them and running test suites. For
> this I need
>
> - A set of packages that are known to work with latest GHC
> - A way to build them and run their test suites (if I could specify compile and
>   runtime flags that'd be even better)
>
> I think stackage can serve as (1) but I don't know how to do (2). Can anyone
> point me to the right direction? I vaguely remember some nix-based solution for
> this that was being discussed on the IRC channel, but can't recall any details.
>
You are probably remembering head.hackage [1], which has both nix and
non-nix tooling. A few of us have been trying to bring it
up-to-date with GHC 8.6. I suspect the changes in HEAD are small enough
that it will work with newer compilers as well.

Cheers,

- Ben

[1] https://github.com/hvr/head.hackage


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Manuel M T Chakravarty-4
In reply to this post by Ömer Sinan Ağacan
Hi Ömer,

This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at

  https://github.com/tweag/stackage-head

and also the blog post from when the first version went live:

  https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html

Cheers,
Manuel

> Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan <[hidden email]>:
>
> Hi,
>
> I'd like to test some GHC builds + some compile and runtime flag combinations
> against a large set of packages by building them and running test suites. For
> this I need
>
> - A set of packages that are known to work with latest GHC
> - A way to build them and run their test suites (if I could specify compile and
>  runtime flags that'd be even better)
>
> I think stackage can serve as (1) but I don't know how to do (2). Can anyone
> point me to the right direction? I vaguely remember some nix-based solution for
> this that was being discussed on the IRC channel, but can't recall any details.
>
> Thanks,
>
> Ömer
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (891 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Ömer Sinan Ağacan
Thanks for both suggestions. I'll try both and see which one works better.

Ömer

Manuel M T Chakravarty <[hidden email]>, 7 Ağu 2018 Sal, 18:15
tarihinde şunu yazdı:

>
> Hi Ömer,
>
> This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at
>
>   https://github.com/tweag/stackage-head
>
> and also the blog post from when the first version went live:
>
>   https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html
>
> Cheers,
> Manuel
>
> > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan <[hidden email]>:
> >
> > Hi,
> >
> > I'd like to test some GHC builds + some compile and runtime flag combinations
> > against a large set of packages by building them and running test suites. For
> > this I need
> >
> > - A set of packages that are known to work with latest GHC
> > - A way to build them and run their test suites (if I could specify compile and
> >  runtime flags that'd be even better)
> >
> > I think stackage can serve as (1) but I don't know how to do (2). Can anyone
> > point me to the right direction? I vaguely remember some nix-based solution for
> > this that was being discussed on the IRC channel, but can't recall any details.
> >
> > Thanks,
> >
> > Ömer
> > _______________________________________________
> > ghc-devs mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Ömer Sinan Ağacan
Hi Manuel,

I'm trying stackage-head. I'm following the steps for the scheduled build in
.circleci/config.yml. So far steps I took:

- Installed ghc-head (from [1]) to ~/ghc-head
- Installed stackage-build-plan, stackage-curator and stackage-head (with
  -fdev) from git repos, using stack.
- export BUILD_PLAN=nightly-2018-07-30 (from config.yml)
- curl https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json
--output metadata.json
- curl https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml
--output $BUILD_PLAN.yaml

Now I'm doing

- ./.local/bin/stackage-head already-seen --target $BUILD_PLAN
--ghc-metadata metadata.json --outdir build-reports

but it's failing with

    The combination of target and commit is new to me

Any ideas what I'm doing wrong?

Thanks

[1]: https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz

Ömer

Ömer Sinan Ağacan <[hidden email]>, 7 Ağu 2018 Sal, 23:28
tarihinde şunu yazdı:

>
> Thanks for both suggestions. I'll try both and see which one works better.
>
> Ömer
>
> Manuel M T Chakravarty <[hidden email]>, 7 Ağu 2018 Sal, 18:15
> tarihinde şunu yazdı:
> >
> > Hi Ömer,
> >
> > This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at
> >
> >   https://github.com/tweag/stackage-head
> >
> > and also the blog post from when the first version went live:
> >
> >   https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html
> >
> > Cheers,
> > Manuel
> >
> > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan <[hidden email]>:
> > >
> > > Hi,
> > >
> > > I'd like to test some GHC builds + some compile and runtime flag combinations
> > > against a large set of packages by building them and running test suites. For
> > > this I need
> > >
> > > - A set of packages that are known to work with latest GHC
> > > - A way to build them and run their test suites (if I could specify compile and
> > >  runtime flags that'd be even better)
> > >
> > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone
> > > point me to the right direction? I vaguely remember some nix-based solution for
> > > this that was being discussed on the IRC channel, but can't recall any details.
> > >
> > > Thanks,
> > >
> > > Ömer
> > > _______________________________________________
> > > ghc-devs mailing list
> > > [hidden email]
> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Ömer Sinan Ağacan
Ah, I now realize that that command is supposed to print that output. I'll
continue following the steps and keep you updated if I get stuck again.

Ömer

Ömer Sinan Ağacan <[hidden email]>, 9 Ağu 2018 Per, 13:20
tarihinde şunu yazdı:

>
> Hi Manuel,
>
> I'm trying stackage-head. I'm following the steps for the scheduled build in
> .circleci/config.yml. So far steps I took:
>
> - Installed ghc-head (from [1]) to ~/ghc-head
> - Installed stackage-build-plan, stackage-curator and stackage-head (with
>   -fdev) from git repos, using stack.
> - export BUILD_PLAN=nightly-2018-07-30 (from config.yml)
> - curl https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json
> --output metadata.json
> - curl https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml
> --output $BUILD_PLAN.yaml
>
> Now I'm doing
>
> - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN
> --ghc-metadata metadata.json --outdir build-reports
>
> but it's failing with
>
>     The combination of target and commit is new to me
>
> Any ideas what I'm doing wrong?
>
> Thanks
>
> [1]: https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz
>
> Ömer
>
> Ömer Sinan Ağacan <[hidden email]>, 7 Ağu 2018 Sal, 23:28
> tarihinde şunu yazdı:
> >
> > Thanks for both suggestions. I'll try both and see which one works better.
> >
> > Ömer
> >
> > Manuel M T Chakravarty <[hidden email]>, 7 Ağu 2018 Sal, 18:15
> > tarihinde şunu yazdı:
> > >
> > > Hi Ömer,
> > >
> > > This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at
> > >
> > >   https://github.com/tweag/stackage-head
> > >
> > > and also the blog post from when the first version went live:
> > >
> > >   https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html
> > >
> > > Cheers,
> > > Manuel
> > >
> > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan <[hidden email]>:
> > > >
> > > > Hi,
> > > >
> > > > I'd like to test some GHC builds + some compile and runtime flag combinations
> > > > against a large set of packages by building them and running test suites. For
> > > > this I need
> > > >
> > > > - A set of packages that are known to work with latest GHC
> > > > - A way to build them and run their test suites (if I could specify compile and
> > > >  runtime flags that'd be even better)
> > > >
> > > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone
> > > > point me to the right direction? I vaguely remember some nix-based solution for
> > > > this that was being discussed on the IRC channel, but can't recall any details.
> > > >
> > > > Thanks,
> > > >
> > > > Ömer
> > > > _______________________________________________
> > > > ghc-devs mailing list
> > > > [hidden email]
> > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > >
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Ömer Sinan Ağacan
Hi,

This is working great, I just generated my first report. One problem is stm-2.4
doesn't compile with GHC HEAD, we need stm-2.5.0.0. But that's not published on
Hackage yet, and latest nightly still uses stm-2.4.5.0. I wonder if there's
anything that can be done about this. Apparently stm blocks 82 packages (I
don't know if that's counting transitively or just packages that are directly
blocked by stm). Any ideas about this?

Ömer

Ömer Sinan Ağacan <[hidden email]>, 9 Ağu 2018 Per, 14:45
tarihinde şunu yazdı:

>
> Ah, I now realize that that command is supposed to print that output. I'll
> continue following the steps and keep you updated if I get stuck again.
>
> Ömer
>
> Ömer Sinan Ağacan <[hidden email]>, 9 Ağu 2018 Per, 13:20
> tarihinde şunu yazdı:
> >
> > Hi Manuel,
> >
> > I'm trying stackage-head. I'm following the steps for the scheduled build in
> > .circleci/config.yml. So far steps I took:
> >
> > - Installed ghc-head (from [1]) to ~/ghc-head
> > - Installed stackage-build-plan, stackage-curator and stackage-head (with
> >   -fdev) from git repos, using stack.
> > - export BUILD_PLAN=nightly-2018-07-30 (from config.yml)
> > - curl https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json
> > --output metadata.json
> > - curl https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml
> > --output $BUILD_PLAN.yaml
> >
> > Now I'm doing
> >
> > - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN
> > --ghc-metadata metadata.json --outdir build-reports
> >
> > but it's failing with
> >
> >     The combination of target and commit is new to me
> >
> > Any ideas what I'm doing wrong?
> >
> > Thanks
> >
> > [1]: https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz
> >
> > Ömer
> >
> > Ömer Sinan Ağacan <[hidden email]>, 7 Ağu 2018 Sal, 23:28
> > tarihinde şunu yazdı:
> > >
> > > Thanks for both suggestions. I'll try both and see which one works better.
> > >
> > > Ömer
> > >
> > > Manuel M T Chakravarty <[hidden email]>, 7 Ağu 2018 Sal, 18:15
> > > tarihinde şunu yazdı:
> > > >
> > > > Hi Ömer,
> > > >
> > > > This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at
> > > >
> > > >   https://github.com/tweag/stackage-head
> > > >
> > > > and also the blog post from when the first version went live:
> > > >
> > > >   https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html
> > > >
> > > > Cheers,
> > > > Manuel
> > > >
> > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan <[hidden email]>:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I'd like to test some GHC builds + some compile and runtime flag combinations
> > > > > against a large set of packages by building them and running test suites. For
> > > > > this I need
> > > > >
> > > > > - A set of packages that are known to work with latest GHC
> > > > > - A way to build them and run their test suites (if I could specify compile and
> > > > >  runtime flags that'd be even better)
> > > > >
> > > > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone
> > > > > point me to the right direction? I vaguely remember some nix-based solution for
> > > > > this that was being discussed on the IRC channel, but can't recall any details.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Ömer
> > > > > _______________________________________________
> > > > > ghc-devs mailing list
> > > > > [hidden email]
> > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > > >
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Artem Pelenitsyn
Hello Ömer,

Just a week ago I asked very similar question: how to install the test suite dependencies after breaking changes in GHC:


But no one replied :(

The task seems to be not solvable even if an affected package (stm in your case and primitive in mine) has already adopted in its master the breaking change but has no corresponding release on Hackage (which will always be the case, because how would you release depending on base which hasn't been released yet).

Some package managers in other languages (Julia's Pkg, go install) allow you to depend directly on master branch of a package if you wish to. I wonder if there is a road here for cabal to take in order to support this kind of unsafe installs.

--
Best wishes,
Artem

On Fri, 10 Aug 2018, 11:40 Ömer Sinan Ağacan, <[hidden email]> wrote:
Hi,

This is working great, I just generated my first report. One problem is stm-2.4
doesn't compile with GHC HEAD, we need stm-2.5.0.0. But that's not published on
Hackage yet, and latest nightly still uses stm-2.4.5.0. I wonder if there's
anything that can be done about this. Apparently stm blocks 82 packages (I
don't know if that's counting transitively or just packages that are directly
blocked by stm). Any ideas about this?

Ömer

Ömer Sinan Ağacan <[hidden email]>, 9 Ağu 2018 Per, 14:45
tarihinde şunu yazdı:
>
> Ah, I now realize that that command is supposed to print that output. I'll
> continue following the steps and keep you updated if I get stuck again.
>
> Ömer
>
> Ömer Sinan Ağacan <[hidden email]>, 9 Ağu 2018 Per, 13:20
> tarihinde şunu yazdı:
> >
> > Hi Manuel,
> >
> > I'm trying stackage-head. I'm following the steps for the scheduled build in
> > .circleci/config.yml. So far steps I took:
> >
> > - Installed ghc-head (from [1]) to ~/ghc-head
> > - Installed stackage-build-plan, stackage-curator and stackage-head (with
> >   -fdev) from git repos, using stack.
> > - export BUILD_PLAN=nightly-2018-07-30 (from config.yml)
> > - curl https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json
> > --output metadata.json
> > - curl https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml
> > --output $BUILD_PLAN.yaml
> >
> > Now I'm doing
> >
> > - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN
> > --ghc-metadata metadata.json --outdir build-reports
> >
> > but it's failing with
> >
> >     The combination of target and commit is new to me
> >
> > Any ideas what I'm doing wrong?
> >
> > Thanks
> >
> > [1]: https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz
> >
> > Ömer
> >
> > Ömer Sinan Ağacan <[hidden email]>, 7 Ağu 2018 Sal, 23:28
> > tarihinde şunu yazdı:
> > >
> > > Thanks for both suggestions. I'll try both and see which one works better.
> > >
> > > Ömer
> > >
> > > Manuel M T Chakravarty <[hidden email]>, 7 Ağu 2018 Sal, 18:15
> > > tarihinde şunu yazdı:
> > > >
> > > > Hi Ömer,
> > > >
> > > > This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at
> > > >
> > > >   https://github.com/tweag/stackage-head
> > > >
> > > > and also the blog post from when the first version went live:
> > > >
> > > >   https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html
> > > >
> > > > Cheers,
> > > > Manuel
> > > >
> > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan <[hidden email]>:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I'd like to test some GHC builds + some compile and runtime flag combinations
> > > > > against a large set of packages by building them and running test suites. For
> > > > > this I need
> > > > >
> > > > > - A set of packages that are known to work with latest GHC
> > > > > - A way to build them and run their test suites (if I could specify compile and
> > > > >  runtime flags that'd be even better)
> > > > >
> > > > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone
> > > > > point me to the right direction? I vaguely remember some nix-based solution for
> > > > > this that was being discussed on the IRC channel, but can't recall any details.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Ömer
> > > > > _______________________________________________
> > > > > ghc-devs mailing list
> > > > > [hidden email]
> > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > > >
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Herbert Valerio Riedel-3
Hi Artem,

On Fri, Aug 10, 2018 at 11:05 AM Artem Pelenitsyn
<[hidden email]> wrote:
> The task seems to be not solvable even if an affected package (stm in your case and primitive in mine) has already adopted in its master the breaking change but has no corresponding release on Hackage (which will always be the case, because how would you release depending on base which hasn't been released yet).
>
> Some package managers in other languages (Julia's Pkg, go install) allow you to depend directly on master branch of a package if you wish to. I wonder if there is a road here for cabal to take in order to support this kind of unsafe installs.

As a matter of fact we have a couple of options here:

- Add the http://cand.hackage.haskell.org/ package index to your
config and get access to the stm-2.5.0.0 package release candidate
(i.e. http://hackage.haskell.org/package/stm-2.5.0.0/candidate/stm-2.5.0.0.tar.gz)

- We could have added `stm-2.5.0.0` to the head.hackage overlay (so
far the "overlay" has only modified existing packages from the primary
hackage index)

- Starting with cabal-2.4, we support adding package tarballs to
`cabal.project`; and this should work also for specifiying remote
tarball locations such as
http://hackage.haskell.org/package/stm-2.5.0.0/candidate/stm-2.5.0.0.tar.gz

- Starting with cabal 2.4, we do support remote git repo deps in
`cabal.project`, see e.g.

     https://github.com/hvr/cardano-sl/blob/wip/cabal-project/cabal.project#L41-L156

   for an extensive real-world example.


There's work planned to refine/improve these options and make them
more convenient/expressive. Help is appreciated if somebody wants to
help us get there sooner.


-- hvr
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Ömer Sinan Ağacan
In reply to this post by Ömer Sinan Ağacan
I also briefly looked at hackage.head. As far as I understand it doesn't
out-of-the-box provide a way to build a large set of packages, right? It'd be
useful if I had a package that I want to test against GHC HEAD but currently it
doesn't help me, unless I'm missing something.

Ömer

Ömer Sinan Ağacan <[hidden email]>, 10 Ağu 2018 Cum, 11:39
tarihinde şunu yazdı:

>
> Hi,
>
> This is working great, I just generated my first report. One problem is stm-2.4
> doesn't compile with GHC HEAD, we need stm-2.5.0.0. But that's not published on
> Hackage yet, and latest nightly still uses stm-2.4.5.0. I wonder if there's
> anything that can be done about this. Apparently stm blocks 82 packages (I
> don't know if that's counting transitively or just packages that are directly
> blocked by stm). Any ideas about this?
>
> Ömer
>
> Ömer Sinan Ağacan <[hidden email]>, 9 Ağu 2018 Per, 14:45
> tarihinde şunu yazdı:
> >
> > Ah, I now realize that that command is supposed to print that output. I'll
> > continue following the steps and keep you updated if I get stuck again.
> >
> > Ömer
> >
> > Ömer Sinan Ağacan <[hidden email]>, 9 Ağu 2018 Per, 13:20
> > tarihinde şunu yazdı:
> > >
> > > Hi Manuel,
> > >
> > > I'm trying stackage-head. I'm following the steps for the scheduled build in
> > > .circleci/config.yml. So far steps I took:
> > >
> > > - Installed ghc-head (from [1]) to ~/ghc-head
> > > - Installed stackage-build-plan, stackage-curator and stackage-head (with
> > >   -fdev) from git repos, using stack.
> > > - export BUILD_PLAN=nightly-2018-07-30 (from config.yml)
> > > - curl https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json
> > > --output metadata.json
> > > - curl https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml
> > > --output $BUILD_PLAN.yaml
> > >
> > > Now I'm doing
> > >
> > > - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN
> > > --ghc-metadata metadata.json --outdir build-reports
> > >
> > > but it's failing with
> > >
> > >     The combination of target and commit is new to me
> > >
> > > Any ideas what I'm doing wrong?
> > >
> > > Thanks
> > >
> > > [1]: https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz
> > >
> > > Ömer
> > >
> > > Ömer Sinan Ağacan <[hidden email]>, 7 Ağu 2018 Sal, 23:28
> > > tarihinde şunu yazdı:
> > > >
> > > > Thanks for both suggestions. I'll try both and see which one works better.
> > > >
> > > > Ömer
> > > >
> > > > Manuel M T Chakravarty <[hidden email]>, 7 Ağu 2018 Sal, 18:15
> > > > tarihinde şunu yazdı:
> > > > >
> > > > > Hi Ömer,
> > > > >
> > > > > This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at
> > > > >
> > > > >   https://github.com/tweag/stackage-head
> > > > >
> > > > > and also the blog post from when the first version went live:
> > > > >
> > > > >   https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html
> > > > >
> > > > > Cheers,
> > > > > Manuel
> > > > >
> > > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan <[hidden email]>:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'd like to test some GHC builds + some compile and runtime flag combinations
> > > > > > against a large set of packages by building them and running test suites. For
> > > > > > this I need
> > > > > >
> > > > > > - A set of packages that are known to work with latest GHC
> > > > > > - A way to build them and run their test suites (if I could specify compile and
> > > > > >  runtime flags that'd be even better)
> > > > > >
> > > > > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone
> > > > > > point me to the right direction? I vaguely remember some nix-based solution for
> > > > > > this that was being discussed on the IRC channel, but can't recall any details.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Ömer
> > > > > > _______________________________________________
> > > > > > ghc-devs mailing list
> > > > > > [hidden email]
> > > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > > > >
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Ben Gamari-2
On August 10, 2018 7:55:38 AM EDT, "Ömer Sinan Ağacan" <[hidden email]> wrote:

>I also briefly looked at hackage.head. As far as I understand it
>doesn't
>out-of-the-box provide a way to build a large set of packages, right?
>It'd be
>useful if I had a package that I want to test against GHC HEAD but
>currently it
>doesn't help me, unless I'm missing something.
>
>Ömer
>
>Ömer Sinan Ağacan <[hidden email]>, 10 Ağu 2018 Cum, 11:39
>tarihinde şunu yazdı:
>>
>> Hi,
>>
>> This is working great, I just generated my first report. One problem
>is stm-2.4
>> doesn't compile with GHC HEAD, we need stm-2.5.0.0. But that's not
>published on
>> Hackage yet, and latest nightly still uses stm-2.4.5.0. I wonder if
>there's
>> anything that can be done about this. Apparently stm blocks 82
>packages (I
>> don't know if that's counting transitively or just packages that are
>directly
>> blocked by stm). Any ideas about this?
>>
>> Ömer
>>
>> Ömer Sinan Ağacan <[hidden email]>, 9 Ağu 2018 Per, 14:45
>> tarihinde şunu yazdı:
>> >
>> > Ah, I now realize that that command is supposed to print that
>output. I'll
>> > continue following the steps and keep you updated if I get stuck
>again.
>> >
>> > Ömer
>> >
>> > Ömer Sinan Ağacan <[hidden email]>, 9 Ağu 2018 Per, 13:20
>> > tarihinde şunu yazdı:
>> > >
>> > > Hi Manuel,
>> > >
>> > > I'm trying stackage-head. I'm following the steps for the
>scheduled build in
>> > > .circleci/config.yml. So far steps I took:
>> > >
>> > > - Installed ghc-head (from [1]) to ~/ghc-head
>> > > - Installed stackage-build-plan, stackage-curator and
>stackage-head (with
>> > >   -fdev) from git repos, using stack.
>> > > - export BUILD_PLAN=nightly-2018-07-30 (from config.yml)
>> > > - curl
>https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json
>> > > --output metadata.json
>> > > - curl
>https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml
>> > > --output $BUILD_PLAN.yaml
>> > >
>> > > Now I'm doing
>> > >
>> > > - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN
>> > > --ghc-metadata metadata.json --outdir build-reports
>> > >
>> > > but it's failing with
>> > >
>> > >     The combination of target and commit is new to me
>> > >
>> > > Any ideas what I'm doing wrong?
>> > >
>> > > Thanks
>> > >
>> > > [1]:
>https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz
>> > >
>> > > Ömer
>> > >
>> > > Ömer Sinan Ağacan <[hidden email]>, 7 Ağu 2018 Sal, 23:28
>> > > tarihinde şunu yazdı:
>> > > >
>> > > > Thanks for both suggestions. I'll try both and see which one
>works better.
>> > > >
>> > > > Ömer
>> > > >
>> > > > Manuel M T Chakravarty <[hidden email]>, 7 Ağu 2018 Sal,
>18:15
>> > > > tarihinde şunu yazdı:
>> > > > >
>> > > > > Hi Ömer,
>> > > > >
>> > > > > This is exactly the motivation for the Stackage HEAD works
>that we have pushed at Tweag I/O in the context of the GHC DevOps
>group. Have a look at
>> > > > >
>> > > > >   https://github.com/tweag/stackage-head
>> > > > >
>> > > > > and also the blog post from when the first version went live:
>> > > > >
>> > > > >  
>https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html
>> > > > >
>> > > > > Cheers,
>> > > > > Manuel
>> > > > >
>> > > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan
><[hidden email]>:
>> > > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > I'd like to test some GHC builds + some compile and runtime
>flag combinations
>> > > > > > against a large set of packages by building them and
>running test suites. For
>> > > > > > this I need
>> > > > > >
>> > > > > > - A set of packages that are known to work with latest GHC
>> > > > > > - A way to build them and run their test suites (if I could
>specify compile and
>> > > > > >  runtime flags that'd be even better)
>> > > > > >
>> > > > > > I think stackage can serve as (1) but I don't know how to
>do (2). Can anyone
>> > > > > > point me to the right direction? I vaguely remember some
>nix-based solution for
>> > > > > > this that was being discussed on the IRC channel, but can't
>recall any details.
>> > > > > >
>> > > > > > Thanks,
>> > > > > >
>> > > > > > Ömer
>> > > > > > _______________________________________________
>> > > > > > ghc-devs mailing list
>> > > > > > [hidden email]
>> > > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> > > > >
>_______________________________________________
>ghc-devs mailing list
>[hidden email]
>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Head.hackage doesn't have an out of the box package set but it is quite straightforward to construct such a set. While I now tend to use nix, in the past I generally just constructed a dummy cabal package listing the packages of interest as dependencies.

There are two approaches to choosing a set of packages: extract the packages from Stackage's build-constraints.yaml or just additively build up a set from the top of your head. I find the latter is often more realistic; stackage is now large enough that even getting a fraction of it to build with a prerelease compiler can be a significant undertaking.
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Any ways to test a GHC build against large set of packages (including test suites)?

Joachim Breitner-2
Hi,

Am Freitag, den 10.08.2018, 20:06 -0400 schrieb Ben Gamari:

> Head.hackage doesn't have an out of the box package set but it is
> quite straightforward to construct such a set. While I now tend to
> use nix, in the past I generally just constructed a dummy cabal
> package listing the packages of interest as dependencies.
>
> There are two approaches to choosing a set of packages: extract the
> packages from Stackage's build-constraints.yaml or just additively
> build up a set from the top of your head. I find the latter is often
> more realistic; stackage is now large enough that even getting a
> fraction of it to build with a prerelease compiler can be a
> significant undertaking.
here is another package set worth considering, if you want a
representative collection of packages with (somewhat) significance: The
packages distributed by Debian:
https://salsa.debian.org/haskell-team/package-plan/raw/master/packages.txt

Cheers,
Joachim

--
Joachim Breitner
  [hidden email]
  http://www.joachim-breitner.de/

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment