Perf test viewer

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Perf test viewer

Sylvain Henry-2
Hello everyone,

Since testsuite performance results are stored into Git notes, they are
more difficult to visualize. At least with values in .T files we could
see the changes over time, but now increase/decrease are only indicated
into commit messages (but not necessarily by how much). The only tool we
have afaik is the perf_notes.py script [1] but it's not very interactive.

So, long story short, I've started another one which is more interactive
(in the browser). An instance is running on my server: http://hsyl20.fr:4222

It should be updated every 2 hours with newer metrics/commits from CI.
Sources are here: https://github.com/hsyl20/had

Any feedback/PR is welcome!

Cheers,
Sylvain

[1]
https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-tests/performance-tests#comparing-commits


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Perf test viewer

Ben Gamari-2
Sylvain Henry <[hidden email]> writes:

> Hello everyone,
>
> Since testsuite performance results are stored into Git notes, they are
> more difficult to visualize. At least with values in .T files we could
> see the changes over time, but now increase/decrease are only indicated
> into commit messages (but not necessarily by how much). The only tool we
> have afaik is the perf_notes.py script [1] but it's not very interactive.
>
> So, long story short, I've started another one which is more interactive
> (in the browser). An instance is running on my server: http://hsyl20.fr:4222
>
Hi Sylvain,

This looks great!

I would also note that all of the performance metrics produced by the
testsuite, nofib, and (soon) head.hackage are preserved in a PostgreSQL
database on gitlab.haskell.org which can be accessed by way of Postgrest
[1]. The schema can be found here [2]. You will also find the import
tools in the same repository (but they are quite rough around the edges;
you have been warned).

The schema is probably most conveniently used via the `results_view`
view. For instance, one can request all metrics for commits in the last
month with:

    $ curl https://perf.ghc.haskell.org/db/results_view?commit_date=gt.2020-09-01

Another useful view is `deltas`, which provides compares metrics for
pairs of commits. This I have found quite useful in spotting
regressions.

Apart from curl, there are a few other options for consuming this
information:

 * I have an Python notebook which has some convenient helpers for
   quickly plotting things in `ghc-utils` [3]. I find this option to
   have the greatest power/weight ratio for most use-cases.

 * There is an incredibly hacky attempt at a web interface here [4]
   (source here [5]) although it's probably best to bury this sad effort
 
 * Sometimes I have needed to fall back to SQL for some queries; if
   someone would find this useful we can arrange read-only SQL access on
   an individual basis.

Regardless, this project has been gradually evolving as the need arises
for the last few years. It's been quite useful but I think it's
potential is quite under-realized.

Cheers,

- Ben


[1] https://postgrest.org/
[2] https://github.com/bgamari/ghc-perf-import/blob/master/import/schema.sql
[3] https://github.com/bgamari/ghc-perf-import/blob/master/plot.ipynb
[4] http://home.smart-cactus.org/ghc-perf/
[5] https://github.com/bgamari/ghc-perf-import/tree/master/web

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment