[GHC] #14711: Machine readable output of coverage

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
           Reporter:  Koterpillar    |             Owner:  (none)
               Type:  feature        |            Status:  new
  request                            |
           Priority:  normal         |         Milestone:
          Component:  Code Coverage  |           Version:  8.2.2
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 When integrating with a CI system, it might be useful to fail the build,
 alert or take other actions based on coverage percentage. To do that, hpc
 needs to output a summary coverage report in a machine readable format.

 For example, the current output:

 {{{
  38% expressions used (1779/4624)
  57% boolean coverage (16/28)
       56% guards (9/16), 6 always True, 1 always False
       58% 'if' conditions (7/12), 2 always False, 3 unevaluated
      100% qualifiers (0/0)
   7% alternatives used (103/1410)
  75% local declarations used (102/136)
  40% top-level declarations used (185/457)
 }}}

 can be represented as:

 {{{
 {
   expressions: { total: 4624, used: 1779, percentage: 0.38 },
   boolean: { total: 28, used: 16, percentage: 0.57 },
   <...>
 }
 }}}

 This output can be then parsed to decide whether to fail the build,
 produce a coverage graph over time, etc.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
        Reporter:  Koterpillar       |                Owner:  (none)
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Code Coverage     |              Version:  8.2.2
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bgamari):

 Indeed this sounds quite useful. Let me know if you want help taking a
 stab at this.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
        Reporter:  Koterpillar       |                Owner:  (none)
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Code Coverage     |              Version:  8.2.2
      Resolution:                    |             Keywords:  newcomers
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by bgamari):

 * keywords:   => newcomers


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:2>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
        Reporter:  Koterpillar       |                Owner:  (none)
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Code Coverage     |              Version:  8.2.2
      Resolution:                    |             Keywords:  newcomers
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by Koterpillar):

 I'll try to do this, reading through the newcomers documentation
 currently.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:3>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
        Reporter:  Koterpillar       |                Owner:  (none)
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Code Coverage     |              Version:  8.2.2
      Resolution:                    |             Keywords:  newcomers
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by jml):

 There's already an `--xml-output` flag for `report`:

 {{{
 $ hpc help report
 Usage: hpc report [OPTION] .. <TIX_FILE> [<MODULE> [<MODULE> ..]]
 Output textual report about program coverage


 Options:

     --per-module                  show module level detail
     --decl-list                   show unused decls
     --exclude=[PACKAGE:][MODULE]  exclude MODULE and/or PACKAGE
     --include=[PACKAGE:][MODULE]  include MODULE and/or PACKAGE
     --srcdir=DIR                  path to source directory of .hs files
                                   multi-use of srcdir possible
     --hpcdir=DIR                  append sub-directory that contains .mix
 files
                                   default .hpc [rarely used]
     --reset-hpcdirs               empty the list of hpcdir's
                                   [rarely used]
     --xml-output                  show output in XML
     --verbosity=[0-2]             verbosity level, 0-2
                                   default 1

 $ hpc version
 hpc tools, version 0.67

 $ hpc report $(stack path --stack-yaml=stack-8.2.yaml --local-hpc-root
 )/graphql-api/graphql-api-tests/graphql-api-tests.tix --hpcdir=$(stack
 path --stack-yaml=stack-8.2.yaml --dist-dir)/hpc/ --xml-output
 <?xml version="1.0" encoding="UTF-8"?>
 <coverage name="/Users/jml/src/graphql-api/.stack-
 work/install/x86_64-osx/lts-10.4/8.2.2/hpc/graphql-api/graphql-api-tests
 /graphql-api-tests.tix">
   <summary>
     <exprs boxes="3774" count="2290"/>
     <booleans boxes="19" true="4" false="0" count="15"/>
     <guards boxes="13" true="4" false="0" count="9"/>
     <conditionals boxes="4" true="0" false="0" count="4"/>
     <qualifiers boxes="2" true="0" false="0" count="2"/>
     <alts boxes="342" count="168"/>
     <local boxes="84" count="69"/>
     <toplevel boxes="1051" count="366"/>
   </summary>
 </coverage>
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:4>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
        Reporter:  Koterpillar       |                Owner:  (none)
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Code Coverage     |              Version:  8.2.2
      Resolution:                    |             Keywords:  newcomers
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by potomak):

 Can we consider this task as invalid or does it require to include also a
 JSON representation of the coverage output?

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:5>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
        Reporter:  Koterpillar       |                Owner:  (none)
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Code Coverage     |              Version:  8.2.2
      Resolution:                    |             Keywords:  newcomers
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4522
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by jproyo):

 * differential:   => Phab:D4522


Comment:

 Hi Team,

 I am a newcomer and i have been trying to add this functionality.

 I have committed a review to Phabricator D4522 but i am not getting build
 pass because since i added a new flag to `hpc report` output command help
 i had to change `libraries/hpc` submodule testsuite in order to get the
 test pass successfully.

 I have committed the review with arc diff as the wiki indicates but it is
 complaining because my submodule changes are not in the upstream.
 Obviously i didnt want to push my submodule changes to not affect master
 branch build until review is done. Apart from this i dont have push
 permissions.

 How can i commit a review to Phab which contains changes in some
 component, in this case hpc, and in a submodule either?

 Best,

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:6>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
        Reporter:  Koterpillar       |                Owner:  jproyo
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Code Coverage     |              Version:  8.2.2
      Resolution:                    |             Keywords:  newcomers
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4522
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by Phyx-):

 * owner:  (none) => jproyo


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:7>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
        Reporter:  Koterpillar       |                Owner:  jproyo
            Type:  feature request   |               Status:  patch
        Priority:  normal            |            Milestone:
       Component:  Code Coverage     |              Version:  8.2.2
      Resolution:                    |             Keywords:  newcomers
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4522
       Wiki Page:                    |  Phab:4523
-------------------------------------+-------------------------------------
Changes (by Phyx-):

 * status:  new => patch
 * differential:  Phab:D4522 => Phab:D4522 Phab:4523


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:8>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
        Reporter:  Koterpillar       |                Owner:  jproyo
            Type:  feature request   |               Status:  patch
        Priority:  normal            |            Milestone:
       Component:  Code Coverage     |              Version:  8.2.2
      Resolution:                    |             Keywords:  newcomers
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4522
       Wiki Page:                    |  Phab:D4523
-------------------------------------+-------------------------------------
Changes (by Phyx-):

 * differential:  Phab:D4522 Phab:4523 => Phab:D4522 Phab:D4523


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:9>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14711: Machine readable output of coverage

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14711: Machine readable output of coverage
-------------------------------------+-------------------------------------
        Reporter:  Koterpillar       |                Owner:  jproyo
            Type:  feature request   |               Status:  patch
        Priority:  normal            |            Milestone:
       Component:  Code Coverage     |              Version:  8.2.2
      Resolution:                    |             Keywords:  newcomers
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4522
       Wiki Page:                    |  Phab:D4523 Phab:D4524
-------------------------------------+-------------------------------------
Changes (by jproyo):

 * differential:  Phab:D4522 Phab:D4523 => Phab:D4522 Phab:D4523 Phab:D4524


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14711#comment:10>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets