[GHC] #14660: Improve +RTS -t --machine-readable

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[GHC] #14660: Improve +RTS -t --machine-readable

GHC - devs mailing list
#14660: Improve +RTS -t --machine-readable
-------------------------------------+-------------------------------------
           Reporter:  duog           |             Owner:  (none)
               Type:  task           |            Status:  new
           Priority:  normal         |         Milestone:
          Component:  Runtime        |           Version:  8.3
  System                             |
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 The output of +RTS -t --machine-readable is a shown Haskell list, so we
 can easily add fields. Let's add everything shown by +RTS -s .
 I've posted a rough patch that does that at Phab:D4303.

 If this is a good idea, I'd like to also add the following:
 * a data structure and parsing function to GHC.Stats in base.
 * The ghc version as a field in the --machine-readable output
 * The rts ways as a field in the --machine-readable output

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14660>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14660: Improve +RTS -t --machine-readable

GHC - devs mailing list
#14660: Improve +RTS -t --machine-readable
-------------------------------------+-------------------------------------
        Reporter:  duog              |                Owner:  (none)
            Type:  task              |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Runtime System    |              Version:  8.3
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by mpickering):

 It could just be output as a JSON record rather than relying on having to
 use Haskell tooling to parse it.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14660#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14660: Improve +RTS -t --machine-readable

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14660: Improve +RTS -t --machine-readable
-------------------------------------+-------------------------------------
        Reporter:  duog              |                Owner:  (none)
            Type:  task              |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Runtime System    |              Version:  8.3
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by duog):

 Using a JSON record would break existing tooling, which is a little on the
 nose since this output has been documented as being forwards compatible.

 It would be good to have a nice structured record for the per-generations
 stats though.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14660#comment:2>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14660: Improve +RTS -t --machine-readable

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14660: Improve +RTS -t --machine-readable
-------------------------------------+-------------------------------------
        Reporter:  duog              |                Owner:  (none)
            Type:  task              |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Runtime System    |              Version:  8.3
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4303
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by duog):

 * differential:   => Phab:D4303


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14660#comment:3>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14660: Improve +RTS -t --machine-readable

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14660: Improve +RTS -t --machine-readable
-------------------------------------+-------------------------------------
        Reporter:  duog              |                Owner:  (none)
            Type:  task              |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Runtime System    |              Version:  8.3
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4303
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by Ben Gamari <ben@…>):

 In [changeset:"2d4bda2e4ac68816baba0afab00da6f769ea75a7/ghc"
 2d4bda2e/ghc]:
 {{{
 #!CommitTicketReference repository="ghc"
 revision="2d4bda2e4ac68816baba0afab00da6f769ea75a7"
 rts, base: Refactor stats.c to improve --machine-readable report

 There should be no change in the output of the '+RTS -s' (summary)
 report, or the 'RTS -t' (one-line) report.

 All data shown in the summary report is now shown in the machine
 readable report.

 All data in RTSStats is now shown in the machine readable report.

 init times are added to RTSStats and added to GHC.Stats.

 Example of the new output:
 ```
  [("bytes allocated", "375016384")
  ,("num_GCs", "113")
  ,("average_bytes_used", "148348")
  ,("max_bytes_used", "206552")
  ,("num_byte_usage_samples", "2")
  ,("peak_megabytes_allocated", "6")
  ,("init_cpu_seconds", "0.001642")
  ,("init_wall_seconds", "0.001027")
  ,("mut_cpu_seconds", "3.020166")
  ,("mut_wall_seconds", "0.757244")
  ,("GC_cpu_seconds", "0.037750")
  ,("GC_wall_seconds", "0.009569")
  ,("exit_cpu_seconds", "0.000890")
  ,("exit_wall_seconds", "0.002551")
  ,("total_cpu_seconds", "3.060452")
  ,("total_wall_seconds", "0.770395")
  ,("major_gcs", "2")
  ,("allocated_bytes", "375016384")
  ,("max_live_bytes", "206552")
  ,("max_large_objects_bytes", "159344")
  ,("max_compact_bytes", "0")
  ,("max_slop_bytes", "59688")
  ,("max_mem_in_use_bytes", "6291456")
  ,("cumulative_live_bytes", "296696")
  ,("copied_bytes", "541024")
  ,("par_copied_bytes", "493976")
  ,("cumulative_par_max_copied_bytes", "104104")
  ,("cumulative_par_balanced_copied_bytes", "274456")
  ,("fragmentation_bytes", "2112")
  ,("alloc_rate", "124170795")
  ,("productivity_cpu_percent", "0.986838")
  ,("productivity_wall_percent", "0.982935")
  ,("bound_task_count", "1")
  ,("sparks_count", "5836258")
  ,("sparks_converted", "237")
  ,("sparks_overflowed", "1990408")
  ,("sparks_dud ", "0")
  ,("sparks_gcd", "3455553")
  ,("sparks_fizzled", "390060")
  ,("work_balance", "0.555606")
  ,("n_capabilities", "4")
  ,("task_count", "10")
  ,("peak_worker_count", "9")
  ,("worker_count", "9")
  ,("gc_alloc_block_sync_spin", "162")
  ,("gc_alloc_block_sync_yield", "0")
  ,("gc_alloc_block_sync_spin", "162")
  ,("gc_spin_spin", "18840855")
  ,("gc_spin_yield", "10355")
  ,("mut_spin_spin", "70331392")
  ,("mut_spin_yield", "61700")
  ,("waitForGcThreads_spin", "241")
  ,("waitForGcThreads_yield", "2797")
  ,("whitehole_gc_spin", "0")
  ,("whitehole_lockClosure_spin", "0")
  ,("whitehole_lockClosure_yield", "0")
  ,("whitehole_executeMessage_spin", "0")
  ,("whitehole_threadPaused_spin", "0")
  ,("any_work", "1667")
  ,("no_work", "1662")
  ,("scav_find_work", "1026")
  ,("gen_0_collections", "111")
  ,("gen_0_par_collections", "111")
  ,("gen_0_cpu_seconds", "0.036126")
  ,("gen_0_wall_seconds", "0.036126")
  ,("gen_0_max_pause_seconds", "0.036126")
  ,("gen_0_avg_pause_seconds", "0.000081")
  ,("gen_0_sync_spin", "21")
  ,("gen_0_sync_yield", "0")
  ,("gen_1_collections", "2")
  ,("gen_1_par_collections", "1")
  ,("gen_1_cpu_seconds", "0.001624")
  ,("gen_1_wall_seconds", "0.001624")
  ,("gen_1_max_pause_seconds", "0.001624")
  ,("gen_1_avg_pause_seconds", "0.000272")
  ,("gen_1_sync_spin", "3")
  ,("gen_1_sync_yield", "0")
  ]
 ```

 Test Plan: Ensure that one-line and summary reports are unchanged.

 Reviewers: bgamari, erikd, simonmar, hvr

 Reviewed By: simonmar

 Subscribers: rwbarton, thomie, carter

 GHC Trac Issues: #14660

 Differential Revision: https://phabricator.haskell.org/D4303
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14660#comment:4>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14660: Improve +RTS -t --machine-readable

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14660: Improve +RTS -t --machine-readable
-------------------------------------+-------------------------------------
        Reporter:  duog              |                Owner:  (none)
            Type:  task              |               Status:  closed
        Priority:  normal            |            Milestone:
       Component:  Runtime System    |              Version:  8.3
      Resolution:  fixed             |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4303
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by bgamari):

 * status:  new => closed
 * resolution:   => fixed


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14660#comment:5>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14660: Improve +RTS -t --machine-readable

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14660: Improve +RTS -t --machine-readable
-------------------------------------+-------------------------------------
        Reporter:  duog              |                Owner:  (none)
            Type:  task              |               Status:  closed
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.3
      Resolution:  fixed             |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4303
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Changes (by bgamari):

 * milestone:   => 8.6.1


--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14660#comment:6>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets
Reply | Threaded
Open this post in threaded view
|

Re: [GHC] #14660: Improve +RTS -t --machine-readable

GHC - devs mailing list
In reply to this post by GHC - devs mailing list
#14660: Improve +RTS -t --machine-readable
-------------------------------------+-------------------------------------
        Reporter:  duog              |                Owner:  (none)
            Type:  task              |               Status:  closed
        Priority:  normal            |            Milestone:  8.6.1
       Component:  Runtime System    |              Version:  8.3
      Resolution:  fixed             |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D4303
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by Ben Gamari <ben@…>):

 In [changeset:"f0b258bc7e2d4ef32a20c61b7285a21f7680660e/ghc" f0b258b/ghc]:
 {{{
 #!CommitTicketReference repository="ghc"
 revision="f0b258bc7e2d4ef32a20c61b7285a21f7680660e"
 rts, base: Refactor stats.c to improve --machine-readable report

 There should be no change in the output of the '+RTS -s' (summary)
 report, or
 the 'RTS -t' (one-line) report.

 All data shown in the summary report is now shown in the machine
 readable
 report.

 All data in RTSStats is now shown in the machine readable report.

 init times are added to RTSStats and added to GHC.Stats.

 Example of the new output:
 ```
  [("bytes allocated", "375016384")
  ,("num_GCs", "113")
  ,("average_bytes_used", "148348")
  ,("max_bytes_used", "206552")
  ,("num_byte_usage_samples", "2")
  ,("peak_megabytes_allocated", "6")
  ,("init_cpu_seconds", "0.001642")
  ,("init_wall_seconds", "0.001027")
  ,("mut_cpu_seconds", "3.020166")
  ,("mut_wall_seconds", "0.757244")
  ,("GC_cpu_seconds", "0.037750")
  ,("GC_wall_seconds", "0.009569")
  ,("exit_cpu_seconds", "0.000890")
  ,("exit_wall_seconds", "0.002551")
  ,("total_cpu_seconds", "3.060452")
  ,("total_wall_seconds", "0.770395")
  ,("major_gcs", "2")
  ,("allocated_bytes", "375016384")
  ,("max_live_bytes", "206552")
  ,("max_large_objects_bytes", "159344")
  ,("max_compact_bytes", "0")
  ,("max_slop_bytes", "59688")
  ,("max_mem_in_use_bytes", "6291456")
  ,("cumulative_live_bytes", "296696")
  ,("copied_bytes", "541024")
  ,("par_copied_bytes", "493976")
  ,("cumulative_par_max_copied_bytes", "104104")
  ,("cumulative_par_balanced_copied_bytes", "274456")
  ,("fragmentation_bytes", "2112")
  ,("alloc_rate", "124170795")
  ,("productivity_cpu_percent", "0.986838")
  ,("productivity_wall_percent", "0.982935")
  ,("bound_task_count", "1")
  ,("sparks_count", "5836258")
  ,("sparks_converted", "237")
  ,("sparks_overflowed", "1990408")
  ,("sparks_dud ", "0")
  ,("sparks_gcd", "3455553")
  ,("sparks_fizzled", "390060")
  ,("work_balance", "0.555606")
  ,("n_capabilities", "4")
  ,("task_count", "10")
  ,("peak_worker_count", "9")
  ,("worker_count", "9")
  ,("gc_alloc_block_sync_spin", "162")
  ,("gc_alloc_block_sync_yield", "0")
  ,("gc_alloc_block_sync_spin", "162")
  ,("gc_spin_spin", "18840855")
  ,("gc_spin_yield", "10355")
  ,("mut_spin_spin", "70331392")
  ,("mut_spin_yield", "61700")
  ,("waitForGcThreads_spin", "241")
  ,("waitForGcThreads_yield", "2797")
  ,("whitehole_gc_spin", "0")
  ,("whitehole_lockClosure_spin", "0")
  ,("whitehole_lockClosure_yield", "0")
  ,("whitehole_executeMessage_spin", "0")
  ,("whitehole_threadPaused_spin", "0")
  ,("any_work", "1667")
  ,("no_work", "1662")
  ,("scav_find_work", "1026")
  ,("gen_0_collections", "111")
  ,("gen_0_par_collections", "111")
  ,("gen_0_cpu_seconds", "0.036126")
  ,("gen_0_wall_seconds", "0.036126")
  ,("gen_0_max_pause_seconds", "0.036126")
  ,("gen_0_avg_pause_seconds", "0.000081")
  ,("gen_0_sync_spin", "21")
  ,("gen_0_sync_yield", "0")
  ,("gen_1_collections", "2")
  ,("gen_1_par_collections", "1")
  ,("gen_1_cpu_seconds", "0.001624")
  ,("gen_1_wall_seconds", "0.001624")
  ,("gen_1_max_pause_seconds", "0.001624")
  ,("gen_1_avg_pause_seconds", "0.000272")
  ,("gen_1_sync_spin", "3")
  ,("gen_1_sync_yield", "0")
  ]
 ```

 Test Plan: Ensure that one-line and summary reports are unchanged.

 Reviewers: erikd, simonmar, hvr

 Subscribers: duog, carter, thomie, rwbarton

 GHC Trac Issues: #14660

 Differential Revision: https://phabricator.haskell.org/D4529
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14660#comment:7>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

_______________________________________________
ghc-tickets mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets