profiling a constructor?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

profiling a constructor?

Richard Eisenberg-5
Hi devs,

Is there a way to count the number of times a particular constructor is allocated? I might want to know, say, the total number of cons cells allocated over the course of a program, or (in my actual case) the total number of FunTy cells allocated over the course of a program.

I can try to do this with an SCC. But it's very clunky:
* The constructor may be used in a number of places; each such place would need the SCC.
* SCCs can interfere with optimizations. In my use case, this would negate the usefulness of the exercise entirely, as I think some of the structures I wish to observe should never come into being, due to optimizations (e.g. case-of-known-constructor after inlining).

Surely, the RTS must be able to count the number of times a constructor is used. But is there any way to access such a feature? If others agree that this would sometimes be helpful, perhaps we can build the feature. Now is not the first time I've wanted this.

Thanks!
Richard
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: profiling a constructor?

Ben Gamari-3
Yes, you can use +RTS -hT, which is available even in the non-profiled runtime. This breaks down the heap by closure type. Of course, you don't have visibility into the source of the allocation, but this is often still quite useful.

— Ben

On May 23, 2020 8:58:30 AM EDT, Richard Eisenberg <[hidden email]> wrote:
Hi devs,

Is there a way to count the number of times a particular constructor is allocated? I might want to know, say, the total number of cons cells allocated over the course of a program, or (in my actual case) the total number of FunTy cells allocated over the course of a program.

I can try to do this with an SCC. But it's very clunky:
* The constructor may be used in a number of places; each such place would need the SCC.
* SCCs can interfere with optimizations. In my use case, this would negate the usefulness of the exercise entirely, as I think some of the structures I wish to observe should never come into being, due to optimizations (e.g. case-of-known-constructor after inlining).

Surely, the RTS must be able to count the number of times a constructor is used. But is there any way to access such a feature? If others agree that this would sometimes be helpful, perhaps we can build the feature. Now is not the first time I've wanted this.

Thanks!
Richard
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: profiling a constructor?

Andreas Klebinger
In reply to this post by Richard Eisenberg-5
> Surely, the RTS must be able to count the number of times a constructor is used.
It can't. For one there are different kinds of "uses" for constructors.
* Allocations - They might be dead by the time we gc the nursery, so the
RTS never get's to see them.
* Accessing the Constructor? That's even harder to track.
* The constructor being present during GC? One can do this using heap
profiling (as ben described).

There are also top level constructors which currently don't generate
code at all (just static data).

So currently there is no such feature.

For allocations in particular we could implement one on top of the ticky
profiler.
It operates on the STG => Cmm boundry so doesn't affect core optimizations.

There we could for every runtime constructor allocation emit code which
will bump a counter for that specific constructor.

I imagine this wouldn't that hard either. But it does require
modification of the compiler.

Cheers,
Andreas



Richard Eisenberg schrieb am 23.05.2020 um 14:58:

> Hi devs,
>
> Is there a way to count the number of times a particular constructor is allocated? I might want to know, say, the total number of cons cells allocated over the course of a program, or (in my actual case) the total number of FunTy cells allocated over the course of a program.
>
> I can try to do this with an SCC. But it's very clunky:
> * The constructor may be used in a number of places; each such place would need the SCC.
> * SCCs can interfere with optimizations. In my use case, this would negate the usefulness of the exercise entirely, as I think some of the structures I wish to observe should never come into being, due to optimizations (e.g. case-of-known-constructor after inlining).
>
> Surely, the RTS must be able to count the number of times a constructor is used. But is there any way to access such a feature? If others agree that this would sometimes be helpful, perhaps we can build the feature. Now is not the first time I've wanted this.
>
> Thanks!
> Richard
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: profiling a constructor?

Ben Gamari-2
On May 23, 2020 9:32:32 AM EDT, Andreas Klebinger <[hidden email]> wrote:
>> Surely, the RTS must be able to count the number of times a
>constructor is used.
>It can't. For one there are different kinds of "uses" for constructors.

This is a fair point. I had assumed that Richard was debugging a residency issue but i suppose that was an ungrounded assumption on my part.

>* Allocations - They might be dead by the time we gc the nursery, so
>the
>RTS never get's to see them.

Indeed I thought that the ticky-ticky profiler could do precisely this but it seems I was mistaken; it can only report thunks. Indeed it doesn't seem like it would be hard to introduce this.


>* Accessing the Constructor? That's even harder to track.

Indeed, this is not possible as written. However, the LDV profiler allows a slightly weaker version on this (reporting whether the closure as been read at all).

Allowing full tracking of closure reads this would incur a significant overhead.

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: profiling a constructor?

GHC - devs mailing list
In reply to this post by Richard Eisenberg-5
This would be a sensible thing to be able to count with -ticky.  Not hard to add either.

(Richard is asking for number of allocations, not what is live at any moment.)

Simon

|  -----Original Message-----
|  From: ghc-devs <[hidden email]> On Behalf Of Richard
|  Eisenberg
|  Sent: 23 May 2020 13:59
|  To: GHC developers <[hidden email]>
|  Subject: profiling a constructor?
|  
|  Hi devs,
|  
|  Is there a way to count the number of times a particular constructor is
|  allocated? I might want to know, say, the total number of cons cells
|  allocated over the course of a program, or (in my actual case) the total
|  number of FunTy cells allocated over the course of a program.
|  
|  I can try to do this with an SCC. But it's very clunky:
|  * The constructor may be used in a number of places; each such place
|  would need the SCC.
|  * SCCs can interfere with optimizations. In my use case, this would
|  negate the usefulness of the exercise entirely, as I think some of the
|  structures I wish to observe should never come into being, due to
|  optimizations (e.g. case-of-known-constructor after inlining).
|  
|  Surely, the RTS must be able to count the number of times a constructor
|  is used. But is there any way to access such a feature? If others agree
|  that this would sometimes be helpful, perhaps we can build the feature.
|  Now is not the first time I've wanted this.
|  
|  Thanks!
|  Richard
|  _______________________________________________
|  ghc-devs mailing list
|  [hidden email]
|  http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: profiling a constructor?

Richard Eisenberg-5
Thanks for all the responses. Yes, I'm looking for the number of allocations of a constructor; this is an allocations issue, not a retention/liveness issue. I'm not concerned about accesses (not even sure what that would mean). Adding this to -ticky would be very helpful -- and not just for me. For example, it would help us to know more precisely why !2249 (implementing BoxedRep) is mired in performance trouble: we could see how many more TyConApp nodes are created.

If this would be easy, is there someone who can guide me how to implement it? I am almost as clueless as someone who has never before gazed on GHC's code in these areas. (Or, I would be grateful for someone else to implement it, too.)

Thanks,
Richard

> On May 23, 2020, at 10:11 PM, Simon Peyton Jones <[hidden email]> wrote:
>
> This would be a sensible thing to be able to count with -ticky.  Not hard to add either.
>
> (Richard is asking for number of allocations, not what is live at any moment.)
>
> Simon
>
> |  -----Original Message-----
> |  From: ghc-devs <[hidden email]> On Behalf Of Richard
> |  Eisenberg
> |  Sent: 23 May 2020 13:59
> |  To: GHC developers <[hidden email]>
> |  Subject: profiling a constructor?
> |  
> |  Hi devs,
> |  
> |  Is there a way to count the number of times a particular constructor is
> |  allocated? I might want to know, say, the total number of cons cells
> |  allocated over the course of a program, or (in my actual case) the total
> |  number of FunTy cells allocated over the course of a program.
> |  
> |  I can try to do this with an SCC. But it's very clunky:
> |  * The constructor may be used in a number of places; each such place
> |  would need the SCC.
> |  * SCCs can interfere with optimizations. In my use case, this would
> |  negate the usefulness of the exercise entirely, as I think some of the
> |  structures I wish to observe should never come into being, due to
> |  optimizations (e.g. case-of-known-constructor after inlining).
> |  
> |  Surely, the RTS must be able to count the number of times a constructor
> |  is used. But is there any way to access such a feature? If others agree
> |  that this would sometimes be helpful, perhaps we can build the feature.
> |  Now is not the first time I've wanted this.
> |  
> |  Thanks!
> |  Richard
> |  _______________________________________________
> |  ghc-devs mailing list
> |  [hidden email]
> |  http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: profiling a constructor?

Ben Gamari-2
Richard Eisenberg <[hidden email]> writes:

> Thanks for all the responses. Yes, I'm looking for the number of
> allocations of a constructor; this is an allocations issue, not a
> retention/liveness issue. I'm not concerned about accesses (not even
> sure what that would mean). Adding this to -ticky would be very
> helpful -- and not just for me. For example, it would help us to know
> more precisely why !2249 (implementing BoxedRep) is mired in
> performance trouble: we could see how many more TyConApp nodes are
> created.
>
> If this would be easy, is there someone who can guide me how to
> implement it? I am almost as clueless as someone who has never before
> gazed on GHC's code in these areas. (Or, I would be grateful for
> someone else to implement it, too.)
>
Actually, looking more closely at the implementation of ticky, it looks
like we *do* track DataCon allocations (the giveaway was the mention of
withNewTickyCounterCon in GHC.StgToCmm.Bind.cgRhs). For instance,

    let con = Just x
    in ...

would produce a ticky counter named "con". The problem is that there is
no convenient way to get from the Id name (e.g. `con`) back to the
datacon that it allocates (e.g. `Just`). You could of course try to
parse this out of -ddump-stg output but... yuck.

Perhaps the ticker name should preserve this information. I've pushed a
completely un-build-tested attempt at this as !3340. Like most of
ticky's output, it won't be the easiest thing to mechanically parse but
it should be possible.

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment