Heap allocation in the RTS

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Heap allocation in the RTS

David Feuer
I'm working on re-implementing the stable name system. For the new design, it seems much cleaner to allocate stable names in lookupStableName (in rts/StableName.c) rather than in the C-- code that calls it. But I haven't seen RTS code that does heap allocation. Is it possible? If so, how do I do it?

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Heap allocation in the RTS

Ömer Sinan Ağacan
Have you seen Storage.c:allocateMightFail ?

Ömer


David Feuer <[hidden email]>, 20 Eyl 2018 Per, 11:32 tarihinde
şunu yazdı:
>
> I'm working on re-implementing the stable name system. For the new design, it seems much cleaner to allocate stable names in lookupStableName (in rts/StableName.c) rather than in the C-- code that calls it. But I haven't seen RTS code that does heap allocation. Is it possible? If so, how do I do it?
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Heap allocation in the RTS

David Feuer
Aha! Okay. How do I get the current capability to pass to that? I guess I should probably perform a heap check before calling lookupSta bleName for simplicity, at least to start.

On Thu, Sep 20, 2018, 5:16 AM Ömer Sinan Ağacan <[hidden email]> wrote:
Have you seen Storage.c:allocateMightFail ?

Ömer


David Feuer <[hidden email]>, 20 Eyl 2018 Per, 11:32 tarihinde
şunu yazdı:
>
> I'm working on re-implementing the stable name system. For the new design, it seems much cleaner to allocate stable names in lookupStableName (in rts/StableName.c) rather than in the C-- code that calls it. But I haven't seen RTS code that does heap allocation. Is it possible? If so, how do I do it?
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Heap allocation in the RTS

Ömer Sinan Ağacan
allocateMightFail does the heap check for you and returns NULL. For the current
capability you can use MyCapability() in Cmm and pass the value to the RTS
function you're implementing.

Ömer

David Feuer <[hidden email]>, 20 Eyl 2018 Per, 12:26 tarihinde
şunu yazdı:

>
> Aha! Okay. How do I get the current capability to pass to that? I guess I should probably perform a heap check before calling lookupSta bleName for simplicity, at least to start.
>
> On Thu, Sep 20, 2018, 5:16 AM Ömer Sinan Ağacan <[hidden email]> wrote:
>>
>> Have you seen Storage.c:allocateMightFail ?
>>
>> Ömer
>>
>>
>> David Feuer <[hidden email]>, 20 Eyl 2018 Per, 11:32 tarihinde
>> şunu yazdı:
>> >
>> > I'm working on re-implementing the stable name system. For the new design, it seems much cleaner to allocate stable names in lookupStableName (in rts/StableName.c) rather than in the C-- code that calls it. But I haven't seen RTS code that does heap allocation. Is it possible? If so, how do I do it?
>> > _______________________________________________
>> > ghc-devs mailing list
>> > [hidden email]
>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Heap allocation in the RTS

David Feuer
If it returns NULL, then I need to back off what I'm doing and trigger a GC. How do I do the latter?

On Thu, Sep 20, 2018, 5:31 AM Ömer Sinan Ağacan <[hidden email]> wrote:
allocateMightFail does the heap check for you and returns NULL. For the current
capability you can use MyCapability() in Cmm and pass the value to the RTS
function you're implementing.

Ömer

David Feuer <[hidden email]>, 20 Eyl 2018 Per, 12:26 tarihinde
şunu yazdı:
>
> Aha! Okay. How do I get the current capability to pass to that? I guess I should probably perform a heap check before calling lookupSta bleName for simplicity, at least to start.
>
> On Thu, Sep 20, 2018, 5:16 AM Ömer Sinan Ağacan <[hidden email]> wrote:
>>
>> Have you seen Storage.c:allocateMightFail ?
>>
>> Ömer
>>
>>
>> David Feuer <[hidden email]>, 20 Eyl 2018 Per, 11:32 tarihinde
>> şunu yazdı:
>> >
>> > I'm working on re-implementing the stable name system. For the new design, it seems much cleaner to allocate stable names in lookupStableName (in rts/StableName.c) rather than in the C-- code that calls it. But I haven't seen RTS code that does heap allocation. Is it possible? If so, how do I do it?
>> > _______________________________________________
>> > ghc-devs mailing list
>> > [hidden email]
>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Heap allocation in the RTS

Ömer Sinan Ağacan
allocateMightFail allocates new nursery blocks as long as you don't hit the
heap limit, so it returns NULL less often than you might think. In particular,
it doesn't return NULL when the nursery is full, instead it allocates a new
block and adds it to the nursery.

I'd do the GC triggering part in Cmm code instead of C code -- I'm not sure if
it's possible to do this in C code. There should be some functions in
PrimOps.cmm that do heap allocation, maybe look there. I'd look for uses of
ALLOC_PRIM. The file HeapStackCheck.cmm may also be helpful (may at least give
an idea of how a GC is triggered).

Ömer

David Feuer <[hidden email]>, 20 Eyl 2018 Per, 12:34 tarihinde
şunu yazdı:

>
> If it returns NULL, then I need to back off what I'm doing and trigger a GC. How do I do the latter?
>
> On Thu, Sep 20, 2018, 5:31 AM Ömer Sinan Ağacan <[hidden email]> wrote:
>>
>> allocateMightFail does the heap check for you and returns NULL. For the current
>> capability you can use MyCapability() in Cmm and pass the value to the RTS
>> function you're implementing.
>>
>> Ömer
>>
>> David Feuer <[hidden email]>, 20 Eyl 2018 Per, 12:26 tarihinde
>> şunu yazdı:
>> >
>> > Aha! Okay. How do I get the current capability to pass to that? I guess I should probably perform a heap check before calling lookupSta bleName for simplicity, at least to start.
>> >
>> > On Thu, Sep 20, 2018, 5:16 AM Ömer Sinan Ağacan <[hidden email]> wrote:
>> >>
>> >> Have you seen Storage.c:allocateMightFail ?
>> >>
>> >> Ömer
>> >>
>> >>
>> >> David Feuer <[hidden email]>, 20 Eyl 2018 Per, 11:32 tarihinde
>> >> şunu yazdı:
>> >> >
>> >> > I'm working on re-implementing the stable name system. For the new design, it seems much cleaner to allocate stable names in lookupStableName (in rts/StableName.c) rather than in the C-- code that calls it. But I haven't seen RTS code that does heap allocation. Is it possible? If so, how do I do it?
>> >> > _______________________________________________
>> >> > ghc-devs mailing list
>> >> > [hidden email]
>> >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Heap allocation in the RTS

David Feuer
I'm not sure I understand. Are you saying allocateMightFail ignores the usual nursery size? That's not my intention. It would actually be just fine to simply fail if GC would be required--I can then back off, fail out to CMM, trigger a GC there, and retry. Or I could perform an extra heap check before I start; that's a little silly, but I doubt it'll be expensive enough to really matter here.

On Thu, Sep 20, 2018, 5:42 AM Ömer Sinan Ağacan <[hidden email]> wrote:
allocateMightFail allocates new nursery blocks as long as you don't hit the
heap limit, so it returns NULL less often than you might think. In particular,
it doesn't return NULL when the nursery is full, instead it allocates a new
block and adds it to the nursery.

I'd do the GC triggering part in Cmm code instead of C code -- I'm not sure if
it's possible to do this in C code. There should be some functions in
PrimOps.cmm that do heap allocation, maybe look there. I'd look for uses of
ALLOC_PRIM. The file HeapStackCheck.cmm may also be helpful (may at least give
an idea of how a GC is triggered).

Ömer

David Feuer <[hidden email]>, 20 Eyl 2018 Per, 12:34 tarihinde
şunu yazdı:
>
> If it returns NULL, then I need to back off what I'm doing and trigger a GC. How do I do the latter?
>
> On Thu, Sep 20, 2018, 5:31 AM Ömer Sinan Ağacan <[hidden email]> wrote:
>>
>> allocateMightFail does the heap check for you and returns NULL. For the current
>> capability you can use MyCapability() in Cmm and pass the value to the RTS
>> function you're implementing.
>>
>> Ömer
>>
>> David Feuer <[hidden email]>, 20 Eyl 2018 Per, 12:26 tarihinde
>> şunu yazdı:
>> >
>> > Aha! Okay. How do I get the current capability to pass to that? I guess I should probably perform a heap check before calling lookupSta bleName for simplicity, at least to start.
>> >
>> > On Thu, Sep 20, 2018, 5:16 AM Ömer Sinan Ağacan <[hidden email]> wrote:
>> >>
>> >> Have you seen Storage.c:allocateMightFail ?
>> >>
>> >> Ömer
>> >>
>> >>
>> >> David Feuer <[hidden email]>, 20 Eyl 2018 Per, 11:32 tarihinde
>> >> şunu yazdı:
>> >> >
>> >> > I'm working on re-implementing the stable name system. For the new design, it seems much cleaner to allocate stable names in lookupStableName (in rts/StableName.c) rather than in the C-- code that calls it. But I haven't seen RTS code that does heap allocation. Is it possible? If so, how do I do it?
>> >> > _______________________________________________
>> >> > ghc-devs mailing list
>> >> > [hidden email]
>> >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Heap allocation in the RTS

Ömer Sinan Ağacan
> Are you saying allocateMightFail ignores the usual nursery size?

Right, so normally in Cmm you'd do something like `if (Hp + size > HpLim) {
trigger GC }`, but allocateMightFail adds more blocks to the nursery instead.
Maybe just look at the code, it's quite simple.

I don't know how to check Hp in the RTS and trigger a GC. I'd do that part in
Cmm as there are lots of Cmm functions that do this already (in PrimOps.cmm
and maybe elsewhere).

Ömer
David Feuer <[hidden email]>, 20 Eyl 2018 Per, 12:50 tarihinde
şunu yazdı:

>
> I'm not sure I understand. Are you saying allocateMightFail ignores the usual nursery size? That's not my intention. It would actually be just fine to simply fail if GC would be required--I can then back off, fail out to CMM, trigger a GC there, and retry. Or I could perform an extra heap check before I start; that's a little silly, but I doubt it'll be expensive enough to really matter here.
>
> On Thu, Sep 20, 2018, 5:42 AM Ömer Sinan Ağacan <[hidden email]> wrote:
>>
>> allocateMightFail allocates new nursery blocks as long as you don't hit the
>> heap limit, so it returns NULL less often than you might think. In particular,
>> it doesn't return NULL when the nursery is full, instead it allocates a new
>> block and adds it to the nursery.
>>
>> I'd do the GC triggering part in Cmm code instead of C code -- I'm not sure if
>> it's possible to do this in C code. There should be some functions in
>> PrimOps.cmm that do heap allocation, maybe look there. I'd look for uses of
>> ALLOC_PRIM. The file HeapStackCheck.cmm may also be helpful (may at least give
>> an idea of how a GC is triggered).
>>
>> Ömer
>>
>> David Feuer <[hidden email]>, 20 Eyl 2018 Per, 12:34 tarihinde
>> şunu yazdı:
>> >
>> > If it returns NULL, then I need to back off what I'm doing and trigger a GC. How do I do the latter?
>> >
>> > On Thu, Sep 20, 2018, 5:31 AM Ömer Sinan Ağacan <[hidden email]> wrote:
>> >>
>> >> allocateMightFail does the heap check for you and returns NULL. For the current
>> >> capability you can use MyCapability() in Cmm and pass the value to the RTS
>> >> function you're implementing.
>> >>
>> >> Ömer
>> >>
>> >> David Feuer <[hidden email]>, 20 Eyl 2018 Per, 12:26 tarihinde
>> >> şunu yazdı:
>> >> >
>> >> > Aha! Okay. How do I get the current capability to pass to that? I guess I should probably perform a heap check before calling lookupSta bleName for simplicity, at least to start.
>> >> >
>> >> > On Thu, Sep 20, 2018, 5:16 AM Ömer Sinan Ağacan <[hidden email]> wrote:
>> >> >>
>> >> >> Have you seen Storage.c:allocateMightFail ?
>> >> >>
>> >> >> Ömer
>> >> >>
>> >> >>
>> >> >> David Feuer <[hidden email]>, 20 Eyl 2018 Per, 11:32 tarihinde
>> >> >> şunu yazdı:
>> >> >> >
>> >> >> > I'm working on re-implementing the stable name system. For the new design, it seems much cleaner to allocate stable names in lookupStableName (in rts/StableName.c) rather than in the C-- code that calls it. But I haven't seen RTS code that does heap allocation. Is it possible? If so, how do I do it?
>> >> >> > _______________________________________________
>> >> >> > ghc-devs mailing list
>> >> >> > [hidden email]
>> >> >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs