Min closure payload size?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Min closure payload size?

Ömer Sinan Ağacan
Hi,

I was trying to understand why some info tables that have no ptrs and nptrs like
GCD_CAF end up with 1 nptrs in the generated info table and found this code in
Constants.h:

    /* -----------------------------------------------------------------------------
       Minimum closure sizes

       This is the minimum number of words in the payload of a
       heap-allocated closure, so that the closure has enough room to be
       overwritten with a forwarding pointer during garbage collection.
       --------------------------------------------------------------------------
*/

    #define MIN_PAYLOAD_SIZE 1

We use this in a few places in the compiler and add at least one word space in
the payload. However the comment is actually wrong, forwarding pointers are made
by tagging the info ptr field so we don't need a word in the payload for
forwarding pointers. I tried updating this as 0 but that caused a lot of test
failures (mostly in GHCi). I'm wondering if I'm missing anything or is it just
some code assuming min payload size 1 without using this macro.

Any ideas?

Ömer
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Min closure payload size?

Ömer Sinan Ağacan
I just came across a closure that is according to this code is not valid:

    >>> print *get_itbl(0x7b2870)
    $8 = {
      layout = {
        payload = {
          ptrs = 0,
          nptrs = 0
        },
        bitmap = 0,
        large_bitmap_offset = 0,
        __pad_large_bitmap_offset = 0,
        selector_offset = 0
      },
      type = 21,
      srt = 3856568,
      code = 0x404ef0 <r1Al_info>
"H\215E\360L9\370rDH\203\354\bL\211\350H\211\336H\211\307\061\300\350|\034\062"
    }

This is a THUNK_STATIC with 0 ptrs and nptrs in the payload.

Ömer

Ömer Sinan Ağacan <[hidden email]>, 4 Şub 2019 Pzt, 16:23
tarihinde şunu yazdı:

>
> Hi,
>
> I was trying to understand why some info tables that have no ptrs and nptrs like
> GCD_CAF end up with 1 nptrs in the generated info table and found this code in
> Constants.h:
>
>     /* -----------------------------------------------------------------------------
>        Minimum closure sizes
>
>        This is the minimum number of words in the payload of a
>        heap-allocated closure, so that the closure has enough room to be
>        overwritten with a forwarding pointer during garbage collection.
>        --------------------------------------------------------------------------
> */
>
>     #define MIN_PAYLOAD_SIZE 1
>
> We use this in a few places in the compiler and add at least one word space in
> the payload. However the comment is actually wrong, forwarding pointers are made
> by tagging the info ptr field so we don't need a word in the payload for
> forwarding pointers. I tried updating this as 0 but that caused a lot of test
> failures (mostly in GHCi). I'm wondering if I'm missing anything or is it just
> some code assuming min payload size 1 without using this macro.
>
> Any ideas?
>
> Ömer
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: Min closure payload size?

GHC - devs mailing list
I'm relying on Simon M here.  I'm out of my depth!

Simon

| -----Original Message-----
| From: ghc-devs <[hidden email]> On Behalf Of Ömer Sinan
| Agacan
| Sent: 05 February 2019 13:38
| To: ghc-devs <[hidden email]>
| Subject: Re: Min closure payload size?
|
| I just came across a closure that is according to this code is not valid:
|
|     >>> print *get_itbl(0x7b2870)
|     $8 = {
|       layout = {
|         payload = {
|           ptrs = 0,
|           nptrs = 0
|         },
|         bitmap = 0,
|         large_bitmap_offset = 0,
|         __pad_large_bitmap_offset = 0,
|         selector_offset = 0
|       },
|       type = 21,
|       srt = 3856568,
|       code = 0x404ef0 <r1Al_info>
| "H\215E\360L9\370rDH\203\354\bL\211\350H\211\336H\211\307\061\300\350|\034\
| 062"
|     }
|
| This is a THUNK_STATIC with 0 ptrs and nptrs in the payload.
|
| Ömer
|
| Ömer Sinan Ağacan <[hidden email]>, 4 Şub 2019 Pzt, 16:23
| tarihinde şunu yazdı:
| >
| > Hi,
| >
| > I was trying to understand why some info tables that have no ptrs and
| nptrs like
| > GCD_CAF end up with 1 nptrs in the generated info table and found this
| code in
| > Constants.h:
| >
| >     /* ------------------------------------------------------------------
| -----------
| >        Minimum closure sizes
| >
| >        This is the minimum number of words in the payload of a
| >        heap-allocated closure, so that the closure has enough room to be
| >        overwritten with a forwarding pointer during garbage collection.
| >        ------------------------------------------------------------------
| --------
| > */
| >
| >     #define MIN_PAYLOAD_SIZE 1
| >
| > We use this in a few places in the compiler and add at least one word
| space in
| > the payload. However the comment is actually wrong, forwarding pointers
| are made
| > by tagging the info ptr field so we don't need a word in the payload for
| > forwarding pointers. I tried updating this as 0 but that caused a lot of
| test
| > failures (mostly in GHCi). I'm wondering if I'm missing anything or is it
| just
| > some code assuming min payload size 1 without using this macro.
| >
| > Any ideas?
| >
| > Ömer
| _______________________________________________
| ghc-devs mailing list
| [hidden email]
| https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haske
| ll.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
| devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C4297c3983d594168ad0b08d68
| b6f3d32%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636849707211774471&amp
| ;sdata=H4sLNvWnJHxdHo1hjdC0fU3pUL3K1AUjV4nC3tHlBEU%3D&amp;reserved=0
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Min closure payload size?

Gabor Greif-2
In reply to this post by Ömer Sinan Ağacan
Just guessing here, maybe this thunk type lives in (read-only?) static
sections, and as such it will never be overwritten with forwarding
pointers?

    Gabor

On 2/5/19, Ömer Sinan Ağacan <[hidden email]> wrote:

> I just came across a closure that is according to this code is not valid:
>
>     >>> print *get_itbl(0x7b2870)
>     $8 = {
>       layout = {
>         payload = {
>           ptrs = 0,
>           nptrs = 0
>         },
>         bitmap = 0,
>         large_bitmap_offset = 0,
>         __pad_large_bitmap_offset = 0,
>         selector_offset = 0
>       },
>       type = 21,
>       srt = 3856568,
>       code = 0x404ef0 <r1Al_info>
> "H\215E\360L9\370rDH\203\354\bL\211\350H\211\336H\211\307\061\300\350|\034\062"
>     }
>
> This is a THUNK_STATIC with 0 ptrs and nptrs in the payload.
>
> Ömer
>
> Ömer Sinan Ağacan <[hidden email]>, 4 Şub 2019 Pzt, 16:23
> tarihinde şunu yazdı:
>>
>> Hi,
>>
>> I was trying to understand why some info tables that have no ptrs and
>> nptrs like
>> GCD_CAF end up with 1 nptrs in the generated info table and found this
>> code in
>> Constants.h:
>>
>>     /*
>> -----------------------------------------------------------------------------
>>        Minimum closure sizes
>>
>>        This is the minimum number of words in the payload of a
>>        heap-allocated closure, so that the closure has enough room to be
>>        overwritten with a forwarding pointer during garbage collection.
>>
>> --------------------------------------------------------------------------
>> */
>>
>>     #define MIN_PAYLOAD_SIZE 1
>>
>> We use this in a few places in the compiler and add at least one word
>> space in
>> the payload. However the comment is actually wrong, forwarding pointers
>> are made
>> by tagging the info ptr field so we don't need a word in the payload for
>> forwarding pointers. I tried updating this as 0 but that caused a lot of
>> test
>> failures (mostly in GHCi). I'm wondering if I'm missing anything or is it
>> just
>> some code assuming min payload size 1 without using this macro.
>>
>> Any ideas?
>>
>> Ömer
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Min closure payload size?

Ömer Sinan Ağacan
I don't think so, for two reasons:

- We update static thunks just fine so I don't think they're in a read-only
  section.

- Forwarding pointers are needed when moving objects, and we don't move static
  objects, so we don't need to make them forwarding pointers (I think you
  confused forwarding pointers with indirections generated by thunk updates?).

Ömer

Gabor Greif <[hidden email]>, 6 Şub 2019 Çar, 03:47 tarihinde şunu yazdı:

>
> Just guessing here, maybe this thunk type lives in (read-only?) static
> sections, and as such it will never be overwritten with forwarding
> pointers?
>
>     Gabor
>
> On 2/5/19, Ömer Sinan Ağacan <[hidden email]> wrote:
> > I just came across a closure that is according to this code is not valid:
> >
> >     >>> print *get_itbl(0x7b2870)
> >     $8 = {
> >       layout = {
> >         payload = {
> >           ptrs = 0,
> >           nptrs = 0
> >         },
> >         bitmap = 0,
> >         large_bitmap_offset = 0,
> >         __pad_large_bitmap_offset = 0,
> >         selector_offset = 0
> >       },
> >       type = 21,
> >       srt = 3856568,
> >       code = 0x404ef0 <r1Al_info>
> > "H\215E\360L9\370rDH\203\354\bL\211\350H\211\336H\211\307\061\300\350|\034\062"
> >     }
> >
> > This is a THUNK_STATIC with 0 ptrs and nptrs in the payload.
> >
> > Ömer
> >
> > Ömer Sinan Ağacan <[hidden email]>, 4 Şub 2019 Pzt, 16:23
> > tarihinde şunu yazdı:
> >>
> >> Hi,
> >>
> >> I was trying to understand why some info tables that have no ptrs and
> >> nptrs like
> >> GCD_CAF end up with 1 nptrs in the generated info table and found this
> >> code in
> >> Constants.h:
> >>
> >>     /*
> >> -----------------------------------------------------------------------------
> >>        Minimum closure sizes
> >>
> >>        This is the minimum number of words in the payload of a
> >>        heap-allocated closure, so that the closure has enough room to be
> >>        overwritten with a forwarding pointer during garbage collection.
> >>
> >> --------------------------------------------------------------------------
> >> */
> >>
> >>     #define MIN_PAYLOAD_SIZE 1
> >>
> >> We use this in a few places in the compiler and add at least one word
> >> space in
> >> the payload. However the comment is actually wrong, forwarding pointers
> >> are made
> >> by tagging the info ptr field so we don't need a word in the payload for
> >> forwarding pointers. I tried updating this as 0 but that caused a lot of
> >> test
> >> failures (mostly in GHCi). I'm wondering if I'm missing anything or is it
> >> just
> >> some code assuming min payload size 1 without using this macro.
> >>
> >> Any ideas?
> >>
> >> Ömer
> > _______________________________________________
> > ghc-devs mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Min closure payload size?

Ömer Sinan Ağacan
In reply to this post by Ömer Sinan Ağacan
It took 7 months to figure this one out. The compacting GC (which is enabled by
default when heap residency is above 30%) needs two bits per closure: one for
the actual "liveness" mark bit, another one for whether the object's new
location (computed during two scans in compacting GC) is in the next block in
the heap chain, instead of the current block (which the "free" pointer points
to).

This bit is important becuase in the second pass we get info pointer of an
object and update any references to it (so that they point to its new location)
in one pass (the "unthreading" business), and we can't compute the size of the
object being moved (to see whether we can move the object to the current free
location or we have to move to the next block in the heap) before we get the
info table. So if we want to avoid this second mark bit we'd need to do one pass
to get the info table, compute the size, see if it fits in the current block
etc. and then do another pass to actually "unthread" (i.e. update references to
the object with the new location).

However I was able to lift this restriction by just bumping the bitmap size, to
allocate two bits per word in the heap (instead of one). It seems to work fine
(passes the test suite), I'll test this a little bit more and then submit a MR.
This reduces closure sizes of objects with no payload, e.g.:

    {-# LANGUAGE MagicHash #-}

    import GHC.Int
    import GHC.Prim

    main = print (I# (closureSize# True))

This now prints 1 instead of 2 as before.

Ömer

Ömer Sinan Ağacan <[hidden email]>, 4 Şub 2019 Pzt, 16:23
tarihinde şunu yazdı:

>
> Hi,
>
> I was trying to understand why some info tables that have no ptrs and nptrs like
> GCD_CAF end up with 1 nptrs in the generated info table and found this code in
> Constants.h:
>
>     /* -----------------------------------------------------------------------------
>        Minimum closure sizes
>
>        This is the minimum number of words in the payload of a
>        heap-allocated closure, so that the closure has enough room to be
>        overwritten with a forwarding pointer during garbage collection.
>        --------------------------------------------------------------------------
> */
>
>     #define MIN_PAYLOAD_SIZE 1
>
> We use this in a few places in the compiler and add at least one word space in
> the payload. However the comment is actually wrong, forwarding pointers are made
> by tagging the info ptr field so we don't need a word in the payload for
> forwarding pointers. I tried updating this as 0 but that caused a lot of test
> failures (mostly in GHCi). I'm wondering if I'm missing anything or is it just
> some code assuming min payload size 1 without using this macro.
>
> Any ideas?
>
> Ömer
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Min closure payload size?

Ömer Sinan Ağacan
https://gitlab.haskell.org/ghc/ghc/merge_requests/1701

Ömer

Ömer Sinan Ağacan <[hidden email]>, 10 Eyl 2019 Sal, 15:14
tarihinde şunu yazdı:

>
> It took 7 months to figure this one out. The compacting GC (which is enabled by
> default when heap residency is above 30%) needs two bits per closure: one for
> the actual "liveness" mark bit, another one for whether the object's new
> location (computed during two scans in compacting GC) is in the next block in
> the heap chain, instead of the current block (which the "free" pointer points
> to).
>
> This bit is important becuase in the second pass we get info pointer of an
> object and update any references to it (so that they point to its new location)
> in one pass (the "unthreading" business), and we can't compute the size of the
> object being moved (to see whether we can move the object to the current free
> location or we have to move to the next block in the heap) before we get the
> info table. So if we want to avoid this second mark bit we'd need to do one pass
> to get the info table, compute the size, see if it fits in the current block
> etc. and then do another pass to actually "unthread" (i.e. update references to
> the object with the new location).
>
> However I was able to lift this restriction by just bumping the bitmap size, to
> allocate two bits per word in the heap (instead of one). It seems to work fine
> (passes the test suite), I'll test this a little bit more and then submit a MR.
> This reduces closure sizes of objects with no payload, e.g.:
>
>     {-# LANGUAGE MagicHash #-}
>
>     import GHC.Int
>     import GHC.Prim
>
>     main = print (I# (closureSize# True))
>
> This now prints 1 instead of 2 as before.
>
> Ömer
>
> Ömer Sinan Ağacan <[hidden email]>, 4 Şub 2019 Pzt, 16:23
> tarihinde şunu yazdı:
> >
> > Hi,
> >
> > I was trying to understand why some info tables that have no ptrs and nptrs like
> > GCD_CAF end up with 1 nptrs in the generated info table and found this code in
> > Constants.h:
> >
> >     /* -----------------------------------------------------------------------------
> >        Minimum closure sizes
> >
> >        This is the minimum number of words in the payload of a
> >        heap-allocated closure, so that the closure has enough room to be
> >        overwritten with a forwarding pointer during garbage collection.
> >        --------------------------------------------------------------------------
> > */
> >
> >     #define MIN_PAYLOAD_SIZE 1
> >
> > We use this in a few places in the compiler and add at least one word space in
> > the payload. However the comment is actually wrong, forwarding pointers are made
> > by tagging the info ptr field so we don't need a word in the payload for
> > forwarding pointers. I tried updating this as 0 but that caused a lot of test
> > failures (mostly in GHCi). I'm wondering if I'm missing anything or is it just
> > some code assuming min payload size 1 without using this macro.
> >
> > Any ideas?
> >
> > Ömer
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs