GHC imposes an upper bound of 62 on the size of tuples one can use [1]. Currently, this upper bound applies to both boxed and unboxed tuples alike. For example, if you try to create an unboxed tuple of arity 64, then GHC will throw an error:
error: A 64-tuple is too large for GHC (max size is 62) Workaround: use nested tuples or define a data type | | f = (#,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,#) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ However, I discovered recently that there are places where GHC *does* use unboxed tuples with arity greater than 62. For example, the GHC.Prim.unpackInt8X64# [2] function returns an unboxed tuple of size 64. I was confused for a while about how this was even possible, but I realized later than GHC only enforces the tuple size limit in expressions and patterns [3]. Simply having a type signature with a large unboxed tuple is fine in and of itself, and since unpackInt8X64# is implemented as a primop, no large unboxed tuples are ever used in the "body" of the function. (Indeed, primops don't have function bodies in the conventional sense.) Other functions in GHC.Prim that use unboxed tuples of arity 64 include unpackWord8X64# [4], packInt8X64# [5], and packWord8X64# [6]. But this makes me wonder: how on earth is it even possible to *use* unpackInt8X64#? The only thing that comes to mind is to pattern-match on the unboxed tuple that it returns, but if you try to do that, you'll get a "64-tuple is too large for GHC" error like shown above. What's more, I can't find any test cases in the GHC test suite that show how unpackInt8X64# or its cousins are supposed to be used. What am I missing? The comments for these functions mention "Warning: this is only available on LLVM", so perhaps someone familiar with the LLVM backend knows what the answer is? For some context, this came up in !4097, where I am attempting to extend the "too large for GHC" error message to include types, in addition to expressions and patterns [7]. However, doing so causes Haddock to error when processing unpackInt8X64# et al. Ryan S. ----- [1] https://gitlab.haskell.org/ghc/ghc/-/blob/a1f34d37b47826e86343e368a5c00f1a4b1f2bce/compiler/GHC/Settings/Constants.hs#L13-15 [2] https://hackage.haskell.org/package/ghc-prim-0.6.1/docs/GHC-Prim.html#v:unpackInt8X64-35- [3] https://gitlab.haskell.org/ghc/ghc/-/issues/18723 [4] https://hackage.haskell.org/package/ghc-prim-0.6.1/docs/GHC-Prim.html#v:unpackWord8X64-35- [5] https://hackage.haskell.org/package/ghc-prim-0.6.1/docs/GHC-Prim.html#v:packInt8X64-35- [6] https://hackage.haskell.org/package/ghc-prim-0.6.1/docs/GHC-Prim.html#v:packWord8X64-35- [7] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4097#note_301191 _______________________________________________ ghc-devs mailing list [hidden email] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs |
On September 25, 2020 6:21:23 PM EDT, Ryan Scott <[hidden email]> wrote:
... >However, I discovered recently that there are places where GHC *does* >use >unboxed tuples with arity greater than 62. For example, the >GHC.Prim.unpackInt8X64# [2] function returns an unboxed tuple of size >64. I >was confused for a while about how this was even possible, but I >realized >later than GHC only enforces the tuple size limit in expressions and >patterns [3]. Simply having a type signature with a large unboxed tuple >is >fine in and of itself, and since unpackInt8X64# is implemented as a >primop, >no large unboxed tuples are ever used in the "body" of the function. >(Indeed, primops don't have function bodies in the conventional sense.) >Other functions in GHC.Prim that use unboxed tuples of arity 64 include >unpackWord8X64# [4], packInt8X64# [5], and packWord8X64# [6]. > >But this makes me wonder: how on earth is it even possible to *use* >unpackInt8X64#? I strongly suspect that the answer here is "you can't yet no one has noticed until now." The SIMD operations were essentially introduced as a technology preview and therefore never had proper tests added. Only a subset of these operations have any tests at all and I doubt anyone has attempted to use the 64-wide operations, which are rather specialized. Cheers, - Ben _______________________________________________ ghc-devs mailing list [hidden email] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs |
I had a feeling that this might be the case. Unfortunately, this technology preview is actively blocking progress on
!4097, which leaves me at a loss for what to do. I can see two ways forward: 1. Remove
unpackInt8X64#
and friends. 2. Reconsider whether the tuple size limit should apply to unboxed tuples. Perhaps this size limit only makes sense for boxed tuples? This comment [1] suggests that defining a boxed tuple of size greater than 62 induces a segfault, but it's unclear to me if the same thing happens for unboxed tuples. Ryan S. ----- On Sat, Sep 26, 2020 at 7:54 AM Ben Gamari <[hidden email]> wrote: On September 25, 2020 6:21:23 PM EDT, Ryan Scott <[hidden email]> wrote: _______________________________________________ ghc-devs mailing list [hidden email] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs |
Luite is currently working on unboxed tuple support in the interpreter. This will also be limited, as getting a generic solution for arbitrary sized tuples raises a lot of complications. Thus form a practical point of view, I’d go for (1) ;-) We’ll need to rethink and get SIMD proper support at some point though, the lack of such is rather sad. On Sat, 26 Sep 2020 at 8:27 PM, Ryan Scott <[hidden email]> wrote:
_______________________________________________ ghc-devs mailing list [hidden email] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs |
In reply to this post by Ryan Scott
I think it would be worth trying to add tuples up to width 64. The only real cost here is the interface file size of GHC.Tuple and if adding a 63-wide tuple really does induce a crash then that is a bug in its own right that deserves investigation.
- Ben On September 26, 2020 8:26:32 AM EDT, Ryan Scott <[hidden email]> wrote:
-- Sent from my Android device with K-9 Mail. Please excuse my brevity. _______________________________________________ ghc-devs mailing list [hidden email] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs |
I think as long as it's bounded it's ok. On Sat, Sep 26, 2020 at 8:52 PM Ben Gamari <[hidden email]> wrote:
_______________________________________________ ghc-devs mailing list [hidden email] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs |
In reply to this post by Ryan Scott
Ryan, the comment in GHC.Tuple seems to be misleading today, as suggested by this SO post: https://stackoverflow.com/questions/46412823/why-are-ghc-tuples-limited-to-size-62
I was going to add it to the comment at some point but was hesitant as I hadn’t verified the information myself. V
_______________________________________________ ghc-devs mailing list [hidden email] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs |
In reply to this post by Moritz Angermann-2
The complication for arbitrary sized unboxed tuples in the interpreter
was mainly because we don't want to generate new Cmm on the fly for each tuple size. That one probably doesn't apply here, but maybe others do. It looks like a fully general solution would require a change to the calling convention, which required more changes than I hoped. For now it there will be a limitation in tuple size; the number of words passed on the stack in the native calling convention: https://gitlab.haskell.org/luite/ghc/-/commit/f83d08f0247bd7fd590b9be450c6d2d0968caa3c#6ca006a5d6dfdfdb97d0dd72db322e3f6eaa6214_198_217 It's probably not a big problem if GHCi has some size limit here, even if GHC doesn't (or has a higher limit) Bumping tuple size limits a bit to make the primops usable (testable?) looks okay to me. Luite On Sat, Sep 26, 2020 at 2:39 PM Moritz Angermann <[hidden email]> wrote: > > Luite is currently working on unboxed tuple support in the interpreter. This will also be limited, as getting a generic solution for arbitrary sized tuples raises a lot of complications. > > Thus form a practical point of view, I’d go for (1) ;-) > > We’ll need to rethink and get SIMD proper support at some point though, the lack of such is rather sad. > > On Sat, 26 Sep 2020 at 8:27 PM, Ryan Scott <[hidden email]> wrote: >> >> I had a feeling that this might be the case. Unfortunately, this technology preview is actively blocking progress on >> >> !4097, which leaves me at a loss for what to do. I can see two ways forward: >> >> 1. Remove >> >> unpackInt8X64# >> >> >> >> and friends. >> 2. Reconsider whether the tuple size limit should apply to unboxed tuples. Perhaps this size limit only makes sense for boxed tuples? This comment [1] suggests that defining a boxed tuple of size greater than 62 induces a segfault, but it's unclear to me if the same thing happens for unboxed tuples. >> >> Ryan S. >> ----- >> [1] https://gitlab.haskell.org/ghc/ghc/-/blob/a1f34d37b47826e86343e368a5c00f1a4b1f2bce/libraries/ghc-prim/GHC/Tuple.hs#L170 >> >> >> >> >> On Sat, Sep 26, 2020 at 7:54 AM Ben Gamari <[hidden email]> wrote: >>> >>> On September 25, 2020 6:21:23 PM EDT, Ryan Scott <[hidden email]> wrote: >>> >>> >>> ... >>> >>> >>> >However, I discovered recently that there are places where GHC *does* >>> >>> >>> >use >>> >>> >>> >unboxed tuples with arity greater than 62. For example, the >>> >>> >>> >GHC.Prim.unpackInt8X64# [2] function returns an unboxed tuple of size >>> >>> >>> >64. I >>> >>> >>> >was confused for a while about how this was even possible, but I >>> >>> >>> >realized >>> >>> >>> >later than GHC only enforces the tuple size limit in expressions and >>> >>> >>> >patterns [3]. Simply having a type signature with a large unboxed tuple >>> >>> >>> >is >>> >>> >>> >fine in and of itself, and since unpackInt8X64# is implemented as a >>> >>> >>> >primop, >>> >>> >>> >no large unboxed tuples are ever used in the "body" of the function. >>> >>> >>> >(Indeed, primops don't have function bodies in the conventional sense.) >>> >>> >>> >Other functions in GHC.Prim that use unboxed tuples of arity 64 include >>> >>> >>> >unpackWord8X64# [4], packInt8X64# [5], and packWord8X64# [6]. >>> >>> >>> > >>> >>> >>> >But this makes me wonder: how on earth is it even possible to *use* >>> >>> >>> >unpackInt8X64#? >>> >>> >>> >>> >>> >>> >>> >>> >>> I strongly suspect that the answer here is "you can't yet no one has noticed until now." The SIMD operations were essentially introduced as a technology preview and therefore never had proper tests added. Only a subset of these operations have any tests at all and I doubt anyone has attempted to use the 64-wide operations, which are rather specialized. >>> >>> >>> >>> >>> >>> Cheers, >>> >>> >>> >>> >>> >>> - Ben >>> >>> >> >> >> _______________________________________________ >> >> ghc-devs mailing list >> >> [hidden email] >> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> ghc-devs mailing list [hidden email] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs |
In reply to this post by Ryan Scott
Hmm. It’s not clear to me why we have *any* limitation on the size of unboxed tuples.
Why do we need a limit at all? Simon From: ghc-devs <[hidden email]>
On Behalf Of Ryan Scott
GHC imposes an upper bound of 62 on the size of tuples one can use [1]. Currently, this upper bound applies to both boxed and unboxed tuples alike. For example, if you try to create an unboxed tuple of arity 64, then GHC will throw an error: _______________________________________________ ghc-devs mailing list [hidden email] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs |
In reply to this post by Vilem-Benjamin Liepelt
>
Ryan, the comment in GHC.Tuple seems to be misleading today, as suggested by this SO post: https://stackoverflow.com/questions/46412823/why-are-ghc-tuples-limited-to-size-62 > I was going to add it to the comment at some point but was hesitant as I hadn’t verified the information myself. For what it's worth, Ben has removed this comment in !4146 [1], where he increases the maximum tuple size to 64. This is enough to unblock me on progressing with
!4097, so I'm happy. There seems to be a more general question of whether it makes sense to enforce a size limit at all for unboxed tuples, but I'll leave that question to people more knowledgeable than I am. Ryan S. -----
_______________________________________________ ghc-devs mailing list [hidden email] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs |
Free forum by Nabble | Edit this page |