Template Haskell changes to names and package keys

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Template Haskell changes to names and package keys

Edward Z. Yang
In GHC 7.10, we changed the internal representation of names to
be based on package keys (base_XXXXXX) rather than package IDs
(base-4.7.0.1), however, we forgot to update the Template Haskell
API to track these changes.  This lead to some bugs in TH
code which was synthesizing names by using package name and
version directly, e.g. https://ghc.haskell.org/trac/ghc/ticket/10279

We now propose the following changes to the TH API in order to
track these changes:

    1. Currently, a TH NameG contains a PkgName, defined as:

        newtype PkgName = PkgName String

       This is badly misleading, even in the old world order, since
       these needed version numbers as well. We propose that this be
       renamed to PkgKey:

        newtype PkgKey = PkgKey String
        mkPackageKey :: String -> PackageKey
        mkPackageKey = PkgKey

    2. Package keys are somewhat hard to synthesize, so we also
       offer an API for querying the package database of the GHC which
       is compiling your code for information about packages.  So,
       we introduce a new abstract data type:

        data Package
        packageKey :: Package -> PkgKey

       and some functions for getting packages:

        searchPackage :: String -- Package name
                      -> String -- Version
                      -> Q [Package]

        reifyPackage :: PkgKey -> Q Package

       We could add other functions (e.g., return all packages with a
       package name).

    3. Commonly, a user wants to get the package key of the current
       package.  Following Simon's suggestion, this will be done by
       augmenting ModuleInfo:

        data ModuleInfo =
            ModuleInfo { mi_this_mod :: Module -- new
                       , mi_imports :: [Module] }

       We'll also add a function for accessing the module package key:

        modulePackageKey :: Module -> PkgKey

       And a convenience function for accessing the current module:

        thisPackageKey :: Q PkgKey
        thisPackageKey = fmap (modulePackageKey . mi_this_mod) qReifyModule

        thisPackage :: Q Package
        thisPackage = reifyPackage =<< thisPackageKey

Discussion period: 1 month

Thanks,
Edward

(apologies to cc'd folks, I sent from my wrong email address)
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Dan Burton
Can you give (or link to) the rationale for the internal representation change? What use cases (or new corner cases for ghc-7.10) justify the added complexity here?

-- Dan Burton

On Fri, May 1, 2015 at 10:09 AM, Edward Z. Yang <[hidden email]> wrote:
In GHC 7.10, we changed the internal representation of names to
be based on package keys (base_XXXXXX) rather than package IDs
(base-4.7.0.1), however, we forgot to update the Template Haskell
API to track these changes.  This lead to some bugs in TH
code which was synthesizing names by using package name and
version directly, e.g. https://ghc.haskell.org/trac/ghc/ticket/10279

We now propose the following changes to the TH API in order to
track these changes:

    1. Currently, a TH NameG contains a PkgName, defined as:

        newtype PkgName = PkgName String

       This is badly misleading, even in the old world order, since
       these needed version numbers as well. We propose that this be
       renamed to PkgKey:

        newtype PkgKey = PkgKey String
        mkPackageKey :: String -> PackageKey
        mkPackageKey = PkgKey

    2. Package keys are somewhat hard to synthesize, so we also
       offer an API for querying the package database of the GHC which
       is compiling your code for information about packages.  So,
       we introduce a new abstract data type:

        data Package
        packageKey :: Package -> PkgKey

       and some functions for getting packages:

        searchPackage :: String -- Package name
                      -> String -- Version
                      -> Q [Package]

        reifyPackage :: PkgKey -> Q Package

       We could add other functions (e.g., return all packages with a
       package name).

    3. Commonly, a user wants to get the package key of the current
       package.  Following Simon's suggestion, this will be done by
       augmenting ModuleInfo:

        data ModuleInfo =
            ModuleInfo { mi_this_mod :: Module -- new
                       , mi_imports :: [Module] }

       We'll also add a function for accessing the module package key:

        modulePackageKey :: Module -> PkgKey

       And a convenience function for accessing the current module:

        thisPackageKey :: Q PkgKey
        thisPackageKey = fmap (modulePackageKey . mi_this_mod) qReifyModule

        thisPackage :: Q Package
        thisPackage = reifyPackage =<< thisPackageKey

Discussion period: 1 month

Thanks,
Edward

(apologies to cc'd folks, I sent from my wrong email address)
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Edward Z. Yang
Hello Dan,

https://ghc.haskell.org/trac/ghc/ticket/9265 I think summarizes
the reason why we shifted to this.  It's also important infrastructural
groundwork for a more sophisticated module system in Haskell.

Edward

Excerpts from Dan Burton's message of 2015-05-01 11:54:05 -0700:

> Can you give (or link to) the rationale for the internal representation
> change? What use cases (or new corner cases for ghc-7.10) justify the added
> complexity here?
>
> -- Dan Burton
>
> On Fri, May 1, 2015 at 10:09 AM, Edward Z. Yang <[hidden email]> wrote:
>
> > In GHC 7.10, we changed the internal representation of names to
> > be based on package keys (base_XXXXXX) rather than package IDs
> > (base-4.7.0.1), however, we forgot to update the Template Haskell
> > API to track these changes.  This lead to some bugs in TH
> > code which was synthesizing names by using package name and
> > version directly, e.g. https://ghc.haskell.org/trac/ghc/ticket/10279
> >
> > We now propose the following changes to the TH API in order to
> > track these changes:
> >
> >     1. Currently, a TH NameG contains a PkgName, defined as:
> >
> >         newtype PkgName = PkgName String
> >
> >        This is badly misleading, even in the old world order, since
> >        these needed version numbers as well. We propose that this be
> >        renamed to PkgKey:
> >
> >         newtype PkgKey = PkgKey String
> >         mkPackageKey :: String -> PackageKey
> >         mkPackageKey = PkgKey
> >
> >     2. Package keys are somewhat hard to synthesize, so we also
> >        offer an API for querying the package database of the GHC which
> >        is compiling your code for information about packages.  So,
> >        we introduce a new abstract data type:
> >
> >         data Package
> >         packageKey :: Package -> PkgKey
> >
> >        and some functions for getting packages:
> >
> >         searchPackage :: String -- Package name
> >                       -> String -- Version
> >                       -> Q [Package]
> >
> >         reifyPackage :: PkgKey -> Q Package
> >
> >        We could add other functions (e.g., return all packages with a
> >        package name).
> >
> >     3. Commonly, a user wants to get the package key of the current
> >        package.  Following Simon's suggestion, this will be done by
> >        augmenting ModuleInfo:
> >
> >         data ModuleInfo =
> >             ModuleInfo { mi_this_mod :: Module -- new
> >                        , mi_imports :: [Module] }
> >
> >        We'll also add a function for accessing the module package key:
> >
> >         modulePackageKey :: Module -> PkgKey
> >
> >        And a convenience function for accessing the current module:
> >
> >         thisPackageKey :: Q PkgKey
> >         thisPackageKey = fmap (modulePackageKey . mi_this_mod) qReifyModule
> >
> >         thisPackage :: Q Package
> >         thisPackage = reifyPackage =<< thisPackageKey
> >
> > Discussion period: 1 month
> >
> > Thanks,
> > Edward
> >
> > (apologies to cc'd folks, I sent from my wrong email address)
> > _______________________________________________
> > Libraries mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
> >
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Dan Burton
Thanks Edward.

#3 looks like an obvious win.

#1 looks like harmless bikeshedding. (Keep the old mkPkgName and pkgString and `type PkgName = PkgKey` as synonyms for a deprecation cycle or two?).

As for #2, searchPackage struck me as odd at first. What could a TH user do with a list of packages that are all "the same version"? As specified, it's rather opaque. However, it occurs to me that if the Package data type has some information about, for example, the [Package] that it was built against, then perhaps some useful things could be done via TH with this information.

So upon second look, it seems to me that the proposed points are really not all that complex. +1 from me, fwiw

-- Dan Burton

On Fri, May 1, 2015 at 1:08 PM, Edward Z. Yang <[hidden email]> wrote:
Hello Dan,

https://ghc.haskell.org/trac/ghc/ticket/9265 I think summarizes
the reason why we shifted to this.  It's also important infrastructural
groundwork for a more sophisticated module system in Haskell.

Edward

Excerpts from Dan Burton's message of 2015-05-01 11:54:05 -0700:
> Can you give (or link to) the rationale for the internal representation
> change? What use cases (or new corner cases for ghc-7.10) justify the added
> complexity here?
>
> -- Dan Burton
>
> On Fri, May 1, 2015 at 10:09 AM, Edward Z. Yang <[hidden email]> wrote:
>
> > In GHC 7.10, we changed the internal representation of names to
> > be based on package keys (base_XXXXXX) rather than package IDs
> > (base-4.7.0.1), however, we forgot to update the Template Haskell
> > API to track these changes.  This lead to some bugs in TH
> > code which was synthesizing names by using package name and
> > version directly, e.g. https://ghc.haskell.org/trac/ghc/ticket/10279
> >
> > We now propose the following changes to the TH API in order to
> > track these changes:
> >
> >     1. Currently, a TH NameG contains a PkgName, defined as:
> >
> >         newtype PkgName = PkgName String
> >
> >        This is badly misleading, even in the old world order, since
> >        these needed version numbers as well. We propose that this be
> >        renamed to PkgKey:
> >
> >         newtype PkgKey = PkgKey String
> >         mkPackageKey :: String -> PackageKey
> >         mkPackageKey = PkgKey
> >
> >     2. Package keys are somewhat hard to synthesize, so we also
> >        offer an API for querying the package database of the GHC which
> >        is compiling your code for information about packages.  So,
> >        we introduce a new abstract data type:
> >
> >         data Package
> >         packageKey :: Package -> PkgKey
> >
> >        and some functions for getting packages:
> >
> >         searchPackage :: String -- Package name
> >                       -> String -- Version
> >                       -> Q [Package]
> >
> >         reifyPackage :: PkgKey -> Q Package
> >
> >        We could add other functions (e.g., return all packages with a
> >        package name).
> >
> >     3. Commonly, a user wants to get the package key of the current
> >        package.  Following Simon's suggestion, this will be done by
> >        augmenting ModuleInfo:
> >
> >         data ModuleInfo =
> >             ModuleInfo { mi_this_mod :: Module -- new
> >                        , mi_imports :: [Module] }
> >
> >        We'll also add a function for accessing the module package key:
> >
> >         modulePackageKey :: Module -> PkgKey
> >
> >        And a convenience function for accessing the current module:
> >
> >         thisPackageKey :: Q PkgKey
> >         thisPackageKey = fmap (modulePackageKey . mi_this_mod) qReifyModule
> >
> >         thisPackage :: Q Package
> >         thisPackage = reifyPackage =<< thisPackageKey
> >
> > Discussion period: 1 month
> >
> > Thanks,
> > Edward
> >
> > (apologies to cc'd folks, I sent from my wrong email address)
> > _______________________________________________
> > Libraries mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
> >


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Michael Sloan
In reply to this post by Edward Z. Yang
+1 on #2 and #3.

Being able to get all the packages that have a particular name sounds good!  Note that if "-hide-package" / "-hide-all-packages" is used, I wouldn't want those packages in the results.

Considering that we also have the Module / ModuleInfo datatypes, to me it makes sense that we'd also be able to get a [Module] from a Package.


Like Dan, I'm a little less keen on #1.  Here's why:

* PkgName is in the internal "Language.Haskell.TH.Syntax" module

* It isn't actually documented what it means, so it's meaning can be freely bent.  There's no guarantee that you should be able to generate them.

* I couldn't find any examples of its usage that would be broken by this new semantics.  I've done a search on github[1] to find usages of PkgName, and I only found one use[2] that uses PkgName which would be affected by this change.  This example is in the TH lib itself, and so would hopefully be fixed by this change.

On the other hand, since it's rarely used, such an API breakage wouldn't be that impactful, so it's not that big of a deal.


On Fri, May 1, 2015 at 10:09 AM, Edward Z. Yang <[hidden email]> wrote:
In GHC 7.10, we changed the internal representation of names to
be based on package keys (base_XXXXXX) rather than package IDs
(base-4.7.0.1), however, we forgot to update the Template Haskell
API to track these changes.  This lead to some bugs in TH
code which was synthesizing names by using package name and
version directly, e.g. https://ghc.haskell.org/trac/ghc/ticket/10279

We now propose the following changes to the TH API in order to
track these changes:

    1. Currently, a TH NameG contains a PkgName, defined as:

        newtype PkgName = PkgName String

       This is badly misleading, even in the old world order, since
       these needed version numbers as well. We propose that this be
       renamed to PkgKey:

        newtype PkgKey = PkgKey String
        mkPackageKey :: String -> PackageKey
        mkPackageKey = PkgKey

    2. Package keys are somewhat hard to synthesize, so we also
       offer an API for querying the package database of the GHC which
       is compiling your code for information about packages.  So,
       we introduce a new abstract data type:

        data Package
        packageKey :: Package -> PkgKey

       and some functions for getting packages:

        searchPackage :: String -- Package name
                      -> String -- Version
                      -> Q [Package]

        reifyPackage :: PkgKey -> Q Package

       We could add other functions (e.g., return all packages with a
       package name).

    3. Commonly, a user wants to get the package key of the current
       package.  Following Simon's suggestion, this will be done by
       augmenting ModuleInfo:

        data ModuleInfo =
            ModuleInfo { mi_this_mod :: Module -- new
                       , mi_imports :: [Module] }

       We'll also add a function for accessing the module package key:

        modulePackageKey :: Module -> PkgKey

       And a convenience function for accessing the current module:

        thisPackageKey :: Q PkgKey
        thisPackageKey = fmap (modulePackageKey . mi_this_mod) qReifyModule

        thisPackage :: Q Package
        thisPackage = reifyPackage =<< thisPackageKey

Discussion period: 1 month

Thanks,
Edward

(apologies to cc'd folks, I sent from my wrong email address)
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Edward Z. Yang
In reply to this post by Dan Burton
Excerpts from Dan Burton's message of 2015-05-01 15:05:18 -0700:
> #1 looks like harmless bikeshedding. (Keep the old mkPkgName and pkgString
> and `type PkgName = PkgKey` as synonyms for a deprecation cycle or two?).

It's not that harmless: there really was a semantic change, and most
users of mkPkgName did, in fact, break when GHC 7.10 was released (this
change was prompted by two separate reports of TH code which was
synthesizing these IDs breaking.)

> As for #2, searchPackage struck me as odd at first. What could a TH user do
> with a list of packages that are all "the same version"? As specified, it's
> rather opaque. However, it occurs to me that if the Package data type has
> some information about, for example, the [Package] that it was built
> against, then perhaps some useful things could be done via TH with this
> information.

Yes. Admittedly, I have a hard time thinking of uses for this function,
but it doesn't cost us too much to add.

Edward
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Edward Z. Yang
In reply to this post by Michael Sloan
> Being able to get all the packages that have a particular name sounds
> good!  Note that if "-hide-package" / "-hide-all-packages" is used, I
> wouldn't want those packages in the results.

This is not entirely clear; you could imagine a package also coming with
an 'exposed' bit, so you could filter for exposed/not exposed pacakges.

> Like Dan, I'm a little less keen on #1.  Here's why:
>
> * PkgName is in the internal "Language.Haskell.TH.Syntax" module
>
> * It isn't actually documented what it means, so it's meaning can be freely
> bent.  There's no guarantee that you should be able to generate them.
>
> * I couldn't find any examples of its usage that would be broken by this
> new semantics.  I've done a search on github[1] to find usages of PkgName,
> and I only found one use[2] that uses PkgName which would be affected by
> this change.  This example is in the TH lib itself, and so would hopefully
> be fixed by this change.
>
> On the other hand, since it's rarely used, such an API breakage wouldn't be
> that impactful, so it's not that big of a deal.

Actually, this proposal was prompted by two separate reports of
breakage:

    https://github.com/ekmett/lens/issues/496
    https://ghc.haskell.org/trac/ghc/ticket/10279

So, like it or not, people seem to be depending on this API, which means
it's worth at least discussing a little when we change it :)

Cheers,
Edward
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Michael Sloan


On Fri, May 1, 2015 at 5:48 PM, Edward Z. Yang <[hidden email]> wrote:
> Being able to get all the packages that have a particular name sounds
> good!  Note that if "-hide-package" / "-hide-all-packages" is used, I
> wouldn't want those packages in the results.

This is not entirely clear; you could imagine a package also coming with
an 'exposed' bit, so you could filter for exposed/not exposed pacakges.

> Like Dan, I'm a little less keen on #1.  Here's why:
>
> * PkgName is in the internal "Language.Haskell.TH.Syntax" module
>
> * It isn't actually documented what it means, so it's meaning can be freely
> bent.  There's no guarantee that you should be able to generate them.
>
> * I couldn't find any examples of its usage that would be broken by this
> new semantics.  I've done a search on github[1] to find usages of PkgName,
> and I only found one use[2] that uses PkgName which would be affected by
> this change.  This example is in the TH lib itself, and so would hopefully
> be fixed by this change.
>
> On the other hand, since it's rarely used, such an API breakage wouldn't be
> that impactful, so it's not that big of a deal.

Actually, this proposal was prompted by two separate reports of
breakage:

    https://github.com/ekmett/lens/issues/496
    https://ghc.haskell.org/trac/ghc/ticket/10279

So, like it or not, people seem to be depending on this API, which means
it's worth at least discussing a little when we change it :)

Hmm, while that is a rather exotic circumstance (building a stage1 cross compiler), it is legitimate.  Also, it occurs to me that one benefit of breaking compatibility is not only for old code, but also future code.  This will ensure that authors of new TH code are aware of this distinction if the attempt to build on older GHCs.

So, I don't really feel strongly about it, and changing PkgKey seems fine.  TH doesn't need to be the stablest of APIs.
 
Cheers,
Edward


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Bertram Felgenhauer-2
In reply to this post by Michael Sloan
Hi Michael,

> Being able to get all the packages that have a particular name sounds
> good!  Note that if "-hide-package" / "-hide-all-packages" is used, I
> wouldn't want those packages in the results.

Same here.

> Like Dan, I'm a little less keen on #1.  Here's why:
>
[...]
>
> * I couldn't find any examples of its usage that would be broken by this
> new semantics.  I've done a search on github[1] to find usages of PkgName,
> and I only found one use[2] that uses PkgName which would be affected by
> this change.  This example is in the TH lib itself, and so would hopefully
> be fixed by this change.

Did you find any uses where an existing PkgName is reused to construct
a new name? I believe this would be an important data point for deciding
whether renaming the constructor is a good idea or not. Basically I'm
asking whether the majority of uses of PkgName "out in the wild" right
now is correct or incorrect.

Cheers,

Bertram
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Edward Kmett-2
In reply to this post by Edward Z. Yang
I'm fully on board with cleaning up the story around package keys, so +1 on the mission, but I have very few preferences on the particulars.

-Edward

On Fri, May 1, 2015 at 1:06 PM, Edward Z. Yang <[hidden email]> wrote:
In GHC 7.10, we changed the internal representation of names to
be based on package keys (base_XXXXXX) rather than package IDs
(base-4.7.0.1), however, we forgot to update the Template Haskell
API to track these changes.  This lead to some bugs in TH
code which was synthesizing names by using package name and
version directly, e.g. https://ghc.haskell.org/trac/ghc/ticket/10279

We now propose the following changes to the TH API in order to
track these changes:

    1. Currently, a TH NameG contains a PkgName, defined as:

        newtype PkgName = PkgName String

       This is badly misleading, even in the old world order, since
       these needed version numbers as well. We propose that this be
       renamed to PkgKey:

        newtype PkgKey = PkgKey String
        mkPackageKey :: String -> PackageKey
        mkPackageKey = PkgKey

    2. Package keys are somewhat hard to synthesize, so we also
       offer an API for querying the package database of the GHC which
       is compiling your code for information about packages.  So,
       we introduce a new abstract data type:

        data Package
        packageKey :: Package -> PkgKey

       and some functions for getting packages:

        searchPackage :: String -- Package name
                      -> String -- Version
                      -> Q [Package]

        reifyPackage :: PkgKey -> Q Package

       We could add other functions (e.g., return all packages with a
       package name).

    3. Commonly, a user wants to get the package key of the current
       package.  Following Simon's suggestion, this will be done by
       augmenting ModuleInfo:

        data ModuleInfo =
            ModuleInfo { mi_this_mod :: Module -- new
                       , mi_imports :: [Module] }

       We'll also add a function for accessing the module package key:

        modulePackageKey :: Module -> PkgKey

       And a convenience function for accessing the current module:

        thisPackageKey :: Q PkgKey
        thisPackageKey = fmap (modulePackageKey . mi_this_mod) qReifyModule

        thisPackage :: Q Package
        thisPackage = reifyPackage =<< thisPackageKey

Discussion period: 1 month

Thanks,
Edward



_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Michael Sloan
In reply to this post by Bertram Felgenhauer-2


On Sat, May 2, 2015 at 2:38 AM, Bertram Felgenhauer <[hidden email]> wrote:
Hi Michael,

> Being able to get all the packages that have a particular name sounds
> good!  Note that if "-hide-package" / "-hide-all-packages" is used, I
> wouldn't want those packages in the results.

Same here.

> Like Dan, I'm a little less keen on #1.  Here's why:
>
[...]
>
> * I couldn't find any examples of its usage that would be broken by this
> new semantics.  I've done a search on github[1] to find usages of PkgName,
> and I only found one use[2] that uses PkgName which would be affected by
> this change.  This example is in the TH lib itself, and so would hopefully
> be fixed by this change.

Did you find any uses where an existing PkgName is reused to construct
a new name? I believe this would be an important data point for deciding
whether renaming the constructor is a good idea or not. Basically I'm
asking whether the majority of uses of PkgName "out in the wild" right
now is correct or incorrect.

I didn't search for that, since I was searching for explicit uses of PkgName.  The results for NameG are pretty interesting, though:


It seems like most of the results are either `Lift`ing a PkgName or re-using a PkgName from another name.  However, there are a fair number of uses which explicitly construct the name, though.  In particular, there are a fair number of usages of this to generate references to the tuple constructors in GHC.Prim.

Interestingly, these usages of (mkPkgName "ghc-prim") don't have version numbers at all.  This implies that the behavior is to select the version of the package which is currently being used.

Considering how much more convenient this is than manually looking up package names, I'm strongly in favor of keeping PkgName and its current behavior, and extending it to handle package keys.  Ideally, also documenting the different formats it expects.  In particular, "name", "name-version", and "name-version-key" (I'm guessing).

The issue here is that we may be linking against two different packages which are named the same thing, right?  This was always a possibility, and having package keys just makes it so that we need the ability to disambiguate packages beyond version numbers.

So, how about for this rare-ish case where there we're linking against two different versions of the foo package, and the TH manually generates a (PkgName "foo"), we generate an ambiguity error.

If we force users to search for and specify fully unambiguous package names then that pushes the boilerplate of package selection out to TH usage.  It seems like pretty much everyone will want "select the version of the package that I'm using."  That said, I'm also in favor of adding the function mentioned in #2, for those cases where the default isn't sufficient (I'm anticipating very few of these).

-Michael

Cheers,

Bertram
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Edward Z. Yang
> I didn't search for that, since I was searching for explicit uses of
> PkgName.  The results for NameG are pretty interesting, though:
>
> https://github.com/search?l=haskell&q=NameG&type=Code&utf8=%E2%9C%93
>
> It seems like most of the results are either `Lift`ing a PkgName or
> re-using a PkgName from another name.  However, there are a fair number of
> uses which explicitly construct the name, though.  In particular, there are
> a fair number of usages of this to generate references to the tuple
> constructors in GHC.Prim.
>
> Interestingly, these usages of (mkPkgName "ghc-prim") don't have version
> numbers at all.  This implies that the behavior is to select the version of
> the package which is currently being used.

Well, the reason they don't have version numbers is because ghc-prim
(and a few other packages) are internally considered wired-in packages,
so their package keys are always some specific, hard-coded value.
There is no "implied behavior", it's just an implementation detail that
this TH API happens to expose.  (There is one concrete consequence of
this, which is that it's not possible to have two versions of ghc-prim
simultaneously in the same executable.)

Prior to 7.10, this manifested as ghc-prim having PkgName
"ghc-prim", but lens having PkgName "lens-0.3.2".

> So, how about for this rare-ish case where there we're linking against two
> different versions of the foo package, and the TH manually generates a
> (PkgName "foo"), we generate an ambiguity error.
>
> If we force users to search for and specify fully unambiguous package names
> then that pushes the boilerplate of package selection out to TH usage.  It
> seems like pretty much everyone will want "select the version of the
> package that I'm using."  That said, I'm also in favor of adding the
> function mentioned in #2, for those cases where the default isn't
> sufficient (I'm anticipating very few of these).

I looked at the results.

    https://github.com/bitemyapp/hackage-packages/blob/fd9649f426254c0581bd976789a1c384eda0e3c9/lighttpd-conf-0.4/src/Lighttpd/Conf/Instances/Lift.hs
    This is just very wrong, they should clearly switch to thisPackageKey when we have it

    https://github.com/s9gf4ult/letqq/blob/472d665cf64ec60b5f02f200c2b4c54fc6580f3f/src/THTH.hs
    I'm not really sure what this is supposed to be, besides some
    meta-meta-programming library. It's clearly prototype code and I
    don't think they will mind if it stops working.

    https://github.com/suhailshergill/liboleg/blob/57673d01c66ab9f284579a40aed059ed4617ce6c/Data/Symbolic/TypedCodeAux.hs
    https://github.com/bitemyapp/hackage-packages/blob/fd9649f426254c0581bd976789a1c384eda0e3c9/liboleg-2010.1.10.0/Data/Symbolic/TypedCodeAux.hs
    So, this piece of code is something that really ought to be in the
    standard library, if it isn't already.  Basically, the problem is
    that when you quote an identifier, e.g. [| foo |], you get the
    *un-renamed* syntax (a variable foo) rather than the renamed syntax
    (somepkg:Data.Foo.foo).  This is very useful, too useful to be in
    another library.

    https://github.com/andrep/hoc/blob/bfd0391bf0dab4bda5d6a5f7845fab19f8e4b9a9/hoc/HOC/HOC/TH.hs
    https://github.com/mokus0/hoc/blob/b6fa3906b8e1e61bed0435a8d497a132e5905227/hoc/HOC/HOC/TH.hs
    Wouldn't be broken.

    https://github.com/DavidAlphaFox/ghc/blob/6a5d9fa147b8abc370159d87e9c3dac87171cbd5/libraries/template-haskell/Language/Haskell/TH/Quote.hs
    This (and the next page of results) are from the template-haskell
    library. We can fix these ourselves when we make these changes.

To sum up, I still think we should rename, and I think we should
also add some functionality to the standard library for getting the
renamed version of a TH syntax tree.

Edward
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Richard Eisenberg-2
In reply to this post by Edward Z. Yang
Thanks for spearheading this update.

I'm +1 on #1, in particular because I don't think TH should try too hard to have a stable API.

I'm +0.5 on #2, because the interface as designed below is too weak. I'm +1 for a stronger interface. For example:

packageVersionString :: Package -> String   -- maybe use a newtype here? or the proper Cabal datatype?
packageName          :: Package -> String   -- use the old PkgName here? or is that confusing?

packageDependencies :: Package -> Q [Package]
  -- could be very useful in printing debugging information. Then, if you're writing a library that
  -- is sensitive to dependency versions, you can print this info out when you panic

I'm -1 on #3 if it will break code. You can already get current package information through the `qLocation` function.

One need I've had a few times is to be able to get a version number for the current package, just for UI nicety. It would be great if this change could support such an operation (like my packageVersionString, above).

Thanks!
Richard

PS: I wonder if this proposal and debate wouldn't be easier to follow in wiki format. That way, the design could evolve over time...

On May 1, 2015, at 1:09 PM, Edward Z. Yang <[hidden email]> wrote:

> In GHC 7.10, we changed the internal representation of names to
> be based on package keys (base_XXXXXX) rather than package IDs
> (base-4.7.0.1), however, we forgot to update the Template Haskell
> API to track these changes.  This lead to some bugs in TH
> code which was synthesizing names by using package name and
> version directly, e.g. https://ghc.haskell.org/trac/ghc/ticket/10279
>
> We now propose the following changes to the TH API in order to
> track these changes:
>
>    1. Currently, a TH NameG contains a PkgName, defined as:
>
>        newtype PkgName = PkgName String
>
>       This is badly misleading, even in the old world order, since
>       these needed version numbers as well. We propose that this be
>       renamed to PkgKey:
>
>        newtype PkgKey = PkgKey String
>        mkPackageKey :: String -> PackageKey
>        mkPackageKey = PkgKey
>
>    2. Package keys are somewhat hard to synthesize, so we also
>       offer an API for querying the package database of the GHC which
>       is compiling your code for information about packages.  So,
>       we introduce a new abstract data type:
>
>        data Package
>        packageKey :: Package -> PkgKey
>
>       and some functions for getting packages:
>
>        searchPackage :: String -- Package name
>                      -> String -- Version
>                      -> Q [Package]
>
>        reifyPackage :: PkgKey -> Q Package
>
>       We could add other functions (e.g., return all packages with a
>       package name).
>
>    3. Commonly, a user wants to get the package key of the current
>       package.  Following Simon's suggestion, this will be done by
>       augmenting ModuleInfo:
>
>        data ModuleInfo =
>            ModuleInfo { mi_this_mod :: Module -- new
>                       , mi_imports :: [Module] }
>
>       We'll also add a function for accessing the module package key:
>
>        modulePackageKey :: Module -> PkgKey
>
>       And a convenience function for accessing the current module:
>
>        thisPackageKey :: Q PkgKey
>        thisPackageKey = fmap (modulePackageKey . mi_this_mod) qReifyModule
>
>        thisPackage :: Q Package
>        thisPackage = reifyPackage =<< thisPackageKey
>
> Discussion period: 1 month
>
> Thanks,
> Edward
>
> (apologies to cc'd folks, I sent from my wrong email address)
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Edward Z. Yang
In reply to this post by Edward Z. Yang
Excerpts from Edward Z. Yang's message of 2015-05-03 13:22:00 -0700:
>     https://github.com/suhailshergill/liboleg/blob/57673d01c66ab9f284579a40aed059ed4617ce6c/Data/Symbolic/TypedCodeAux.hs
>     https://github.com/bitemyapp/hackage-packages/blob/fd9649f426254c0581bd976789a1c384eda0e3c9/liboleg-2010.1.10.0/Data/Symbolic/TypedCodeAux.hs
>     So, this piece of code is something that really ought to be in the
>     standard library, if it isn't already.  Basically, the problem is
>     that when you quote an identifier, e.g. [| foo |], you get the
>     *un-renamed* syntax (a variable foo) rather than the renamed syntax
>     (somepkg:Data.Foo.foo).  This is very useful, too useful to be in
>     another library.

I have to backtrack on this statement; I misinterpreted what this code
is doing; the problem here is we need a Lift instance for the Exp data
type.  We should just add this instance (or use the Data generated one).

Edward
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Edward Z. Yang
In reply to this post by Richard Eisenberg-2
Hello Richard,

Thanks for the comments.

I've created a wiki page here:
https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/PackageKeyChanges

> packageVersionString :: Package -> String   -- maybe use a newtype here? or the proper Cabal datatype?
> packageName          :: Package -> String   -- use the old PkgName here? or is that confusing?
>
> packageDependencies :: Package -> Q [Package]
>   -- could be very useful in printing debugging information. Then, if you're writing a library that
>   -- is sensitive to dependency versions, you can print this info out when you panic

OK, we can plan on adding this information.

> I'm -1 on #3 if it will break code. You can already get current
> package information through the `qLocation` function.

This is true. However, qLocation returns a String. Should we change
it to return a PkgKey? (Could break users of the information.)

I don't know know if the name and version should get newtyped.  If we
do newtype them, should qLocation also be updated accordingly?

Thanks,
Edward
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: Template Haskell changes to names and package keys

Simon Peyton Jones
In reply to this post by Edward Z. Yang
Why do we need two types: PkgKey and Package?   Can't we make them the same?

It's good that Package be abstract (as proposed below). Then we are free to add more metadata to it

I'd urge that Module too should be abstract, for the same reason


Simon


|  -----Original Message-----
|  From: Libraries [mailto:[hidden email]] On Behalf Of
|  Edward Z. Yang
|  Sent: 01 May 2015 18:09
|  To: libraries
|  Subject: Template Haskell changes to names and package keys
|  
|  In GHC 7.10, we changed the internal representation of names to be
|  based on package keys (base_XXXXXX) rather than package IDs (base-
|  4.7.0.1), however, we forgot to update the Template Haskell API to
|  track these changes.  This lead to some bugs in TH code which was
|  synthesizing names by using package name and version directly, e.g.
|  https://ghc.haskell.org/trac/ghc/ticket/10279
|  
|  We now propose the following changes to the TH API in order to track
|  these changes:
|  
|      1. Currently, a TH NameG contains a PkgName, defined as:
|  
|          newtype PkgName = PkgName String
|  
|         This is badly misleading, even in the old world order, since
|         these needed version numbers as well. We propose that this be
|         renamed to PkgKey:
|  
|          newtype PkgKey = PkgKey String
|          mkPackageKey :: String -> PackageKey
|          mkPackageKey = PkgKey
|  
|      2. Package keys are somewhat hard to synthesize, so we also
|         offer an API for querying the package database of the GHC which
|         is compiling your code for information about packages.  So,
|         we introduce a new abstract data type:
|  
|          data Package
|          packageKey :: Package -> PkgKey
|  
|         and some functions for getting packages:
|  
|          searchPackage :: String -- Package name
|                        -> String -- Version
|                        -> Q [Package]
|  
|          reifyPackage :: PkgKey -> Q Package
|  
|         We could add other functions (e.g., return all packages with a
|         package name).
|  
|      3. Commonly, a user wants to get the package key of the current
|         package.  Following Simon's suggestion, this will be done by
|         augmenting ModuleInfo:
|  
|          data ModuleInfo =
|              ModuleInfo { mi_this_mod :: Module -- new
|                         , mi_imports :: [Module] }
|  
|         We'll also add a function for accessing the module package key:
|  
|          modulePackageKey :: Module -> PkgKey
|  
|         And a convenience function for accessing the current module:
|  
|          thisPackageKey :: Q PkgKey
|          thisPackageKey = fmap (modulePackageKey . mi_this_mod)
|  qReifyModule
|  
|          thisPackage :: Q Package
|          thisPackage = reifyPackage =<< thisPackageKey
|  
|  Discussion period: 1 month
|  
|  Thanks,
|  Edward
|  
|  (apologies to cc'd folks, I sent from my wrong email address)
|  _______________________________________________
|  Libraries mailing list
|  [hidden email]
|  http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: Template Haskell changes to names and package keys

Simon Peyton Jones
In reply to this post by Michael Sloan

I didn't search for that, since I was searching for explicit uses of PkgName.  The results for NameG are pretty interesting, though:

https://github.com/search?l=haskell&q=NameG&type=Code&utf8=%E2%9C%93

It seems like most of the results are either `Lift`ing a PkgName or re-using a PkgName from another name.  However, there are a fair number of uses which explicitly construct the name, though.  In particular, there are a fair number of usages of this to generate references to the tuple constructors in GHC.Prim.

 

Very good exercise!  Looking for how an existing API is used (perhaps in a clumsy way, because of the inadequacies of the existing API) is a good guide to improving it.

 

e.g. If tuple names are an issue, let’s provide a TH API for getting their names!!

 

Simon

 

 

From: Libraries [mailto:[hidden email]] On Behalf Of Michael Sloan
Sent: 02 May 2015 23:46
To: Bertram Felgenhauer
Cc: Haskell Libraries
Subject: Re: Template Haskell changes to names and package keys

 

 

 

On Sat, May 2, 2015 at 2:38 AM, Bertram Felgenhauer <[hidden email]> wrote:

Hi Michael,

> Being able to get all the packages that have a particular name sounds
> good!  Note that if "-hide-package" / "-hide-all-packages" is used, I
> wouldn't want those packages in the results.

Same here.

> Like Dan, I'm a little less keen on #1.  Here's why:
>
[...]
>
> * I couldn't find any examples of its usage that would be broken by this
> new semantics.  I've done a search on github[1] to find usages of PkgName,
> and I only found one use[2] that uses PkgName which would be affected by
> this change.  This example is in the TH lib itself, and so would hopefully
> be fixed by this change.

Did you find any uses where an existing PkgName is reused to construct
a new name? I believe this would be an important data point for deciding
whether renaming the constructor is a good idea or not. Basically I'm
asking whether the majority of uses of PkgName "out in the wild" right
now is correct or incorrect.


I didn't search for that, since I was searching for explicit uses of PkgName.  The results for NameG are pretty interesting, though:

 

 

It seems like most of the results are either `Lift`ing a PkgName or re-using a PkgName from another name.  However, there are a fair number of uses which explicitly construct the name, though.  In particular, there are a fair number of usages of this to generate references to the tuple constructors in GHC.Prim.

 

Interestingly, these usages of (mkPkgName "ghc-prim") don't have version numbers at all.  This implies that the behavior is to select the version of the package which is currently being used.

 

Considering how much more convenient this is than manually looking up package names, I'm strongly in favor of keeping PkgName and its current behavior, and extending it to handle package keys.  Ideally, also documenting the different formats it expects.  In particular, "name", "name-version", and "name-version-key" (I'm guessing).

 

The issue here is that we may be linking against two different packages which are named the same thing, right?  This was always a possibility, and having package keys just makes it so that we need the ability to disambiguate packages beyond version numbers.

 

So, how about for this rare-ish case where there we're linking against two different versions of the foo package, and the TH manually generates a (PkgName "foo"), we generate an ambiguity error.

 

If we force users to search for and specify fully unambiguous package names then that pushes the boilerplate of package selection out to TH usage.  It seems like pretty much everyone will want "select the version of the package that I'm using."  That said, I'm also in favor of adding the function mentioned in #2, for those cases where the default isn't sufficient (I'm anticipating very few of these).

 

-Michael

 

Cheers,

Bertram

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

 


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: Template Haskell changes to names and package keys

Edward Z. Yang
Excerpts from Simon Peyton Jones's message of 2015-05-05 01:15:50 -0700:
> Very good exercise!  Looking for how an existing API is used (perhaps in a clumsy way, because of the inadequacies of the existing API) is a good guide to improving it.
>
> e.g. If tuple names are an issue, let’s provide a TH API for getting their names!!

Hello Simon,

The right and proper way of getting a tuple name (well, constructor
really, but that's the only reason people want names) should be:

    [| (,) |]

But people sometimes don't want to use quotes, e.g. as in
https://github.com/ekmett/lens/issues/496 where they want to
work with stage1 GHC.

So in this case, https://ghc.haskell.org/trac/ghc/ticket/10382 (making
quotes work with stage1 GHC) will help a lot.

Edward
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Template Haskell changes to names and package keys

Adam Vogt
On Tue, May 5, 2015 at 10:50 AM, Edward Z. Yang <[hidden email]> wrote:
Excerpts from Simon Peyton Jones's message of 2015-05-05 01:15:50 -0700:
> Very good exercise!  Looking for how an existing API is used (perhaps in a clumsy way, because of the inadequacies of the existing API) is a good guide to improving it.
>
> e.g. If tuple names are an issue, let’s provide a TH API for getting their names!!

Hello Simon,

The right and proper way of getting a tuple name (well, constructor
really, but that's the only reason people want names) should be:

    [| (,) |]

But people sometimes don't want to use quotes, e.g. as in
https://github.com/ekmett/lens/issues/496 where they want to
work with stage1 GHC.

So in this case, https://ghc.haskell.org/trac/ghc/ticket/10382 (making
quotes work with stage1 GHC) will help a lot.

Language.Haskell.TH already exports tupleTypeName tupleDataName

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries