invoking front-end phases from within a GHC plugin?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

invoking front-end phases from within a GHC plugin?

Nicolas Frisby
I'd like to generate new declarations/bindings from within a GHC plugin. My
current concrete goal is just to add a new vanilla data definition, but
long-term it could be more open-ended. Crucially, these new declarations
will reference variables bound by the original module and its imports.

Long-term, it'd be nice if we could invoke the parser/renamer/typechecker
in order to do so. Broadly, the required plumbing seems doable. I wrote a
custom initTc that initializes the TcGblEnv from the ModGuts that the
plugin receives. This is enough to invoke the
renamer/typechecker/desugarer. The resulting new ModGuts can then be merged
into the original ModGuts, perhaps after some Simplification.

Has anyone explored/designed down this path? I've got some questions about
a snag I hit.

Parsing seems fine (superficially) but the renamer fails. I'm re-using the
global Rdr environment from the plugin's input ModGuts, so that free vars
from the new syntax will reference other bindings in the original module
and its imports. However, that environment is keyed on the uniques of
FastStrings, ultimately raising "name not found" errors.

This happens because the global variable for the FastString Table is not
being shared between the actual compiler and the new image of the GHC
library that the plugin is using. One solution seems to be including the
FastString Table in the  "reinitializeGlobals" mechanism. This would
require a GHC patch.

Questions:

1) Was the FastString Table intentionally left out of the
reinitializeGlobals mechanism?

2) Are there similar obstacles lurking beyond this one that people know
about? How incompatible is the invocation of front-end phases wrt the
current plugin architecture?

My design assumes the HscEnv and ModGuts received by the plugin include
enough information to reconstruct a meaningful TcGblEnv representing the
original module. This seems to be at least nearly true. I am, however, not
very well-versed with how imports are handled (EPS and such), so I don't
know if the FastString Table issue is just the tip of the iceberg.

Thanks for your thoughts.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130620/f1a4782b/attachment.htm>

Reply | Threaded
Open this post in threaded view
|

invoking front-end phases from within a GHC plugin?

Nicolas Frisby
Just a quick update: I patched a local copy of 7.6.3 to add
FastString.string_table to the CoreMonad.reinitializeGlobals mechanism.
After that change, I can invoke the front-end
(parser,renamer,typechecker,desugarer) from within my plugin on a simple
example.

Original module:

> module Test where
>
> data X = X Int Int

Syntax parser from within the plugin:

> data Y = Y X X
> f (Y (X a b) (X c d)) = a + b + c + d

Perhaps this simple example is luckily avoiding some nasty pitfalls, but it
seems to be working.


On Thu, Jun 20, 2013 at 11:27 AM, Nicolas Frisby
<nicolas.frisby at gmail.com>wrote:

> I'd like to generate new declarations/bindings from within a GHC plugin.
> My current concrete goal is just to add a new vanilla data definition, but
> long-term it could be more open-ended. Crucially, these new declarations
> will reference variables bound by the original module and its imports.
>
> Long-term, it'd be nice if we could invoke the parser/renamer/typechecker
> in order to do so. Broadly, the required plumbing seems doable. I wrote a
> custom initTc that initializes the TcGblEnv from the ModGuts that the
> plugin receives. This is enough to invoke the
> renamer/typechecker/desugarer. The resulting new ModGuts can then be merged
> into the original ModGuts, perhaps after some Simplification.
>
> Has anyone explored/designed down this path? I've got some questions about
> a snag I hit.
>
> Parsing seems fine (superficially) but the renamer fails. I'm re-using the
> global Rdr environment from the plugin's input ModGuts, so that free vars
> from the new syntax will reference other bindings in the original module
> and its imports. However, that environment is keyed on the uniques of
> FastStrings, ultimately raising "name not found" errors.
>
> This happens because the global variable for the FastString Table is not
> being shared between the actual compiler and the new image of the GHC
> library that the plugin is using. One solution seems to be including the
> FastString Table in the  "reinitializeGlobals" mechanism. This would
> require a GHC patch.
>
> Questions:
>
> 1) Was the FastString Table intentionally left out of the
> reinitializeGlobals mechanism?
>
> 2) Are there similar obstacles lurking beyond this one that people know
> about? How incompatible is the invocation of front-end phases wrt the
> current plugin architecture?
>
> My design assumes the HscEnv and ModGuts received by the plugin include
> enough information to reconstruct a meaningful TcGblEnv representing the
> original module. This seems to be at least nearly true. I am, however, not
> very well-versed with how imports are handled (EPS and such), so I don't
> know if the FastString Table issue is just the tip of the iceberg.
>
> Thanks for your thoughts.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130624/4beab12b/attachment.htm>

Reply | Threaded
Open this post in threaded view
|

invoking front-end phases from within a GHC plugin?

Carter Schonwald
This sounds somewhat complementary to some of the Template Haskell work
being done by Geoff Mainland to support Manuel's Inline Objective C style
template haskell machinery. Have you looked at that work at all? Sounds
like there might be some overlap!



On Mon, Jun 24, 2013 at 10:56 PM, Nicolas Frisby
<nicolas.frisby at gmail.com>wrote:

> Just a quick update: I patched a local copy of 7.6.3 to add
> FastString.string_table to the CoreMonad.reinitializeGlobals mechanism.
> After that change, I can invoke the front-end
> (parser,renamer,typechecker,desugarer) from within my plugin on a simple
> example.
>
> Original module:
>
> > module Test where
> >
> > data X = X Int Int
>
> Syntax parser from within the plugin:
>
> > data Y = Y X X
> > f (Y (X a b) (X c d)) = a + b + c + d
>
> Perhaps this simple example is luckily avoiding some nasty pitfalls, but
> it seems to be working.
>
>
> On Thu, Jun 20, 2013 at 11:27 AM, Nicolas Frisby <nicolas.frisby at gmail.com
> > wrote:
>
>> I'd like to generate new declarations/bindings from within a GHC plugin.
>> My current concrete goal is just to add a new vanilla data definition, but
>> long-term it could be more open-ended. Crucially, these new declarations
>> will reference variables bound by the original module and its imports.
>>
>> Long-term, it'd be nice if we could invoke the parser/renamer/typechecker
>> in order to do so. Broadly, the required plumbing seems doable. I wrote a
>> custom initTc that initializes the TcGblEnv from the ModGuts that the
>> plugin receives. This is enough to invoke the
>> renamer/typechecker/desugarer. The resulting new ModGuts can then be merged
>> into the original ModGuts, perhaps after some Simplification.
>>
>> Has anyone explored/designed down this path? I've got some questions
>> about a snag I hit.
>>
>> Parsing seems fine (superficially) but the renamer fails. I'm re-using
>> the global Rdr environment from the plugin's input ModGuts, so that free
>> vars from the new syntax will reference other bindings in the original
>> module and its imports. However, that environment is keyed on the uniques
>> of FastStrings, ultimately raising "name not found" errors.
>>
>> This happens because the global variable for the FastString Table is not
>> being shared between the actual compiler and the new image of the GHC
>> library that the plugin is using. One solution seems to be including the
>> FastString Table in the  "reinitializeGlobals" mechanism. This would
>> require a GHC patch.
>>
>> Questions:
>>
>> 1) Was the FastString Table intentionally left out of the
>> reinitializeGlobals mechanism?
>>
>> 2) Are there similar obstacles lurking beyond this one that people know
>> about? How incompatible is the invocation of front-end phases wrt the
>> current plugin architecture?
>>
>> My design assumes the HscEnv and ModGuts received by the plugin include
>> enough information to reconstruct a meaningful TcGblEnv representing the
>> original module. This seems to be at least nearly true. I am, however, not
>> very well-versed with how imports are handled (EPS and such), so I don't
>> know if the FastString Table issue is just the tip of the iceberg.
>>
>> Thanks for your thoughts.
>>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130626/03658401/attachment-0001.htm>

Reply | Threaded
Open this post in threaded view
|

invoking front-end phases from within a GHC plugin?

Nicolas Frisby
I hadn't, thanks for the pointer. At first glance, the methodologies look
distinct, however. My use case does not involve metaprogramming per se.
Thanks again for the response though; looks like an interesting project to
track.
On Jun 26, 2013 2:44 AM, "Carter Schonwald" <carter.schonwald at gmail.com>
wrote:

> This sounds somewhat complementary to some of the Template Haskell work
> being done by Geoff Mainland to support Manuel's Inline Objective C style
> template haskell machinery. Have you looked at that work at all? Sounds
> like there might be some overlap!
>
>
>
> On Mon, Jun 24, 2013 at 10:56 PM, Nicolas Frisby <nicolas.frisby at gmail.com
> > wrote:
>
>> Just a quick update: I patched a local copy of 7.6.3 to add
>> FastString.string_table to the CoreMonad.reinitializeGlobals mechanism.
>> After that change, I can invoke the front-end
>> (parser,renamer,typechecker,desugarer) from within my plugin on a simple
>> example.
>>
>> Original module:
>>
>> > module Test where
>> >
>> > data X = X Int Int
>>
>> Syntax parser from within the plugin:
>>
>> > data Y = Y X X
>> > f (Y (X a b) (X c d)) = a + b + c + d
>>
>> Perhaps this simple example is luckily avoiding some nasty pitfalls, but
>> it seems to be working.
>>
>>
>> On Thu, Jun 20, 2013 at 11:27 AM, Nicolas Frisby <
>> nicolas.frisby at gmail.com> wrote:
>>
>>> I'd like to generate new declarations/bindings from within a GHC plugin.
>>> My current concrete goal is just to add a new vanilla data definition, but
>>> long-term it could be more open-ended. Crucially, these new declarations
>>> will reference variables bound by the original module and its imports.
>>>
>>> Long-term, it'd be nice if we could invoke the
>>> parser/renamer/typechecker in order to do so. Broadly, the required
>>> plumbing seems doable. I wrote a custom initTc that initializes the
>>> TcGblEnv from the ModGuts that the plugin receives. This is enough to
>>> invoke the renamer/typechecker/desugarer. The resulting new ModGuts can
>>> then be merged into the original ModGuts, perhaps after some Simplification.
>>>
>>> Has anyone explored/designed down this path? I've got some questions
>>> about a snag I hit.
>>>
>>> Parsing seems fine (superficially) but the renamer fails. I'm re-using
>>> the global Rdr environment from the plugin's input ModGuts, so that free
>>> vars from the new syntax will reference other bindings in the original
>>> module and its imports. However, that environment is keyed on the uniques
>>> of FastStrings, ultimately raising "name not found" errors.
>>>
>>> This happens because the global variable for the FastString Table is not
>>> being shared between the actual compiler and the new image of the GHC
>>> library that the plugin is using. One solution seems to be including the
>>> FastString Table in the  "reinitializeGlobals" mechanism. This would
>>> require a GHC patch.
>>>
>>> Questions:
>>>
>>> 1) Was the FastString Table intentionally left out of the
>>> reinitializeGlobals mechanism?
>>>
>>> 2) Are there similar obstacles lurking beyond this one that people know
>>> about? How incompatible is the invocation of front-end phases wrt the
>>> current plugin architecture?
>>>
>>> My design assumes the HscEnv and ModGuts received by the plugin include
>>> enough information to reconstruct a meaningful TcGblEnv representing the
>>> original module. This seems to be at least nearly true. I am, however, not
>>> very well-versed with how imports are handled (EPS and such), so I don't
>>> know if the FastString Table issue is just the tip of the iceberg.
>>>
>>> Thanks for your thoughts.
>>>
>>
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130626/e67b5b93/attachment.htm>