True multi stage Haskell

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

True multi stage Haskell

Tim Sheard
After many years of hoping someone else would do this, I would like to
make GHC into a true multi-stage programming language. Here is how I
thought I might approach this.

1) Use the GHC as a library module.
2) Use the LLVM backend.

I have no experience with either of these tools.
Lets start simple, How would I write functions like

compile :: String -> IO PtrToLLVMCode  -- where the string is a small
Haskell program.
llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode
jumpTo:: PtrToMachineCode -> IO ans   -- where ans is the "type" of the
string.


Any thoughts on how to get started? What papers to read, examples to
look at?

I'd love to move to some more disciplined input type, a sort of (mono)
typed program
representation (with similar complexity) to Template Haskell Exp type.

where (Exp t) is a data structure representing a Haskell program of type t.

All offers of advice accepted. I'm trying to get started soon, and good
advice
about what to avoid is especially welcome.  If any one wanted to help
with this,
that would be great.

Tim Sheard

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: True multi stage Haskell

Matthew Pickering
Hi Tim,

When you say "multi-stage programming language" do you have a specific
calculus in mind? I think this can mean lots of different things to
different people.

This is a topic I have been interested in recently.

Cheers,

Matt

On Fri, Nov 17, 2017 at 6:05 PM, Tim Sheard <[hidden email]> wrote:

> After many years of hoping someone else would do this, I would like to
> make GHC into a true multi-stage programming language. Here is how I
> thought I might approach this.
>
> 1) Use the GHC as a library module.
> 2) Use the LLVM backend.
>
> I have no experience with either of these tools.
> Lets start simple, How would I write functions like
>
> compile :: String -> IO PtrToLLVMCode  -- where the string is a small
> Haskell program.
> llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode
> jumpTo:: PtrToMachineCode -> IO ans   -- where ans is the "type" of the
> string.
>
>
> Any thoughts on how to get started? What papers to read, examples to look
> at?
>
> I'd love to move to some more disciplined input type, a sort of (mono) typed
> program
> representation (with similar complexity) to Template Haskell Exp type.
>
> where (Exp t) is a data structure representing a Haskell program of type t.
>
> All offers of advice accepted. I'm trying to get started soon, and good
> advice
> about what to avoid is especially welcome.  If any one wanted to help with
> this,
> that would be great.
>
> Tim Sheard
>
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: True multi stage Haskell

Iavor Diatchki
Hello Tim,

I don't know the answers to your questions exactly, but I've played around with something that I think might be related, so I thought I'd share.   The example code is here:


This module contains a function that compiles (at runtime) a Haskell module represented as a String to the value of whatever `main` happened to be defined in this module.
Actually, in this case the expression is taken as a `Doc` , which is the result of pretty printing something, but this doc is simply "show"-ed to make a string (see `modText`).

The function works in roughly two steps:
    1. Compile the module:
        1.1. Setup some compiler flags, this has to do with the dependencies of the module (i.e., what packages it depends on, etc)
        1.2. Define a "target":  this is what GHC will be compiling.  There is a bit of weirdness here:  GHC can take the input to compile either from a text file, or from an in-memory buffer.  The second is what we want here.  Unfortunately, for some reason GHC still insists on there being a file around (its contents is completely irrelevant) I think it just want to check when it was last modified :-)    This is why we create an empty file.     This is something that should probably be fixed in GHC.
        1.3 Load all the targets:  this is what will actually compile the module
    2. Get the value of `main` in this module:  this is done in the `Succeed` branch of the case, where we create an expression `M.main` and tell GHC to evaluate it.

Depending on how you need things to be setup, just the second step might be enough (i.e., call `compileExpr`).   You'd still need to do enough to tell GHC in what context it will be compiling the expression though.

I am not an expert on this topic, but I'd be happy to try to answer more questions if I can.

-Iavor



On Fri, Nov 17, 2017 at 10:42 AM Matthew Pickering <[hidden email]> wrote:
Hi Tim,

When you say "multi-stage programming language" do you have a specific
calculus in mind? I think this can mean lots of different things to
different people.

This is a topic I have been interested in recently.

Cheers,

Matt

On Fri, Nov 17, 2017 at 6:05 PM, Tim Sheard <[hidden email]> wrote:
> After many years of hoping someone else would do this, I would like to
> make GHC into a true multi-stage programming language. Here is how I
> thought I might approach this.
>
> 1) Use the GHC as a library module.
> 2) Use the LLVM backend.
>
> I have no experience with either of these tools.
> Lets start simple, How would I write functions like
>
> compile :: String -> IO PtrToLLVMCode  -- where the string is a small
> Haskell program.
> llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode
> jumpTo:: PtrToMachineCode -> IO ans   -- where ans is the "type" of the
> string.
>
>
> Any thoughts on how to get started? What papers to read, examples to look
> at?
>
> I'd love to move to some more disciplined input type, a sort of (mono) typed
> program
> representation (with similar complexity) to Template Haskell Exp type.
>
> where (Exp t) is a data structure representing a Haskell program of type t.
>
> All offers of advice accepted. I'm trying to get started soon, and good
> advice
> about what to avoid is especially welcome.  If any one wanted to help with
> this,
> that would be great.
>
> Tim Sheard
>
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: True multi stage Haskell

Shayan Najd Javadipour-2
In reply to this post by Tim Sheard
Hi Tim, 

About using GHC as a library module, have you see the ongoing work on "native metaprogramming" in GHC at

and a child project at
 
There, online code generation is not of the highest priority, rather we are trying to open up the compiler from top to down: 
making available GHC's AST and then the parser and the typechecker have priority over the other functionalities down the pipeline (like the code generator).

If any one wanted to help with this, that would be great.

I am interested to hear more about the approach you have in mind, and would be happy to help.

Yours, 
  Shayan

On Fri, Nov 17, 2017 at 6:05 PM, Tim Sheard <[hidden email]> wrote:
After many years of hoping someone else would do this, I would like to
make GHC into a true multi-stage programming language. Here is how I
thought I might approach this.

1) Use the GHC as a library module.
2) Use the LLVM backend.

I have no experience with either of these tools.
Lets start simple, How would I write functions like

compile :: String -> IO PtrToLLVMCode  -- where the string is a small Haskell program.
llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode
jumpTo:: PtrToMachineCode -> IO ans   -- where ans is the "type" of the string.


Any thoughts on how to get started? What papers to read, examples to look at?

I'd love to move to some more disciplined input type, a sort of (mono) typed program
representation (with similar complexity) to Template Haskell Exp type.

where (Exp t) is a data structure representing a Haskell program of type t.

All offers of advice accepted. I'm trying to get started soon, and good advice
about what to avoid is especially welcome.  If any one wanted to help with this,
that would be great.

Tim Sheard

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: True multi stage Haskell

Joachim Breitner-2
In reply to this post by Tim Sheard
Hi Tim,

Am Freitag, den 17.11.2017, 13:05 -0500 schrieb Tim Sheard:

> After many years of hoping someone else would do this, I would like to
> make GHC into a true multi-stage programming language. Here is how I
> thought I might approach this.
>
> 1) Use the GHC as a library module.
> 2) Use the LLVM backend.
>
> I have no experience with either of these tools.
>
> Lets start simple, How would I write functions like
>
> compile :: String -> IO PtrToLLVMCode  -- where the string is a small
> Haskell program.
> llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode
> jumpTo:: PtrToMachineCode -> IO ans   -- where ans is the "type" of the
> string.
maybe not quite what you are looking for, but veggies, at
https://github.com/nomeata/veggies
uses the GHC API build a “new” Haskell compiler with its own Core→LLVM
pass.

Cheers,
Joachim
--
Joachim Breitner
  [hidden email]
  http://www.joachim-breitner.de/

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: True multi stage Haskell

John Lato-2
In reply to this post by Tim Sheard
Hi Tim,

Several years ago I wrote a proof of concept of one way to implement this in Haskell, http://johnlato.blogspot.com/2012/10/runtime-meta-programming-in-haskell.html.

I used TH to automatically lift expressions into a newtype-wrapped ExpQ that could be used for staged evaluation. More than one stage would probably have been tedious though, and a major problem with any non-trivial code was getting all the imports in place. I do think it would have been possible to automatically track imports though.

Of course that was 5 years ago, so state of the art has moved on. TH has better facilities for this now, and the ghc API probably has something better too.


On Fri, Nov 17, 2017, 10:06 Tim Sheard <[hidden email]> wrote:
After many years of hoping someone else would do this, I would like to
make GHC into a true multi-stage programming language. Here is how I
thought I might approach this.

1) Use the GHC as a library module.
2) Use the LLVM backend.

I have no experience with either of these tools.
Lets start simple, How would I write functions like

compile :: String -> IO PtrToLLVMCode  -- where the string is a small
Haskell program.
llvmCompile :: PtrToLLVMCode -> IO PtrToMachineCode
jumpTo:: PtrToMachineCode -> IO ans   -- where ans is the "type" of the
string.


Any thoughts on how to get started? What papers to read, examples to
look at?

I'd love to move to some more disciplined input type, a sort of (mono)
typed program
representation (with similar complexity) to Template Haskell Exp type.

where (Exp t) is a data structure representing a Haskell program of type t.

All offers of advice accepted. I'm trying to get started soon, and good
advice
about what to avoid is especially welcome.  If any one wanted to help
with this,
that would be great.

Tim Sheard

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs