splicing types

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

splicing types

Robert Greayer
Hi,

I've been using TH to implement a library that generates code in a 'foreign' language that corresponds to a haskell ADT, as well as the haskell & foreign code required to transmit (i.e. serialize and deserialize) values back and forth between a haskell program and the foreign program. I've run into some situations in which it would be useful for there to be a way to control the code generation declaratively by providing additional meta-data to the code generator, beyond just the ADT(s) themselves.  For example, suppose in a group of ADTs, certain fields should be omitted when serialized, and therefore not represented in the 'foreign' types, e.g.:

> data Bar {
>         val1 :: String,
>         val2 :: Int -- omit this in serialization
>     }

As a 'first order' stab at handling this issue, I thought of using type aliases, e.g

> type Omitted a = a
...
>         val2 :: Omitted Int

which could be generalized to a sort of 'annotation type', e.g.

> type Ann a b = a
> type Omitted = () -- a particular annotation
...
>        val2 ::  Ann Int Omitted

you could have multiple annotations, e.g.
>        val2 :: Ann Int (Omitted,Ann1,Ann2) -- an Int with three annotations

The code generator 'sees' the aliases, and can therefore act on the ones it understands.

This works, but I immediately run into situations where more flexibility is needed.  Indeed, in the example I've chose ('omit'), all works ok when serializing the data, but what about *de* serializing (in haskell)?  Some value needs to be chosen for the omitted field.  An arbitrary default of (say) zero may not be appropriate.  What I'd really like to be able to do is parameterize my annotations with values.  'Omitted' would be parameterized with the default value to use, e.g.

> type Omitted a = ()
...
>        val2 :: Ann Int (Omitted 1000)

But of course this doesn't work, as I'm dealing with types here, not expressions.  I could do something extremely hokey (hokier even than this system of annotations), like:

> type I1000 = ()
...
>        val2 :: Ann Int (Omitted I1000) -- read as 'omitted, with default of 1000'

The code generator would determine the default value based on the name of the type 'Omitted' is applied to.  The user of the annotation system would have the ugly task of creating 'dummy' type aliases for any values that need passing (and it would be quite cumbersome to attempt a naming convention for strings, etc.)

It's at this point that I had the (for me) clever idea of simply writing a 'type level encoder' for any haskell expression, and then splicing in the encoding at the appropriate point, e.g. my integer as a tuple-type of binary digits: (One,Zero,Zero,One) would be a type encoding of '9', given the existence of type (aliases) such as:

> type One = ()
> type Zero = ()

A function to encode an integer into the TH.Syntax representation of such a tuple --

(AppT (AppT (AppT (AppT (TupleT 4) (ConT ''One)) (ConT ''Zero)) (ConT ''Zero)) (ConT ''One))

is fairly trivial, as is a function to decode it back into an integer.  It should (I think) be possible to write a pair of functions:

> encT :: Q Exp -> Q Type
> decT :: Q Type -> Q Exp

to encode/decode arbitrary expressions into/from types.

which would then allow my annotations to look like:

>         val2 :: Ann Int (Omitted $(encT [|1000|]))

and allow me to annotate types with just about anything else, as well, with the ugly details of encoding/decoding hidden from both the user of the annotation system, and any code generation system that makes use of annotations.  Which I found quite pleasing, since this 'quasi-annotations' feature is probably more powerful than some similar, 'intentional' annotation features of other languages, despite (afaik) not being an envisaged use of the language features involved.

But then I tried it out and found that I'm trying to splice into what is currently an invalid splice position for TH.  (Indeed, on the very day that I decided to try my idea out concretely, the relevant ticket affecting type declarations as valid splice positions was downgraded to low priority, and disincluded from GHC 6.10 -- http://hackage.haskell.org/trac/ghc/ticket/1476).  So I was wondering if 1) splicing types is inseparable from the  (harder?) problem of splicing patterns and 2) if splicing types is something that will is likely at some point to be implemented (is it that a good solution isn't yet apparent, or that it's just not important enough for the goals of TH?) and 3) has this idea, or something similar, been explored (regardless of the fact that it won't work with TH as-implemented) before?

Thanks,
rcg


     
_______________________________________________
template-haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/template-haskell
Reply | Threaded
Open this post in threaded view
|

Re: splicing types

Claus Reinke
>> data Bar {
>>         val1 :: String,
>>         val2 :: Int -- omit this in serialization
>>     }
..
> which could be generalized to a sort of 'annotation type', e.g.
>
>> type Ann a b = a
>> type Omitted = () -- a particular annotation
> ...
>>        val2 ::  Ann Int Omitted
>
> you could have multiple annotations, e.g.
>>        val2 :: Ann Int (Omitted,Ann1,Ann2) -- an Int with three annotations

try googling for "phantom types"?

..
>> type Omitted a = ()
> ...
>>        val2 :: Ann Int (Omitted 1000)

if you bake the default into the type, you can have only one
per type per program, so how about using a class to map
from type to default value?

    class Default a where default :: a
   
    valn :: Default a => a

    instance Default Bar where default = Bar undefined 1000
or
    instance Default Int where default = 1000

then you could omit types that have defaults, or use/infer an
omit annotation for fields that the default for the enclosing
type can fill in. Parsing a slightly extended data syntax might
be easier than overloading the type system, but maybe that
is just me?-)

Claus

ps if you absolutely want to go for more type-level
    programming, Oleg has some notes on number-
    parameterized types
    http://okmij.org/ftp/Haskell/number-parameterized-types.html

_______________________________________________
template-haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/template-haskell
Reply | Threaded
Open this post in threaded view
|

RE: splicing types

Simon Peyton Jones
In reply to this post by Robert Greayer
Robert

| disincluded from GHC 6.10 -- http://hackage.haskell.org/trac/ghc/ticket/1476).
| So I was wondering if 1)
| splicing types is inseparable from the  (harder?) problem of splicing patterns
| and 2) if splicing types is something that will is likely at some point
| to be implemented (is it that a good solution isn't yet
| apparent, or that it's just not important enough for the goals of TH?)
| and 3) has this idea, or
| something similar, been explored (regardless of the fact that it won't
| work with TH as-implemented) before?

Interesting.  I hope that others will, like Claus, suggest other ways of looking at your problem.   For example, Max Bolingbroke in his SoC project is thinking about programmer "annotations" or "attributes" that might propagate robustly down the compiler pipeline.
        http://hackage.haskell.org/trac/ghc/wiki/Plugins/Annotations
Your example might serve as an interesting use-case for him.

You mention a special code generator -- who writes that?

>         val2 :: Ann Int (Omitted $(encT [|1000|]))

Could you instead write a declaration splice?
        $(sigEncode "val2" 100)
where sigEncode generates a suitable declaration?


But meanwhile to answer your questions

1) I think that splicing types is very much easier than splicing patterns; unlike the latter, splicing types raises few problems of principle.  The only one I remember is this. If you see this
        f :: Int -> a -> (a,b) -> a
then what you mean (adding the implicit forall) is this
        f :: forall a b. Int -> a -> (a,b) -> a
But if you see
        f :: Int -> a -> $(foo 3)
then what do you mean?  What implicit foralls are added?

2)  I don't think this is insoluble at all, but someone needs to work it all out and implement it.  I have postponed doing this until sufficient user pressure arises!  It's just a bandwidth problem -- I am totally snowed under at the moment.  Would anyone care to help?

3) Claus's pointer to Oleg's stuff is relevant here I think.

Simon



_______________________________________________
template-haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/template-haskell
Reply | Threaded
Open this post in threaded view
|

Re: splicing types

Marc Weber
In reply to this post by Robert Greayer
Hi Robert,

Funny to see that you have had quite much the same ideas as me.
I've been trying to find a way to encode some meta information
describing relations between database tables. My first idea was using
phantom types as well and I started coding and implement a lot of pattern matching ..
Suddenly I noticed that I don't have to encode / decode the whole stuff
when simply creating my own representation (see my other post. There is
an example)..

Do in your case it could look like this:

$(
        let myData = MyData [ MyCon {
                                name = "Bar",
                                types = [ ("val1", 'String, DoSerialize), ("val1", 'String, NoSerialization) ]
                        } ]
        in createSerializationAndDataTypes myData
)

Drawback: You need to learn this different syntax (That was one reason to introduce Quasi-quoting, correct?)
                It's not as easy to read as
                        Omitted Type
Benefits: You only have to write an encoder to create the data type
        which should be easy. And you'll need some kind of representation
        anyway, do you? If you have this you can experiment with many more
        things such as extending haskell-src, DrIft to support real annotations (you
        can invent your very own syntax then ..) etc.

        Haskell doesn't have to box unbox the Omitted data type which
        maybe can be a speed penalty ? (Not sure about this)

        It could look like this then (maybe I'm abusing quasi-quoting here, I haven't used it yet)
[:defineDataAndSerialization |
        data Bar {
                val1 :: String,
                val2 :: Int (@ OmitDefault [|20|]) -- omit this in serialization
            }
]

Marc Weber
_______________________________________________
template-haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/template-haskell
Reply | Threaded
Open this post in threaded view
|

RE: splicing types

Robert Greayer
In reply to this post by Simon Peyton Jones

--- On Tue, 7/1/08, Simon Peyton-Jones <[hidden email]> wrote:

>
> You mention a special code generator -- who writes that?

Anyone writing a library which generates code by reifying/analysing user-supplied code (e.g. Neil Mitchell's derive, http://www-users.cs.york.ac.uk/~ndm/derive/) could potentially make use of programmer supplied meta-data associated with the (library user's) code.  So there's any number of special code generators which could make use of some annotation facility.  Essentially, the author of the code generator specifies the annotations it understands (e.g. in my formulation, 'type Omitted defval = ()') and is otherwise a client of the annotation facility (which, in my formulation, means simply using 'decT' to decode values associated with annotations).  So you have (in my formulation of annotations) an annotation support library which contains encT and decT and probably 'type Ann a b = a', and everything else is just a use case for annotations.  I hope this answers your question.

(My formulation, aside from not working, has the disadvantage of obfuscating declarations; a 'proper' annotations feature, such as the GHC plugins/annotation project might not have this problem.  Also the burden of type checking falls squarely on the author of the code generator.  'Omitted $(encT "hello")' would be perfectly valid at the splice point.  Also not a problem for a 'real' annotations feature, one would think.)

> Could you instead write a declaration splice?
>         $(sigEncode "val2" 100)
> where sigEncode generates a suitable declaration?

Not quite; I elided the surrounding context for brevity:
> data Bar {
>         val1 :: String,
>         val2 :: Ann Int (Omitted $(encT 1000))
>     }

I could generate the entire declaration for Bar (this, I think, is the approach Marc Weber is taking, as he points out in his response to my original message) with a declaration splice, but not the just portion of the declaration that I wanted to annotate, the declaration of the field 'val2', i.e.

> data Bar {
>     val1 :: String,
>     $(sigEncode "val2" 1000) -- still an invalid splice position
> }
----

> 2)  I don't think this is insoluble at all, but someone
> needs to work it all out and implement it.  I have postponed
> doing this until sufficient user pressure arises!  It's
> just a bandwidth problem -- I am totally snowed under at
> the moment.  Would anyone care to help?

Well, I'll certainly spelunk into the GHC to see how this might work.  Whether this will eventually translate into anything resembling 'help' is another question.

Thanks to you, and Claus, for the pointers!

Regards,
rcg


     
_______________________________________________
template-haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/template-haskell
Reply | Threaded
Open this post in threaded view
|

RE: splicing types

Robert Greayer

In trying to explain something, I wrote:

(Omitted $(encT 1000))

and (Omitted $(encT "hello"))

but I meant (Omitted $(encT [|1000|]) and (Omitted $(encT [|"hello"|]).  I.e. encT is a (monomorphic) function of type (Q Exp -> Q Type), as specified in my original message.


     
_______________________________________________
template-haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/template-haskell
Reply | Threaded
Open this post in threaded view
|

Re: splicing types

Bugzilla from alfonso.acosta@gmail.com
In reply to this post by Robert Greayer
I think that the type-level library:
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/type-level

It allows  to encode and make computations over numbers in the
type-level. For example, decimal number 100 is encoded as:

D1 :* D0 :* D0

For syntactic convenience, there's a module containing aliases (i.e.
type synonyms) with which you can write D100 instead of D1 :* D0 :*
D0.

In case the number you want to encode is out of the aliases range, you
can always encode it by hand (i.e. D1 :* D0 :* D0) or splice it with
dec2TypeLevel (http://hackage.haskell.org/packages/archive/type-level/0.2/doc/html/Data-TypeLevel-Num-Aliases-TH.html#v%3Adec2TypeLevel
)

I hope that helps. If you have any questions about the library or need
a new feature, please let me know.

On Mon, Jun 30, 2008 at 10:14 PM, Robert Greayer <[hidden email]> wrote:

> Hi,
>
> I've been using TH to implement a library that generates code in a 'foreign' language that corresponds to a haskell ADT, as well as the haskell & foreign code required to transmit (i.e. serialize and deserialize) values back and forth between a haskell program and the foreign program. I've run into some situations in which it would be useful for there to be a way to control the code generation declaratively by providing additional meta-data to the code generator, beyond just the ADT(s) themselves.  For example, suppose in a group of ADTs, certain fields should be omitted when serialized, and therefore not represented in the 'foreign' types, e.g.:
>
>> data Bar {
>>         val1 :: String,
>>         val2 :: Int -- omit this in serialization
>>     }
>
> As a 'first order' stab at handling this issue, I thought of using type aliases, e.g
>
>> type Omitted a = a
> ...
>>         val2 :: Omitted Int
>
> which could be generalized to a sort of 'annotation type', e.g.
>
>> type Ann a b = a
>> type Omitted = () -- a particular annotation
> ...
>>        val2 ::  Ann Int Omitted
>
> you could have multiple annotations, e.g.
>>        val2 :: Ann Int (Omitted,Ann1,Ann2) -- an Int with three annotations
>
> The code generator 'sees' the aliases, and can therefore act on the ones it understands.
>
> This works, but I immediately run into situations where more flexibility is needed.  Indeed, in the example I've chose ('omit'), all works ok when serializing the data, but what about *de* serializing (in haskell)?  Some value needs to be chosen for the omitted field.  An arbitrary default of (say) zero may not be appropriate.  What I'd really like to be able to do is parameterize my annotations with values.  'Omitted' would be parameterized with the default value to use, e.g.
>
>> type Omitted a = ()
> ...
>>        val2 :: Ann Int (Omitted 1000)
>
> But of course this doesn't work, as I'm dealing with types here, not expressions.  I could do something extremely hokey (hokier even than this system of annotations), like:
>
>> type I1000 = ()
> ...
>>        val2 :: Ann Int (Omitted I1000) -- read as 'omitted, with default of 1000'
>
> The code generator would determine the default value based on the name of the type 'Omitted' is applied to.  The user of the annotation system would have the ugly task of creating 'dummy' type aliases for any values that need passing (and it would be quite cumbersome to attempt a naming convention for strings, etc.)
>
> It's at this point that I had the (for me) clever idea of simply writing a 'type level encoder' for any haskell expression, and then splicing in the encoding at the appropriate point, e.g. my integer as a tuple-type of binary digits: (One,Zero,Zero,One) would be a type encoding of '9', given the existence of type (aliases) such as:
>
>> type One = ()
>> type Zero = ()
>
> A function to encode an integer into the TH.Syntax representation of such a tuple --
>
> (AppT (AppT (AppT (AppT (TupleT 4) (ConT ''One)) (ConT ''Zero)) (ConT ''Zero)) (ConT ''One))
>
> is fairly trivial, as is a function to decode it back into an integer.  It should (I think) be possible to write a pair of functions:
>
>> encT :: Q Exp -> Q Type
>> decT :: Q Type -> Q Exp
>
> to encode/decode arbitrary expressions into/from types.
>
> which would then allow my annotations to look like:
>
>>         val2 :: Ann Int (Omitted $(encT [|1000|]))
>
> and allow me to annotate types with just about anything else, as well, with the ugly details of encoding/decoding hidden from both the user of the annotation system, and any code generation system that makes use of annotations.  Which I found quite pleasing, since this 'quasi-annotations' feature is probably more powerful than some similar, 'intentional' annotation features of other languages, despite (afaik) not being an envisaged use of the language features involved.
>
> But then I tried it out and found that I'm trying to splice into what is currently an invalid splice position for TH.  (Indeed, on the very day that I decided to try my idea out concretely, the relevant ticket affecting type declarations as valid splice positions was downgraded to low priority, and disincluded from GHC 6.10 -- http://hackage.haskell.org/trac/ghc/ticket/1476).  So I was wondering if 1) splicing types is inseparable from the  (harder?) problem of splicing patterns and 2) if splicing types is something that will is likely at some point to be implemented (is it that a good solution isn't yet apparent, or that it's just not important enough for the goals of TH?) and 3) has this idea, or something similar, been explored (regardless of the fact that it won't work with TH as-implemented) before?
>
> Thanks,
> rcg
>
>
>
> _______________________________________________
> template-haskell mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/template-haskell
>
_______________________________________________
template-haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/template-haskell
Reply | Threaded
Open this post in threaded view
|

Re: splicing types

Bugzilla from alfonso.acosta@gmail.com
On Wed, Jul 2, 2008 at 11:37 AM, Alfonso Acosta
<[hidden email]> wrote:
> I think that the type-level library:
> http://hackage.haskell.org/cgi-bin/hackage-scripts/package/type-level

... is what you need
_______________________________________________
template-haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/template-haskell