Arrows and pickler combinators

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Arrows and pickler combinators

Joel Reymont
Folks,

I'm trying to monadify the pickler code. sequ below positively looks  
like >>= but you can't really join both pickle and unpickle into a  
single monad. I would like to keep the ops together, though, as this  
allows me a single specification for both pickling and unpickling.

Cale suggested that PU is really an arrow in that it supports both  
input and output. I could not find an example of such an arrow,  
though. I thought that it could be a "dual arrow" but then could not  
find a description for one.

I would appreciate your suggestions! The original paper is at http://
research.microsoft.com/ ~akenn/fun/picklercombinators.pdf

        Thanks, Joel

P.S.

data PU a = PU
     {
      appP :: Ptr Word8 -> Int -> a -> IO Int,
      appU :: Ptr Word8 -> Int -> IO (a, Int),
      appS :: a -> IO Int
     }

pickle :: PU a -> Ptr Word8 -> Int -> a -> IO Int
pickle p ptr ix value = appP p ptr ix value

unpickle :: PU a -> Ptr Word8 -> Int -> IO (a, Int)
unpickle p ptr ix = appU p ptr ix

sizeup :: PU a -> a -> IO Int
sizeup p value = appS p value

lift :: a -> PU a
lift x = PU (\_ ix _ -> return ix) (\_ ix -> return (x, ix)) (\_ ->  
return 0)

sequ :: (b -> a) -> PU a -> (a -> PU b) -> PU b
sequ f pa k = PU
               (\ptr ix b ->
                    do let a = f b
                           pb = k a
                       ix1 <- appP pa ptr ix a
                       appP pb ptr ix1 b)
               (\ptr ix ->
                    do (a, ix1) <- appU pa ptr ix
                       let pb = k a
                       appU pb ptr ix1)
               (\b ->
                    do let a = f b
                           pb = k a
                       sz1 <- appS pa a
                       sz2 <- appS pb b
                       return $ sz1 + sz2)

pair :: PU a -> PU b -> PU (a,b)
pair pa pb = sequ fst pa (\ a -> sequ snd pb
                           (\ b -> lift $! (a, b)))

--
http://wagerlabs.com/





_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Arrows and pickler combinators

Jeremy Shaw
At Thu, 22 Dec 2005 11:26:51 +0000,
Joel Reymont wrote:
>
> Folks,
>
> I'm trying to monadify the pickler code. sequ below positively looks  
> like >>= but you can't really join both pickle and unpickle into a  
> single monad. I would like to keep the ops together, though, as this  
> allows me a single specification for both pickling and unpickling.

Last weekend, I hacked up a pickling/unpickling library of my own. The
code is currently a little confusing because I decided to change the
naming scheme half way through. So, don't assume to much from the
names of things.

darcs get http://www.n-heptane.com/nhlab/repos/BerkeleyDB 

The file you are interested in is Binary.hs.

-== Short Summary ==-

My library splits the pickling into two parts you can mix and match.

 (1) the part that turns a value into a byte stream
 (2) the part that reads/writes values from/to the byte stream

-== Core of pickler ==-

My pickling/unpickling code is based around the data type:

data BinParser state m a = BinParser { runBinParser :: state -> m (a, state) }

This type is used for both pickling and unpickling (the type needs a
better name). It is abstracted over three types:

 state - for the pickler, state will hold the data that has currently been pickled.
         for the unpickler, state will hold the data to unpickle.

 m - a monad of your choice

 a - the value being pickled/unpickled

The reason for abstracting over state is to allow you to pickle
directly to [Word8], Ptr Word8, or whatever else you wish to
implement. Some times a monad my be needed for reading/writing the
state. For example, Ptr Word8 requires the IO monad. If you don't
really need a monad, then the Identity monad can be used.

-== BinParser Monad Instance ==-

The monad instance for BinParser is pretty straight-forward:

-- A monad instance for BinParser
instance (Monad m) => Monad (BinParser state m) where
    return a = BinParser (\s -> return (a, s))
    bp >>= f = BinParser ( \state ->
                               do (a, state') <- runBinParser bp state
                                  runBinParser (f a) state'
                         )

As I mentioned before, BinParser is used for both pickling *and*
unpickling. Normally we think of Parsers as consuming the state, but
there is no reason the 'parser' can not instead produce the state.

Also, note that this monad instance is not very specific to
pickling/unpickling at all. It is pretty much just a state monad. As a
matter of fact, I hope to be able to switch to Control.Monad.State
when I have time to work on this again.

-== Adding new 'states' to pickle/unpickle ==-

To add a new type of state to pickle/unpickle, you just add a new
instance to this class (once again, needs a better name):

class (Monad m) => BinState s m where
    getWord8 :: BinParser s m Word8
    putWord8 :: Word8 -> BinParser s m ()


For example:

instance BinState (Ptr Word8) IO where
    getWord8 = BinParser $ \p -> do v <- peek p
                                    return (v, advancePtr p 1)
    putWord8 w = BinParser $ \p -> do poke p w
                                      return ((), advancePtr p 1)

-== pickle vs. unpickle ==-

Here is where we actually combine the above to do pickling (once
again, naming should be updated):

class ToBin state m a where
  binary :: a -> BinParser state m ()
  unbinary :: BinParser state m a

Here is a simple pickler for 'Char'

-- May want to store as 4 bytes to support Unicode later.
instance (BinState state m) => ToBin state m Char where
     binary c = putWord8 (fromIntegral (ord c))
     unbinary =
         do w <- getWord8
            return $! (chr (fromIntegral w))

Here is a pickler for lists that shows the monad usage a bit better:

instance (BinState state m, ToBin state m a) => ToBin state m [a] where
    binary l =
        do binary (length l)
           mapM_ binary l
    unbinary = getList

getList  :: (BinState state m, ToBin state m a) => BinParser state m [a]
getList =
    do len <- getInt
       replicateM len unbinary


-== User Friendly Interface ==-

binary/unbinary are not very user friendly interfaces, so we also
define some user friendly interefaces. If I switched to
Control.Monad.State, I could just use the similar interfaces defined
there...

pickleM/unpickleM is useful if your state requires a monad.

-- NOTE: you may need to force the type to get this to work
-- eg. pickleM "hi" :: IO [Word8]
pickleM :: (Monad m, ToBin state m a) => state -> a -> m state
pickleM initState a =
    do (_,finalState) <- (runBinParser (binary a) initState)
       return finalState

-- NOTE: you may need to force the type to get this to work
-- eg. fromBin (unPickleM "hi" :: [Word8]) :: IO String
unpickleM :: (Monad m, ToBin state m a) => state -> m a
unpickleM state =
    do (a,_) <- runBinParser unbinary state
       return a

pickle/unpickle are useful if your state does not need a monad.

-- Some pickler's may not need to run inside a monad, in which case we
-- can use these varients to avoid the monad
pickle :: (ToBin state Identity a) => state -> a -> state
pickle initState value = runIdentity (pickleM initState value)

unpickle :: (ToBin state Identity a) => state -> a
unpickle state = runIdentity (unpickleM state)


-== Example Usage ==-

Here is an example of using the picklers:


-- First define some data types to pickle
data Foo
    = Bar String
    | Baz Int Char
      deriving Show

data FooBar a = FooBar a
            deriving Show

-- Use TH to derive some piclkers
$( mkBinInstance ''Foo )
$( mkBinInstance ''FooBar)

-- try them out
-- NOTE: not sure if I am using the terms monomorphic/polymorphic
-- correctly
main = -- first pickle/unpickle a monomorphic (?) type
    do print (unpickle (pickle [] (Bar "hello") :: [Word8]) :: Foo)
       -- then pickle/unpickle a polymorphic (?) type
       print (unpickle (pickle [] (FooBar (Bar "hello")) :: [Word8]) :: (FooBar Foo))
       -- use a pickler that outputs to (Ptr Word) instead of [Word8]
       allocaBytes 512 $ \ (p :: Ptr Word8) ->
           do encoded <- pickleM p (Baz 4 'd')
              decoded <- unpickleM p :: IO Foo
              print decoded


-== Summary ==-

As I mentioned, the current implementation is a bit of hack-job, but I
think the design is somewhat compelling because of the flexibility
gained by seperating the pickling/unpickling from the mechanism used
to write/read the bytes.

I hope to clean to code up and submit a TMR article eventually.

j.


ps. DStore.hs contains some code for deriving new instances of ToBin (the
pickler/unpickler). I highly recommend you do not look at that code --
I am not sure I even understand how it works anymore :p
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Arrows and pickler combinators

Joel Reymont
Jeremy,

This is a very nice library you've got but... It does not answer my  
question re: arrows and it still requires you to specify pickling and  
unpickling separately. I can have a single spec right now and would  
like to keep that.

        Thanks, Joel

On Dec 22, 2005, at 8:25 PM, Jeremy Shaw wrote:

> Last weekend, I hacked up a pickling/unpickling library of my own. The
> code is currently a little confusing because I decided to change the
> naming scheme half way through. So, don't assume to much from the
> names of things.
>
> darcs get http://www.n-heptane.com/nhlab/repos/BerkeleyDB

--
http://wagerlabs.com/





_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Arrows and pickler combinators

Jeremy Shaw
At Thu, 22 Dec 2005 21:08:15 +0000,
Joel Reymont wrote:
>
> Jeremy,
>
> This is a very nice library you've got but... It does not answer my  
> question re: arrows and it still requires you to specify pickling and  
> unpickling separately. I can have a single spec right now and would  
> like to keep that.

Ah yes, I see what you mean.

Have you seen, "There and Back Again, Arrows for Invertible Programming"

http://www.cs.ru.nl/~clean/download/papers/2005/alia2005-biarrowsHaskellWorkshop.pdf

In section 5 they talk about using biarrows for Polytypic
(de)serialization.

It is not quite what you want (and requires some special compiler
extensions I think), but it may give you some ideas.

I believe the idea is, you specify either the encoder or the decoder
(your choice), and the other direction is automatically derived.

j.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re[2]: Arrows and pickler combinators

Bulat Ziganshin
In reply to this post by Jeremy Shaw
Hello Jeremy,

Thursday, December 22, 2005, 11:25:40 PM, you wrote:

JS> As I mentioned, the current implementation is a bit of hack-job, but I
JS> think the design is somewhat compelling because of the flexibility
JS> gained by seperating the pickling/unpickling from the mechanism used
JS> to write/read the bytes.

JS> I hope to clean to code up and submit a TMR article eventually.

seem that number of serialization libraries is larger that one can
imagine :)  i also wrote my own (thirs) library which in some places
are close to your, in some better and in some worser. for examle, i
want to use any monad instead of fixed IO in current design. on the
other side, i support bit-oriented & byte-oriented serialization, and
whole hierarchy of Stream types

i'm not sure what yor goal was a speed, but if you are interesting -
your design may be not too fast because of using tuples



--
Best regards,
 Bulat                            mailto:[hidden email]



_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe