Say I have a `Pipes a b m r` called 'p'.
 How do I convert it to a `Pipes (ZipList a) (ZipList b) m r`? That is, I want to apply the pipe 'p' and apply zipwise to an input ziplist, without losing the state of each pipe in the ziplist. The reason I want this, is that I want to run multiple attoparsec parsers in parallel in one pass of a file. My approach for this is to use Control.Foldl to combine the attoparsec parsers into a `Control.Foldl.Fold ByteString [PartialParseResult]`, then convert it (using Control.Foldl.purely Pipes.Prelude.scan) to a `Pipe ByteString [PartialParseResult]` My other requirement is that I want to look at intermediate parser output (eg. each line of a file as it's parsed), in order to summarise each line, in a stateful way as I'm keeping track of the current line number. My approach for this is to use Control.Foldl.Fold for the summary logic, and then convert it to a pipe. I am able to create a `Pipe PartialParseResult AnalysisResult m r` for this. However, how do I connect a `Pipe ByteString [PartialParseResult] m r` to a `Pipe PartialParseResult AnalysisResult m r`? Regards, Louis 
One approach I can think of is the use Pipes.Concurrent to manually split the ZipList and then combine then results. Is there a nicer way?
 On Tuesday, 23 August 2016 11:35:50 UTC+10, Louis Pan wrote:

I would do what you're already doing with `Control.Foldl.Fold`. That's the approach I usually recommend for this.

Sorry, could you please elaborate? I'm a bit slow.
 How do I use Control.Foldl.Fold to convert `Pipes a b m r` into `Pipes (ZipList a) (ZipList b) m r` ? On Tuesday, 23 August 2016 13:20:24 UTC+10, Gabriel Gonzalez wrote:

I managed to convert between `P.Producer (ZipList a) m r` and `ZipList (P.Producer a m ()` using the StateT trick used in Pipes.Parse
 Does the following code make sense? Am I breaking any laws? import Control.Applicative import Control.Monad.Trans.Class import Control.Monad.Trans.State.Strict import qualified Pipes as P import qualified Pipes.Lift as PL  ZipList Traversable is only in 4.9.0.0 sequenceAZipList :: Applicative f => ZipList (f a) > f (ZipList a) sequenceAZipList xs = ZipList <$> sequenceA (getZipList xs)   Similar to Pipes.Parse.Parser, except it stores a ZipList of Producers. type ZipParser a m r = forall x . StateT (ZipList (P.Producer a m x)) m r   Draw one element from each underlying Producer, returning 'Nothing' if any of the producers are empty drawZ :: Monad m => ZipParser a m (Maybe (ZipList a)) drawZ = do ps < get rs < lift (sequenceAZipList (P.next <$> ps)) case sequenceAZipList rs of Left _ > pure Nothing Right rs' > do put $ snd <$> rs' pure . Just $ fst <$> rs'   Push back a Ziplist element onto the underlying ZipList of Producers unDrawZ :: Monad m => ZipList a > ZipParser a m () unDrawZ as = modify (\ps > appendA <$> ps <*> as) where appendA p a = do r < p P.yield a pure r toZipList :: Monad m => P.Producer (ZipList a) m r > m (ZipList (P.Producer a m ())) toZipList p = execStateT (toZipParser p) (pure (pure ())) where toZipParser :: Monad m => P.Producer (ZipList a) m r > ZipParser a m r toZipParser p' = do r < lift $ P.next p' On Tuesday, 23 August 2016 15:01:54 UTC+10, Louis Pan wrote:

My code doesn't stream properly  it forces consumption of the original producer before outputting anything. :(
 On Wednesday, 24 August 2016 04:24:03 UTC+10, Louis Pan wrote:

I think I understand what you mean by using Control.Fold.Foldl.
 It's not possible to "applicatively" lift a Pipe from "Pipe a b m r" to "Applicative f => Pipe (f a) (f a) m r", but it is possible to lift a Control.Foldl.Fold from "Fold a b" to "Applicative f => Fold a b > Fold (f a) (f b)" (see below). This means it's better in my case to code the consumer as a Fold (and then convert it to a Pipes Parse), as opposed to a Pipe Consumer.  requires a function to force the applicative to prevent space leaks. liftFold :: Applicative f => (forall x. f x > f x) > L.Fold a b > L.Fold (f a) (f b) liftFold f (L.Fold step begin done) = L.Fold step' begin' done' where step' xs as = f (step <$> xs <*> as) begin' = pure begin done' xs = done <$> xs forceZipList :: ZipList a > ZipList a forceZipList (ZipList xs) = ZipList (forceFoldable xs)  https://roche.info/articles/20150528forcelist forceFoldable :: Foldable t => t a > t a forceFoldable xs = case foldr seq () xs of () > xs liftZipList :: L.Fold a b > L.Fold (ZipList a) (ZipList b) liftZipList = liftFold forceZipList I've confirmed this prevents space leaks by profiling with "+RTS h p" and using hp2ps. However, I'm a novice with the significance of using seq and $!. Am I doing anything dangerously? Is there something I should be careful of? On Wednesday, 24 August 2016 20:16:29 UTC+10, Louis Pan wrote:

Yeah, what you wrote looks correct. The general idea is that you compose `Fold`s and then convert to a `Pipe` at the last minute.
There's another related type which also has nice properties for this sort of thing, which is basically a mealy machine. I usually encode it like this:
Then you can use `focus traverse` to lift a `Scan` to work on anything that is `Traversable` (like `ZipList`):

Hi Gabriel, thank you for taking the time to answer my questions.
 Is your Scan encoding available anywhere? The closest thing I can find is ekmett's "strict Mealy Machine" at https://hackage.haskell.org/package/folds0.7.1/docs/DataFoldL1'.html but it looks quite different. On Saturday, 27 August 2016 02:15:25 UTC+10, Gabriel Gonzalez wrote:

The closest thing is the mealy machine type from this library, which is equivalent, although possibly less efficient (and missing the `focus` utility): https://hackage.haskell.org/package/machines0.6.1/docs/DataMachineMealy.html This is something I've been meaning to write up in a small library at some point, but I keep forgetting.

This library would be awesome to have!
 
Free forum by Nabble  Edit this page 