Either Monad and Laziness

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Either Monad and Laziness

oleg-30

> I am currently trying to rewrite the Graphics.Pgm library from hackage
> to parse the PGM to a lazy array.

Laziness and IO really do not mix.

> The problem is that even using a lazy array structure, because the
> parser returns an Either structure it is only possible to know if the
> parser was successful or not after the whole file is read,

That is one of the problems. Unexpected memory blowups could be
another problem. The drawbacks of lazy IO are well documented by now.

> The behaviour I want to achieve is like this: I want the program when
> compiled to read from a file, parsing the PGM and at the same time
> apply transformations to the entries as they are read and write them
> back to another PGM file.

Such problems are the main motivation for iteratees, conduits, pipes,
etc. Every such library contains procedures for doing exactly what you
want. Please check Hackage. John Lato's iteratee library, for example,
has procedure for handling sound (AIFF) files -- which may be very
big. IterateeM has the TIFF decoder -- which is incremental and
strict. TIFF is much harder to parse than PGM.



_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Either Monad and Laziness

Eric Velten de Melo
Thanks for all the tips! The iteratees seem worth checking out. I'll
see what I can do and will report back if I come up with something.

Eric

On 12 September 2012 03:03,  <[hidden email]> wrote:

>
>> I am currently trying to rewrite the Graphics.Pgm library from hackage
>> to parse the PGM to a lazy array.
>
> Laziness and IO really do not mix.
>
>> The problem is that even using a lazy array structure, because the
>> parser returns an Either structure it is only possible to know if the
>> parser was successful or not after the whole file is read,
>
> That is one of the problems. Unexpected memory blowups could be
> another problem. The drawbacks of lazy IO are well documented by now.
>
>> The behaviour I want to achieve is like this: I want the program when
>> compiled to read from a file, parsing the PGM and at the same time
>> apply transformations to the entries as they are read and write them
>> back to another PGM file.
>
> Such problems are the main motivation for iteratees, conduits, pipes,
> etc. Every such library contains procedures for doing exactly what you
> want. Please check Hackage. John Lato's iteratee library, for example,
> has procedure for handling sound (AIFF) files -- which may be very
> big. IterateeM has the TIFF decoder -- which is incremental and
> strict. TIFF is much harder to parse than PGM.
>
>

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Either Monad and Laziness

Eric Velten de Melo
On 12 September 2012 11:46, Eric Velten de Melo <[hidden email]> wrote:

> Thanks for all the tips! The iteratees seem worth checking out. I'll
> see what I can do and will report back if I come up with something.
>
> Eric
>
> On 12 September 2012 03:03,  <[hidden email]> wrote:
>>
>>> I am currently trying to rewrite the Graphics.Pgm library from hackage
>>> to parse the PGM to a lazy array.
>>
>> Laziness and IO really do not mix.
>>
>>> The problem is that even using a lazy array structure, because the
>>> parser returns an Either structure it is only possible to know if the
>>> parser was successful or not after the whole file is read,
>>
>> That is one of the problems. Unexpected memory blowups could be
>> another problem. The drawbacks of lazy IO are well documented by now.
>>
>>> The behaviour I want to achieve is like this: I want the program when
>>> compiled to read from a file, parsing the PGM and at the same time
>>> apply transformations to the entries as they are read and write them
>>> back to another PGM file.
>>
>> Such problems are the main motivation for iteratees, conduits, pipes,
>> etc. Every such library contains procedures for doing exactly what you
>> want. Please check Hackage. John Lato's iteratee library, for example,
>> has procedure for handling sound (AIFF) files -- which may be very
>> big. IterateeM has the TIFF decoder -- which is incremental and
>> strict. TIFF is much harder to parse than PGM.
>>

It would be really awesome, though, if it were possible to use a
parser written in Parsec with this, in the spirit of avoiding code
rewriting and enhancing expressivity and abstraction.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Either Monad and Laziness

Francesco Mazzoli
At Wed, 12 Sep 2012 12:04:31 -0300,
Eric Velten de Melo wrote:
> It would be really awesome, though, if it were possible to use a
> parser written in Parsec with this, in the spirit of avoiding code
> rewriting and enhancing expressivity and abstraction.

There is <http://hackage.haskell.org/package/attoparsec-conduit> and
<http://hackage.haskell.org/package/attoparsec-enumerator>, which turn
attoparsec parsers into enumerators/conduits, and
<http://hackage.haskell.org/package/attoparsec-parsec>, which is a compatibility
layer between attoaparsec and parsec.  Good luck :).

--
Francesco * Often in error, never in doubt

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Either Monad and Laziness

wren ng thornton
On 9/12/12 5:37 PM, Francesco Mazzoli wrote:

> At Wed, 12 Sep 2012 12:04:31 -0300,
> Eric Velten de Melo wrote:
>> It would be really awesome, though, if it were possible to use a
>> parser written in Parsec with this, in the spirit of avoiding code
>> rewriting and enhancing expressivity and abstraction.
>
> There is <http://hackage.haskell.org/package/attoparsec-conduit> and
> <http://hackage.haskell.org/package/attoparsec-enumerator>, which turn
> attoparsec parsers into enumerators/conduits, and
> <http://hackage.haskell.org/package/attoparsec-parsec>, which is a compatibility
> layer between attoaparsec and parsec.  Good luck :).

Not to mention attoparsec-iteratee, for the iteratee minded folks:

     http://hackage.haskell.org/package/attoparsec-iteratee


--
Live well,
~wren

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Either Monad and Laziness

Eric Velten de Melo
On 13 September 2012 20:29, wren ng thornton <[hidden email]> wrote:

> On 9/12/12 5:37 PM, Francesco Mazzoli wrote:
>>
>> At Wed, 12 Sep 2012 12:04:31 -0300,
>> Eric Velten de Melo wrote:
>>>
>>> It would be really awesome, though, if it were possible to use a
>>> parser written in Parsec with this, in the spirit of avoiding code
>>> rewriting and enhancing expressivity and abstraction.
>>
>>
>> There is <http://hackage.haskell.org/package/attoparsec-conduit> and
>> <http://hackage.haskell.org/package/attoparsec-enumerator>, which turn
>> attoparsec parsers into enumerators/conduits, and
>> <http://hackage.haskell.org/package/attoparsec-parsec>, which is a
>> compatibility
>> layer between attoaparsec and parsec.  Good luck :).
>
>
> Not to mention attoparsec-iteratee, for the iteratee minded folks:
>
>     http://hackage.haskell.org/package/attoparsec-iteratee
>

Hm... I guess I'm spoiled for choice then. :)

But now I'm kinda lost. Is there an easy way to explain the difference between:
-iteratee
-conduit
-enumerator

I'm very curious about everything concerning Haskell and new
interesting abstractions and ways of doing things, but I might not
have the time to delve deeper into that.

>
> --
> Live well,
> ~wren
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Either Monad and Laziness

wren ng thornton
On 9/14/12 5:16 PM, Eric Velten de Melo wrote:
> But now I'm kinda lost. Is there an easy way to explain the difference between:
> -iteratee
> -conduit
> -enumerator

John Lato's iteratee library is the original one based on Oleg
Kiselyov's work. I've used it a fair deal and am quite fond of it.

Some folks didn't like it so much though; whence enumerator, conduit,
pipes, pipes-core,... I've followed the discussions back and forth over
those libraries, but I've not used them nor sat down to compare them
head-to-head.

--
Live well,
~wren

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Either Monad and Laziness

John Lato-2
In reply to this post by oleg-30
> Subject: Re: [Haskell-cafe] Either Monad and Laziness
>
> On 9/14/12 5:16 PM, Eric Velten de Melo wrote:
>> But now I'm kinda lost. Is there an easy way to explain the difference between:
>> -iteratee
>> -conduit
>> -enumerator

I tend to group them into three families.  'iteratee' and 'enumerator'
are fairly directly drawn from Oleg's code, with mostly implementation
differences (at least when compared to the other families).  They've
tended to keep Oleg's original names (iteratee, enumerator,
enumeratee).

The biggest user-visible difference between iteratee and enumerator is
the level of datastream abstraction.  iteratee abstracts over the
stream, and enumerator abstracts over elements of the stream.  The
stream is explicitly a list of elements.  This exposes some of the
details of data chunking to the user, which has both advantages and
disadvantages (iteratee exposes this also, but it's not necessary as
is the case for enumerator).

The second family (chronologically) includes conduit and (maybe)
iterIO.  I've written a little about this group at
http://johnlato.blogspot.sg/2012/06/understandings-of-iteratees.html
Although they serve the same goals in spirit, the implementation may
or may not necessarily be an iteratee/enumerator arrangement (e.g.
conduit).  This is a technical observation, not a criticism, depending
on exactly what you consider to define the style in the first place.
This group has usually renamed functions.  I discuss some of the other
differences on my blog.

The third familiy is all the pipes-* stuff.  This group tends towards
emphasizing the relationship between iteratee/enumerator pairs and
coroutines, and also emphasizing (to use Oleg terminology) composition
of enumeratees.  I've been meaning to write more about this group, but
thus far have been unable to do so.

I'd rather not hijack by evangelizing, but obviously I think iteratee
provides several important advantages over the other options.

John L.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Either Monad and Laziness

Malcolm Wallace-2
In reply to this post by Eric Velten de Melo

On 12 Sep 2012, at 16:04, Eric Velten de Melo wrote:

>>>> The behaviour I want to achieve is like this: I want the program when
>>>> compiled to read from a file, parsing the PGM and at the same time
>>>> apply transformations to the entries as they are read and write them
>>>> back to another PGM file.
>>>
>>> Such problems are the main motivation for iteratees, conduits, pipes,
>>> etc. Every such library contains procedures for doing exactly what you
>>> want.
>>>
>
> It would be really awesome, though, if it were possible to use a
> parser written in Parsec with this, in the spirit of avoiding code
> rewriting and enhancing expressivity and abstraction.

The polyparse library on Hackage is another parser combinator framework that allows lazy incremental parsing.
http://hackage.haskell.org/package/polyparse

A PDF paper/tutorial is here:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.118.1754&rep=rep1&type=pdf

Regards,
    Malcolm

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe