installing streams library

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

installing streams library

Chad Scherrer
It sounds like Bulat has gotten some impressive I/O speedups with his Streams library. I'd like to try this out, but I'm having some trouble installing it. I'm using GHC on Linux.

My first attempt was looking around on this page:
http://www.haskell.org/haskellwiki/Library/Streams

There's a really nice description, but no signs of where to actually get the library. Eventually (thanks to Google) I tracked down this message:
http://article.gmane.org/gmane.comp.lang.haskell.general/13625

From here I was able to download it, but there's no information regarding how this needs to be set up. There are directories "Data", "Examples", and "System", which I assume are supposed to be plugged into the hierarchical module structure, but how do I do that? I thought this might have something to do with Cabal (I've not yet used that), but the Cabal manual talks about a .cabal file, which doesn't exist here.

Does this follow some standard approach that I'm not familiar with? Where should I look to learn more?

Thanks,
--

Chad Scherrer

"Time flies like an arrow; fruit flies like a banana" -- Groucho Marx
_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Jared Updike
I found this in an old post (gotta love GMail search):

> You can find further information about the library at the
> page http://haskell.org/haskellwiki/Library/Streams and
> download it as http://freearc.narod.ru/Streams.tar.gz

from a thread in February entitled:

  Streams: the extensible I/O library (Feb 21)

Google  site:haskell.org bulat extensible I/O library for the thread, etc.

Hope that helps,
  Jared.

On 5/19/06, Chad Scherrer <[hidden email]> wrote:

> It sounds like Bulat has gotten some impressive I/O speedups with his
> Streams library. I'd like to try this out, but I'm having some trouble
> installing it. I'm using GHC on Linux.
>
> My first attempt was looking around on this page:
> http://www.haskell.org/haskellwiki/Library/Streams
>
> There's a really nice description, but no signs of where to actually get the
> library. Eventually (thanks to Google) I tracked down this message:
> http://article.gmane.org/gmane.comp.lang.haskell.general/13625
>
> From here I was able to download it, but there's no information regarding
> how this needs to be set up. There are directories "Data", "Examples", and
> "System", which I assume are supposed to be plugged into the hierarchical
> module structure, but how do I do that? I thought this might have something
> to do with Cabal (I've not yet used that), but the Cabal manual talks about
> a .cabal file, which doesn't exist here.
>
> Does this follow some standard approach that I'm not familiar with? Where
> should I look to learn more?
>
> Thanks,
> --
>
> Chad Scherrer
>
> "Time flies like an arrow; fruit flies like a banana" -- Groucho Marx
> _______________________________________________
> Haskell mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell
>
>
>


--
http://www.updike.org/~jared/
reverse ")-:"
_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Bulat Ziganshin-2
In reply to this post by Chad Scherrer
Hello Chad,

Friday, May 19, 2006, 10:40:56 PM, you wrote:

> It sounds like Bulat has gotten some impressive I/O speedups with
> his Streams library. I'd like to try this out, but I'm having some
> trouble installing it. I'm using GHC on Linux.

yes, and current (still unpublished) version is even better than previous
one

> My first attempt was looking around on this page:
> http://www.haskell.org/haskellwiki/Library/Streams

> There's a really nice description, but no signs of where to
> actually get the library.

the old hawiki don't allowed to publish links to site narod.ru,
because it's public site used also by spammers. i plan to request
some place on the haskell.org site for my projects to make
possible links publishing

> From here I was able to download it, but there's no information
> regarding how this needs to be set up. There are directories "Data",
> "Examples", and "System", which I assume are supposed to be plugged
> into the hierarchical module structure, but how do I do that? I
> thought this might have something to do with Cabal (I've not yet
> used that), but the Cabal manual talks about a .cabal file, which doesn't exist here.

> Does this follow some standard approach that I'm not familiar with?
> Where should I look to learn more?

you will be laughing but i still not learned how cabal should be used
and just pack all the library modules together. you can either copy
Data & System dirs to the place where your program lies or use "-i"
ghc switch to point compiler where it can find additional modules:

ghc --make -i/Haskell/StreamsLib  YourProgram.hs

in this case, ghc will search, for example, module "Data.Ref" in file
/Haskell/StreamsLib/Data/Ref.hs (after looking to ./Data/Ref.hs)

directory Examples contains examples of using library, you should copy
files from this dir to root library directory in order to compile them



--
Best regards,
 Bulat                            mailto:[hidden email]

_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Chad Scherrer
Thanks, Bulat. I'm looking forward to trying it out this weekend.

Is there any indication what fast IO approach might work its way into
the standard libraries? It would be nice for idiomatic Haskell to be
really fast by default, and I'd love to be able to show off the language
shootout implications to coworkers.

Cabal doesn't seem obvious to me. Currently my uninformed point of view
is that the whole thing is too complicated to be worth the benefit it
provides. Since it generally seems popular, I'm guessing that means it's
either much simpler than it seems, or else the benefit provided is just
enormous. Can anyone point me to a sales pitch, or provide some
motivation for me to RTFM?

Thanks,

Chad

On Sat, 2006-05-20 at 10:17 +0400, Bulat Ziganshin wrote:

> Hello Chad,
>
> Friday, May 19, 2006, 10:40:56 PM, you wrote:
>
> > It sounds like Bulat has gotten some impressive I/O speedups with
> > his Streams library. I'd like to try this out, but I'm having some
> > trouble installing it. I'm using GHC on Linux.
>
> yes, and current (still unpublished) version is even better than previous
> one
>
> > My first attempt was looking around on this page:
> > http://www.haskell.org/haskellwiki/Library/Streams
>
> > There's a really nice description, but no signs of where to
> > actually get the library.
>
> the old hawiki don't allowed to publish links to site narod.ru,
> because it's public site used also by spammers. i plan to request
> some place on the haskell.org site for my projects to make
> possible links publishing
>
> > From here I was able to download it, but there's no information
> > regarding how this needs to be set up. There are directories "Data",
> > "Examples", and "System", which I assume are supposed to be plugged
> > into the hierarchical module structure, but how do I do that? I
> > thought this might have something to do with Cabal (I've not yet
> > used that), but the Cabal manual talks about a .cabal file, which doesn't exist here.
>
> > Does this follow some standard approach that I'm not familiar with?
> > Where should I look to learn more?
>
> you will be laughing but i still not learned how cabal should be used
> and just pack all the library modules together. you can either copy
> Data & System dirs to the place where your program lies or use "-i"
> ghc switch to point compiler where it can find additional modules:
>
> ghc --make -i/Haskell/StreamsLib  YourProgram.hs
>
> in this case, ghc will search, for example, module "Data.Ref" in file
> /Haskell/StreamsLib/Data/Ref.hs (after looking to ./Data/Ref.hs)
>
> directory Examples contains examples of using library, you should copy
> files from this dir to root library directory in order to compile them
>
>
>

_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Sebastian Sylvan-2
On 5/20/06, Chad Scherrer <[hidden email]> wrote:

> Thanks, Bulat. I'm looking forward to trying it out this weekend.
>
> Is there any indication what fast IO approach might work its way into
> the standard libraries? It would be nice for idiomatic Haskell to be
> really fast by default, and I'd love to be able to show off the language
> shootout implications to coworkers.
>
> Cabal doesn't seem obvious to me. Currently my uninformed point of view
> is that the whole thing is too complicated to be worth the benefit it
> provides. Since it generally seems popular, I'm guessing that means it's
> either much simpler than it seems, or else the benefit provided is just
> enormous. Can anyone point me to a sales pitch, or provide some
> motivation for me to RTFM?
>

A quick sales pitch: usually you, the library user, can just type:

./runhaskell Setup.hs configure
./runhaskell Setup.hs build
./runhaskell Setup.hs install

And it will Do The Right Thing(TM), which is nice.


/S

--
Sebastian Sylvan
+46(0)736-818655
UIN: 44640862
_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Jon Fairbairn
On 2006-05-20 at 12:00+0200 "Sebastian Sylvan" wrote:
> A quick sales pitch: usually you, the library user, can just type:
>
> ./runhaskell Setup.hs configure
> ./runhaskell Setup.hs build
> ./runhaskell Setup.hs install
>
> And it will Do The Right Thing(TM), which is nice.

This is something I've never understood about the current
fashion. Make allows one to set up rules about what depends
on what, so why can't we just arrange it so that someone who
wants to install the thing just hast to type

./runhaskell Setup.hs install

?  I'm aware that part of this process might require root
privileges, and that it would be bad to do the rest of it as
root, but there are ways around that, surely? Even if not,
the first line ought to be unnecessary, since configure
produces something that build depends upon.

 Jón

--
Jón Fairbairn                              Jon.Fairbairn at cl.cam.ac.uk


_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Bugzilla from robdockins@fastmail.fm
On Saturday 20 May 2006 06:53 am, Jon Fairbairn wrote:

> On 2006-05-20 at 12:00+0200 "Sebastian Sylvan" wrote:
> > A quick sales pitch: usually you, the library user, can just type:
> >
> > ./runhaskell Setup.hs configure
> > ./runhaskell Setup.hs build
> > ./runhaskell Setup.hs install
> >
> > And it will Do The Right Thing(TM), which is nice.
>
> This is something I've never understood about the current
> fashion. Make allows one to set up rules about what depends
> on what, so why can't we just arrange it so that someone who
> wants to install the thing just hast to type
>
> ./runhaskell Setup.hs install
>
> ?  I'm aware that part of this process might require root
> privileges, and that it would be bad to do the rest of it as
> root, but there are ways around that, surely? Even if not,
> the first line ought to be unnecessary, since configure
> produces something that build depends upon.

FWIW, it's almost identical to the incantation necessary for projects based on
autotools, where it usually reads something like:

./configure
make
make install

That means Cabal should fit nicely into package management systems that are
used to dealing with autotools projects (Debian, Gentoo, bsd ports,
darwinports, etc).


>  Jón


Rob Dockins
_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Neil Mitchell
Hi

> FWIW, it's almost identical to the incantation necessary for projects based on
> autotools, where it usually reads something like:
>
> ./configure
> make
> make install

But sadly compared to Windows tools, where its just a case of "double
click" this sounds a bit too complex.

I am hoping that I can register Cabal with Windows in some way to
automate this down to "double click" once more.

Thanks

Neil
_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Jon Fairbairn
In reply to this post by Bugzilla from robdockins@fastmail.fm
On 2006-05-20 at 11:58EDT Robert Dockins wrote:

> On Saturday 20 May 2006 06:53 am, Jon Fairbairn wrote:
> > Make allows one to set up rules about what depends
> > on what, so why can't we just arrange it so that someone who
> > wants to install the thing just hast to type
> >
> > ./runhaskell Setup.hs install
> >
> FWIW, it's almost identical to the incantation necessary
> for projects based on autotools, where it usually reads
> something like:
>
> ./configure
> make
> make install

I know. That's the fashion I'm complaining about!  I think
we shouldn't follow it if we can possibly avoid it.

--
Jón Fairbairn                              Jon.Fairbairn at cl.cam.ac.uk


_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Brian Hulley
In reply to this post by Neil Mitchell
Neil Mitchell wrote:

> Hi
>
>> FWIW, it's almost identical to the incantation necessary for
>> projects based on autotools, where it usually reads something like:
>>
>> ./configure
>> make
>> make install
>
> But sadly compared to Windows tools, where its just a case of "double
> click" this sounds a bit too complex.
>
> I am hoping that I can register Cabal with Windows in some way to
> automate this down to "double click" once more.

It's strange that with the most advanced programming language in existence
we're still unable to automate the task of having to manually type a
sequence of 3 commands at an ancient UNIX command prompt!!! :-)

Regards, Brian.

_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Taral
On 5/20/06, Brian Hulley <[hidden email]> wrote:
> It's strange that with the most advanced programming language in existence
> we're still unable to automate the task of having to manually type a
> sequence of 3 commands at an ancient UNIX command prompt!!! :-)

No, it's not strange. What appears to have been overlooked is that
different people will execute these commands differently:

./configure --prefix=/opt/freeware --sysconfdir=/etc --disable-nls
make
make install DESTDIR=/home/taral/staging

Many people will run "make install" as root but the main compile as
non-root. You can't blindly mix all that together.

--
Taral <[hidden email]>
"You can't prove anything."
    -- Gödel's Incompetence Theorem
_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Duncan Coutts
In reply to this post by Chad Scherrer
On Sat, 2006-05-20 at 00:00 -0700, Chad Scherrer wrote:
> Thanks, Bulat. I'm looking forward to trying it out this weekend.
>
> Is there any indication what fast IO approach might work its way into
> the standard libraries? It would be nice for idiomatic Haskell to be
> really fast by default, and I'd love to be able to show off the language
> shootout implications to coworkers.

Personally I rather like the idea of a new IO library based on lazy byte
strings or in this context it'd be more helpful to call them lazy byte
streams. I believe this could give us a system which is much more
idiomatic Haskell than an explicit buffer/IO based system and with
almost as good performance (by almost I mean at most 10%, not 50%).

The idea is that instead of pushing things like string encoding or
compression into the IO part of the system that we can leave it in the
pure part and get nicer code and still get good performance. That is,
instead of layering encoding etc into a stream that we can write things
like:

print . count . lines . decode utf8 . gunzip  =<< readFile "foo"

and have that run within 10% of the speed of C code.

Donald Stewart has been posting benchmarks to the libraries mailing list
showing that the Data.ByteString and Data.ByteString.Lazy modules can
give us near-C speed and constant memory use on huge files while still
using idiomatic Haskell code.

So instead of layering streams I want to compose functions.

> Cabal doesn't seem obvious to me. Currently my uninformed point of view
> is that the whole thing is too complicated to be worth the benefit it
> provides. Since it generally seems popular, I'm guessing that means it's
> either much simpler than it seems, or else the benefit provided is just
> enormous. Can anyone point me to a sales pitch, or provide some
> motivation for me to RTFM?

The general idea is that it provides a default build system for Haskell
libraries and programs. Otherwise people use various build systems which
are usually broken in various ways for corner cases that their authors
don't encounter.

As someone who packages Haskell progs/libs for Gentoo I can assure you
that the shift to Cabal has been a great improvement. Previously,
packages needed a great deal of manual work and often patching to
package them adequately. That's not to say that Cabal is perfect, but as
we improve it everyone benefits which was not previously the case when
everyone rolled their own build system. For example we still don't
support win32 very well but we should be able to improve that by
improving Cabal (or layering tools on top of Cabal) where as previously
you'd have to go replacing every make-based build system to get it to
work on win32.

Duncan

_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Atila Romero
As a haskell beginner, I saw the lack of a simple and fast standard
library as THE major dificulty.
Yes, more than monads and lazy programming and all other stuff.
This happened because I simply could not create simple programs to
simple task as I needed then in production work.
I work at forensic area, and have to deal with large ammounts of data
(HD images for example).
The processing time is important to me and so is the ability to accept
files of any size.
If an easy fast IO package become part of standard libraries, I am sure
will make my life and of more beginners a lot easier.

Atila


Duncan Coutts wrote:

> On Sat, 2006-05-20 at 00:00 -0700, Chad Scherrer wrote:
>  
>> Thanks, Bulat. I'm looking forward to trying it out this weekend.
>>
>> Is there any indication what fast IO approach might work its way into
>> the standard libraries? It would be nice for idiomatic Haskell to be
>> really fast by default, and I'd love to be able to show off the language
>> shootout implications to coworkers.
>>    
>
> Personally I rather like the idea of a new IO library based on lazy byte
> strings or in this context it'd be more helpful to call them lazy byte
> streams. I believe this could give us a system which is much more
> idiomatic Haskell than an explicit buffer/IO based system and with
> almost as good performance (by almost I mean at most 10%, not 50%).
>
> The idea is that instead of pushing things like string encoding or
> compression into the IO part of the system that we can leave it in the
> pure part and get nicer code and still get good performance. That is,
> instead of layering encoding etc into a stream that we can write things
> like:
>
> print . count . lines . decode utf8 . gunzip  =<< readFile "foo"
>
> and have that run within 10% of the speed of C code.
>
> Donald Stewart has been posting benchmarks to the libraries mailing list
> showing that the Data.ByteString and Data.ByteString.Lazy modules can
> give us near-C speed and constant memory use on huge files while still
> using idiomatic Haskell code.
>
> So instead of layering streams I want to compose functions.
>
>  
>> Cabal doesn't seem obvious to me. Currently my uninformed point of view
>> is that the whole thing is too complicated to be worth the benefit it
>> provides. Since it generally seems popular, I'm guessing that means it's
>> either much simpler than it seems, or else the benefit provided is just
>> enormous. Can anyone point me to a sales pitch, or provide some
>> motivation for me to RTFM?
>>    
>
> The general idea is that it provides a default build system for Haskell
> libraries and programs. Otherwise people use various build systems which
> are usually broken in various ways for corner cases that their authors
> don't encounter.
>
> As someone who packages Haskell progs/libs for Gentoo I can assure you
> that the shift to Cabal has been a great improvement. Previously,
> packages needed a great deal of manual work and often patching to
> package them adequately. That's not to say that Cabal is perfect, but as
> we improve it everyone benefits which was not previously the case when
> everyone rolled their own build system. For example we still don't
> support win32 very well but we should be able to improve that by
> improving Cabal (or layering tools on top of Cabal) where as previously
> you'd have to go replacing every make-based build system to get it to
> work on win32.
>
> Duncan
>
> _______________________________________________
> Haskell mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell
>
>  


       
       
               
____________________________________________________

_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Simon Marlow-5
In reply to this post by Neil Mitchell
Neil Mitchell wrote:

>
>> FWIW, it's almost identical to the incantation necessary for projects
>> based on
>> autotools, where it usually reads something like:
>>
>> ./configure
>> make
>> make install
>
>
> But sadly compared to Windows tools, where its just a case of "double
> click" this sounds a bit too complex.
>
> I am hoping that I can register Cabal with Windows in some way to
> automate this down to "double click" once more.

I'm sure you can.  The important point is that Cabal doesn't require
Cygwin or any other build machinery aside from GHC (or Hugs) itself, so
you *can* build and install libraries on Windows.  This is one Big Win
that you get with Cabal.

When somebody gets around to it, Cabal will be able to automatically
generate a .msi binary windows installer directly from a Cabal package.

Cheers,
        Simon
_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Simon Marlow-5
In reply to this post by Duncan Coutts
Duncan Coutts wrote:

> Personally I rather like the idea of a new IO library based on lazy byte
> strings or in this context it'd be more helpful to call them lazy byte
> streams. I believe this could give us a system which is much more
> idiomatic Haskell than an explicit buffer/IO based system and with
> almost as good performance (by almost I mean at most 10%, not 50%).
>
> The idea is that instead of pushing things like string encoding or
> compression into the IO part of the system that we can leave it in the
> pure part and get nicer code and still get good performance. That is,
> instead of layering encoding etc into a stream that we can write things
> like:
>
> print . count . lines . decode utf8 . gunzip  =<< readFile "foo"
>
> and have that run within 10% of the speed of C code.

What about error handling?  At the moment, the lazy string returned by
readFile is just truncated when an error occurs, and the error itself is
discarded.  I don't think anyone wants this behaviour.

Now you could make purely functional code raise I/O exceptions, but that
gives rise to a few problems: imprecise exceptions are difficult to
program with (need deepSeq etc.), and they aren't widely implemented
(Hugs doesn't have them, JHC doesn't have them and may not get them
IIUC).  Also, laziness gives rise to problems with predictable resource
consumption - many times we've had to explain to people how to make sure
that the file opened by readFile is closed early enough so they don't
run out of file descriptors or run into a locking problem.

Maybe lazy I/O can be made to work, but until/unless we have a good way
to handle errors I'm certain it's not something on which we should be
basing a new I/O library design.

Cheers,
        Simon
_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re[2]: installing streams library

Bulat Ziganshin-2
In reply to this post by Chad Scherrer
Hello Chad,

Saturday, May 20, 2006, 11:00:48 AM, you wrote:

> Is there any indication what fast IO approach might work its way into
> the standard libraries? It would be nice for idiomatic Haskell to be
> really fast by default, and I'd love to be able to show off the language
> shootout implications to coworkers.

first, about the speed. i've just implemented ByteString I/O and it
shows speed of 20-100 mb/s on my box. so, as i've previously said,
the speed of text I/O using ByteStrings+Streams will be comparable to
disk speed itself. that means that my "quest for text I/O speed" now
finished :)

second - i developed Streams to be usable both as add-on lib for
existing hugs/ghc versions and as future part of base libraries what will
replace current Handle-based I/O. in particular, i will include in new
version module what emulates System.IO types and functions. afaik,
Simon Marlow supports my plans. but the current version still lacks
some functionality that System.IO provides:

network I/O via sockets
overlapping I/O in multi-threaded programs
non-blockable I/O
correctness with asynchronous interrupts and 'trace (putStr ..)'
LineBuffering/NoBuffering
large number of functions, such as hUngetChar

the last three things just needs my attention, i know how to do it.
the first three things, though, are from areas where i don't have
any experience. so i think that i now should publish library in it's
current state and interest other developers to work on it. as you
understand, we can't say about replacing System.IO without full
implementation of it's features and it's no good to have two I/O
libraries in the base package :)


and third that i want to say is what Streams isn't only "fast I/O
library", it's general purpose I/O library that includes support for
binary serialization, Char encoding, I/O to memory buffers and strings,
memory-mapped files and so on, so on. i think that its main selling
points is modularity and extensibility, that greatly simplifies
addition of new features, maintenance and improves code readability.
i believe that everyone is able to understand library structure and
add new features what one needs or replace old modules with better new ones



in February i wrote to Andrew Pimlott about this library goals:

Wednesday, February 08, 2006, 8:24:59 PM, you wrote:

>> AP> Bulat, it wouldn't hurt to include a motivation section at the top.  As
>> AP> I understand, it's ultimately all about speed, right?  Otherwise, we
>> AP> would all be happy with lists (and unsafeInterleave*).  So maybe a
>> AP> comparison between Stream and [] should be given.
>>
>> you guessed wrong :)  this library is ultimately about replacing
>> System.IO library (i.e. Handles)

AP> Let me rephrase my question:  Why not just reimplement the Handles API
AP> (with some extensions like binary IO)?  Is there really a need to use a
AP> handle-like API for more than real IO?  If so, what is the need,
AP> expressivity or performance (or both)?  Maybe a use case showing what
AP> you can do with your library, and how you would have to do it otherwise?

now i understood you. actually, my presentation was meant only for a
few people who are already know about limitations of current library,
who are already requested additional features but don't got it. here i
need to give some history:

when a System.IO interface was developed, it implements much less
features than now, and its implementation was enough simple and
straightforward. as time goes, the more and more features was added to
this library: complex buffering scheme, several async i/o
implementations, locking, networking. And at current moment, GHC's
System.IO implementation consists of about 3000 lines with a rather
monolithic structure. you can't easily add new feature or make using
of some "heavyweight" feature optional because it needs to make
changes through the entire library. As the result, GHC users can't get
improvements in this library, required for his work. Some of them even
develop his own libraries what implements just that they need. for
example, Einar Karttunen developed networking library with advanced
async i/o and support for i/o of fast packed strings. But such
solutions is not universal - his library can't be used for file i/o,
fo example, although the code is essentially the same.

what i done? the main merit of Streams library is not implementation
of any particular feature, but birth of framework for the I/O
sub-libraries. and my library essentially is just a collection of such
sublibs. first, for example, implements file i/o, second implements
buffering, third - utf-8 encoding, and so on. the most important
property of all these sublibs is that no one of them is greater than
300 lines. that means that it is far easier to understand, modify or
even replace any of them. and that will have no impact to other part
of library because all these sublibs binded together not via data
fields, but with well defined interfaces

now, implementing any new I/O feature or new I/O source means only
implementing Stream class-comforming interface - all other features,
including locking, buffering, encoding, compression, serialization,
binary and text i/o, async i/o, will become available automatically.
the same for transformers - once implemented gzip compression or
UTF-16 encoding support will become automatically available for all
the I/O sources, present and future. is not that great? :)  moreover,
user apllication or third party lib can easily implement new stream
types or transformers without bothering the original library.

so, this lib in some aspect is meta-meta-instrument, whose capital
will be automatically increasing as new sublibs will appear. just at
this moment its advantages over System.IO is in the following areas:

faster i/o
support for optional utf-8 encoding
binary i/o and serialization
user-controlled locking

if you will look inside the archive, you will find directory Examples,
which demonstrates usage of all these features. as i said in docs, i
also plan to implement other user requests to the System.IO library

another consequence of emerging this library is that all these
features will become available on Hugs and other Haskell compilers,
that never had enough man resources to develop such great library as
GHC's System.IO.

and about using streams in monads other than IO. i really don't know,
whether it will be used or not. at least, for seriazliation it looks
promising. for example, there is functions "encode" and "decode" that
is like show/read pair, but implements binary encoding according to
the instances of Binary class. and of course, it is implemented
through the StringBuffer instance of Stream class, working in the ST
monad.

comparing to the Handles, library provides essentially the same
interface. again, you can find information about swithcing from
Handles to Streams in doc. i plan to provide in future "legacy layer"
which will emulate 100% of System.IO interface, but use the streams
internally. It will be essential for old apps, especially if Streams
will become official and sole GHC i/o library

about internal organization - Streams is somewhat like that if Simon
himself splitted up Handles library to the small independent parts and
then replaced part of them with simpler/faster implementations.
nothing more, except for common Stream class interface, developed by
John Goerzen. my work was mainly to bring the best ideas together :)



--
Best regards,
 Bulat                            mailto:[hidden email]

_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: Re: installing streams library

Ross Paterson
In reply to this post by Simon Marlow-5
On Wed, May 24, 2006 at 10:51:31AM +0100, Simon Marlow wrote:
> Now you could make purely functional code raise I/O exceptions, but that
> gives rise to a few problems: imprecise exceptions are difficult to
> program with (need deepSeq etc.), and they aren't widely implemented
> (Hugs doesn't have them, JHC doesn't have them and may not get them
> IIUC).

Hugs has had imprecise exceptions (but not asynchronous ones) since Nov 2003.

_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: installing streams library

Simon Marlow-5
Ross Paterson wrote:

> On Wed, May 24, 2006 at 10:51:31AM +0100, Simon Marlow wrote:
>
>>Now you could make purely functional code raise I/O exceptions, but that
>>gives rise to a few problems: imprecise exceptions are difficult to
>>program with (need deepSeq etc.), and they aren't widely implemented
>>(Hugs doesn't have them, JHC doesn't have them and may not get them
>>IIUC).
>
>
> Hugs has had imprecise exceptions (but not asynchronous ones) since Nov 2003.

oops, thanks for the correction.

Simon
_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re[2]: Re: installing streams library

Bulat Ziganshin-2
In reply to this post by Ross Paterson
Hello Ross,

Wednesday, May 24, 2006, 4:50:39 PM, you wrote:

>> Now you could make purely functional code raise I/O exceptions, but that
>> gives rise to a few problems: imprecise exceptions are difficult to
>> program with (need deepSeq etc.), and they aren't widely implemented
>> (Hugs doesn't have them, JHC doesn't have them and may not get them
>> IIUC).

> Hugs has had imprecise exceptions (but not asynchronous ones) since Nov 2003.

can you please explain or give a pointer - what is an imprecise
exceptions? i thought that it the synonym for async ones


--
Best regards,
 Bulat                            mailto:[hidden email]

_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
Reply | Threaded
Open this post in threaded view
|

Re: Re: installing streams library

Ben Franksen-2
On Thursday 25 May 2006 11:36, Bulat Ziganshin wrote:

> Hello Ross,
>
> Wednesday, May 24, 2006, 4:50:39 PM, you wrote:
> >> Now you could make purely functional code raise I/O exceptions,
> >> but that gives rise to a few problems: imprecise exceptions are
> >> difficult to program with (need deepSeq etc.), and they aren't
> >> widely implemented (Hugs doesn't have them, JHC doesn't have them
> >> and may not get them IIUC).
> >
> > Hugs has had imprecise exceptions (but not asynchronous ones) since
> > Nov 2003.
>
> can you please explain or give a pointer - what is an imprecise
> exceptions? i thought that it the synonym for async ones

It's explained here

http://research.microsoft.com/~simonpj/Papers/except.ps.gz

Imprecise exceptions are abotu how you handle exceptions coming from
pure code, like devide-by-zero and such. Imagine

 x = (something/0) + (something_else/0)

then which of the two subexpressions is the exception "divide by zero"
associated with? This is not clear in a non-strict language like
Haskell, where evaluation order is not specified. Imprecise exceptions
are a way around that by (conceptually) including /all/ 'possible'
exceptions into a /set/. However, this set is not observable. Only when
catching an imprecise exception, which /must/ happen in the IO monad,
can be consult an external 'oracle' that choses just one representing
element of the set. Of course in a practical implementation this
element will be determined by the actual evaluation order.

Ben
--
You've never looked into my eyes but don't you want to know
What the dark and the wild and the different know -- Melissa Etheridge
_______________________________________________
Haskell mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell
12