Hypothetical Haskell job in New York

classic Classic list List threaded Threaded
80 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

John Goerzen-3
On Thu, Jan 08, 2009 at 10:36:32AM -0700, John A. De Goes wrote:
>
> The number of applications requiring the implementation of a custom web
> server is an insignificant fraction of the number of applications  
> requiring a messaging system. I don't think anyone would dispute  
> Haskell's ability to do low-level, raw networking, of the type that few
> people actually need to do. It's the higher level stuff where there's a
> huge amount of room for improvement.

I disagree on both points.

Haskell has had somewhat of a deficit in the low-level networking
stuff, not even supporting IPv6 in the standard stack until just
recently.  (That is, things like AF_INET6 were not present.)

I think it has pretty much caught up by now though.

On the other hand, I see nothing in Haskell that would prevent its use
for any of your purposes.  There are numerous high-level web
infrastructures already.  Perhaps they are more or less suited to your
needs, but that's a library issue, not a language issue.  Saying "WASH
sucks" or "Happs sucks" is like saying "rails sucks".  May or may not
be true, but it doesn't imply anything about whether Haskell is suited
to that problem domain.  And much as I detest Rails, it doesn't
necessarily mean that Ruby *must* suck for web apps.

-- John
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

Tim Newsham
In reply to this post by John A. De Goes
> On Jan 8, 2009, at 12:56 PM, Tim Newsham wrote:
>> You replied to someone discussing using Haskell at a CDN to implement
>> things like web servers by saying that Haskell wasn't suitable for
>> the task.
>
> That is incorrect. I replied to Achim's message asking for elaboration on
> Haskell's unsuitability. It was a convenient point to discuss Haskell's
> networking deficiencies.

Oops.  My apologies.

> Regards,
> John

Tim Newsham
http://www.thenewsh.com/~newsham/
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: Hypothetical Haskell job in New York

Manlio Perillo-3
In reply to this post by Don Stewart-2
Don Stewart ha scritto:

> manlio_perillo:
>> Tony Hannan ha scritto:
>>> Let me give you more information about this hypothetical job posting.
>>> Our company is a startup CDN
>>> (http://en.wikipedia.org/wiki/Content_Delivery_Network) about 3 years
>>> old and doing well. You would hythothetically be one of 7 programmer who
>>> write all the software involved in a CDN including http server, dns
>>> server, monitoring, load balancing, customer and operator user
>>> interface, etc. The pay depends on experience but is good.
>>>
>> Isn't it better to use Erlang for the http and dns server ?
>>
>> I don't really like the syntax, but you have many things already
>> implemented.
>> Unfortunately Haskell is not yet ready for this task.
>>
>> http://eddie.sourceforge.net/what.html
>> http://yaws.hyber.org/
>>
>
> Umm... http and dns? You're kidding right? Half of hackage.haskell.org
> is devoted to networking tasks.
>

I'm speaking about servers, not clients.

How much of pure Haskell internet servers are used in a production
environment, in the "open internet" (and not in restricted LANs)?

How much traffic they handle?
How hard are to maintain/update/develope/?


Personally, I only know http://hpaste.org/, based on
Server: HAppS/0.8.4


And about HAppS, I'm not an Haskell expert, but reading the source I see
that static files are server (in the HTTP server) using
Data.ByteString.Lazy's hGetContents

Is this ok?



> -- Don
>


Manlio Perillo
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

Manlio Perillo-3
In reply to this post by John Goerzen-3
John Goerzen ha scritto:
> On Thu, Jan 08, 2009 at 10:36:32AM -0700, John A. De Goes wrote:
> [...]
>
> On the other hand, I see nothing in Haskell that would prevent its use
> for any of your purposes.  There are numerous high-level web
> infrastructures already.  Perhaps they are more or less suited to your
> needs, but that's a library issue, not a language issue.  


The question is not about Haskell language.
I think that Haskell is far better than Erlang, and in fact I'm studying
Haskell and not Erlang; and one of the reason I choosed Haskell is for
its support to concurrency.

The problem, IMHO, is with the availability of solid, production ready
servers implemented in Haskell, that can be used as case study.

The major web framework in Haskell is HAppS, if I'm correct, and yet in
the HAppS code I see some things that make me worry about the robustess
of the code.

I you grep for hGetsContent, it appears in place where I'm not sure if
it is safe to use.

One example is in serving static files.
Another example is the multipart parser:

-- | Read a multi-part message from a 'Handle'.
--   Fails on parse errors.
hGetMultipartBody :: String -- ^ Boundary
                   -> Handle
                   -> IO MultiPart
hGetMultipartBody b h =
     do
     s <- BS.hGetContents h
     case parseMultipartBody b s of
         Nothing -> fail "Error parsing multi-part message"
         Just m  -> return m


Now, if you read the paper about iteratee:
http://okmij.org/ftp/Haskell/Iteratee/

you should have doubts about how robust all of this is.



The other problem is with the use of select in the GHC runtime for IO
multiplexing.

I know that things works, but using select in a server that should
support many concurrent requests, is not really what you really want.




Regards  Manlio Perillo
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Low-level networking [Haskell not ready for Foo]

Andrew Coppin
In reply to this post by John Goerzen-3
John Goerzen wrote:

> On Thu, Jan 08, 2009 at 10:36:32AM -0700, John A. De Goes wrote:
>  
>> The number of applications requiring the implementation of a custom web
>> server is an insignificant fraction of the number of applications  
>> requiring a messaging system. I don't think anyone would dispute  
>> Haskell's ability to do low-level, raw networking, of the type that few
>> people actually need to do. It's the higher level stuff where there's a
>> huge amount of room for improvement.
>>    
>
> I disagree on both points.
>
> Haskell has had somewhat of a deficit in the low-level networking
> stuff, not even supporting IPv6 in the standard stack until just
> recently.  (That is, things like AF_INET6 were not present.)
>
> I think it has pretty much caught up by now though.
>  

Any idea how I get Haskell to send ICMP ECHO packets? (And, obviously,
receive the replies.)

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

Bardur Arantsson-2
In reply to this post by Manlio Perillo-3
Manlio Perillo wrote:

> John Goerzen ha scritto:
>> On Thu, Jan 08, 2009 at 10:36:32AM -0700, John A. De Goes wrote:
>> [...]
>>
>> On the other hand, I see nothing in Haskell that would prevent its use
>> for any of your purposes.  There are numerous high-level web
>> infrastructures already.  Perhaps they are more or less suited to your
>> needs, but that's a library issue, not a language issue.  
>
>
> The question is not about Haskell language.
> I think that Haskell is far better than Erlang, and in fact I'm studying
> Haskell and not Erlang; and one of the reason I choosed Haskell is for
> its support to concurrency.
>
> The problem, IMHO, is with the availability of solid, production ready
> servers implemented in Haskell, that can be used as case study.
>
> The major web framework in Haskell is HAppS, if I'm correct, and yet in
> the HAppS code I see some things that make me worry about the robustess
> of the code.
>
[--snip--]

Indeed. I've been looking for a Haskell HTTP server implementation that
can actually handle file serving using strictly limited memory (for a
simple UPnP server, as of yet unreleased) and that also doesn't leak
handles like a sieve, but I haven't found anything yet. I don't know,
maybe my hackage-foo is lacking. In the end I just rolled my own
implementation using the HTTP package for parsing requests and doing all
the socket I/O myself using low-level primitives. It seemed to be the
only way to guarantee reasonable resource usage while serving
multi-gigabyte files to fickle HTTP clients that like to drop
connections willy-nilly.

Don't get me wrong -- the socket support is pretty decent, but there are
also some weird idiosyncrasies, for example requiring that the PortNum
is specified in network byte order and lacking a function to convert
host->network byte order (hton).

Oleg's Iteratee does look very interesting though. Maybe I'll have a go
at trying to use his ideas in my UPnP server.

Cheers,

Bardur Arantsson

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: Hypothetical Haskell job in New York

Henning Thielemann
In reply to this post by Manlio Perillo-3

On Thu, 8 Jan 2009, Manlio Perillo wrote:

> Personally, I only know http://hpaste.org/, based on
> Server: HAppS/0.8.4

I'm using a modified HWS for the parallel webs, e.g. the Real Monad
Transformer:
   http://www.haskell.org.monadtransformer.parallelnetz.de/haskellwiki/Category:Monad
However, recently a lot of accesses made the server quite unusable.
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

John Van Enk
In reply to this post by Bardur Arantsson-2
> PortNum is specified in network byte order and lacking a function to convert host->network byte order (hton).

Perhaps this is another argument for my thread from a while back?


/jve


On Thu, Jan 8, 2009 at 4:37 PM, Bardur Arantsson <[hidden email]> wrote:
Manlio Perillo wrote:
John Goerzen ha scritto:
On Thu, Jan 08, 2009 at 10:36:32AM -0700, John A. De Goes wrote:
[...]

On the other hand, I see nothing in Haskell that would prevent its use
for any of your purposes.  There are numerous high-level web
infrastructures already.  Perhaps they are more or less suited to your
needs, but that's a library issue, not a language issue.  


The question is not about Haskell language.
I think that Haskell is far better than Erlang, and in fact I'm studying Haskell and not Erlang; and one of the reason I choosed Haskell is for its support to concurrency.

The problem, IMHO, is with the availability of solid, production ready servers implemented in Haskell, that can be used as case study.

The major web framework in Haskell is HAppS, if I'm correct, and yet in the HAppS code I see some things that make me worry about the robustess of the code.

[--snip--]

Indeed. I've been looking for a Haskell HTTP server implementation that can actually handle file serving using strictly limited memory (for a simple UPnP server, as of yet unreleased) and that also doesn't leak handles like a sieve, but I haven't found anything yet. I don't know, maybe my hackage-foo is lacking. In the end I just rolled my own implementation using the HTTP package for parsing requests and doing all the socket I/O myself using low-level primitives. It seemed to be the only way to guarantee reasonable resource usage while serving multi-gigabyte files to fickle HTTP clients that like to drop connections willy-nilly.

Don't get me wrong -- the socket support is pretty decent, but there are also some weird idiosyncrasies, for example requiring that the PortNum is specified in network byte order and lacking a function to convert host->network byte order (hton).

Oleg's Iteratee does look very interesting though. Maybe I'll have a go at trying to use his ideas in my UPnP server.

Cheers,

Bardur Arantsson


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

John Goerzen-3
In reply to this post by Bardur Arantsson-2
On Thu, Jan 08, 2009 at 10:37:55PM +0100, Bardur Arantsson wrote:
> Don't get me wrong -- the socket support is pretty decent, but there are  
> also some weird idiosyncrasies, for example requiring that the PortNum  
> is specified in network byte order and lacking a function to convert  
> host->network byte order (hton).

Look at Haddock for PortNumber:

newtype PortNumber
  Constructors
    PortNum Word16  

  Instances

  Enum PortNumber
  Eq PortNumber
  Integral PortNumber
  Num PortNumber
  Ord PortNumber
  Real PortNumber
  Show PortNumber
  Typeable PortNumber
  Storable PortNumber

Try it in ghci:

Prelude Network.Socket> 15 :: PortNumber
15
Prelude Network.Socket> PortNum 15
3840
Prelude Network.Socket> (fromIntegral (15::Int))::PortNumber
15

So, in essence, there are *many* functions that let you do this.  You
should not be needing to construct PortNum by hand.


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

John Goerzen-3
In reply to this post by John A. De Goes
On Thu, Jan 08, 2009 at 11:14:18AM -0700, John A. De Goes wrote:
> But really, what's the point? FFI code is fragile, often uncompilable  
> and unsupported, and doesn't observe the idioms of Haskell nor take  
> advantage of its powerful language features. Rather than coding through

That is an extraordinarily cruel, and inaccurate, sweep of FFI.

I've worked with C bindings to several high-level languages, and I
must say that I like FFI the best of any I've used.  It's easy to use
correctly, stable, and solid.  If anything, it suffers from
under-documentation.

The whole point of FFI is to bring other languages into the Haskell
fold.  So you can, say, talk to a database using its C library and
wind up putting the strongly-typed HaskellDB atop it.  Or you can
write an MD5 algorithm in C and make it look like a regular Haskell
function.

> You can indeed fit a square peg in a round hole, if you pound hard  
> enough. That doesn't mean it's a good thing to do.

And with that, I fully agree.

-- Joh
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: Hypothetical Haskell job in New York

John Goerzen-3
In reply to this post by Manlio Perillo-3
On Thu, Jan 08, 2009 at 09:46:36PM +0100, Manlio Perillo wrote:
> I'm speaking about servers, not clients.
>
> How much of pure Haskell internet servers are used in a production  
> environment, in the "open internet" (and not in restricted LANs)?

Does that really matter?  I tend to judge technology based on its
merits for my work, not on who uses it.  The fact that Google uses
Python didn't impact my decision to start using it, and it also didn't
impact my decision to start using Haskell.

> How much traffic they handle?
> How hard are to maintain/update/develope/?

Those, of course, are pretty good questions.

> Personally, I only know http://hpaste.org/, based on
> Server: HAppS/0.8.4

Take a look at Hackage.  There are quite a few other Haskell web
frameworks as well: everything from the low-level FastCGI to
higher-level HSP and WASH.

> And about HAppS, I'm not an Haskell expert, but reading the source I see  
> that static files are server (in the HTTP server) using  
> Data.ByteString.Lazy's hGetContents
>
> Is this ok?

In what respect?  The fact that something uses
ByteString.Lazy.hGetContents doesn't imply a problem to me.  It's a
useful function.  It can be used properly, or not, just as while or
read() in C can be.

It's not evil incarnate like sprintf() or anything :-)

-- John
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

Bardur Arantsson-2
In reply to this post by John Goerzen-3
John Goerzen wrote:

> On Thu, Jan 08, 2009 at 10:37:55PM +0100, Bardur Arantsson wrote:
>> Don't get me wrong -- the socket support is pretty decent, but there are  
>> also some weird idiosyncrasies, for example requiring that the PortNum  
>> is specified in network byte order and lacking a function to convert  
>> host->network byte order (hton).
>
> Look at Haddock for PortNumber:
>
> newtype PortNumber
>   Constructors
>     PortNum Word16  
>
>   Instances
>
>   Enum PortNumber
>   Eq PortNumber
>   Integral PortNumber
>   Num PortNumber
>   Ord PortNumber
>   Real PortNumber
>   Show PortNumber
>   Typeable PortNumber
>   Storable PortNumber
>
> Try it in ghci:
>
> Prelude Network.Socket> 15 :: PortNumber
> 15
> Prelude Network.Socket> PortNum 15
> 3840
> Prelude Network.Socket> (fromIntegral (15::Int))::PortNumber
> 15
>
> So, in essence, there are *many* functions that let you do this.  You
> should not be needing to construct PortNum by hand.

Thanks. For some reason I hadn't thought to use

    (fromIntegral x)::PortNumber

I guess I got stuck on the idea of constructing a PortNum directly and
didn't think beyond that. (Maybe PortNum should really be an abstract
type to force indirect construction...?)

I guess the API isn't all that idiosyncratic after all :).

Cheers,

Bardur Arantsson

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

Henning Thielemann

On Thu, 8 Jan 2009, Bardur Arantsson wrote:

> Thanks. For some reason I hadn't thought to use
>
>   (fromIntegral x)::PortNumber
>
> I guess I got stuck on the idea of constructing a PortNum directly and didn't
> think beyond that. (Maybe PortNum should really be an abstract type to force
> indirect construction...?)

I also think that a Num instance for PortNumber is not a good idea. How
shall (+) and (*) be defined?
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

Austin Seipp
In reply to this post by John A. De Goes
Excerpts from John A. De Goes's message of Thu Jan 08 12:14:18 -0600 2009:
> But really, what's the point? FFI code is fragile, often uncompilable  
> and unsupported, and doesn't observe the idioms of Haskell nor take  
> advantage of its powerful language features.

This is a completely unfair generalization. The FFI is an excellent
way to interoperate with an extraordinary amount of external
libraries, and if you ask me, it's *worth* taking those pieces of C
code and wrapping them up in a nice, safe haskell interface. I will also
mention that Haskell has *the* simplest FFI I have ever used, which to
me only means it's easier to get it right (the fact that there are
customized *languages* like e.g. cython to make writing python
extensions easier makes me wonder.)

I suggest you take a look at the haskell llvm bindings - they are
extraordinarily well documented, and the high level interface uses
*many* haskell idioms that make the library safe and easy to use:

http://hackage.haskell.org/cgi-bin/hackage-scripts/package/llvm-0.4.2.0

This code most certainly takes advantage of powerful features and
idioms that only Haskell can provide. Please do not take your bad
experiences with a few crappy binding (or not even crappy bindings,
perhaps bindings that just aren't very abstract) and extend them to
the bindings that are excellent with a sweeping statement like that.

Austin
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

John Goerzen-3
In reply to this post by Bardur Arantsson-2
Bardur Arantsson wrote:
> Thanks. For some reason I hadn't thought to use
>
>     (fromIntegral x)::PortNumber
>
> I guess I got stuck on the idea of constructing a PortNum directly and
> didn't think beyond that. (Maybe PortNum should really be an abstract

No problem.  I knew exactly what your problem was because I had the
exact same blinders on when I first learned the Haskell networking API.
 It would be helpful to have a pointer to this in the Haddock docs,
because it is non-intuitive to somebody just learning it.

> I guess the API isn't all that idiosyncratic after all :).

Just a touch under-documented ;-)
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hypothetical Haskell job in New York

Bardur Arantsson-2
In reply to this post by John Goerzen-3
John Goerzen wrote:

> On Thu, Jan 08, 2009 at 09:46:36PM +0100, Manlio Perillo wrote:
>> I'm speaking about servers, not clients.
>>
>
>> Personally, I only know http://hpaste.org/, based on
>> Server: HAppS/0.8.4
>
> Take a look at Hackage.  There are quite a few other Haskell web
> frameworks as well: everything from the low-level FastCGI to
> higher-level HSP and WASH.
>

FastCGI is not a HTTP server. WASH seems so include one, but the latest
version ("Wash and go") seems to be from mid-2007 ("tested with GHC 6.6"
as the web page states), unless of course I'm looking at the wrong page.
That doesn't exactly inspire a lot of confidence.

Now, if you're talking about using, say, Apache + FastCGI then you'll
probably have something pretty robust, but I don't think that counts as
a "Haskell server".

Generally my experience has been that most of the Haskell server stuff
hasn't been very mature.

>> And about HAppS, I'm not an Haskell expert, but reading the source I see  
>> that static files are server (in the HTTP server) using  
>> Data.ByteString.Lazy's hGetContents
>>
>> Is this ok?
>
> In what respect?  The fact that something uses
> ByteString.Lazy.hGetContents doesn't imply a problem to me.  It's a
> useful function.  It can be used properly, or not, just as while or
> read() in C can be.

It's a great way to introduce unavoidable handle leaks, that's for sure.

Cheers,

Bardur Arantsson

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Low-level networking [Haskell not ready for Foo]

John Goerzen-3
In reply to this post by Andrew Coppin
Andrew Coppin wrote:

> John Goerzen wrote:
>> On Thu, Jan 08, 2009 at 10:36:32AM -0700, John A. De Goes wrote:
>>  
>>> The number of applications requiring the implementation of a custom web
>>> server is an insignificant fraction of the number of applications  
>>> requiring a messaging system. I don't think anyone would dispute  
>>> Haskell's ability to do low-level, raw networking, of the type that few
>>> people actually need to do. It's the higher level stuff where there's a
>>> huge amount of room for improvement.
>>>    
>> I disagree on both points.
>>
>> Haskell has had somewhat of a deficit in the low-level networking
>> stuff, not even supporting IPv6 in the standard stack until just
>> recently.  (That is, things like AF_INET6 were not present.)
>>
>> I think it has pretty much caught up by now though.
>>  
>
> Any idea how I get Haskell to send ICMP ECHO packets? (And, obviously,
> receive the replies.)

SocketType claims to support Raw, which I think is the conventional
means for doing this.  Whether all the infrastructure for that is there,
I don't know.  I have never worked with raw sockets though, so I may be
leading you down a dark mugger-laden alley here ;-)

-- John
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

Bryan O'Sullivan
In reply to this post by Don Stewart-2
On Thu, Jan 8, 2009 at 10:06 AM, Don Stewart <[hidden email]> wrote:
Note that there even exists an FFI binding to Erlang for Haskell, so
that Haskell nodes can seamlessly interact with other nodes speaking
Erlang's protocol format.

Actually, the erlang package doesn't use the FFI: it speaks the Erlang wire protocol to send and receive terms.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

sclv
In reply to this post by John Goerzen-3

> On Thu, Jan 08, 2009 at 11:14:18AM -0700, John A. De Goes wrote:
>> But really, what's the point? FFI code is fragile, often uncompilable
>> and unsupported, and doesn't observe the idioms of Haskell nor take
>> advantage of its powerful language features. Rather than coding  
>> through
>

Just for clarity's sake, we should specify that the Erlang ffi  
interface that's been worked on (not my project, I've just browsed  
the code) is *not* low-level via C, but rather a set of parsers/
unparsers between haskell data types and the Erlang wire format and a  
set of behaviors for message queuing that between them let a haskell  
program act as a node to Erlang programs, or let Haskell programs  
communicate between themselves as nodes, which just coincidentally  
happen to use the same wire format as Erlang. Which is not to say  
that Erlang does not have specific excellent libraries that allow  
*certain types* of network programming do be done very easily. The  
Haskell library-space has lots of room to grow, and lots of  
inspiration to take from the OTP (although less for "hot-swapping"  
which is somewhat overrated, and more from supervision-tree type  
stuff). However, even now an adequate subset of whatever  
functionality is needed can be whipped up pretty quickly for any  
project requiring only that subset.

Cheers,
S.
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Haskell not ready for Foo [was: Re: Hypothetical Haskell job in New York]

Bryan O'Sullivan
In reply to this post by Manlio Perillo-3
On Thu, Jan 8, 2009 at 1:07 PM, Manlio Perillo <[hidden email]> wrote:
 
Another example is the multipart parser:

-- | Read a multi-part message from a 'Handle'.
--   Fails on parse errors.
hGetMultipartBody :: String -- ^ Boundary
                 -> Handle
                 -> IO MultiPart
hGetMultipartBody b h =
   do
   s <- BS.hGetContents h
   case parseMultipartBody b s of
       Nothing -> fail "Error parsing multi-part message"
       Just m  -> return m

Yes, that's definitely on the scary side of things.

However, you don't have to go all the way to drinking the Iteratee Kool-Aid in order to write safer networking code that is still performant. Here are a few projects I'm actively working on in this area:
  • I'm adding epoll support to the threaded RTS. This is a necessity for server performance.
  • I've added support for sending and receiving lazy ByteStrings to Johan Tibbell's network-bytestring library. A quick benchmark with a toy HTTP server has shown this to be about 2x faster than writing ByteStrings to a Handle (i.e. 14,000 requests per second, vs 7,000).
  • I've got a continuation-based resumable parser combinator module for attoparsec in progress, which uses lazy ByteStrings for blazing performance. You can use this to write protocol parsers in a completely clean way, decoupled from the underlying network receive operations.
While much of this isn't quite ready for use yet, this just represents one person's work, and there are lots of people beavering away actively at corners of the problem space that interest them.

I actually think that we're very close to being in fantastic shape here. I'm working on a memcached client library that uses the above libraries, and it's faster than the absolute best C memcached client (libmemcached), while also far smaller and elegantly layered. As a community, we are developing many proofs that you can have beautiful code without sacrificing performance.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
1234