Project postmortem II /Haskell vs. Erlang/

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Project postmortem II /Haskell vs. Erlang/

Joel Reymont
Simon,

Please see this post for an extended reply:

http://wagerlabs.com/articles/2006/01/01/haskell-vs-erlang-reloaded

        Thanks, Joel

On Dec 29, 2005, at 8:22 AM, Simon Peyton-Jones wrote:

> | Using Haskell for this networking app forced me to focus on all the
> | issues _but_ the business logic. Type constraints, binary IO and
> | serialization, minimizing memory use and fighting laziness, timers,
> | tweaking concurrency and setting up message channels, you name it.
>
> That's a disappointing result.  Mostly I think Haskell lets you
> precisely focus on the logic of your program, because lots else is  
> taken
> care of behind the scenes.  You found precisely the reverse.
>
> It'd be interesting to understand which of these issues are
> - language issues
> - library issues
> - compiler/run-time issues
>
> My (ill-informed) hypothesis is that better libraries would have  
> solved
> much of your problems.  A good example is a fast, generic  
> serialisation
> library.
>
> If you felt able (sometime) to distil your experience under headings
> like the above, only more concretely and precisely, I think it might
> help to motivate Haskellers to start solving them.

--
http://wagerlabs.com/


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Project postmortem II /Haskell vs. Erlang/

Dylan Thurston-3
On Sun, Jan 01, 2006 at 11:12:31PM +0000, Joel Reymont wrote:
> Simon,
>
> Please see this post for an extended reply:
>
> http://wagerlabs.com/articles/2006/01/01/haskell-vs-erlang-reloaded

Looking at this code, I wonder if there are better ways to express
what you really want using static typing.  To wit, with records, you
give an example


data Pot = Pot
    {
     pProfit :: !Word64,
     pAmounts :: ![Word64] -- Word16/
    } deriving (Show, Typeable)

mkPot :: Pot
mkPot =
    Pot
    {
     pProfit = 333,
     pAmounts = []
    }

and complain about "having to explain to the customer how xyFoo is
really different from zFoo when they really mean the same thing".  I
wonder: if they really are the same thing, is there a way to get the
data types to faithfully reflect that?  Can you post a few more
snippets of your data structures?

Peace,
        Dylan

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Project postmortem II /Haskell vs. Erlang/

Ketil Malde-2
Dylan Thurston <[hidden email]> writes:

>> http://wagerlabs.com/articles/2006/01/01/haskell-vs-erlang-reloaded

| Compare Erlang
|
| -record(pot, {
|  profit = 0,
|  amounts = []
| }).

> [...] complain about "having to explain to the customer how xyFoo is
> really different from zFoo when they really mean the same thing".

Isn't the obvious solution to declare a class here?  I.e.

   class HasProfits h where profits :: h -> Word64

   data Pot = Pot { pProfits :: !Word64, pAmounts = ![Word64] }

   instance HasProfits Pot where profits = pProfits

And since you like to count LOC, why not use a more compact representation?

   mkPot = Pot 333 []

If it resides close to the data definition, it's easy to keep the two
in sync.

-k
--
If I haven't seen further, it is by standing in the footprints of giants

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Project postmortem II /Haskell vs. Erlang/

Joel Reymont
In reply to this post by Dylan Thurston-3
Sure. Type classes, as Ketil Malde has suggested.

On Jan 4, 2006, at 2:09 AM, Dylan Thurston wrote:

> Looking at this code, I wonder if there are better ways to express
> what you really want using static typing.  To wit, with records, you
> give an example
>
>
> data Pot = Pot
>     {
>      pProfit :: !Word64,
>      pAmounts :: ![Word64] -- Word16/
>     } deriving (Show, Typeable)

--
http://wagerlabs.com/





_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Project postmortem II /Haskell vs. Erlang/

Joel Reymont
In reply to this post by Ketil Malde-2
Yes, that _is_ obvious but then puts the burden on the programmer to  
define the getters. It also misses the setters issue entirely.

Each field can definitely be made into a class and records can be  
composed dynamically, HList-style. I think HList is _the_ facility  
for doing this.

How do you create a "record" that has a field dictionary for updates  
and retrievals and stores the order of the fields for serialization?  
I think HList does provide for the order of records since it keeps  
them in a list of sorts.

Can serialization of these HList-style records serialization be  
efficient in this case? Would updating fields be efficient?

I tried composing my records using HList and that made GHC run out of  
memory and bomb out. Simon Peyton-Jones fixed the problem promptly  
but since I was not familiar with the profiler at the time I never  
got to measuring the pickling efficiency. I figured if it's that  
tough on the compiler then maybe I should pick an easier path.

        Thanks, Joel

On Jan 4, 2006, at 8:41 AM, Ketil Malde wrote:

> Isn't the obvious solution to declare a class here?  I.e.
>
>    class HasProfits h where profits :: h -> Word64
>
>    data Pot = Pot { pProfits :: !Word64, pAmounts = ![Word64] }
>
>    instance HasProfits Pot where profits = pProfits

--
http://wagerlabs.com/





_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re[2]: Project postmortem II /Haskell vs. Erlang/

Bulat Ziganshin
In reply to this post by Joel Reymont
Hello Joel,

use enclosed unstuff.hs
ghc -O2 --make unstuff.hs -o unstuff -lz
./unstuff trace.dat +RTS -s -A10m

analysis:

1) are you really don't use -O2 switch? :)

2) your origianl program spend 2/3 of it's time in GC. so i used
-A10m to reduce GC times

3) i also placed lock around `unstuff` call to decrease GC times

4) small delay between starting threads allow each thread to start
smoother

of course, try these changes in your real networking code

--
Best regards,
 Bulat                            mailto:[hidden email]
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe

unstuff.hs (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: Project postmortem II /Haskell vs. Erlang/

Joel Reymont
Bulat,

On Jan 4, 2006, at 7:57 PM, Bulat Ziganshin wrote:

> 3) i also placed lock around `unstuff` call to decrease GC times

This sort of invalidates the test. We have already proven that it  
works much better when you do this but it just pushes the delays  
upstream.

I will profile your version in the next few days to compare it to my  
final results.

        Thanks, Joel

--
http://wagerlabs.com/





_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re[4]: Project postmortem II /Haskell vs. Erlang/

Bulat Ziganshin
Hello Joel,

Wednesday, January 04, 2006, 11:31:48 PM, you wrote:

>> 3) i also placed lock around `unstuff` call to decrease GC times

JR> This sort of invalidates the test. We have already proven that it  
JR> works much better when you do this but it just pushes the delays  
JR> upstream.

as i say, try this on real program


--
Best regards,
 Bulat                            mailto:[hidden email]



_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re[5]: Project postmortem II /Haskell vs. Erlang/

Bulat Ziganshin
Hello Bulat,

Thursday, January 05, 2006, 3:14:12 AM, you wrote:

>>> 3) i also placed lock around `unstuff` call to decrease GC times

JR>> This sort of invalidates the test. We have already proven that it  
JR>> works much better when you do this but it just pushes the delays  
JR>> upstream.

on my 1ghz duron, unpickling speed (for sequential code) is about 2
mb/s. with 50 kb packets, it can run 40 packets/s and 120 packets in a
3 sec. my changes to program ensure minimal overhead of threading, so
i can guarantee 120 working threads for this program. your processor
is slightly faster, it will run 150-200 threads. to futher increase
speed, you need either

1) use faster processor or many processors
2) speed up unpickling
3) learn timeout strategy of server and write program according to it

i also recommend you to try FD from my Binary package instead of
Handles because using 1000 Handles may involve a large memory/cpu pressure


--
Best regards,
 Bulat                            mailto:[hidden email]



_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re[5]: Project postmortem II /Haskell vs. Erlang/

Joel Reymont
Could you give us a bit more detail on this?

How does using handles involve large memory/CPU pressure?

On Jan 5, 2006, at 10:01 AM, Bulat Ziganshin wrote:

> i also recommend you to try FD from my Binary package instead of
> Handles because using 1000 Handles may involve a large memory/cpu  
> pressure

--
http://wagerlabs.com/





_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re[7]: Project postmortem II /Haskell vs. Erlang/

Bulat Ziganshin
Hello Joel,

Thursday, January 05, 2006, 2:01:43 PM, you wrote:

JR> Could you give us a bit more detail on this?

forget about this :)  i forget that direct i/o on sockets will block
entire app. GHC organizez its own complex non-blocking machinery

JR> How does using handles involve large memory/CPU pressure?

just look at GHC.Handle, .IO, .IOBase modules

JR> On Jan 5, 2006, at 10:01 AM, Bulat Ziganshin wrote:

>> i also recommend you to try FD from my Binary package instead of
>> Handles because using 1000 Handles may involve a large memory/cpu  
>> pressure

JR> --
JR> http://wagerlabs.com/







--
Best regards,
 Bulat                            mailto:[hidden email]



_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe