Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)

tphyahoo
From: tphyahoo <[hidden email]>
Subject: Help me prove macid can scale! (Macid stress tests results
for happs-tutorial toy job board disappointing so far.)
To: HAppS


HAppS is a new, relatively unproven technology.

So I am asking myself this question. Will HAppS allow me to scale the
toy job board I created for happs-tutorial into a high-volume, high
concurrency, large-user count type job board? You know, the kind that
might make money?

I have done some preliminary testing to answer this question, and so
far the results have been disappointing.

I am hoping that I am doing something wrong, and that there is a way
of using macid effectively for more than just toy applications. If
there are solution for the problems I'm experiencing I will definitely
be integrating this knowledge into the tutorial, so stay tuned.

I am seeking feedback from HAppS experts and educated users on the
following questions:

    * Is building a heavy duty website like monster.com in HAppS a
realistic goal -- say, in the next twelve months?
    * Are there certain types of web apps that are unlikely to work
well with the HAppS web architecture?
    * Are there changes I can make to my toy app's architecture -- be
it data structures, buying new hardware, whatever -- that will enable
me to get good performance against the stress test described below and
in the demo?
    * Are there other HAppS stress tests in the public domain, and
what are the results so far?

A complete description of the stress test I did (basically, insert 200
users with 200 jobs apiece)  along with a runnable demo, is in the
tutorial preview at

http://happstutorial.com:5002/tutorial/macid-stress-test.

(The actual stress test is disabled in the online demo because I don't
want to zap my server.)

But the summary is, it's slow, slow to start, and creates a huge
events file -- 356M. For around 40000 records. That's like, 8
megabytes per job, which is a few lines of text. OK, happs isn't a
database -- but still.

This is all is checked into darcs and tagged:

darcs get --tag='stress test'http://code.haskell.org/happs-tutorial

Thanks in advance for helping make happs usable in the real world!

thomas.
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)

Jason Dusek
  I don't want to be contrarian, but I guess I can't help
  myself. Does MACID have anything to say about failover and
  replication? Isn't that more important than volume?

--
_jsn
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)

Duncan Coutts
On Sun, 2008-10-05 at 15:57 -0700, Jason Dusek wrote:
> I don't want to be contrarian, but I guess I can't help
>   myself. Does MACID have anything to say about failover and
>   replication? Isn't that more important than volume?

HAppS does failover and replication within a cluster. They're working on
sharding, but that's a good deal harder.

Duncan

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)

tphyahoo
There's a thread at happs group about this with good answers

http://groups.google.com/group/HAppS/browse_thread/thread/1031809dca26f349

My top post should probably have been a cc, apologies.

Looks pretty likely that the issues are solvable. Grokking the answers now.

thomas.


2008/10/6 Duncan Coutts <[hidden email]>:

> On Sun, 2008-10-05 at 15:57 -0700, Jason Dusek wrote:
>> I don't want to be contrarian, but I guess I can't help
>>   myself. Does MACID have anything to say about failover and
>>   replication? Isn't that more important than volume?
>
> HAppS does failover and replication within a cluster. They're working on
> sharding, but that's a good deal harder.
>
> Duncan
>
>
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe