haskell in the cloud

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

haskell in the cloud

Dennis Raddle
I have a CPU-intensive Haskell application. I have it working with simple multi-core concurrency. I'm wondering if I can run this on a cloud virtual machine with 128 cores or so, paying by the CPU minute. I'll run it for maybe 15 minutes a day so I'm probably best off paying just for the CPU used.

What platform would be recommended? For ease of use? For best price/performance?

It's a backtracking optimization algorithm that builds data, one element at a time. It's not hard at all to make it concurrent: at the first 3 or so levels of element choices would be about 100 to 500 combinations. Even if the simplest method of running concurrent Haskell on multiple cores doesn't work, I could just divide these first cases into batches and run them on individual machines. 

D

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: haskell in the cloud

David Reaver
Amazon Web Services has a x1.32xlarge EC2 instance with 128 CPU cores and just under 2000 GiB of RAM for about $13 per hour. AWS actually has per-second billing (with a 1 minute minimum) since late last year.

They have lots of other options as well of course. Here is their pricing page: https://aws.amazon.com/ec2/pricing/on-demand/

On Wed, May 23, 2018 at 2:51 AM, Dennis Raddle <[hidden email]> wrote:
I have a CPU-intensive Haskell application. I have it working with simple multi-core concurrency. I'm wondering if I can run this on a cloud virtual machine with 128 cores or so, paying by the CPU minute. I'll run it for maybe 15 minutes a day so I'm probably best off paying just for the CPU used.

What platform would be recommended? For ease of use? For best price/performance?

It's a backtracking optimization algorithm that builds data, one element at a time. It's not hard at all to make it concurrent: at the first 3 or so levels of element choices would be about 100 to 500 combinations. Even if the simplest method of running concurrent Haskell on multiple cores doesn't work, I could just divide these first cases into batches and run them on individual machines. 

D

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: haskell in the cloud

Neil Mayhew
On 2018-05-23 02:24 PM, David Reaver wrote:
Even if the simplest method of running concurrent Haskell on multiple cores doesn't work, I could just divide these first cases into batches and run them on individual machines. 

You might find it's cheaper to use several low-end machines than one very high-end machine. You probably pay a premium for being able to run algorithms that require it all to be on one machine.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: haskell in the cloud

Alexandre Peyroux
In reply to this post by David Reaver

Le mer. 23 mai 2018 à 22:25, David Reaver <[hidden email]> a écrit :
Amazon Web Services has a x1.32xlarge EC2 instance with 128 CPU cores and just under 2000 GiB of RAM for about $13 per hour. AWS actually has per-second billing (with a 1 minute minimum) since late last year.

They have lots of other options as well of course. Here is their pricing page: https://aws.amazon.com/ec2/pricing/on-demand/

On Wed, May 23, 2018 at 2:51 AM, Dennis Raddle <[hidden email]> wrote:
I have a CPU-intensive Haskell application. I have it working with simple multi-core concurrency. I'm wondering if I can run this on a cloud virtual machine with 128 cores or so, paying by the CPU minute. I'll run it for maybe 15 minutes a day so I'm probably best off paying just for the CPU used.

What platform would be recommended? For ease of use? For best price/performance?

It's a backtracking optimization algorithm that builds data, one element at a time. It's not hard at all to make it concurrent: at the first 3 or so levels of element choices would be about 100 to 500 combinations. Even if the simplest method of running concurrent Haskell on multiple cores doesn't work, I could just divide these first cases into batches and run them on individual machines. 

D

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: haskell in the cloud

Dennis Raddle
In reply to this post by David Reaver
Thanks, everyone.

Billing by the second is good. Billing by the hour is not going to work (that seems to be the Scaleway model).

When I asked this question a while back, I was told that there is a cloud service specifically for Haskell programs. Just wondering if that might be a good fit for me.

I probably would benefit the most by running on multiple machines. I'd like to have them coordinate with each other... i.e. one main controller program will initiate and run other programs for a few seconds, then collect the results and start another run.

What's the simplest Haskell library to get this going? Cloud Haskell?

D



On Wed, May 23, 2018 at 1:24 PM, David Reaver <[hidden email]> wrote:
Amazon Web Services has a x1.32xlarge EC2 instance with 128 CPU cores and just under 2000 GiB of RAM for about $13 per hour. AWS actually has per-second billing (with a 1 minute minimum) since late last year.

They have lots of other options as well of course. Here is their pricing page: https://aws.amazon.com/ec2/pricing/on-demand/

On Wed, May 23, 2018 at 2:51 AM, Dennis Raddle <[hidden email]> wrote:
I have a CPU-intensive Haskell application. I have it working with simple multi-core concurrency. I'm wondering if I can run this on a cloud virtual machine with 128 cores or so, paying by the CPU minute. I'll run it for maybe 15 minutes a day so I'm probably best off paying just for the CPU used.

What platform would be recommended? For ease of use? For best price/performance?

It's a backtracking optimization algorithm that builds data, one element at a time. It's not hard at all to make it concurrent: at the first 3 or so levels of element choices would be about 100 to 500 combinations. Even if the simplest method of running concurrent Haskell on multiple cores doesn't work, I could just divide these first cases into batches and run them on individual machines. 

D

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.



_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: haskell in the cloud

Johann Gonzalez
Hello!,

You could also try Google Cloud's n1-ultramem-160 [1] , with 160 vcpus and 3800GB RAM.
As you can only need to run it for a short time it can be a preemptible instance that goes for $5.3 per hour, with per second billing[2]

[1]https://cloud.google.com/compute/pricing#megamem

On Wed, May 23, 2018 at 5:59 PM, Dennis Raddle <[hidden email]> wrote:
Thanks, everyone.

Billing by the second is good. Billing by the hour is not going to work (that seems to be the Scaleway model).

When I asked this question a while back, I was told that there is a cloud service specifically for Haskell programs. Just wondering if that might be a good fit for me.

I probably would benefit the most by running on multiple machines. I'd like to have them coordinate with each other... i.e. one main controller program will initiate and run other programs for a few seconds, then collect the results and start another run.

What's the simplest Haskell library to get this going? Cloud Haskell?

D



On Wed, May 23, 2018 at 1:24 PM, David Reaver <[hidden email]> wrote:
Amazon Web Services has a x1.32xlarge EC2 instance with 128 CPU cores and just under 2000 GiB of RAM for about $13 per hour. AWS actually has per-second billing (with a 1 minute minimum) since late last year.

They have lots of other options as well of course. Here is their pricing page: https://aws.amazon.com/ec2/pricing/on-demand/

On Wed, May 23, 2018 at 2:51 AM, Dennis Raddle <[hidden email]> wrote:
I have a CPU-intensive Haskell application. I have it working with simple multi-core concurrency. I'm wondering if I can run this on a cloud virtual machine with 128 cores or so, paying by the CPU minute. I'll run it for maybe 15 minutes a day so I'm probably best off paying just for the CPU used.

What platform would be recommended? For ease of use? For best price/performance?

It's a backtracking optimization algorithm that builds data, one element at a time. It's not hard at all to make it concurrent: at the first 3 or so levels of element choices would be about 100 to 500 combinations. Even if the simplest method of running concurrent Haskell on multiple cores doesn't work, I could just divide these first cases into batches and run them on individual machines. 

D

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.



_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.



--
Johan Gonzalez
Devops/IT Manager


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: haskell in the cloud

Dennis Raddle
I notice that Amazon calls that "virtual CPUs" not "CPU cores".. is that the same as cores? I.e. will standard multi-core Haskell concurrency methods work? Anyone know?

Mike


On Wed, May 23, 2018 at 3:14 PM, Johann Gonzalez <[hidden email]> wrote:
Hello!,

You could also try Google Cloud's n1-ultramem-160 [1] , with 160 vcpus and 3800GB RAM.
As you can only need to run it for a short time it can be a preemptible instance that goes for $5.3 per hour, with per second billing[2]

[1]https://cloud.google.com/compute/pricing#megamem

On Wed, May 23, 2018 at 5:59 PM, Dennis Raddle <[hidden email]> wrote:
Thanks, everyone.

Billing by the second is good. Billing by the hour is not going to work (that seems to be the Scaleway model).

When I asked this question a while back, I was told that there is a cloud service specifically for Haskell programs. Just wondering if that might be a good fit for me.

I probably would benefit the most by running on multiple machines. I'd like to have them coordinate with each other... i.e. one main controller program will initiate and run other programs for a few seconds, then collect the results and start another run.

What's the simplest Haskell library to get this going? Cloud Haskell?

D



On Wed, May 23, 2018 at 1:24 PM, David Reaver <[hidden email]> wrote:
Amazon Web Services has a x1.32xlarge EC2 instance with 128 CPU cores and just under 2000 GiB of RAM for about $13 per hour. AWS actually has per-second billing (with a 1 minute minimum) since late last year.

They have lots of other options as well of course. Here is their pricing page: https://aws.amazon.com/ec2/pricing/on-demand/

On Wed, May 23, 2018 at 2:51 AM, Dennis Raddle <[hidden email]> wrote:
I have a CPU-intensive Haskell application. I have it working with simple multi-core concurrency. I'm wondering if I can run this on a cloud virtual machine with 128 cores or so, paying by the CPU minute. I'll run it for maybe 15 minutes a day so I'm probably best off paying just for the CPU used.

What platform would be recommended? For ease of use? For best price/performance?

It's a backtracking optimization algorithm that builds data, one element at a time. It's not hard at all to make it concurrent: at the first 3 or so levels of element choices would be about 100 to 500 combinations. Even if the simplest method of running concurrent Haskell on multiple cores doesn't work, I could just divide these first cases into batches and run them on individual machines. 

D

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.



_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.



--
Johan Gonzalez
Devops/IT Manager



_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: haskell in the cloud

Alexander Kjeldaas
In reply to this post by Dennis Raddle


On Wed, May 23, 2018 at 11:59 PM, Dennis Raddle <[hidden email]> wrote:
Thanks, everyone.

Billing by the second is good. Billing by the hour is not going to work (that seems to be the Scaleway model).


You pay a significant premium for the per second billing.  The price-efficient EC2 server is going to be c5.18xlarge

Those are $0.048 / (core*hour).   Compare that to Hetzner Cloud cores, also Skylake, which clocks in at $0.0073 / (core*hour).

When you factor in server setup time of 1-3 minutes, then you can only use your "ultra-cheap" EC2 machine for a few minutes before it would be cheaper to rent the Hetzner server for an hour!

Alexander

 

When I asked this question a while back, I was told that there is a cloud service specifically for Haskell programs. Just wondering if that might be a good fit for me.

I probably would benefit the most by running on multiple machines. I'd like to have them coordinate with each other... i.e. one main controller program will initiate and run other programs for a few seconds, then collect the results and start another run.

What's the simplest Haskell library to get this going? Cloud Haskell?


 
D



On Wed, May 23, 2018 at 1:24 PM, David Reaver <[hidden email]> wrote:
Amazon Web Services has a x1.32xlarge EC2 instance with 128 CPU cores and just under 2000 GiB of RAM for about $13 per hour. AWS actually has per-second billing (with a 1 minute minimum) since late last year.

They have lots of other options as well of course. Here is their pricing page: https://aws.amazon.com/ec2/pricing/on-demand/

On Wed, May 23, 2018 at 2:51 AM, Dennis Raddle <[hidden email]> wrote:
I have a CPU-intensive Haskell application. I have it working with simple multi-core concurrency. I'm wondering if I can run this on a cloud virtual machine with 128 cores or so, paying by the CPU minute. I'll run it for maybe 15 minutes a day so I'm probably best off paying just for the CPU used.

What platform would be recommended? For ease of use? For best price/performance?

It's a backtracking optimization algorithm that builds data, one element at a time. It's not hard at all to make it concurrent: at the first 3 or so levels of element choices would be about 100 to 500 combinations. Even if the simplest method of running concurrent Haskell on multiple cores doesn't work, I could just divide these first cases into batches and run them on individual machines. 

D

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.



_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: haskell in the cloud

Alfred Matthews
Nice.

On Thu, 24 May 2018 at 02:45 Alexander Kjeldaas <[hidden email]> wrote:
On Wed, May 23, 2018 at 11:59 PM, Dennis Raddle <[hidden email]> wrote:
Thanks, everyone.

Billing by the second is good. Billing by the hour is not going to work (that seems to be the Scaleway model).


You pay a significant premium for the per second billing.  The price-efficient EC2 server is going to be c5.18xlarge

Those are $0.048 / (core*hour).   Compare that to Hetzner Cloud cores, also Skylake, which clocks in at $0.0073 / (core*hour).

When you factor in server setup time of 1-3 minutes, then you can only use your "ultra-cheap" EC2 machine for a few minutes before it would be cheaper to rent the Hetzner server for an hour!

Alexander

 

When I asked this question a while back, I was told that there is a cloud service specifically for Haskell programs. Just wondering if that might be a good fit for me.

I probably would benefit the most by running on multiple machines. I'd like to have them coordinate with each other... i.e. one main controller program will initiate and run other programs for a few seconds, then collect the results and start another run.

What's the simplest Haskell library to get this going? Cloud Haskell?


 
D



On Wed, May 23, 2018 at 1:24 PM, David Reaver <[hidden email]> wrote:
Amazon Web Services has a x1.32xlarge EC2 instance with 128 CPU cores and just under 2000 GiB of RAM for about $13 per hour. AWS actually has per-second billing (with a 1 minute minimum) since late last year.

They have lots of other options as well of course. Here is their pricing page: https://aws.amazon.com/ec2/pricing/on-demand/

On Wed, May 23, 2018 at 2:51 AM, Dennis Raddle <[hidden email]> wrote:
I have a CPU-intensive Haskell application. I have it working with simple multi-core concurrency. I'm wondering if I can run this on a cloud virtual machine with 128 cores or so, paying by the CPU minute. I'll run it for maybe 15 minutes a day so I'm probably best off paying just for the CPU used.

What platform would be recommended? For ease of use? For best price/performance?

It's a backtracking optimization algorithm that builds data, one element at a time. It's not hard at all to make it concurrent: at the first 3 or so levels of element choices would be about 100 to 500 combinations. Even if the simplest method of running concurrent Haskell on multiple cores doesn't work, I could just divide these first cases into batches and run them on individual machines. 

D

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.



_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
--


Alfred.


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.