Increased memory usage with GHC 7.10.1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
36 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Increased memory usage with GHC 7.10.1

Jan Stolarek
Forall hi,

I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed installations of GHC and
this means I had to rebuild Agda and Idris because the binaries built with GHC 7.8.4 were stored
inside deactivated 7.8.4 sandbox. Sadly, I had problems building both Agda and Idris due to GHC
taking up all of available memory.

With Idris the problematic module was Idris.ElabTerm (~2900LOC). The interesting part of the story
is that when I do a clean build of Idris GHC consumes all of memory when compiling that module
and I have to kill the build. But when I restart the build after killing GHC the module is
compiled using a reasonable amount of memory and within reasonable time.

With Agda the problematic module is Agda.TypeChecking.Serialise (~2000LOC). The trick with killing
the build and restarting it didn't work in this case. I had to compile Agda with GHC 7.8.4 (which
works without problems though the mentioned module still requires a lot of memory) and alter my
setup so that Agda binary is not stored inside GHC sandbox.

I wonder if any of you came across similar issues with GHC 7.10.1? Do we have any performance data
that allows to compare memory usage and performance of GHC 7.10.1 with previous stable releases?

All of the above happened on 64bit Debian Wheezy with 2GB of RAM.

Janek

---
Politechnika Łódzka
Lodz University of Technology

Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę
prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.

This email contains information intended solely for the use of the individual to whom it is addressed.
If you are not the intended recipient or if you have received this message in error,
please notify the sender and delete it from your system.
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Jan Stolarek
An update frrom my second machine, this time with 4GB of RAM. Compiling Agda ran out of memory
(again Agda.TypeChecking.Serialise module) and I had to kill the build. But once I restarted the
build the module was compiled succesfully in a matter of minutes and using around 50% of memory.
This looks like some kind of memory leak in GHC.

Janek

Dnia środa, 1 kwietnia 2015, Jan Stolarek napisał:

> Forall hi,
>
> I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed
> installations of GHC and this means I had to rebuild Agda and Idris because
> the binaries built with GHC 7.8.4 were stored inside deactivated 7.8.4
> sandbox. Sadly, I had problems building both Agda and Idris due to GHC
> taking up all of available memory.
>
> With Idris the problematic module was Idris.ElabTerm (~2900LOC). The
> interesting part of the story is that when I do a clean build of Idris GHC
> consumes all of memory when compiling that module and I have to kill the
> build. But when I restart the build after killing GHC the module is
> compiled using a reasonable amount of memory and within reasonable time.
>
> With Agda the problematic module is Agda.TypeChecking.Serialise (~2000LOC).
> The trick with killing the build and restarting it didn't work in this
> case. I had to compile Agda with GHC 7.8.4 (which works without problems
> though the mentioned module still requires a lot of memory) and alter my
> setup so that Agda binary is not stored inside GHC sandbox.
>
> I wonder if any of you came across similar issues with GHC 7.10.1? Do we
> have any performance data that allows to compare memory usage and
> performance of GHC 7.10.1 with previous stable releases?
>
> All of the above happened on 64bit Debian Wheezy with 2GB of RAM.
>
> Janek
>
> ---
> Politechnika Łódzka
> Lodz University of Technology
>
> Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
> pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
>
> This email contains information intended solely for the use of the
> individual to whom it is addressed. If you are not the intended recipient
> or if you have received this message in error, please notify the sender and
> delete it from your system.
> _______________________________________________
> Glasgow-haskell-users mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users



---
Politechnika Łódzka
Lodz University of Technology

Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę
prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.

This email contains information intended solely for the use of the individual to whom it is addressed.
If you are not the intended recipient or if you have received this message in error,
please notify the sender and delete it from your system.
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Richard Eisenberg-2
Post a bug report! :)

On Apr 2, 2015, at 8:19 AM, Jan Stolarek <[hidden email]> wrote:

> An update frrom my second machine, this time with 4GB of RAM. Compiling Agda ran out of memory
> (again Agda.TypeChecking.Serialise module) and I had to kill the build. But once I restarted the
> build the module was compiled succesfully in a matter of minutes and using around 50% of memory.
> This looks like some kind of memory leak in GHC.
>
> Janek
>
> Dnia środa, 1 kwietnia 2015, Jan Stolarek napisał:
>> Forall hi,
>>
>> I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed
>> installations of GHC and this means I had to rebuild Agda and Idris because
>> the binaries built with GHC 7.8.4 were stored inside deactivated 7.8.4
>> sandbox. Sadly, I had problems building both Agda and Idris due to GHC
>> taking up all of available memory.
>>
>> With Idris the problematic module was Idris.ElabTerm (~2900LOC). The
>> interesting part of the story is that when I do a clean build of Idris GHC
>> consumes all of memory when compiling that module and I have to kill the
>> build. But when I restart the build after killing GHC the module is
>> compiled using a reasonable amount of memory and within reasonable time.
>>
>> With Agda the problematic module is Agda.TypeChecking.Serialise (~2000LOC).
>> The trick with killing the build and restarting it didn't work in this
>> case. I had to compile Agda with GHC 7.8.4 (which works without problems
>> though the mentioned module still requires a lot of memory) and alter my
>> setup so that Agda binary is not stored inside GHC sandbox.
>>
>> I wonder if any of you came across similar issues with GHC 7.10.1? Do we
>> have any performance data that allows to compare memory usage and
>> performance of GHC 7.10.1 with previous stable releases?
>>
>> All of the above happened on 64bit Debian Wheezy with 2GB of RAM.
>>
>> Janek
>>
>> ---
>> Politechnika Łódzka
>> Lodz University of Technology
>>
>> Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
>> Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
>> pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
>>
>> This email contains information intended solely for the use of the
>> individual to whom it is addressed. If you are not the intended recipient
>> or if you have received this message in error, please notify the sender and
>> delete it from your system.
>> _______________________________________________
>> Glasgow-haskell-users mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
>
>
>
> ---
> Politechnika Łódzka
> Lodz University of Technology
>
> Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę
> prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
>
> This email contains information intended solely for the use of the individual to whom it is addressed.
> If you are not the intended recipient or if you have received this message in error,
> please notify the sender and delete it from your system.
> _______________________________________________
> Glasgow-haskell-users mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
>

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Jan Stolarek
I will. But I was curious whether this is only me or is anyone else seeing similar behaviour. And
what about performance comparisson between 7.8.4 and 7.10.1? Do we have any numbers?

Janek

Dnia czwartek, 2 kwietnia 2015, Richard Eisenberg napisał:

> Post a bug report! :)
>
> On Apr 2, 2015, at 8:19 AM, Jan Stolarek <[hidden email]> wrote:
> > An update frrom my second machine, this time with 4GB of RAM. Compiling
> > Agda ran out of memory (again Agda.TypeChecking.Serialise module) and I
> > had to kill the build. But once I restarted the build the module was
> > compiled succesfully in a matter of minutes and using around 50% of
> > memory. This looks like some kind of memory leak in GHC.
> >
> > Janek
> >
> > Dnia środa, 1 kwietnia 2015, Jan Stolarek napisał:
> >> Forall hi,
> >>
> >> I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed
> >> installations of GHC and this means I had to rebuild Agda and Idris
> >> because the binaries built with GHC 7.8.4 were stored inside deactivated
> >> 7.8.4 sandbox. Sadly, I had problems building both Agda and Idris due to
> >> GHC taking up all of available memory.
> >>
> >> With Idris the problematic module was Idris.ElabTerm (~2900LOC). The
> >> interesting part of the story is that when I do a clean build of Idris
> >> GHC consumes all of memory when compiling that module and I have to kill
> >> the build. But when I restart the build after killing GHC the module is
> >> compiled using a reasonable amount of memory and within reasonable time.
> >>
> >> With Agda the problematic module is Agda.TypeChecking.Serialise
> >> (~2000LOC). The trick with killing the build and restarting it didn't
> >> work in this case. I had to compile Agda with GHC 7.8.4 (which works
> >> without problems though the mentioned module still requires a lot of
> >> memory) and alter my setup so that Agda binary is not stored inside GHC
> >> sandbox.
> >>
> >> I wonder if any of you came across similar issues with GHC 7.10.1? Do we
> >> have any performance data that allows to compare memory usage and
> >> performance of GHC 7.10.1 with previous stable releases?
> >>
> >> All of the above happened on 64bit Debian Wheezy with 2GB of RAM.
> >>
> >> Janek
> >>
> >> ---
> >> Politechnika Łódzka
> >> Lodz University of Technology
> >>
> >> Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> >> Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
> >> pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
> >>
> >> This email contains information intended solely for the use of the
> >> individual to whom it is addressed. If you are not the intended
> >> recipient or if you have received this message in error, please notify
> >> the sender and delete it from your system.
> >> _______________________________________________
> >> Glasgow-haskell-users mailing list
> >> [hidden email]
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
> >
> > ---
> > Politechnika Łódzka
> > Lodz University of Technology
> >
> > Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> > Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
> > pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
> >
> > This email contains information intended solely for the use of the
> > individual to whom it is addressed. If you are not the intended recipient
> > or if you have received this message in error, please notify the sender
> > and delete it from your system.
> > _______________________________________________
> > Glasgow-haskell-users mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users



---
Politechnika Łódzka
Lodz University of Technology

Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę
prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.

This email contains information intended solely for the use of the individual to whom it is addressed.
If you are not the intended recipient or if you have received this message in error,
please notify the sender and delete it from your system.
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Dan Doel
You aren't the only one. The vector test suite also has these kind of issues. In its case, it's hard for me to tell how big the code is, because template haskell is being used to generate it. However, I don't think the template haskell is what's using the additional performance, because the test suite is compiled with both -O0 and -O2, and only the latter runs into performance issues.

In vector's case, 7.6.3 compiles significantly faster than 7.8.4, which is in turn significantly faster than 7.10.1. And memory usage follows the same pattern (with -O2 at least). 7.6.3 uses ~400MB of memory, 7.8.4 1-2GB and 7.10.1 3-4GB (if I'm remembering correctly). And while I can build the tests on 7.10 in around 5 minutes, travis times out building them after around half an hour.

On Thu, Apr 2, 2015 at 8:47 AM, Jan Stolarek <[hidden email]> wrote:
I will. But I was curious whether this is only me or is anyone else seeing similar behaviour. And
what about performance comparisson between 7.8.4 and 7.10.1? Do we have any numbers?

Janek

Dnia czwartek, 2 kwietnia 2015, Richard Eisenberg napisał:
> Post a bug report! :)
>
> On Apr 2, 2015, at 8:19 AM, Jan Stolarek <[hidden email]> wrote:
> > An update frrom my second machine, this time with 4GB of RAM. Compiling
> > Agda ran out of memory (again Agda.TypeChecking.Serialise module) and I
> > had to kill the build. But once I restarted the build the module was
> > compiled succesfully in a matter of minutes and using around 50% of
> > memory. This looks like some kind of memory leak in GHC.
> >
> > Janek
> >
> > Dnia środa, 1 kwietnia 2015, Jan Stolarek napisał:
> >> Forall hi,
> >>
> >> I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed
> >> installations of GHC and this means I had to rebuild Agda and Idris
> >> because the binaries built with GHC 7.8.4 were stored inside deactivated
> >> 7.8.4 sandbox. Sadly, I had problems building both Agda and Idris due to
> >> GHC taking up all of available memory.
> >>
> >> With Idris the problematic module was Idris.ElabTerm (~2900LOC). The
> >> interesting part of the story is that when I do a clean build of Idris
> >> GHC consumes all of memory when compiling that module and I have to kill
> >> the build. But when I restart the build after killing GHC the module is
> >> compiled using a reasonable amount of memory and within reasonable time.
> >>
> >> With Agda the problematic module is Agda.TypeChecking.Serialise
> >> (~2000LOC). The trick with killing the build and restarting it didn't
> >> work in this case. I had to compile Agda with GHC 7.8.4 (which works
> >> without problems though the mentioned module still requires a lot of
> >> memory) and alter my setup so that Agda binary is not stored inside GHC
> >> sandbox.
> >>
> >> I wonder if any of you came across similar issues with GHC 7.10.1? Do we
> >> have any performance data that allows to compare memory usage and
> >> performance of GHC 7.10.1 with previous stable releases?
> >>
> >> All of the above happened on 64bit Debian Wheezy with 2GB of RAM.
> >>
> >> Janek
> >>
> >> ---
> >> Politechnika Łódzka
> >> Lodz University of Technology
> >>
> >> Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> >> Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
> >> pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
> >>
> >> This email contains information intended solely for the use of the
> >> individual to whom it is addressed. If you are not the intended
> >> recipient or if you have received this message in error, please notify
> >> the sender and delete it from your system.
> >> _______________________________________________
> >> Glasgow-haskell-users mailing list
> >> [hidden email]
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
> >
> > ---
> > Politechnika Łódzka
> > Lodz University of Technology
> >
> > Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> > Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
> > pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
> >
> > This email contains information intended solely for the use of the
> > individual to whom it is addressed. If you are not the intended recipient
> > or if you have received this message in error, please notify the sender
> > and delete it from your system.
> > _______________________________________________
> > Glasgow-haskell-users mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users



---
Politechnika Łódzka
Lodz University of Technology

Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę
prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.

This email contains information intended solely for the use of the individual to whom it is addressed.
If you are not the intended recipient or if you have received this message in error,
please notify the sender and delete it from your system.
_______________________________________________


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

George Colpitts
In reply to this post by Jan Stolarek
I'm curious why the amount of RAM is relevant as all of our OS have virtual memory so it is only the size of the heap and the amount of swap that should be relevant for an Out Of Memory error, right? How big is your heap? Amount of RAM should only affect speed (i.e. if there is excessive paging) but should not affect Out Of Memory right?

On Thu, Apr 2, 2015 at 9:47 AM, Jan Stolarek <[hidden email]> wrote:
I will. But I was curious whether this is only me or is anyone else seeing similar behaviour. And
what about performance comparisson between 7.8.4 and 7.10.1? Do we have any numbers?

Janek

Dnia czwartek, 2 kwietnia 2015, Richard Eisenberg napisał:
> Post a bug report! :)
>
> On Apr 2, 2015, at 8:19 AM, Jan Stolarek <[hidden email]> wrote:
> > An update frrom my second machine, this time with 4GB of RAM. Compiling
> > Agda ran out of memory (again Agda.TypeChecking.Serialise module) and I
> > had to kill the build. But once I restarted the build the module was
> > compiled succesfully in a matter of minutes and using around 50% of
> > memory. This looks like some kind of memory leak in GHC.
> >
> > Janek
> >
> > Dnia środa, 1 kwietnia 2015, Jan Stolarek napisał:
> >> Forall hi,
> >>
> >> I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed
> >> installations of GHC and this means I had to rebuild Agda and Idris
> >> because the binaries built with GHC 7.8.4 were stored inside deactivated
> >> 7.8.4 sandbox. Sadly, I had problems building both Agda and Idris due to
> >> GHC taking up all of available memory.
> >>
> >> With Idris the problematic module was Idris.ElabTerm (~2900LOC). The
> >> interesting part of the story is that when I do a clean build of Idris
> >> GHC consumes all of memory when compiling that module and I have to kill
> >> the build. But when I restart the build after killing GHC the module is
> >> compiled using a reasonable amount of memory and within reasonable time.
> >>
> >> With Agda the problematic module is Agda.TypeChecking.Serialise
> >> (~2000LOC). The trick with killing the build and restarting it didn't
> >> work in this case. I had to compile Agda with GHC 7.8.4 (which works
> >> without problems though the mentioned module still requires a lot of
> >> memory) and alter my setup so that Agda binary is not stored inside GHC
> >> sandbox.
> >>
> >> I wonder if any of you came across similar issues with GHC 7.10.1? Do we
> >> have any performance data that allows to compare memory usage and
> >> performance of GHC 7.10.1 with previous stable releases?
> >>
> >> All of the above happened on 64bit Debian Wheezy with 2GB of RAM.
> >>
> >> Janek
> >>
> >> ---
> >> Politechnika Łódzka
> >> Lodz University of Technology
> >>
> >> Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> >> Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
> >> pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
> >>
> >> This email contains information intended solely for the use of the
> >> individual to whom it is addressed. If you are not the intended
> >> recipient or if you have received this message in error, please notify
> >> the sender and delete it from your system.
> >> _______________________________________________
> >> Glasgow-haskell-users mailing list
> >> [hidden email]
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
> >
> > ---
> > Politechnika Łódzka
> > Lodz University of Technology
> >
> > Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> > Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
> > pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
> >
> > This email contains information intended solely for the use of the
> > individual to whom it is addressed. If you are not the intended recipient
> > or if you have received this message in error, please notify the sender
> > and delete it from your system.
> > _______________________________________________
> > Glasgow-haskell-users mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users



---
Politechnika Łódzka
Lodz University of Technology

Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę
prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.

This email contains information intended solely for the use of the individual to whom it is addressed.
If you are not the intended recipient or if you have received this message in error,
please notify the sender and delete it from your system.
_______________________________________________


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Jan Stolarek
> I'm curious why the amount of RAM is relevant as all of our OS have virtual
> memory so it is only the size of the heap and the amount of swap that
> should be relevant for an Out Of Memory error, right? How big is your heap?
> Amount of RAM should only affect speed (i.e. if there is excessive paging)
> but should not affect Out Of Memory right?
Actually, I never allowed GHC to completely run out of memory. I just killed the build process
when it consumed all of physical RAM memory and started growing into swap. At that point machine
becomes barely usable and the build practically stalls as CPU usage drops to 0. FYI: both
machines have size of swap equal to size of RAM.

Janek


---
Politechnika Łódzka
Lodz University of Technology

Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę
prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.

This email contains information intended solely for the use of the individual to whom it is addressed.
If you are not the intended recipient or if you have received this message in error,
please notify the sender and delete it from your system.
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Michael Jones
In reply to this post by George Colpitts
I have run out of memory before when compiling on small machines using GHC 7.8, where small machines have 4GB RAM, no swap, say small Dual Core Atom, almost embedded design. That forced me to compile on a laptop and mount file systems to run it. But since Ubuntu runs well on a NUC, it is nice to be able to run the compiler on it, even if a bit slow.


On Apr 2, 2015, at 9:08 AM, George Colpitts <[hidden email]> wrote:

I'm curious why the amount of RAM is relevant as all of our OS have virtual memory so it is only the size of the heap and the amount of swap that should be relevant for an Out Of Memory error, right? How big is your heap? Amount of RAM should only affect speed (i.e. if there is excessive paging) but should not affect Out Of Memory right?

On Thu, Apr 2, 2015 at 9:47 AM, Jan Stolarek <[hidden email]> wrote:
I will. But I was curious whether this is only me or is anyone else seeing similar behaviour. And
what about performance comparisson between 7.8.4 and 7.10.1? Do we have any numbers?

Janek

Dnia czwartek, 2 kwietnia 2015, Richard Eisenberg napisał:

> Post a bug report! :)
>
> On Apr 2, 2015, at 8:19 AM, Jan Stolarek <[hidden email]> wrote:
> > An update frrom my second machine, this time with 4GB of RAM. Compiling
> > Agda ran out of memory (again Agda.TypeChecking.Serialise module) and I
> > had to kill the build. But once I restarted the build the module was
> > compiled succesfully in a matter of minutes and using around 50% of
> > memory. This looks like some kind of memory leak in GHC.
> >
> > Janek
> >
> > Dnia środa, 1 kwietnia 2015, Jan Stolarek napisał:
> >> Forall hi,
> >>
> >> I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed
> >> installations of GHC and this means I had to rebuild Agda and Idris
> >> because the binaries built with GHC 7.8.4 were stored inside deactivated
> >> 7.8.4 sandbox. Sadly, I had problems building both Agda and Idris due to
> >> GHC taking up all of available memory.
> >>
> >> With Idris the problematic module was Idris.ElabTerm (~2900LOC). The
> >> interesting part of the story is that when I do a clean build of Idris
> >> GHC consumes all of memory when compiling that module and I have to kill
> >> the build. But when I restart the build after killing GHC the module is
> >> compiled using a reasonable amount of memory and within reasonable time.
> >>
> >> With Agda the problematic module is Agda.TypeChecking.Serialise
> >> (~2000LOC). The trick with killing the build and restarting it didn't
> >> work in this case. I had to compile Agda with GHC 7.8.4 (which works
> >> without problems though the mentioned module still requires a lot of
> >> memory) and alter my setup so that Agda binary is not stored inside GHC
> >> sandbox.
> >>
> >> I wonder if any of you came across similar issues with GHC 7.10.1? Do we
> >> have any performance data that allows to compare memory usage and
> >> performance of GHC 7.10.1 with previous stable releases?
> >>
> >> All of the above happened on 64bit Debian Wheezy with 2GB of RAM.
> >>
> >> Janek
> >>
> >> ---
> >> Politechnika Łódzka
> >> Lodz University of Technology
> >>
> >> Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> >> Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
> >> pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
> >>
> >> This email contains information intended solely for the use of the
> >> individual to whom it is addressed. If you are not the intended
> >> recipient or if you have received this message in error, please notify
> >> the sender and delete it from your system.
> >> _______________________________________________
> >> Glasgow-haskell-users mailing list
> >> [hidden email]
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
> >
> > ---
> > Politechnika Łódzka
> > Lodz University of Technology
> >
> > Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> > Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
> > pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
> >
> > This email contains information intended solely for the use of the
> > individual to whom it is addressed. If you are not the intended recipient
> > or if you have received this message in error, please notify the sender
> > and delete it from your system.
> > _______________________________________________
> > Glasgow-haskell-users mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users



---
Politechnika Łódzka
Lodz University of Technology

Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę
prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.

This email contains information intended solely for the use of the individual to whom it is addressed.
If you are not the intended recipient or if you have received this message in error,
please notify the sender and delete it from your system.
_______________________________________________

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Andrés Sicard-Ramírez-2
In reply to this post by Jan Stolarek
In a 64-bit machine with Ubuntu 12.04 and 4 GB of RAM, I can compile
Agda using GHC 7.10.1 without problems. Agda's compilation time
increased ~26% with respect to GHC 7.8.4.


On 2 April 2015 at 07:19, Jan Stolarek <[hidden email]> wrote:

> An update frrom my second machine, this time with 4GB of RAM. Compiling Agda ran out of memory
> (again Agda.TypeChecking.Serialise module) and I had to kill the build. But once I restarted the
> build the module was compiled succesfully in a matter of minutes and using around 50% of memory.
> This looks like some kind of memory leak in GHC.
>
> Janek
>
> Dnia środa, 1 kwietnia 2015, Jan Stolarek napisał:
>> Forall hi,
>>
>> I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed
>> installations of GHC and this means I had to rebuild Agda and Idris because
>> the binaries built with GHC 7.8.4 were stored inside deactivated 7.8.4
>> sandbox. Sadly, I had problems building both Agda and Idris due to GHC
>> taking up all of available memory.
>>
>> With Idris the problematic module was Idris.ElabTerm (~2900LOC). The
>> interesting part of the story is that when I do a clean build of Idris GHC
>> consumes all of memory when compiling that module and I have to kill the
>> build. But when I restart the build after killing GHC the module is
>> compiled using a reasonable amount of memory and within reasonable time.
>>
>> With Agda the problematic module is Agda.TypeChecking.Serialise (~2000LOC).
>> The trick with killing the build and restarting it didn't work in this
>> case. I had to compile Agda with GHC 7.8.4 (which works without problems
>> though the mentioned module still requires a lot of memory) and alter my
>> setup so that Agda binary is not stored inside GHC sandbox.
>>
>> I wonder if any of you came across similar issues with GHC 7.10.1? Do we
>> have any performance data that allows to compare memory usage and
>> performance of GHC 7.10.1 with previous stable releases?
>>
>> All of the above happened on 64bit Debian Wheezy with 2GB of RAM.
>>
>> Janek
>>
>> ---
>> Politechnika Łódzka
>> Lodz University of Technology
>>
>> Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
>> Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez
>> pomyłkę prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
>>
>> This email contains information intended solely for the use of the
>> individual to whom it is addressed. If you are not the intended recipient
>> or if you have received this message in error, please notify the sender and
>> delete it from your system.
>> _______________________________________________
>> Glasgow-haskell-users mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
>
>
>
> ---
> Politechnika Łódzka
> Lodz University of Technology
>
> Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
> Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę
> prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
>
> This email contains information intended solely for the use of the individual to whom it is addressed.
> If you are not the intended recipient or if you have received this message in error,
> please notify the sender and delete it from your system.
> _______________________________________________
> Glasgow-haskell-users mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users



--
Andrés
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Bertram Felgenhauer-2
In reply to this post by George Colpitts
George Colpitts wrote:
> I'm curious why the amount of RAM is relevant as all of our OS have virtual
> memory so it is only the size of the heap and the amount of swap that
> should be relevant for an Out Of Memory error, right?

The computer may not be your own. VPSs are essentially priced based on
RAM available to the virtual server, and have limited swapping space,
so this is an area where increased memory consumption hurts. Building
binaries elsewhere is usually an option, but more painful than doing
it on the VPS itself.

Cheers,

Bertram
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

David Feuer
On a machine with an SSD instead of a hard disk, swapping greatly reduces the lifespan of the storage device.

On Fri, Apr 3, 2015 at 10:14 AM, Bertram Felgenhauer <[hidden email]> wrote:
George Colpitts wrote:
> I'm curious why the amount of RAM is relevant as all of our OS have virtual
> memory so it is only the size of the heap and the amount of swap that
> should be relevant for an Out Of Memory error, right?

The computer may not be your own. VPSs are essentially priced based on
RAM available to the virtual server, and have limited swapping space,
so this is an area where increased memory consumption hurts. Building
binaries elsewhere is usually an option, but more painful than doing
it on the VPS itself.

Cheers,

Bertram
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Michal Terepeta
So I've tried to compile Idris/Agda with prof compilers but this
didn't quite work out due to deps not compiling (apparently it's not
possible to use template haskell with a profiled compiler).

Out of curiosity I had a look at compiling haskell-src-exts since that
takes quite a while. I've used ghc HEAD and 7.8.4 (both built with
BuildFlavour=prof & bootstrapped with a standard ghc 7.8.4) and it's
interesting -- the current HEAD takes quite a bit longer and allocates
way more than 7.8.4. One of the main things that stand out is the
CallArity analysis (which IIRC was not there in 7.8.4). So unless I
messed something up with measuring, the analysis seem to be
pretty expensive.

Anyway, the results are below.

Cheers,
Michal


** HEAD

Sun Apr 12 15:52 2015 Time and Allocation Profiling Report  (Final)

  ghc +RTS -p -RTS [...]

total time  =      147.84 secs   (147841 ticks @ 1000 us, 1 processor)
total alloc = 172,378,600,408 bytes  (excludes profiling overheads)

COST CENTRE       MODULE       %time %alloc

SimplTopBinds     SimplCore     32.4   28.8
CallArity         SimplCore     18.4   25.6
lintAnnots        CoreLint       4.5    4.6
CoreTidy          HscMain        4.5    5.1
pprNativeCode     AsmCodeGen     3.2    3.4
OccAnal           SimplCore      3.2    3.1
occAnalBind.assoc OccurAnal      2.6    2.5
StgCmm            HscMain        2.3    1.9
Simplify          SimplCore      2.1    0.2
RegAlloc          AsmCodeGen     2.1    2.4
FloatOutwards     SimplCore      2.0    1.6
regLiveness       AsmCodeGen     1.9    1.9
tc_rn_src_decls   TcRnDriver     1.8    1.3
sink              CmmPipeline    1.7    1.5
NewStranal        SimplCore      1.3    1.5
genMachCode       AsmCodeGen     1.1    1.0
layoutStack       CmmPipeline    1.0    1.0


** HEAD with -fno-call-arity

Sun Apr 12 18:16 2015 Time and Allocation Profiling Report  (Final)

  ghc +RTS -p -RTS [...] -fno-call-arity

total time  =      113.71 secs   (113714 ticks @ 1000 us, 1 processor)
total alloc = 121,884,896,720 bytes  (excludes profiling overheads)

COST CENTRE       MODULE          %time %alloc

SimplTopBinds     SimplCore        37.2   36.6
CoreTidy          HscMain           6.0    7.3
lintAnnots        CoreLint          5.8    6.5
pprNativeCode     AsmCodeGen        4.1    4.8
OccAnal           SimplCore         3.6    3.8
occAnalBind.assoc OccurAnal         2.9    3.2
StgCmm            HscMain           2.9    2.6
RegAlloc          AsmCodeGen        2.6    3.4
FloatOutwards     SimplCore         2.6    2.3
regLiveness       AsmCodeGen        2.5    2.8
tc_rn_src_decls   TcRnDriver        2.4    1.9
Simplify          SimplCore         2.4    0.3
sink              CmmPipeline       2.1    2.2
NewStranal        SimplCore         1.7    2.1
genMachCode       AsmCodeGen        1.4    1.4
layoutStack       CmmPipeline       1.4    1.4
NativeCodeGen     CodeOutput        1.1    1.2
FloatInwards      SimplCore         1.1    1.4
do_block          Hoopl.Dataflow    1.0    0.6
Digraph.scc       Digraph           0.8    1.3


** 7.8.4

Sun Apr 12 15:41 2015 Time and Allocation Profiling Report  (Final)

  ghc +RTS -p -RTS [...]

total time  =       93.11 secs   (93112 ticks @ 1000 us, 1 processor)
total alloc = 103,135,975,120 bytes  (excludes profiling overheads)

COST CENTRE                 MODULE         %time %alloc

SimplTopBinds               SimplCore       38.5   37.4
pprNativeCode               AsmCodeGen       6.2    7.2
StgCmm                      HscMain          3.9    4.2
RegAlloc                    AsmCodeGen       3.7    5.1
occAnalBind.assoc           OccurAnal        3.3    3.6
OccAnal                     SimplCore        3.3    3.6
regLiveness                 AsmCodeGen       3.1    3.4
FloatOutwards               SimplCore        2.9    2.4
sink                        CmmPipeline      2.8    2.8
Simplify                    SimplCore        2.6    0.3
tc_rn_src_decls             TcRnDriver       2.4    2.1
genMachCode                 AsmCodeGen       1.9    2.0
NewStranal                  SimplCore        1.8    2.1
layoutStack                 CmmPipeline      1.8    1.8
Core2Core                   HscMain          1.3    1.2
deSugar                     HscMain          1.1    1.1
do_block                    Hoopl.Dataflow   1.1    0.7
CoreTidy                    HscMain          1.0    1.1
CorePrep                    HscMain          1.0    1.1
Digraph.scc                 Digraph          0.9    1.5
versioninfo                 MkIface          0.9    1.0
zonkEvBndr_zonkTcTypeToType TcHsSyn          0.6    1.4


On Fri, Apr 3, 2015 at 4:49 PM David Feuer <[hidden email]> wrote:
On a machine with an SSD instead of a hard disk, swapping greatly reduces the lifespan of the storage device.

On Fri, Apr 3, 2015 at 10:14 AM, Bertram Felgenhauer <[hidden email]> wrote:
George Colpitts wrote:
> I'm curious why the amount of RAM is relevant as all of our OS have virtual
> memory so it is only the size of the heap and the amount of swap that
> should be relevant for an Out Of Memory error, right?

The computer may not be your own. VPSs are essentially priced based on
RAM available to the virtual server, and have limited swapping space,
so this is an area where increased memory consumption hurts. Building
binaries elsewhere is usually an option, but more painful than doing
it on the VPS itself.

Cheers,

Bertram
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Christiaan Baaij
Hi,

I wonder if this might be in any way related to the HUGE amount of new terms/types/coercions created by the specialiser as documented in:

I don’t have a profiled version of GHC, so I was wondering if you could run your tests with a ‘-fno-specialise’, and see how everything performs then?

Cheers,

Christiaan

On 12 Apr 2015, at 18:28, Michal Terepeta <[hidden email]> wrote:

So I've tried to compile Idris/Agda with prof compilers but this
didn't quite work out due to deps not compiling (apparently it's not
possible to use template haskell with a profiled compiler).

Out of curiosity I had a look at compiling haskell-src-exts since that
takes quite a while. I've used ghc HEAD and 7.8.4 (both built with
BuildFlavour=prof & bootstrapped with a standard ghc 7.8.4) and it's
interesting -- the current HEAD takes quite a bit longer and allocates
way more than 7.8.4. One of the main things that stand out is the
CallArity analysis (which IIRC was not there in 7.8.4). So unless I
messed something up with measuring, the analysis seem to be
pretty expensive.

Anyway, the results are below.

Cheers,
Michal


** HEAD

Sun Apr 12 15:52 2015 Time and Allocation Profiling Report  (Final)

  ghc +RTS -p -RTS [...]

total time  =      147.84 secs   (147841 ticks @ 1000 us, 1 processor)
total alloc = 172,378,600,408 bytes  (excludes profiling overheads)

COST CENTRE       MODULE       %time %alloc

SimplTopBinds     SimplCore     32.4   28.8
CallArity         SimplCore     18.4   25.6
lintAnnots        CoreLint       4.5    4.6
CoreTidy          HscMain        4.5    5.1
pprNativeCode     AsmCodeGen     3.2    3.4
OccAnal           SimplCore      3.2    3.1
occAnalBind.assoc OccurAnal      2.6    2.5
StgCmm            HscMain        2.3    1.9
Simplify          SimplCore      2.1    0.2
RegAlloc          AsmCodeGen     2.1    2.4
FloatOutwards     SimplCore      2.0    1.6
regLiveness       AsmCodeGen     1.9    1.9
tc_rn_src_decls   TcRnDriver     1.8    1.3
sink              CmmPipeline    1.7    1.5
NewStranal        SimplCore      1.3    1.5
genMachCode       AsmCodeGen     1.1    1.0
layoutStack       CmmPipeline    1.0    1.0


** HEAD with -fno-call-arity

Sun Apr 12 18:16 2015 Time and Allocation Profiling Report  (Final)

  ghc +RTS -p -RTS [...] -fno-call-arity

total time  =      113.71 secs   (113714 ticks @ 1000 us, 1 processor)
total alloc = 121,884,896,720 bytes  (excludes profiling overheads)

COST CENTRE       MODULE          %time %alloc

SimplTopBinds     SimplCore        37.2   36.6
CoreTidy          HscMain           6.0    7.3
lintAnnots        CoreLint          5.8    6.5
pprNativeCode     AsmCodeGen        4.1    4.8
OccAnal           SimplCore         3.6    3.8
occAnalBind.assoc OccurAnal         2.9    3.2
StgCmm            HscMain           2.9    2.6
RegAlloc          AsmCodeGen        2.6    3.4
FloatOutwards     SimplCore         2.6    2.3
regLiveness       AsmCodeGen        2.5    2.8
tc_rn_src_decls   TcRnDriver        2.4    1.9
Simplify          SimplCore         2.4    0.3
sink              CmmPipeline       2.1    2.2
NewStranal        SimplCore         1.7    2.1
genMachCode       AsmCodeGen        1.4    1.4
layoutStack       CmmPipeline       1.4    1.4
NativeCodeGen     CodeOutput        1.1    1.2
FloatInwards      SimplCore         1.1    1.4
do_block          Hoopl.Dataflow    1.0    0.6
Digraph.scc       Digraph           0.8    1.3


** 7.8.4

Sun Apr 12 15:41 2015 Time and Allocation Profiling Report  (Final)

  ghc +RTS -p -RTS [...]

total time  =       93.11 secs   (93112 ticks @ 1000 us, 1 processor)
total alloc = 103,135,975,120 bytes  (excludes profiling overheads)

COST CENTRE                 MODULE         %time %alloc

SimplTopBinds               SimplCore       38.5   37.4
pprNativeCode               AsmCodeGen       6.2    7.2
StgCmm                      HscMain          3.9    4.2
RegAlloc                    AsmCodeGen       3.7    5.1
occAnalBind.assoc           OccurAnal        3.3    3.6
OccAnal                     SimplCore        3.3    3.6
regLiveness                 AsmCodeGen       3.1    3.4
FloatOutwards               SimplCore        2.9    2.4
sink                        CmmPipeline      2.8    2.8
Simplify                    SimplCore        2.6    0.3
tc_rn_src_decls             TcRnDriver       2.4    2.1
genMachCode                 AsmCodeGen       1.9    2.0
NewStranal                  SimplCore        1.8    2.1
layoutStack                 CmmPipeline      1.8    1.8
Core2Core                   HscMain          1.3    1.2
deSugar                     HscMain          1.1    1.1
do_block                    Hoopl.Dataflow   1.1    0.7
CoreTidy                    HscMain          1.0    1.1
CorePrep                    HscMain          1.0    1.1
Digraph.scc                 Digraph          0.9    1.5
versioninfo                 MkIface          0.9    1.0
zonkEvBndr_zonkTcTypeToType TcHsSyn          0.6    1.4


On Fri, Apr 3, 2015 at 4:49 PM David Feuer <[hidden email]> wrote:
On a machine with an SSD instead of a hard disk, swapping greatly reduces the lifespan of the storage device.

On Fri, Apr 3, 2015 at 10:14 AM, Bertram Felgenhauer <[hidden email]> wrote:
George Colpitts wrote:
> I'm curious why the amount of RAM is relevant as all of our OS have virtual
> memory so it is only the size of the heap and the amount of swap that
> should be relevant for an Out Of Memory error, right?

The computer may not be your own. VPSs are essentially priced based on
RAM available to the virtual server, and have limited swapping space,
so this is an area where increased memory consumption hurts. Building
binaries elsewhere is usually an option, but more painful than doing
it on the VPS itself.

Cheers,

Bertram
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Michal Terepeta
On Mon, Apr 13, 2015 at 12:20 PM, Christiaan Baaij
<[hidden email]> wrote:

>
> Hi,
>
> I wonder if this might be in any way related to the HUGE amount of new terms/types/coercions created by the specialiser as documented in:
> https://ghc.haskell.org/trac/ghc/ticket/9630#comment:10
>
> I don’t have a profiled version of GHC, so I was wondering if you could run your tests with a ‘-fno-specialise’, and see how everything performs then?
>
> Cheers,
>
> Christiaan

Unfortunately trac is timing out for me, so I'll have a look at the
issues later...

As for compiling haskell-src-exts with both -fno-specialise and -fno-call-arity,
we're pretty much at the same level as GHC 7.8.4.



Mon Apr 13 20:25 2015 Time and Allocation Profiling Report  (Final)

  ghc +RTS -p -RTS  [...] -fno-specialise -fno-call-arity

total time  =       89.93 secs   (89928 ticks @ 1000 us, 1 processor)
total alloc = 93,495,685,792 bytes  (excludes profiling overheads)

COST CENTRE       MODULE          %time %alloc

SimplTopBinds     SimplCore        38.7   38.6
pprNativeCode     AsmCodeGen        5.1    5.9
StgCmm            HscMain           3.7    3.3
occAnalBind.assoc OccurAnal         3.2    3.6
OccAnal           SimplCore         3.2    3.6
tc_rn_src_decls   TcRnDriver        3.1    2.5
RegAlloc          AsmCodeGen        3.1    4.2
regLiveness       AsmCodeGen        3.0    3.5
FloatOutwards     SimplCore         2.7    2.4
sink              CmmPipeline       2.6    2.7
Simplify          SimplCore         2.5    0.1
NewStranal        SimplCore         1.9    2.3
genMachCode       AsmCodeGen        1.8    1.7
layoutStack       CmmPipeline       1.6    1.7
NativeCodeGen     CodeOutput        1.4    1.5
FloatInwards      SimplCore         1.2    1.6
CoreTidy          HscMain           1.2    1.2
deSugar           HscMain           1.2    1.2
do_block          Hoopl.Dataflow    1.1    0.7
CorePrep          HscMain           1.0    1.1
versioninfo       MkIface           0.9    1.0
Parser            HscMain           0.9    1.2
Digraph.scc       Digraph           0.9    1.5
canEvVar          TcCanonical       0.7    1.0
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Christiaan Baaij
Actually, I meant only with -fno-specialise.

On 13 April 2015 at 21:09, Michal Terepeta <[hidden email]> wrote:
On Mon, Apr 13, 2015 at 12:20 PM, Christiaan Baaij
<[hidden email]> wrote:
>
> Hi,
>
> I wonder if this might be in any way related to the HUGE amount of new terms/types/coercions created by the specialiser as documented in:
> https://ghc.haskell.org/trac/ghc/ticket/9630#comment:10
>
> I don’t have a profiled version of GHC, so I was wondering if you could run your tests with a ‘-fno-specialise’, and see how everything performs then?
>
> Cheers,
>
> Christiaan

Unfortunately trac is timing out for me, so I'll have a look at the
issues later...

As for compiling haskell-src-exts with both -fno-specialise and -fno-call-arity,
we're pretty much at the same level as GHC 7.8.4.



Mon Apr 13 20:25 2015 Time and Allocation Profiling Report  (Final)

  ghc +RTS -p -RTS  [...] -fno-specialise -fno-call-arity

total time  =       89.93 secs   (89928 ticks @ 1000 us, 1 processor)
total alloc = 93,495,685,792 bytes  (excludes profiling overheads)

COST CENTRE       MODULE          %time %alloc

SimplTopBinds     SimplCore        38.7   38.6
pprNativeCode     AsmCodeGen        5.1    5.9
StgCmm            HscMain           3.7    3.3
occAnalBind.assoc OccurAnal         3.2    3.6
OccAnal           SimplCore         3.2    3.6
tc_rn_src_decls   TcRnDriver        3.1    2.5
RegAlloc          AsmCodeGen        3.1    4.2
regLiveness       AsmCodeGen        3.0    3.5
FloatOutwards     SimplCore         2.7    2.4
sink              CmmPipeline       2.6    2.7
Simplify          SimplCore         2.5    0.1
NewStranal        SimplCore         1.9    2.3
genMachCode       AsmCodeGen        1.8    1.7
layoutStack       CmmPipeline       1.6    1.7
NativeCodeGen     CodeOutput        1.4    1.5
FloatInwards      SimplCore         1.2    1.6
CoreTidy          HscMain           1.2    1.2
deSugar           HscMain           1.2    1.2
do_block          Hoopl.Dataflow    1.1    0.7
CorePrep          HscMain           1.0    1.1
versioninfo       MkIface           0.9    1.0
Parser            HscMain           0.9    1.2
Digraph.scc       Digraph           0.9    1.5
canEvVar          TcCanonical       0.7    1.0


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Michal Terepeta
On Mon, Apr 13, 2015 at 10:34 PM, Christiaan Baaij
<[hidden email]> wrote:
> Actually, I meant only with -fno-specialise.

Done. Helps quite a bit but CallArity is still a pretty expensive.


Tue Apr 14 21:46 2015 Time and Allocation Profiling Report  (Final)

  ghc +RTS -p -RTS [...] -fno-specialise

total time  =      109.78 secs   (109784 ticks @ 1000 us, 1 processor)
total alloc = 124,469,615,048 bytes  (excludes profiling overheads)

COST CENTRE       MODULE       %time %alloc

SimplTopBinds     SimplCore     35.4   32.4
CallArity         SimplCore     13.4   20.7
pprNativeCode     AsmCodeGen     4.1    4.5
OccAnal           SimplCore      3.1    3.1
StgCmm            HscMain        3.0    2.5
occAnalBind.assoc OccurAnal      3.0    3.1
RegAlloc          AsmCodeGen     2.6    3.2
tc_rn_src_decls   TcRnDriver     2.5    1.8
regLiveness       AsmCodeGen     2.4    2.6
Simplify          SimplCore      2.3    0.1
FloatOutwards     SimplCore      2.3    1.8
sink              CmmPipeline    2.2    2.0
genMachCode       AsmCodeGen     1.5    1.3
NewStranal        SimplCore      1.5    1.7
layoutStack       CmmPipeline    1.3    1.3
NativeCodeGen     CodeOutput     1.2    1.1
FloatInwards      SimplCore      1.0    1.2
Digraph.scc       Digraph        0.8    1.2
_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Joachim Breitner-2
Hi,

Am Dienstag, den 14.04.2015, 21:54 +0200 schrieb Michal Terepeta:
> On Mon, Apr 13, 2015 at 10:34 PM, Christiaan Baaij
> <[hidden email]> wrote:
> > Actually, I meant only with -fno-specialise.
>
> Done. Helps quite a bit but CallArity is still a pretty expensive.

I’m on that, and I think I have a quite neat fix for it. I’ll report on
that in the trac ticket:
https://ghc.haskell.org/trac/ghc/ticket/10293#comment:4

Greetings,
Joachim

--
Joachim “nomeata” Breitner
  [hidden email]http://www.joachim-breitner.de/
  Jabber: [hidden email]  • GPG-Key: 0xF0FBF51F
  Debian Developer: [hidden email]

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Richard Eisenberg-2
I've pasted Michal's numbers in #9630, which seems like a good place to track this. Michal, would you mind fleshing out a bit precisely what you did to get those numbers? That would be helpful (though you've already been very helpful indeed in getting the data together)!

Thanks,
Richard

On Apr 14, 2015, at 9:09 PM, Joachim Breitner <[hidden email]> wrote:

> Hi,
>
> Am Dienstag, den 14.04.2015, 21:54 +0200 schrieb Michal Terepeta:
>> On Mon, Apr 13, 2015 at 10:34 PM, Christiaan Baaij
>> <[hidden email]> wrote:
>>> Actually, I meant only with -fno-specialise.
>>
>> Done. Helps quite a bit but CallArity is still a pretty expensive.
>
> I’m on that, and I think I have a quite neat fix for it. I’ll report on
> that in the trac ticket:
> https://ghc.haskell.org/trac/ghc/ticket/10293#comment:4
>
> Greetings,
> Joachim
>
> --
> Joachim “nomeata” Breitner
>  [hidden email]http://www.joachim-breitner.de/
>  Jabber: [hidden email]  • GPG-Key: 0xF0FBF51F
>  Debian Developer: [hidden email]
> _______________________________________________
> Glasgow-haskell-users mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

Michal Terepeta
Hi Richard,

Thanks updating the ticket! What I did was: build GHC HEAD/7.8.4 with build flavor 'prof' then get the haskell-src-exts sources, install the dependencies and finally add +RTS -p -RTS to the cabal file and compile it, the resulting ghc.prof contains the profiling information.

For anyone interested about CallArity slowness, the problem is tracked in https://ghc.haskell.org/trac/ghc/ticket/10293

Cheers,
Michal

On Wed, Apr 15, 2015 at 1:40 AM Richard Eisenberg <[hidden email]> wrote:
I've pasted Michal's numbers in #9630, which seems like a good place to track this. Michal, would you mind fleshing out a bit precisely what you did to get those numbers? That would be helpful (though you've already been very helpful indeed in getting the data together)!

Thanks,
Richard

On Apr 14, 2015, at 9:09 PM, Joachim Breitner <[hidden email]> wrote:

> Hi,
>
> Am Dienstag, den 14.04.2015, 21:54 +0200 schrieb Michal Terepeta:
>> On Mon, Apr 13, 2015 at 10:34 PM, Christiaan Baaij
>> <[hidden email]> wrote:
>>> Actually, I meant only with -fno-specialise.
>>
>> Done. Helps quite a bit but CallArity is still a pretty expensive.
>
> I’m on that, and I think I have a quite neat fix for it. I’ll report on
> that in the trac ticket:
> https://ghc.haskell.org/trac/ghc/ticket/10293#comment:4
>
> Greetings,
> Joachim
>
> --
> Joachim “nomeata” Breitner
[hidden email]http://www.joachim-breitner.de/
>  Jabber: [hidden email]  • GPG-Key: 0xF0FBF51F
>  Debian Developer: [hidden email]
> _______________________________________________
> Glasgow-haskell-users mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: Increased memory usage with GHC 7.10.1

David Laing
Hi all,

I've been playing around with profiling GHC recently, so I thought I'd chime in with a pointer that might save people searching for the right docs - you could configure a cabal sandbox to work with multiple version of ghc, which I've found useful:

https://www.haskell.org/cabal/users-guide/installing-packages.html#sandboxes-advanced-usage

Cheers,

Dave

On Fri, Apr 17, 2015 at 6:01 AM, Michal Terepeta <[hidden email]> wrote:
Hi Richard,

Thanks updating the ticket! What I did was: build GHC HEAD/7.8.4 with build flavor 'prof' then get the haskell-src-exts sources, install the dependencies and finally add +RTS -p -RTS to the cabal file and compile it, the resulting ghc.prof contains the profiling information.

For anyone interested about CallArity slowness, the problem is tracked in https://ghc.haskell.org/trac/ghc/ticket/10293

Cheers,
Michal

On Wed, Apr 15, 2015 at 1:40 AM Richard Eisenberg <[hidden email]> wrote:
I've pasted Michal's numbers in #9630, which seems like a good place to track this. Michal, would you mind fleshing out a bit precisely what you did to get those numbers? That would be helpful (though you've already been very helpful indeed in getting the data together)!

Thanks,
Richard

On Apr 14, 2015, at 9:09 PM, Joachim Breitner <[hidden email]> wrote:

> Hi,
>
> Am Dienstag, den 14.04.2015, 21:54 +0200 schrieb Michal Terepeta:
>> On Mon, Apr 13, 2015 at 10:34 PM, Christiaan Baaij
>> <[hidden email]> wrote:
>>> Actually, I meant only with -fno-specialise.
>>
>> Done. Helps quite a bit but CallArity is still a pretty expensive.
>
> I’m on that, and I think I have a quite neat fix for it. I’ll report on
> that in the trac ticket:
> https://ghc.haskell.org/trac/ghc/ticket/10293#comment:4
>
> Greetings,
> Joachim
>
> --
> Joachim “nomeata” Breitner
[hidden email]http://www.joachim-breitner.de/
>  Jabber: [hidden email]  • GPG-Key: 0xF0FBF51F
>  Debian Developer: [hidden email]
> _______________________________________________
> Glasgow-haskell-users mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users



_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
12