ghc-in-ghci memory usage

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

ghc-in-ghci memory usage

Matthew Pickering
I just tried using the ghc-in-ghci script again and it appears that
the memory usage problems have been resolved. Probably thanks to Simon
Marlow who fixed a lot of space leaks in ghc.

A reminder of how to use it:

./utils/ghc-in-ghci/run.sh -j<n>

will load ghc into ghci so you can `:r` to recompile when you make a change.

Cheers,

Matt
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: ghc-in-ghci memory usage

Michael Sloan
Great, I'm glad it's working well for you!

I've realized that when working on ghc-in-ghci, I didn't know about
freezing stage 1 to speed up GHC builds.  For me the primary
motivation was that otherwise builds took quite long.  Of course,
ghc-in-ghci is still quite useful, due to being able to use the repl,
avoiding static link times, etc.

I'm not sure what the best way would be to effectively communicate
with newcomers about freezing stage 1, but it seems important for
build times.  It may be easy to get dissuaded if every development
iteration involves a ton of waiting for the build.  Perhaps hadrian
can be more intelligent about avoiding stage 1 rebuilds?  I realize
that's likely to be tricky from a correctness perspective.

One thing I'm keen on for ghc-in-ghci is getting it to load without
-fobject-code.  This would often mean a much longer initial start (no
use of stored object files), but should make reloads quite a lot
faster, since it's generating bytecode instead.  The main tricky bit
there is the use of unboxed tuples, since ghci cannot bytecode-compile
code that uses them.  So, this either means adding support for unboxed
tuples to bytecode, which seems quite challenging, or having something
clever that only uses object-code compilation where needed.

-Michael

On Tue, Oct 30, 2018 at 7:56 AM Matthew Pickering
<[hidden email]> wrote:

>
> I just tried using the ghc-in-ghci script again and it appears that
> the memory usage problems have been resolved. Probably thanks to Simon
> Marlow who fixed a lot of space leaks in ghc.
>
> A reminder of how to use it:
>
> ./utils/ghc-in-ghci/run.sh -j<n>
>
> will load ghc into ghci so you can `:r` to recompile when you make a change.
>
> Cheers,
>
> Matt
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: ghc-in-ghci memory usage

Matthew Pickering
The information about freezing stage2 is in the first paragraph after
the basics on the newcomers page. So it is already very prominent.

https://ghc.haskell.org/trac/ghc/wiki/Newcomers#Fastrebuilding

That being said, this will all change when hadrian is the default and
I'm not sure how to achieve this with Hadrian. I'm sure this will have
been accounted for.

Cheers,

Matt
On Wed, Oct 31, 2018 at 7:53 PM Michael Sloan <[hidden email]> wrote:

>
> Great, I'm glad it's working well for you!
>
> I've realized that when working on ghc-in-ghci, I didn't know about
> freezing stage 1 to speed up GHC builds.  For me the primary
> motivation was that otherwise builds took quite long.  Of course,
> ghc-in-ghci is still quite useful, due to being able to use the repl,
> avoiding static link times, etc.
>
> I'm not sure what the best way would be to effectively communicate
> with newcomers about freezing stage 1, but it seems important for
> build times.  It may be easy to get dissuaded if every development
> iteration involves a ton of waiting for the build.  Perhaps hadrian
> can be more intelligent about avoiding stage 1 rebuilds?  I realize
> that's likely to be tricky from a correctness perspective.
>
> One thing I'm keen on for ghc-in-ghci is getting it to load without
> -fobject-code.  This would often mean a much longer initial start (no
> use of stored object files), but should make reloads quite a lot
> faster, since it's generating bytecode instead.  The main tricky bit
> there is the use of unboxed tuples, since ghci cannot bytecode-compile
> code that uses them.  So, this either means adding support for unboxed
> tuples to bytecode, which seems quite challenging, or having something
> clever that only uses object-code compilation where needed.
>
> -Michael
>
> On Tue, Oct 30, 2018 at 7:56 AM Matthew Pickering
> <[hidden email]> wrote:
> >
> > I just tried using the ghc-in-ghci script again and it appears that
> > the memory usage problems have been resolved. Probably thanks to Simon
> > Marlow who fixed a lot of space leaks in ghc.
> >
> > A reminder of how to use it:
> >
> > ./utils/ghc-in-ghci/run.sh -j<n>
> >
> > will load ghc into ghci so you can `:r` to recompile when you make a change.
> >
> > Cheers,
> >
> > Matt
> > _______________________________________________
> > ghc-devs mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: ghc-in-ghci memory usage

Alp Mestanogullari-2

Hadrian has the `--freeze1` flag, whose description is: freeze Stage1 GHC, i.e. do not rebuild it even if some of its source files are out-of-date. This allows to significantly reduce the rebuild time when you are working on a feature that affects both Stage1 and Stage2 compilers, but may lead to incorrect build results. To unfreeze Stage1 GHC simply drop the --freeze1 flag and Hadrian will rebuild all out-of-date files.

(from https://ghc.haskell.org/trac/ghc/wiki/Building/Hadrian/QuickStart#Commandlineoptions)

On 31/10/2018 21:54, Matthew Pickering wrote:
The information about freezing stage2 is in the first paragraph after
the basics on the newcomers page. So it is already very prominent.

https://ghc.haskell.org/trac/ghc/wiki/Newcomers#Fastrebuilding

That being said, this will all change when hadrian is the default and
I'm not sure how to achieve this with Hadrian. I'm sure this will have
been accounted for.

Cheers,

Matt
On Wed, Oct 31, 2018 at 7:53 PM Michael Sloan [hidden email] wrote:
Great, I'm glad it's working well for you!

I've realized that when working on ghc-in-ghci, I didn't know about
freezing stage 1 to speed up GHC builds.  For me the primary
motivation was that otherwise builds took quite long.  Of course,
ghc-in-ghci is still quite useful, due to being able to use the repl,
avoiding static link times, etc.

I'm not sure what the best way would be to effectively communicate
with newcomers about freezing stage 1, but it seems important for
build times.  It may be easy to get dissuaded if every development
iteration involves a ton of waiting for the build.  Perhaps hadrian
can be more intelligent about avoiding stage 1 rebuilds?  I realize
that's likely to be tricky from a correctness perspective.

One thing I'm keen on for ghc-in-ghci is getting it to load without
-fobject-code.  This would often mean a much longer initial start (no
use of stored object files), but should make reloads quite a lot
faster, since it's generating bytecode instead.  The main tricky bit
there is the use of unboxed tuples, since ghci cannot bytecode-compile
code that uses them.  So, this either means adding support for unboxed
tuples to bytecode, which seems quite challenging, or having something
clever that only uses object-code compilation where needed.

-Michael

On Tue, Oct 30, 2018 at 7:56 AM Matthew Pickering
[hidden email] wrote:
I just tried using the ghc-in-ghci script again and it appears that
the memory usage problems have been resolved. Probably thanks to Simon
Marlow who fixed a lot of space leaks in ghc.

A reminder of how to use it:

./utils/ghc-in-ghci/run.sh -j<n>

will load ghc into ghci so you can `:r` to recompile when you make a change.

Cheers,

Matt
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-- 
Alp Mestanogullari, Haskell Consultant
Well-Typed LLP, https://www.well-typed.com/

Registered in England and Wales, OC335890
118 Wymering Mansions, Wymering Road, London, W9 2NF, England

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs