GHCi recompilation avoidance UI

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

GHCi recompilation avoidance UI

David Feuer-2
Changes in GHC 8.2.1 lead to a lot of recompilation, because GHCi now refuses to load optimized
code unless -fobject-code (and optimization flags) are enabled. I propose the following slight
modification to https://ghc.haskell.org/trac/ghc/ticket/13604#comment:48

1. Optimization flags (except -O0) imply -fobject-code. This ensures that GHC respects optimization flags regardless of --interactive.

2. Even when -fobject-code is on, :load *M will load M as bytecode. This provides the "escape hatch" from -fobject-code that you need to use debugging features, etc.

3. New -fignore-optim-changes and -fignore-hpc-changes (​​Phab:D4123) flags should enable users to put together object code and bytecode with diverse optimization levels/options and HPC options while still updating automatically based on source changes and whether profiling is enabled.

Does this seem like a reasonable way forward?

David Feuer
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: GHCi recompilation avoidance UI

Evan Laforge
I updated the trac ticket, but here's a copy-paste:

I still don't feel like 1 is necessary, I'd rather flags cause other
flags to be ignored with a warning, rather than turn on other flags.
But that's just a vague preference, with no strong evidence for it.
Maybe it could emit a warning if you didn't put -fobject-code in
explicitly, e.g. "-O implies -fobject-code, adding that flag." So as
long as we accept 1, then 2 and 3 follow naturally. Given that, I
support this UI.

Thanks for looking into it!
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: GHCi recompilation avoidance UI

GHC - devs mailing list
In reply to this post by David Feuer-2
I'm lost.

* What causes the undesired behaviour in GHC 8.2.1?
  Is it this?
    - GHCi wants to load module B with flags F
    - There is a B.o but the flags differ
    - So GHCi recompiles B to bytecode

* How does (1) help?

* Does (2) propose a new command (:load *M vs :load M)?
  If so, how does it help.

* Is (3), what is the spec of the proposed new flags?
  Is it this?
  - GHC wants to compile module B with flags F
  - There is a B.o compiled with flags F'
  - And B.hs (and its dependencies) have not changed
  - Then normally we'd recompile F unless F = F'
  - But with -fignore-optim-changes we would weaken that equality to
    ignore differences between F and F' that only affected optimisation
    level (the list of such flags would need to be written out)
  - And similarly for -fignore-hpc-changes


Precision is all!

Thanks

Simon

|  -----Original Message-----
|  From: ghc-devs [mailto:[hidden email]] On Behalf Of David
|  Feuer
|  Sent: 31 October 2017 15:42
|  To: [hidden email]
|  Subject: GHCi recompilation avoidance UI
|  
|  Changes in GHC 8.2.1 lead to a lot of recompilation, because GHCi now
|  refuses to load optimized code unless -fobject-code (and optimization
|  flags) are enabled. I propose the following slight modification to
|  https://ghc.haskell.org/trac/ghc/ticket/13604#comment:48
|  
|  1. Optimization flags (except -O0) imply -fobject-code. This ensures
|  that GHC respects optimization flags regardless of --interactive.
|  
|  2. Even when -fobject-code is on, :load *M will load M as bytecode.
|  This provides the "escape hatch" from -fobject-code that you need to
|  use debugging features, etc.
|  
|  3. New -fignore-optim-changes and -fignore-hpc-changes (​​Phab:D4123)
|  flags should enable users to put together object code and bytecode with
|  diverse optimization levels/options and HPC options while still
|  updating automatically based on source changes and whether profiling is
|  enabled.
|  
|  Does this seem like a reasonable way forward?
|  
|  David Feuer
|  _______________________________________________
|  ghc-devs mailing list
|  [hidden email]
|  https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.ha
|  skell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cd533ac396d6a4a6ff68208d52
|  076040e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636450613629249637
|  &sdata=9duvpS1xcNXXUm%2BOJgWg4jFBCgvT3SqPJMU2k8Rdxj0%3D&reserved=0
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: GHCi recompilation avoidance UI

Evan Laforge
On Wed, Nov 1, 2017 at 1:46 AM, Simon Peyton Jones via ghc-devs
<[hidden email]> wrote:
> I'm lost.
>
> * What causes the undesired behaviour in GHC 8.2.1?
>   Is it this?
>     - GHCi wants to load module B with flags F
>     - There is a B.o but the flags differ
>     - So GHCi recompiles B to bytecode
>
> * How does (1) help?

From my perspective, what happened is that GHC 8.2 started including
the -O and -fhpc flags in the hash for recompilation avoidance, so
it's started to notice when those particular flags differ.  However,
ghci has always rejected those two flags, courtesy of
DynFlags.makeDynFlagsConsistent.  The result is that it's now
impossible for ghci to load modules compiled with those flags, since
it will never have the right flags.  The reason that's a problem is
that it's really convenient to run tests (compiled with -fhpc)
interactively, and use the GHC API to interpret code when the rest of
the system is compiled with -O.  That latter usage might be on thin
ice according to the comment on makeDynFlagsConsistent, but in
practice it's worked fine for a long time.

I'll let David answer about how #1 helps, but it looks like maybe ghci
could have supported -O just fine with -fobject-code, so it should
have been using that to determine whether or not to reject -O, not
--interactive.

> * Does (2) propose a new command (:load *M vs :load M)?
>   If so, how does it help.

:load *M has always been present, so I'm not sure what is the
difference.  Maybe it didn't override -fobject-code before.  I'll let
David answer this and #3.

But I think the effect of the flags in #3 would be "go back to <8.2",
which is to say, omit -O and -fhpc from the interface hash.  So yes, I
think your description sounds right.
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: GHCi recompilation avoidance UI

Simon Marlow-7
In reply to this post by David Feuer-2
On 31 October 2017 at 15:42, David Feuer <[hidden email]> wrote:
Changes in GHC 8.2.1 lead to a lot of recompilation, because GHCi now refuses to load optimized
code unless -fobject-code (and optimization flags) are enabled. I propose the following slight
modification to https://ghc.haskell.org/trac/ghc/ticket/13604#comment:48

1. Optimization flags (except -O0) imply -fobject-code. This ensures that GHC respects optimization flags regardless of --interactive.

2. Even when -fobject-code is on, :load *M will load M as bytecode. This provides the "escape hatch" from -fobject-code that you need to use debugging features, etc.

Yes, I think this is probably what we want. I'm not sure how smooth it will be to implement though.
 
3. New -fignore-optim-changes and -fignore-hpc-changes (​​Phab:D4123) flags should enable users to put together object code and bytecode with diverse optimization levels/options and HPC options while still updating automatically based on source changes and whether profiling is enabled.

As I mentioned on the diff, I think we'll want at least -fignore-optim-changes to be the default, so that GHCi does the expected thing when you have compiled object files.

Cheers
Simon

 

Does this seem like a reasonable way forward?

David Feuer
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: GHCi recompilation avoidance UI

David Feuer-2
I started digging back into this today, particularly considering Simon PJ's view
that it's a bit odd for optimization flags to imply -fobject-code (specifically
because we could potentially support optimization for the bytecode
interpreter some day). I'm left even more lost about exactly what we want.
I believe it's fairly clear that, as Simon M wrote,

> [W]e'll want at least -fignore-optim-changes to be the default, so that GHCi
> does the expected thing when you have compiled object files.

Based on Simon PJ's comment, I believe we want to *continue* to discard
optimization flags when -fobject-code is not enabled. As for my suggestion in (2),
I spent the last couple hours attempting to figure out what would be necessary
to allow :load *M to load a module  interpreted even when using -fobject-code,
but found myself utterly lost in the module loading logic. I see that the IIModule
constructor is deeply involved in this, but I haven't been able to figure out
where/how that interacts with -fobject-code to determine whether the module
will actually be loaded interpreted or compiled. Can someone give me a clue?

On Thursday, November 2, 2017 10:21:07 AM EST Simon Marlow wrote:

> On 31 October 2017 at 15:42, David Feuer <[hidden email]> wrote:
>
> > Changes in GHC 8.2.1 lead to a lot of recompilation, because GHCi now
> > refuses to load optimized
> > code unless -fobject-code (and optimization flags) are enabled. I propose
> > the following slight
> > modification to https://ghc.haskell.org/trac/ghc/ticket/13604#comment:48
> >
> > 1. Optimization flags (except -O0) imply -fobject-code. This ensures that
> > GHC respects optimization flags regardless of --interactive.
> >
> > 2. Even when -fobject-code is on, :load *M will load M as bytecode. This
> > provides the "escape hatch" from -fobject-code that you need to use
> > debugging features, etc.
> >
>
> Yes, I think this is probably what we want. I'm not sure how smooth it will
> be to implement though.
>
>
> > 3. New -fignore-optim-changes and -fignore-hpc-changes (​​Phab:D4123)
> > flags should enable users to put together object code and bytecode with
> > diverse optimization levels/options and HPC options while still updating
> > automatically based on source changes and whether profiling is enabled.
> >
>
> As I mentioned on the diff, I think we'll want at least
> -fignore-optim-changes to be the default, so that GHCi does the expected
> thing when you have compiled object files.
>
> Cheers
> Simon
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: GHCi recompilation avoidance UI

Ben Gamari-3
David Feuer <[hidden email]> writes:

> I started digging back into this today, particularly considering Simon PJ's view
> that it's a bit odd for optimization flags to imply -fobject-code (specifically
> because we could potentially support optimization for the bytecode
> interpreter some day). I'm left even more lost about exactly what we want.
> I believe it's fairly clear that, as Simon M wrote,
>
>> [W]e'll want at least -fignore-optim-changes to be the default, so that GHCi
>> does the expected thing when you have compiled object files.
>
> Based on Simon PJ's comment, I believe we want to *continue* to discard
> optimization flags when -fobject-code is not enabled. As for my suggestion in (2),
> I spent the last couple hours attempting to figure out what would be necessary
> to allow :load *M to load a module  interpreted even when using -fobject-code,
> but found myself utterly lost in the module loading logic. I see that the IIModule
> constructor is deeply involved in this, but I haven't been able to figure out
> where/how that interacts with -fobject-code to determine whether the module
> will actually be loaded interpreted or compiled. Can someone give me a clue?
>
It seems to me like the place to start would be occurrences of the
targetAllowObjCode field of Target.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GHCi recompilation avoidance UI

Simon Marlow-7
In reply to this post by David Feuer-2
David,

Perhaps it would be good to defer changing the behaviour of :load *M (I believe you that it's hard, that code is quite convoluted) and for now just focus on making GHCi able to load compiled object code again, which I think is a much simpler problem?

Cheers
Simon


On 21 November 2017 at 21:49, David Feuer <[hidden email]> wrote:
I started digging back into this today, particularly considering Simon PJ's view
that it's a bit odd for optimization flags to imply -fobject-code (specifically
because we could potentially support optimization for the bytecode
interpreter some day). I'm left even more lost about exactly what we want.
I believe it's fairly clear that, as Simon M wrote,

> [W]e'll want at least -fignore-optim-changes to be the default, so that GHCi
> does the expected thing when you have compiled object files.

Based on Simon PJ's comment, I believe we want to *continue* to discard
optimization flags when -fobject-code is not enabled. As for my suggestion in (2),
I spent the last couple hours attempting to figure out what would be necessary
to allow :load *M to load a module  interpreted even when using -fobject-code,
but found myself utterly lost in the module loading logic. I see that the IIModule
constructor is deeply involved in this, but I haven't been able to figure out
where/how that interacts with -fobject-code to determine whether the module
will actually be loaded interpreted or compiled. Can someone give me a clue?

On Thursday, November 2, 2017 10:21:07 AM EST Simon Marlow wrote:
> On 31 October 2017 at 15:42, David Feuer <[hidden email]> wrote:
>
> > Changes in GHC 8.2.1 lead to a lot of recompilation, because GHCi now
> > refuses to load optimized
> > code unless -fobject-code (and optimization flags) are enabled. I propose
> > the following slight
> > modification to https://ghc.haskell.org/trac/ghc/ticket/13604#comment:48
> >
> > 1. Optimization flags (except -O0) imply -fobject-code. This ensures that
> > GHC respects optimization flags regardless of --interactive.
> >
> > 2. Even when -fobject-code is on, :load *M will load M as bytecode. This
> > provides the "escape hatch" from -fobject-code that you need to use
> > debugging features, etc.
> >
>
> Yes, I think this is probably what we want. I'm not sure how smooth it will
> be to implement though.
>
>
> > 3. New -fignore-optim-changes and -fignore-hpc-changes (​​Phab:D4123)
> > flags should enable users to put together object code and bytecode with
> > diverse optimization levels/options and HPC options while still updating
> > automatically based on source changes and whether profiling is enabled.
> >
>
> As I mentioned on the diff, I think we'll want at least
> -fignore-optim-changes to be the default, so that GHCi does the expected
> thing when you have compiled object files.
>
> Cheers
> Simon


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: GHCi recompilation avoidance UI

Evan Laforge
https://ghc.haskell.org/trac/ghc/ticket/13604 was just closed, I look
forward to trying it out.  Thanks to dfeuer for the fix!

On Wed, Nov 22, 2017 at 12:41 AM, Simon Marlow <[hidden email]> wrote:

> David,
>
> Perhaps it would be good to defer changing the behaviour of :load *M (I
> believe you that it's hard, that code is quite convoluted) and for now just
> focus on making GHCi able to load compiled object code again, which I think
> is a much simpler problem?
>
> Cheers
> Simon
>
>
> On 21 November 2017 at 21:49, David Feuer <[hidden email]> wrote:
>>
>> I started digging back into this today, particularly considering Simon
>> PJ's view
>> that it's a bit odd for optimization flags to imply -fobject-code
>> (specifically
>> because we could potentially support optimization for the bytecode
>> interpreter some day). I'm left even more lost about exactly what we want.
>> I believe it's fairly clear that, as Simon M wrote,
>>
>> > [W]e'll want at least -fignore-optim-changes to be the default, so that
>> > GHCi
>> > does the expected thing when you have compiled object files.
>>
>> Based on Simon PJ's comment, I believe we want to *continue* to discard
>> optimization flags when -fobject-code is not enabled. As for my suggestion
>> in (2),
>> I spent the last couple hours attempting to figure out what would be
>> necessary
>> to allow :load *M to load a module  interpreted even when using
>> -fobject-code,
>> but found myself utterly lost in the module loading logic. I see that the
>> IIModule
>> constructor is deeply involved in this, but I haven't been able to figure
>> out
>> where/how that interacts with -fobject-code to determine whether the
>> module
>> will actually be loaded interpreted or compiled. Can someone give me a
>> clue?
>>
>> On Thursday, November 2, 2017 10:21:07 AM EST Simon Marlow wrote:
>> > On 31 October 2017 at 15:42, David Feuer <[hidden email]> wrote:
>> >
>> > > Changes in GHC 8.2.1 lead to a lot of recompilation, because GHCi now
>> > > refuses to load optimized
>> > > code unless -fobject-code (and optimization flags) are enabled. I
>> > > propose
>> > > the following slight
>> > > modification to
>> > > https://ghc.haskell.org/trac/ghc/ticket/13604#comment:48
>> > >
>> > > 1. Optimization flags (except -O0) imply -fobject-code. This ensures
>> > > that
>> > > GHC respects optimization flags regardless of --interactive.
>> > >
>> > > 2. Even when -fobject-code is on, :load *M will load M as bytecode.
>> > > This
>> > > provides the "escape hatch" from -fobject-code that you need to use
>> > > debugging features, etc.
>> > >
>> >
>> > Yes, I think this is probably what we want. I'm not sure how smooth it
>> > will
>> > be to implement though.
>> >
>> >
>> > > 3. New -fignore-optim-changes and -fignore-hpc-changes (Phab:D4123)
>> > > flags should enable users to put together object code and bytecode
>> > > with
>> > > diverse optimization levels/options and HPC options while still
>> > > updating
>> > > automatically based on source changes and whether profiling is
>> > > enabled.
>> > >
>> >
>> > As I mentioned on the diff, I think we'll want at least
>> > -fignore-optim-changes to be the default, so that GHCi does the expected
>> > thing when you have compiled object files.
>> >
>> > Cheers
>> > Simon
>
>
>
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs