GHC typecheck API

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

GHC typecheck API

Németh Boldizsár
Dear GHC developers,

I'm developing a framework for development tools for Haskell. I use the
GHC API to parse and typecheck the source files. I recently started to
work on a quick-fix (automatic program correction) for Haskell source
code (correcting parenthesis problems, like 'putStrLn "xxx" ++ show a'
==> 'putStrLn ("xxx" ++ show a)'). To do that I need to get the typed
syntax tree even if the program contains type or rename errors. I could
only do this by a nasty hack, adding a new TH module finalizer
(tcg_th_modfinalizers) to extract the type checker's state before it
fails. Is there a "correct" way to do this? I would not like this
refactorings to be unusable if the GHC API changes.

By the way, if you know of any similar attempt please let me know.

Sincerely,
Boldizsár Németh
haskelltools.org

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: GHC typecheck API

GHC - devs mailing list
This sounds like a good project!

For the most part things look good:

* Most type checker errors arise from *type constraints*. The type checkder tries to solve these, but returns an elaborated syntax tree (i.e. typechecked, and annotated with types) even if constraint solving fails.

* Some renamer errors are like this, notably out-of-scope variables.  (They just show up as another constraint.)

However there is historical baggage.  Back in the beginning, most errors were treated by throwing an exception in the typechecker monad; such exceptions can be caught, so that we can get more than one error from the file, but no syntax tree is returned.  Example
   let f = <expression> in <body>
If there was an error in <expression> we'd throw an exception, catch it at the 'let', give 'f' the type
   f :: forall a. a
and continue to typecheck <body>.

The trouble with the exception stuff is that you don't get an elaborated syntax tree.

So: I think you can get some of the way today, just by returning the tree anyway even if there is an error to report.  But it'd take a bit more work to make more and more errors into things that don't throw an exception.  (Look for failTc, failRn in thd code.)

I'm not very familiar with the GHC API for this part, but others will be.  I'm certain it can be improved, so rather than hacking around what is there already, do propose and implement improvements.

Simon

  | -----Original Message-----
| From: ghc-devs [mailto:[hidden email]] On Behalf Of Németh
| Boldizsár
| Sent: 30 November 2017 05:42
| To: [hidden email]
| Subject: GHC typecheck API
|
| Dear GHC developers,
|
| I'm developing a framework for development tools for Haskell. I use the
| GHC API to parse and typecheck the source files. I recently started to
| work on a quick-fix (automatic program correction) for Haskell source
| code (correcting parenthesis problems, like 'putStrLn "xxx" ++ show a'
| ==> 'putStrLn ("xxx" ++ show a)'). To do that I need to get the typed
| syntax tree even if the program contains type or rename errors. I could
| only do this by a nasty hack, adding a new TH module finalizer
| (tcg_th_modfinalizers) to extract the type checker's state before it
| fails. Is there a "correct" way to do this? I would not like this
| refactorings to be unusable if the GHC API changes.
|
| By the way, if you know of any similar attempt please let me know.
|
| Sincerely,
| Boldizsár Németh
| haskelltools.org
|
| _______________________________________________
| ghc-devs mailing list
| [hidden email]
| https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
| ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
| devs&data=02%7C01%7Csimonpj%40microsoft.com%7C364b8db55e064415650d08d537b
| 5323f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636476173738772433&sda
| ta=Kf5rZYwht8ZGBtGNNH6Q44wLIeefxzHn2UfDIdrNNMU%3D&reserved=0
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: GHC typecheck API

Németh Boldizsár

Thank you for the suggestions!

Setting the -fdefer-type-errors flag is indeed a good way to do it (with also adding Opt_DeferTypedHoles and Opt_DeferOutOfScopeVariables for other kind of errors).

Boldizsár


2017.12.01. 4:12 keltezéssel, Christopher Done írta:
I suppose setting -fdefer-type-errors would also be handy in this scenario!

On Thu, 30 Nov 2017 at 15:37, Simon Peyton Jones via ghc-devs <[hidden email]> wrote:
This sounds like a good project!

For the most part things look good:

* Most type checker errors arise from *type constraints*. The type checkder tries to solve these, but returns an elaborated syntax tree (i.e. typechecked, and annotated with types) even if constraint solving fails.

* Some renamer errors are like this, notably out-of-scope variables.  (They just show up as another constraint.)

However there is historical baggage.  Back in the beginning, most errors were treated by throwing an exception in the typechecker monad; such exceptions can be caught, so that we can get more than one error from the file, but no syntax tree is returned.  Example
   let f = <expression> in <body>
If there was an error in <expression> we'd throw an exception, catch it at the 'let', give 'f' the type
   f :: forall a. a
and continue to typecheck <body>.

The trouble with the exception stuff is that you don't get an elaborated syntax tree.

So: I think you can get some of the way today, just by returning the tree anyway even if there is an error to report.  But it'd take a bit more work to make more and more errors into things that don't throw an exception.  (Look for failTc, failRn in thd code.)

I'm not very familiar with the GHC API for this part, but others will be.  I'm certain it can be improved, so rather than hacking around what is there already, do propose and implement improvements.

Simon

  | -----Original Message-----
| From: ghc-devs [mailto:[hidden email]] On Behalf Of Németh
| Boldizsár
| Sent: 30 November 2017 05:42
| To: [hidden email]
| Subject: GHC typecheck API
|
| Dear GHC developers,
|
| I'm developing a framework for development tools for Haskell. I use the
| GHC API to parse and typecheck the source files. I recently started to
| work on a quick-fix (automatic program correction) for Haskell source
| code (correcting parenthesis problems, like 'putStrLn "xxx" ++ show a'
| ==> 'putStrLn ("xxx" ++ show a)'). To do that I need to get the typed
| syntax tree even if the program contains type or rename errors. I could
| only do this by a nasty hack, adding a new TH module finalizer
| (tcg_th_modfinalizers) to extract the type checker's state before it
| fails. Is there a "correct" way to do this? I would not like this
| refactorings to be unusable if the GHC API changes.
|
| By the way, if you know of any similar attempt please let me know.
|
| Sincerely,
| Boldizsár Németh
| haskelltools.org
|
| _______________________________________________
| ghc-devs mailing list
| [hidden email]
| https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
| ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
| devs&data=02%7C01%7Csimonpj%40microsoft.com%7C364b8db55e064415650d08d537b
| 5323f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636476173738772433&sda
| ta=Kf5rZYwht8ZGBtGNNH6Q44wLIeefxzHn2UfDIdrNNMU%3D&reserved=0
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: GHC typecheck API

Christopher Done-2
We're currently experimenting with this for Intero. There does seem to be a significant performance reduction with -fdefer-type-errors:


It's a classic cost benefit scenario: the benefits are much more type info available, including go to definition and things like that being more immediately up to date. The cost is having to wait more time.

I believe if there's a good compromise to this, it'll probably involve sometimes enabling -fdefer-type-errors and most of the time turning it off for immediate feedback.



On 1 December 2017 at 02:49, Németh Boldizsár <[hidden email]> wrote:

Thank you for the suggestions!

Setting the -fdefer-type-errors flag is indeed a good way to do it (with also adding Opt_DeferTypedHoles and Opt_DeferOutOfScopeVariables for other kind of errors).

Boldizsár


2017.12.01. 4:12 keltezéssel, Christopher Done írta:
I suppose setting -fdefer-type-errors would also be handy in this scenario!

On Thu, 30 Nov 2017 at 15:37, Simon Peyton Jones via ghc-devs <[hidden email]> wrote:
This sounds like a good project!

For the most part things look good:

* Most type checker errors arise from *type constraints*. The type checkder tries to solve these, but returns an elaborated syntax tree (i.e. typechecked, and annotated with types) even if constraint solving fails.

* Some renamer errors are like this, notably out-of-scope variables.  (They just show up as another constraint.)

However there is historical baggage.  Back in the beginning, most errors were treated by throwing an exception in the typechecker monad; such exceptions can be caught, so that we can get more than one error from the file, but no syntax tree is returned.  Example
   let f = <expression> in <body>
If there was an error in <expression> we'd throw an exception, catch it at the 'let', give 'f' the type
   f :: forall a. a
and continue to typecheck <body>.

The trouble with the exception stuff is that you don't get an elaborated syntax tree.

So: I think you can get some of the way today, just by returning the tree anyway even if there is an error to report.  But it'd take a bit more work to make more and more errors into things that don't throw an exception.  (Look for failTc, failRn in thd code.)

I'm not very familiar with the GHC API for this part, but others will be.  I'm certain it can be improved, so rather than hacking around what is there already, do propose and implement improvements.

Simon

  | -----Original Message-----
| From: ghc-devs [mailto:[hidden email]] On Behalf Of Németh
| Boldizsár
| Sent: 30 November 2017 05:42
| To: [hidden email]
| Subject: GHC typecheck API
|
| Dear GHC developers,
|
| I'm developing a framework for development tools for Haskell. I use the
| GHC API to parse and typecheck the source files. I recently started to
| work on a quick-fix (automatic program correction) for Haskell source
| code (correcting parenthesis problems, like 'putStrLn "xxx" ++ show a'
| ==> 'putStrLn ("xxx" ++ show a)'). To do that I need to get the typed
| syntax tree even if the program contains type or rename errors. I could
| only do this by a nasty hack, adding a new TH module finalizer
| (tcg_th_modfinalizers) to extract the type checker's state before it
| fails. Is there a "correct" way to do this? I would not like this
| refactorings to be unusable if the GHC API changes.
|
| By the way, if you know of any similar attempt please let me know.
|
| Sincerely,
| Boldizsár Németh
| haskelltools.org
|
| _______________________________________________
| ghc-devs mailing list
| [hidden email]
| https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
| ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
| devs&data=02%7C01%7Csimonpj%40microsoft.com%7C364b8db55e064415650d08d537b
| 5323f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636476173738772433&sda
| ta=Kf5rZYwht8ZGBtGNNH6Q44wLIeefxzHn2UfDIdrNNMU%3D&reserved=0
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs



_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: GHC typecheck API

GHC - devs mailing list

There does seem to be a significant performance reduction with -fdefer-type-errors:

https://github.com/commercialhaskell/intero/pull/495#issuecomment-348474127

 

That’s odd. I would expect zero effect if there aren’t any errors, and not much even if ther are.  It’s not clear whether the report is to do with compile time or runtime.

 

If you can boil down a test case, maybe a ticket would be good

 

From: Christopher Done [mailto:[hidden email]]
Sent: 01 December 2017 12:01
To: Németh Boldizsár <[hidden email]>
Cc: Simon Peyton Jones <[hidden email]>; [hidden email]
Subject: Re: GHC typecheck API

 

We're currently experimenting with this for Intero. There does seem to be a significant performance reduction with -fdefer-type-errors:

 

 

It's a classic cost benefit scenario: the benefits are much more type info available, including go to definition and things like that being more immediately up to date. The cost is having to wait more time.

 

I believe if there's a good compromise to this, it'll probably involve sometimes enabling -fdefer-type-errors and most of the time turning it off for immediate feedback.

 

 

 

On 1 December 2017 at 02:49, Németh Boldizsár <[hidden email]> wrote:

Thank you for the suggestions!

Setting the -fdefer-type-errors flag is indeed a good way to do it (with also adding Opt_DeferTypedHoles and Opt_DeferOutOfScopeVariables for other kind of errors).

Boldizsár

 

2017.12.01. 4:12 keltezéssel, Christopher Done írta:

I suppose setting -fdefer-type-errors would also be handy in this scenario!

 

On Thu, 30 Nov 2017 at 15:37, Simon Peyton Jones via ghc-devs <[hidden email]> wrote:

This sounds like a good project!

For the most part things look good:

* Most type checker errors arise from *type constraints*. The type checkder tries to solve these, but returns an elaborated syntax tree (i.e. typechecked, and annotated with types) even if constraint solving fails.

* Some renamer errors are like this, notably out-of-scope variables.  (They just show up as another constraint.)

However there is historical baggage.  Back in the beginning, most errors were treated by throwing an exception in the typechecker monad; such exceptions can be caught, so that we can get more than one error from the file, but no syntax tree is returned.  Example
   let f = <expression> in <body>
If there was an error in <expression> we'd throw an exception, catch it at the 'let', give 'f' the type
   f :: forall a. a
and continue to typecheck <body>.

The trouble with the exception stuff is that you don't get an elaborated syntax tree.

So: I think you can get some of the way today, just by returning the tree anyway even if there is an error to report.  But it'd take a bit more work to make more and more errors into things that don't throw an exception.  (Look for failTc, failRn in thd code.)

I'm not very familiar with the GHC API for this part, but others will be.  I'm certain it can be improved, so rather than hacking around what is there already, do propose and implement improvements.

Simon

  | -----Original Message-----
| From: ghc-devs [mailto:[hidden email]] On Behalf Of Németh
| Boldizsár
| Sent: 30 November 2017 05:42
| To: [hidden email]
| Subject: GHC typecheck API
|
| Dear GHC developers,
|
| I'm developing a framework for development tools for Haskell. I use the
| GHC API to parse and typecheck the source files. I recently started to
| work on a quick-fix (automatic program correction) for Haskell source
| code (correcting parenthesis problems, like 'putStrLn "xxx" ++ show a'
| ==> 'putStrLn ("xxx" ++ show a)'). To do that I need to get the typed
| syntax tree even if the program contains type or rename errors. I could
| only do this by a nasty hack, adding a new TH module finalizer
| (tcg_th_modfinalizers) to extract the type checker's state before it
| fails. Is there a "correct" way to do this? I would not like this
| refactorings to be unusable if the GHC API changes.
|
| By the way, if you know of any similar attempt please let me know.
|
| Sincerely,
| Boldizsár Németh
| haskelltools.org
|
| _______________________________________________
| ghc-devs mailing list
| [hidden email]
| https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
| ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
| devs&data=02%7C01%7Csimonpj%40microsoft.com%7C364b8db55e064415650d08d537b
| 5323f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636476173738772433&sda
| ta=Kf5rZYwht8ZGBtGNNH6Q44wLIeefxzHn2UfDIdrNNMU%3D&reserved=0
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

 

 


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: GHC typecheck API

Christopher Done-2
The report is timing how long a `:load` takes. 

Sure, if/when I reproduce this on some open source module I'll make a ticket with a test case! 


On 1 December 2017 at 12:03, Simon Peyton Jones <[hidden email]> wrote:

There does seem to be a significant performance reduction with -fdefer-type-errors:

https://github.com/commercialhaskell/intero/pull/495#issuecomment-348474127

 

That’s odd. I would expect zero effect if there aren’t any errors, and not much even if ther are.  It’s not clear whether the report is to do with compile time or runtime.

 

If you can boil down a test case, maybe a ticket would be good

 

From: Christopher Done [mailto:[hidden email]]
Sent: 01 December 2017 12:01
To: Németh Boldizsár <[hidden email]>
Cc: Simon Peyton Jones <[hidden email]>; [hidden email]
Subject: Re: GHC typecheck API

 

We're currently experimenting with this for Intero. There does seem to be a significant performance reduction with -fdefer-type-errors:

 

 

It's a classic cost benefit scenario: the benefits are much more type info available, including go to definition and things like that being more immediately up to date. The cost is having to wait more time.

 

I believe if there's a good compromise to this, it'll probably involve sometimes enabling -fdefer-type-errors and most of the time turning it off for immediate feedback.

 

 

 

On 1 December 2017 at 02:49, Németh Boldizsár <[hidden email]> wrote:

Thank you for the suggestions!

Setting the -fdefer-type-errors flag is indeed a good way to do it (with also adding Opt_DeferTypedHoles and Opt_DeferOutOfScopeVariables for other kind of errors).

Boldizsár

 

2017.12.01. 4:12 keltezéssel, Christopher Done írta:

I suppose setting -fdefer-type-errors would also be handy in this scenario!

 

On Thu, 30 Nov 2017 at 15:37, Simon Peyton Jones via ghc-devs <[hidden email]> wrote:

This sounds like a good project!

For the most part things look good:

* Most type checker errors arise from *type constraints*. The type checkder tries to solve these, but returns an elaborated syntax tree (i.e. typechecked, and annotated with types) even if constraint solving fails.

* Some renamer errors are like this, notably out-of-scope variables.  (They just show up as another constraint.)

However there is historical baggage.  Back in the beginning, most errors were treated by throwing an exception in the typechecker monad; such exceptions can be caught, so that we can get more than one error from the file, but no syntax tree is returned.  Example
   let f = <expression> in <body>
If there was an error in <expression> we'd throw an exception, catch it at the 'let', give 'f' the type
   f :: forall a. a
and continue to typecheck <body>.

The trouble with the exception stuff is that you don't get an elaborated syntax tree.

So: I think you can get some of the way today, just by returning the tree anyway even if there is an error to report.  But it'd take a bit more work to make more and more errors into things that don't throw an exception.  (Look for failTc, failRn in thd code.)

I'm not very familiar with the GHC API for this part, but others will be.  I'm certain it can be improved, so rather than hacking around what is there already, do propose and implement improvements.

Simon

  | -----Original Message-----
| From: ghc-devs [mailto:[hidden email]] On Behalf Of Németh
| Boldizsár
| Sent: 30 November 2017 05:42
| To: [hidden email]
| Subject: GHC typecheck API
|
| Dear GHC developers,
|
| I'm developing a framework for development tools for Haskell. I use the
| GHC API to parse and typecheck the source files. I recently started to
| work on a quick-fix (automatic program correction) for Haskell source
| code (correcting parenthesis problems, like 'putStrLn "xxx" ++ show a'
| ==> 'putStrLn ("xxx" ++ show a)'). To do that I need to get the typed
| syntax tree even if the program contains type or rename errors. I could
| only do this by a nasty hack, adding a new TH module finalizer
| (tcg_th_modfinalizers) to extract the type checker's state before it
| fails. Is there a "correct" way to do this? I would not like this
| refactorings to be unusable if the GHC API changes.
|
| By the way, if you know of any similar attempt please let me know.
|
| Sincerely,
| Boldizsár Németh
| haskelltools.org
|
| _______________________________________________
| ghc-devs mailing list
| [hidden email]
| https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
| ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
| devs&data=02%7C01%7Csimonpj%40microsoft.com%7C364b8db55e064415650d08d537b
| 5323f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636476173738772433&sda
| ta=Kf5rZYwht8ZGBtGNNH6Q44wLIeefxzHn2UfDIdrNNMU%3D&reserved=0
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

 

 



_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs