Proposal: add HasCallStack for all partial functions in base

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Proposal: add HasCallStack for all partial functions in base

LuoChen

Motivation

Partial functions in base (especially Prelude) often cause runtime errors and is hard to locate.

(here is a document about the concept of totality and partial function, ignore this if you are familiar with them)

For example, consider the following piece of code:

import GHC.Stack

foo :: HasCallStack => [Int] -> Int
foo xs = last xs + 1

xs :: [Int]
xs = []

main :: IO ()
main = do
    print $ foo xs

In this case, the error message will tell nothing about foo, and the HasCallStack constraint is totally helpless, because the call stack is cut off by the call to last which without HasCallStack constraint.

My current workaround is define my own wrapper functions with HasCallStack constraint for some mostly used partial functions to make them traceable, and use the wrapper (the traceable version) whenever I need them.

e.g.

last' :: HasCallStack => [a] -> a
last' xs = case xs of [] -> error "abuse last"; _ -> last xs

So, IMHO, if our goal is to make errors in haskell traceable, then only providing HasCallStack mechanism is not enough, we have to provide traceable base package and prelude at the same time.

Further more, all untraceable partial functions are considered to be harmful, and should not be exported by any package. Because an improper call to an untraceable partial function will cut off the call stack, and here is a demonstration about that.

On the other hand, is it ever necessary for us to add HasCallStack for a total function? Or we can ask, is it possible that a call to a total function cause runtime error? Maybe it’s a NO, since a total function will also crash when the machine is Out Of Memory, but that is also the only situation I can find out. So I suggest that we add HasCallStack constraint only for partial functions, and IMHO this could be a good balance for better debugging experience and less runtime overhead.

Proposal

  1. add HasCallStack constraint for all partial functions in base package
  2. suggest all programmers to add HasCallStack constraint for their exported partial functions when they release a package
  3. provide a compiler option -fignore-hascallstack to toggle off the effect of HasCallStack constraint in case somebody need best performance

Other Considerations

How to get a full list of partial functions provided by the base package?

I wanted to provide a full list of partial functions exported by the base package in this post, but I find it is hard, since Haskell have no totality checking mechanism like Idris have, and there are no consistent keyword like “total” or “partial” in document, so it takes a lot of work to list all the partial functions of a package by check every item manually. Maybe we can work on this list later — when it’s turned out to be worth after some discussion.

Here is part of the list that I have tidied for several modules of the base package.

How to encourage all package contributors to obey the rule (see proposal #2)?

I don’t know, but I think there may be some other rules to obey when contributing packages. Maybe we can just add this into the list.

Obviously, the final perfect solution should be let the compiler to check the totality of functions, and automatically add HasCallStack for the ones which the compiler cannot confirm it’s totality. But this seems too far away from us, since we still doesn’t have dependent type haskell yet.

How to deal with recursive partial functions?

Since the HasCallStack constraint affects the performance (not only because of the runtime overhead, but also it’s influence on optimization strategy), It is best not to add HasCallStack on recursive functions.

In most of the cases, we can just check the input shallowly before everything start, just like how we deal with the non-recursive ones.

But in some other cases, we need to go deep to recognize the invalidity of the input. A trivial solution is just perform a deep check before everything start, but the checking phase seems expensive.

The best solution, IMHO, is to make the recursive part a total function, wrap the return value into Maybe or some similar things, and the partial version is just the total version combined fromJust. In this way, we avoid the single input checking phase and left the error awareing logic where it was before this translation.



_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Henning Thielemann

On Sun, 16 Jun 2019, LuoChen wrote:

> On the other hand, is it ever necessary for us to add HasCallStack for a
> total function? Or we can ask, is it possible that a call to a total
> function cause runtime error? Maybe it’s a NO, since a total function
> will also crash when the machine is Out Of Memory, but that is also the
> only situation I can find out.

A non-IO function 'f' cannot crash because of memory exhaustion. The IO
function calling 'f' is the one that crashes. A call stack would not help
because it is not a programming error in 'f'.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Haskell - Libraries mailing list
In reply to this post by LuoChen
I like the idea of marking partial functions with HasCallStack.
But I'd like to point out that we also have class methods where some but not all implementations are partial, such as foldr1 in Foldable.

Should methods like these also get a HasCallStack constraint?

Am So., 16. Juni 2019 um 14:30 Uhr schrieb LuoChen <[hidden email]>:

Motivation

Partial functions in base (especially Prelude) often cause runtime errors and is hard to locate.

(here is a document about the concept of totality and partial function, ignore this if you are familiar with them)

For example, consider the following piece of code:

import GHC.Stack

foo :: HasCallStack => [Int] -> Int
foo xs = last xs + 1

xs :: [Int]
xs = []

main :: IO ()
main = do
    print $ foo xs

In this case, the error message will tell nothing about foo, and the HasCallStack constraint is totally helpless, because the call stack is cut off by the call to last which without HasCallStack constraint.

My current workaround is define my own wrapper functions with HasCallStack constraint for some mostly used partial functions to make them traceable, and use the wrapper (the traceable version) whenever I need them.

e.g.

last' :: HasCallStack => [a] -> a
last' xs = case xs of [] -> error "abuse last"; _ -> last xs

So, IMHO, if our goal is to make errors in haskell traceable, then only providing HasCallStack mechanism is not enough, we have to provide traceable base package and prelude at the same time.

Further more, all untraceable partial functions are considered to be harmful, and should not be exported by any package. Because an improper call to an untraceable partial function will cut off the call stack, and here is a demonstration about that.

On the other hand, is it ever necessary for us to add HasCallStack for a total function? Or we can ask, is it possible that a call to a total function cause runtime error? Maybe it’s a NO, since a total function will also crash when the machine is Out Of Memory, but that is also the only situation I can find out. So I suggest that we add HasCallStack constraint only for partial functions, and IMHO this could be a good balance for better debugging experience and less runtime overhead.

Proposal

  1. add HasCallStack constraint for all partial functions in base package
  2. suggest all programmers to add HasCallStack constraint for their exported partial functions when they release a package
  3. provide a compiler option -fignore-hascallstack to toggle off the effect of HasCallStack constraint in case somebody need best performance

Other Considerations

How to get a full list of partial functions provided by the base package?

I wanted to provide a full list of partial functions exported by the base package in this post, but I find it is hard, since Haskell have no totality checking mechanism like Idris have, and there are no consistent keyword like “total” or “partial” in document, so it takes a lot of work to list all the partial functions of a package by check every item manually. Maybe we can work on this list later — when it’s turned out to be worth after some discussion.

Here is part of the list that I have tidied for several modules of the base package.

How to encourage all package contributors to obey the rule (see proposal #2)?

I don’t know, but I think there may be some other rules to obey when contributing packages. Maybe we can just add this into the list.

Obviously, the final perfect solution should be let the compiler to check the totality of functions, and automatically add HasCallStack for the ones which the compiler cannot confirm it’s totality. But this seems too far away from us, since we still doesn’t have dependent type haskell yet.

How to deal with recursive partial functions?

Since the HasCallStack constraint affects the performance (not only because of the runtime overhead, but also it’s influence on optimization strategy), It is best not to add HasCallStack on recursive functions.

In most of the cases, we can just check the input shallowly before everything start, just like how we deal with the non-recursive ones.

But in some other cases, we need to go deep to recognize the invalidity of the input. A trivial solution is just perform a deep check before everything start, but the checking phase seems expensive.

The best solution, IMHO, is to make the recursive part a total function, wrap the return value into Maybe or some similar things, and the partial version is just the total version combined fromJust. In this way, we avoid the single input checking phase and left the error awareing logic where it was before this translation.


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Henning Thielemann

On Sun, 16 Jun 2019, Simon Jakobi via Libraries wrote:

> I like the idea of marking partial functions with HasCallStack.
> But I'd like to point out that we also have class methods where some but not all implementations are partial,
> such as foldr1 in Foldable.
>
> Should methods like these also get a HasCallStack constraint?

I think only the partial implementations should get that constraint.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Haskell - Libraries mailing list
>  I think only the partial implementations should get that constraint.

Oops, I had the misconception that one couldn't add a HasCallStack constraint to method implementations.

I agree though!


Am So., 16. Juni 2019 um 18:26 Uhr schrieb Henning Thielemann <[hidden email]>:

On Sun, 16 Jun 2019, Simon Jakobi via Libraries wrote:

> I like the idea of marking partial functions with HasCallStack.
> But I'd like to point out that we also have class methods where some but not all implementations are partial,
> such as foldr1 in Foldable.
>
> Should methods like these also get a HasCallStack constraint?

I think only the partial implementations should get that constraint.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Henning Thielemann

On Sun, 16 Jun 2019, Simon Jakobi wrote:

> >  I think only the partial implementations should get that constraint.

> Oops, I had the misconception that one couldn't add a HasCallStack
> constraint to method implementations.

I think you have to declare:

instance Foldable [] where
    foldl1 = foldl1List

foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
foldl1List = ...

Would this work?
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Oliver Charles-3
Surely then the call stack is only foldr1, and not foldr1's callsite?

On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:

On Sun, 16 Jun 2019, Simon Jakobi wrote:

> >  I think only the partial implementations should get that constraint.

> Oops, I had the misconception that one couldn't add a HasCallStack
> constraint to method implementations.

I think you have to declare:

instance Foldable [] where
    foldl1 = foldl1List

foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
foldl1List = ...

Would this work?_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

LuoChen

I have just confirmed that we can specify HasCallStack separately for different instance of same typeclass, and here is the test code.


Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
Surely then the call stack is only foldr1, and not foldr1's callsite?

On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:

On Sun, 16 Jun 2019, Simon Jakobi wrote:

> >  I think only the partial implementations should get that constraint.

> Oops, I had the misconception that one couldn't add a HasCallStack
> constraint to method implementations.

I think you have to declare:

instance Foldable [] where
    foldl1 = foldl1List

foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
foldl1List = ...

Would this work?_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Edward Kmett-2
Re: your code. This is passing the callStack to each instance and dropping it on the floor for the cases where you ignore the constraint.

I’m starting to warm to the idea of putting HasCallStack constraints on the obviously partial combinators in base if we can demonstrate that the performance impact isn't bad in practice, and even really, to some extent if there is a somewhat middling impact on the performance of code that leans on these hard to debug combinators, so long as the performance of their more total siblings remains unaffected. The impact on the perceived debuggability of Haskell seems _likely_ to significantly outweigh the performance concerns. 

Heck, off of the HEAD of cabal, which we’re encouraging folks to build to play with the ghc 8.8.1 alpha, just today we ran into an issue where the very build system we are using spat out an oh so informative “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that didnt include any explicit bounds.

It seems worth implementing and measuring quite how bad the impact would be before pulling the trigger for good. The impact should be reasonable if the constraints only really live right at the outside of base though, then users can choose to propagate the constraint further outwards, just like they get to do with “error” today.

Slightly more controversially, MonadFail’s fail could be similarly modified with near zero user facing impact other than a bit of additional static info flowing to explicit callstacks to make it take a HasCallStack constraint. The code in existing instances would all still work. This one I think would need more benchmarking though, as recovering from a fail is a far “lighter” affair than catching an exception, but I’d be happy to raise it to the CLC as a question, especially if we can get benchmarks in hand.

—Edward


On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:

I have just confirmed that we can specify HasCallStack separately for different instance of same typeclass, and here is the test code.


Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
Surely then the call stack is only foldr1, and not foldr1's callsite?

On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:

On Sun, 16 Jun 2019, Simon Jakobi wrote:

> >  I think only the partial implementations should get that constraint.

> Oops, I had the misconception that one couldn't add a HasCallStack
> constraint to method implementations.

I think you have to declare:

instance Foldable [] where
    foldl1 = foldl1List

foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
foldl1List = ...

Would this work?_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

David Feuer
As I recall, the main things to watch out for, performance-wise, are recursive definitions. When we throw an error from a library function, we *typically* mean that the caller made a mistake. We don't want to build up the call stacks we'd need to debug the library function itself, unless we're actually doing so (in which case we can edit it in, of course).

On Mon, Jun 17, 2019, 1:09 PM Edward Kmett <[hidden email]> wrote:
Re: your code. This is passing the callStack to each instance and dropping it on the floor for the cases where you ignore the constraint.

I’m starting to warm to the idea of putting HasCallStack constraints on the obviously partial combinators in base if we can demonstrate that the performance impact isn't bad in practice, and even really, to some extent if there is a somewhat middling impact on the performance of code that leans on these hard to debug combinators, so long as the performance of their more total siblings remains unaffected. The impact on the perceived debuggability of Haskell seems _likely_ to significantly outweigh the performance concerns. 

Heck, off of the HEAD of cabal, which we’re encouraging folks to build to play with the ghc 8.8.1 alpha, just today we ran into an issue where the very build system we are using spat out an oh so informative “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that didnt include any explicit bounds.

It seems worth implementing and measuring quite how bad the impact would be before pulling the trigger for good. The impact should be reasonable if the constraints only really live right at the outside of base though, then users can choose to propagate the constraint further outwards, just like they get to do with “error” today.

Slightly more controversially, MonadFail’s fail could be similarly modified with near zero user facing impact other than a bit of additional static info flowing to explicit callstacks to make it take a HasCallStack constraint. The code in existing instances would all still work. This one I think would need more benchmarking though, as recovering from a fail is a far “lighter” affair than catching an exception, but I’d be happy to raise it to the CLC as a question, especially if we can get benchmarks in hand.

—Edward


On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:

I have just confirmed that we can specify HasCallStack separately for different instance of same typeclass, and here is the test code.


Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
Surely then the call stack is only foldr1, and not foldr1's callsite?

On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:

On Sun, 16 Jun 2019, Simon Jakobi wrote:

> >  I think only the partial implementations should get that constraint.

> Oops, I had the misconception that one couldn't add a HasCallStack
> constraint to method implementations.

I think you have to declare:

instance Foldable [] where
    foldl1 = foldl1List

foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
foldl1List = ...

Would this work?_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Eric Seidel-3
In reply to this post by Edward Kmett-2
I did some very basic benchmarks back when I implemented HasCallStack. What I remember finding back then was that there was a small overhead for non-recursive functions like `head` but a substantial overhead for recursive functions. (My benchmarks are available at https://github.com/gridaphobe/located-base/tree/master/bench.)

So if we want to add HasCallStack to partial `base` functions, I'd suggest extra care around recursive functions ((!!) comes to mind). Perhaps rewrite them to use an inner recursive loop that does not extend the CallStack (this might produce nicer stacktraces anyway).

Eric

On Mon, Jun 17, 2019, at 13:09, Edward Kmett wrote:

> Re: your code. This is passing the callStack to each instance and
> dropping it on the floor for the cases where you ignore the constraint.
>
> I’m starting to warm to the idea of putting HasCallStack constraints on
> the obviously partial combinators in base if we can demonstrate that
> the performance impact isn't bad in practice, and even really, to some
> extent if there is a somewhat middling impact on the performance of
> code that leans on these hard to debug combinators, so long as the
> performance of their more total siblings remains unaffected. The impact
> on the perceived debuggability of Haskell seems _likely_ to
> significantly outweigh the performance concerns.
>
> Heck, off of the HEAD of cabal, which we’re encouraging folks to build
> to play with the ghc 8.8.1 alpha, just today we ran into an issue where
> the very build system we are using spat out an oh so informative
> “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that
> didnt include any explicit bounds.
>
> It seems worth implementing and measuring quite how bad the impact
> would be before pulling the trigger for good. The impact should be
> reasonable if the constraints only really live right at the outside of
> base though, then users can choose to propagate the constraint further
> outwards, just like they get to do with “error” today.
>
> Slightly more controversially, MonadFail’s fail could be similarly
> modified with near zero user facing impact other than a bit of
> additional static info flowing to explicit callstacks to make it take a
> HasCallStack constraint. The code in existing instances would all still
> work. This one I think would need more benchmarking though, as
> recovering from a fail is a far “lighter” affair than catching an
> exception, but I’d be happy to raise it to the CLC as a question,
> especially if we can get benchmarks in hand.
>
> —Edward
>
>
> On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:
>
> > I have just confirmed that we can specify `HasCallStack` separately for different instance of same typeclass, and here <https://gist.github.com/luochen1990/bc556d0fc6a1355e5d1bd27a81114870> is the test code.
>
> >
> > Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
> >> Surely then the call stack is only foldr1, and not foldr1's callsite?
> >>
> >> On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:
> >>>
> >>>  On Sun, 16 Jun 2019, Simon Jakobi wrote:
> >>>
> >>>  > > I think only the partial implementations should get that constraint.
> >>>
> >>>  > Oops, I had the misconception that one couldn't add a HasCallStack
> >>>  > constraint to method implementations.
> >>>
> >>>  I think you have to declare:
> >>>
> >>>  instance Foldable [] where
> >>>  foldl1 = foldl1List
> >>>
> >>>  foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
> >>>  foldl1List = ...
> >>>
> >>>  Would this work?_______________________________________________
> >>>  Libraries mailing list
> >>> [hidden email]
> >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
> > _______________________________________________
> > Libraries mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Edward Kmett-2
I definitely do not want to push them through recursive functions! But e.g. foldr1 builds its error just once despite being recursive. We’d need to be somewhat careful there, but otherwise the cost should basically be comparable to a cons.

-Edward

> On Jun 17, 2019, at 10:02 PM, Eric Seidel <[hidden email]> wrote:
>
> I did some very basic benchmarks back when I implemented HasCallStack. What I remember finding back then was that there was a small overhead for non-recursive functions like `head` but a substantial overhead for recursive functions. (My benchmarks are available at https://github.com/gridaphobe/located-base/tree/master/bench.)
>
> So if we want to add HasCallStack to partial `base` functions, I'd suggest extra care around recursive functions ((!!) comes to mind). Perhaps rewrite them to use an inner recursive loop that does not extend the CallStack (this might produce nicer stacktraces anyway).
>
> Eric
>
>> On Mon, Jun 17, 2019, at 13:09, Edward Kmett wrote:
>> Re: your code. This is passing the callStack to each instance and
>> dropping it on the floor for the cases where you ignore the constraint.
>>
>> I’m starting to warm to the idea of putting HasCallStack constraints on
>> the obviously partial combinators in base if we can demonstrate that
>> the performance impact isn't bad in practice, and even really, to some
>> extent if there is a somewhat middling impact on the performance of
>> code that leans on these hard to debug combinators, so long as the
>> performance of their more total siblings remains unaffected. The impact
>> on the perceived debuggability of Haskell seems _likely_ to
>> significantly outweigh the performance concerns.
>>
>> Heck, off of the HEAD of cabal, which we’re encouraging folks to build
>> to play with the ghc 8.8.1 alpha, just today we ran into an issue where
>> the very build system we are using spat out an oh so informative
>> “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that
>> didnt include any explicit bounds.
>>
>> It seems worth implementing and measuring quite how bad the impact
>> would be before pulling the trigger for good. The impact should be
>> reasonable if the constraints only really live right at the outside of
>> base though, then users can choose to propagate the constraint further
>> outwards, just like they get to do with “error” today.
>>
>> Slightly more controversially, MonadFail’s fail could be similarly
>> modified with near zero user facing impact other than a bit of
>> additional static info flowing to explicit callstacks to make it take a
>> HasCallStack constraint. The code in existing instances would all still
>> work. This one I think would need more benchmarking though, as
>> recovering from a fail is a far “lighter” affair than catching an
>> exception, but I’d be happy to raise it to the CLC as a question,
>> especially if we can get benchmarks in hand.
>>
>> —Edward
>>
>>
>>> On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:
>>>
>>> I have just confirmed that we can specify `HasCallStack` separately for different instance of same typeclass, and here <https://gist.github.com/luochen1990/bc556d0fc6a1355e5d1bd27a81114870> is the test code.
>>
>>>
>>> Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
>>>> Surely then the call stack is only foldr1, and not foldr1's callsite?
>>>>
>>>>> On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:
>>>>>
>>>>> On Sun, 16 Jun 2019, Simon Jakobi wrote:
>>>>>
>>>>>>> I think only the partial implementations should get that constraint.
>>>>>
>>>>>> Oops, I had the misconception that one couldn't add a HasCallStack
>>>>>> constraint to method implementations.
>>>>>
>>>>> I think you have to declare:
>>>>>
>>>>> instance Foldable [] where
>>>>> foldl1 = foldl1List
>>>>>
>>>>> foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
>>>>> foldl1List = ...
>>>>>
>>>>> Would this work?_______________________________________________
>>>>> Libraries mailing list
>>>>> [hidden email]
>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>> _______________________________________________
>>> Libraries mailing list
>>> [hidden email]
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

LuoChen
In reply to this post by David Feuer
@david It seem that we doesn't have such a mechanism yet. AFAIK with `HasCallStack` we can only build a call stack from the deepest bottom.

Is it possible to build a call stack from middle of the call chain to the top, without containing the deepest parts?
e.g. f1 -> f2 -> f3 -> error  (f -> g means f call g) , is it possible to build a call stack only contains f1 and f2, without f3 and error?

David Feuer <[hidden email]> 于2019年6月18日周二 上午3:57写道:
As I recall, the main things to watch out for, performance-wise, are recursive definitions. When we throw an error from a library function, we *typically* mean that the caller made a mistake. We don't want to build up the call stacks we'd need to debug the library function itself, unless we're actually doing so (in which case we can edit it in, of course).

On Mon, Jun 17, 2019, 1:09 PM Edward Kmett <[hidden email]> wrote:
Re: your code. This is passing the callStack to each instance and dropping it on the floor for the cases where you ignore the constraint.

I’m starting to warm to the idea of putting HasCallStack constraints on the obviously partial combinators in base if we can demonstrate that the performance impact isn't bad in practice, and even really, to some extent if there is a somewhat middling impact on the performance of code that leans on these hard to debug combinators, so long as the performance of their more total siblings remains unaffected. The impact on the perceived debuggability of Haskell seems _likely_ to significantly outweigh the performance concerns. 

Heck, off of the HEAD of cabal, which we’re encouraging folks to build to play with the ghc 8.8.1 alpha, just today we ran into an issue where the very build system we are using spat out an oh so informative “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that didnt include any explicit bounds.

It seems worth implementing and measuring quite how bad the impact would be before pulling the trigger for good. The impact should be reasonable if the constraints only really live right at the outside of base though, then users can choose to propagate the constraint further outwards, just like they get to do with “error” today.

Slightly more controversially, MonadFail’s fail could be similarly modified with near zero user facing impact other than a bit of additional static info flowing to explicit callstacks to make it take a HasCallStack constraint. The code in existing instances would all still work. This one I think would need more benchmarking though, as recovering from a fail is a far “lighter” affair than catching an exception, but I’d be happy to raise it to the CLC as a question, especially if we can get benchmarks in hand.

—Edward


On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:

I have just confirmed that we can specify HasCallStack separately for different instance of same typeclass, and here is the test code.


Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
Surely then the call stack is only foldr1, and not foldr1's callsite?

On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:

On Sun, 16 Jun 2019, Simon Jakobi wrote:

> >  I think only the partial implementations should get that constraint.

> Oops, I had the misconception that one couldn't add a HasCallStack
> constraint to method implementations.

I think you have to declare:

instance Foldable [] where
    foldl1 = foldl1List

foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
foldl1List = ...

Would this work?_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Eric Seidel-3
You can use withFrozenCallstack to avoid adding new entries to the stacktrace (but you still need HasCallstack constraints all the way down to error, of course). 

http://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-Stack.html#v:withFrozenCallStack

Sent from my iPhone

On Jun 19, 2019, at 00:41, LuoChen <[hidden email]> wrote:

@david It seem that we doesn't have such a mechanism yet. AFAIK with `HasCallStack` we can only build a call stack from the deepest bottom.

Is it possible to build a call stack from middle of the call chain to the top, without containing the deepest parts?
e.g. f1 -> f2 -> f3 -> error  (f -> g means f call g) , is it possible to build a call stack only contains f1 and f2, without f3 and error?

David Feuer <[hidden email]> 于2019年6月18日周二 上午3:57写道:
As I recall, the main things to watch out for, performance-wise, are recursive definitions. When we throw an error from a library function, we *typically* mean that the caller made a mistake. We don't want to build up the call stacks we'd need to debug the library function itself, unless we're actually doing so (in which case we can edit it in, of course).

On Mon, Jun 17, 2019, 1:09 PM Edward Kmett <[hidden email]> wrote:
Re: your code. This is passing the callStack to each instance and dropping it on the floor for the cases where you ignore the constraint.

I’m starting to warm to the idea of putting HasCallStack constraints on the obviously partial combinators in base if we can demonstrate that the performance impact isn't bad in practice, and even really, to some extent if there is a somewhat middling impact on the performance of code that leans on these hard to debug combinators, so long as the performance of their more total siblings remains unaffected. The impact on the perceived debuggability of Haskell seems _likely_ to significantly outweigh the performance concerns. 

Heck, off of the HEAD of cabal, which we’re encouraging folks to build to play with the ghc 8.8.1 alpha, just today we ran into an issue where the very build system we are using spat out an oh so informative “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that didnt include any explicit bounds.

It seems worth implementing and measuring quite how bad the impact would be before pulling the trigger for good. The impact should be reasonable if the constraints only really live right at the outside of base though, then users can choose to propagate the constraint further outwards, just like they get to do with “error” today.

Slightly more controversially, MonadFail’s fail could be similarly modified with near zero user facing impact other than a bit of additional static info flowing to explicit callstacks to make it take a HasCallStack constraint. The code in existing instances would all still work. This one I think would need more benchmarking though, as recovering from a fail is a far “lighter” affair than catching an exception, but I’d be happy to raise it to the CLC as a question, especially if we can get benchmarks in hand.

—Edward


On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:

I have just confirmed that we can specify HasCallStack separately for different instance of same typeclass, and here is the test code.


Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
Surely then the call stack is only foldr1, and not foldr1's callsite?

On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:

On Sun, 16 Jun 2019, Simon Jakobi wrote:

> >  I think only the partial implementations should get that constraint.

> Oops, I had the misconception that one couldn't add a HasCallStack
> constraint to method implementations.

I think you have to declare:

instance Foldable [] where
    foldl1 = foldl1List

foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
foldl1List = ...

Would this work?_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

LuoChen
@eric Thanks, this withFrozenCallstack  works as expected! Maybe we can provide a compiler option to automatically insert `withFrozenCallstack` somewhere when importing other packages, so that we can focus on current working package's call stack, just like david said.

Eric Seidel <[hidden email]> 于2019年6月19日周三 下午7:25写道:
You can use withFrozenCallstack to avoid adding new entries to the stacktrace (but you still need HasCallstack constraints all the way down to error, of course). 

http://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-Stack.html#v:withFrozenCallStack

Sent from my iPhone

On Jun 19, 2019, at 00:41, LuoChen <[hidden email]> wrote:

@david It seem that we doesn't have such a mechanism yet. AFAIK with `HasCallStack` we can only build a call stack from the deepest bottom.

Is it possible to build a call stack from middle of the call chain to the top, without containing the deepest parts?
e.g. f1 -> f2 -> f3 -> error  (f -> g means f call g) , is it possible to build a call stack only contains f1 and f2, without f3 and error?

David Feuer <[hidden email]> 于2019年6月18日周二 上午3:57写道:
As I recall, the main things to watch out for, performance-wise, are recursive definitions. When we throw an error from a library function, we *typically* mean that the caller made a mistake. We don't want to build up the call stacks we'd need to debug the library function itself, unless we're actually doing so (in which case we can edit it in, of course).

On Mon, Jun 17, 2019, 1:09 PM Edward Kmett <[hidden email]> wrote:
Re: your code. This is passing the callStack to each instance and dropping it on the floor for the cases where you ignore the constraint.

I’m starting to warm to the idea of putting HasCallStack constraints on the obviously partial combinators in base if we can demonstrate that the performance impact isn't bad in practice, and even really, to some extent if there is a somewhat middling impact on the performance of code that leans on these hard to debug combinators, so long as the performance of their more total siblings remains unaffected. The impact on the perceived debuggability of Haskell seems _likely_ to significantly outweigh the performance concerns. 

Heck, off of the HEAD of cabal, which we’re encouraging folks to build to play with the ghc 8.8.1 alpha, just today we ran into an issue where the very build system we are using spat out an oh so informative “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that didnt include any explicit bounds.

It seems worth implementing and measuring quite how bad the impact would be before pulling the trigger for good. The impact should be reasonable if the constraints only really live right at the outside of base though, then users can choose to propagate the constraint further outwards, just like they get to do with “error” today.

Slightly more controversially, MonadFail’s fail could be similarly modified with near zero user facing impact other than a bit of additional static info flowing to explicit callstacks to make it take a HasCallStack constraint. The code in existing instances would all still work. This one I think would need more benchmarking though, as recovering from a fail is a far “lighter” affair than catching an exception, but I’d be happy to raise it to the CLC as a question, especially if we can get benchmarks in hand.

—Edward


On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:

I have just confirmed that we can specify HasCallStack separately for different instance of same typeclass, and here is the test code.


Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
Surely then the call stack is only foldr1, and not foldr1's callsite?

On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:

On Sun, 16 Jun 2019, Simon Jakobi wrote:

> >  I think only the partial implementations should get that constraint.

> Oops, I had the misconception that one couldn't add a HasCallStack
> constraint to method implementations.

I think you have to declare:

instance Foldable [] where
    foldl1 = foldl1List

foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
foldl1List = ...

Would this work?_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Matthew Pickering
I find this thread a bit concerning. In my opinion the current best
way to get call stacks on exceptions is with `-xc`. Adding runtime
overhead to every user program is not acceptable.

`HasCallStack` is useful in libraries such as shake or hspec which use
them to good effect to provide callstacks to users on exceptions
controlled by the libraries.

There is an argument that it would be good to disentangle `-xc` from
`prof` but you would still have to compile `base` in this way which
instruments these partial functions (as a user can not do it herself).
I'm not too sure on the details but I don't think -xc uses any
information from the profiling header so the fact it's connected with
-prof seems more like convenience than necessity.

Matt

On Wed, Jun 19, 2019 at 3:48 PM LuoChen <[hidden email]> wrote:

>
> @eric Thanks, this withFrozenCallstack  works as expected! Maybe we can provide a compiler option to automatically insert `withFrozenCallstack` somewhere when importing other packages, so that we can focus on current working package's call stack, just like david said.
>
> Eric Seidel <[hidden email]> 于2019年6月19日周三 下午7:25写道:
>>
>> You can use withFrozenCallstack to avoid adding new entries to the stacktrace (but you still need HasCallstack constraints all the way down to error, of course).
>>
>> http://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-Stack.html#v:withFrozenCallStack
>>
>> Sent from my iPhone
>>
>> On Jun 19, 2019, at 00:41, LuoChen <[hidden email]> wrote:
>>
>> @david It seem that we doesn't have such a mechanism yet. AFAIK with `HasCallStack` we can only build a call stack from the deepest bottom.
>>
>> Is it possible to build a call stack from middle of the call chain to the top, without containing the deepest parts?
>> e.g. f1 -> f2 -> f3 -> error  (f -> g means f call g) , is it possible to build a call stack only contains f1 and f2, without f3 and error?
>>
>> David Feuer <[hidden email]> 于2019年6月18日周二 上午3:57写道:
>>>
>>> As I recall, the main things to watch out for, performance-wise, are recursive definitions. When we throw an error from a library function, we *typically* mean that the caller made a mistake. We don't want to build up the call stacks we'd need to debug the library function itself, unless we're actually doing so (in which case we can edit it in, of course).
>>>
>>> On Mon, Jun 17, 2019, 1:09 PM Edward Kmett <[hidden email]> wrote:
>>>>
>>>> Re: your code. This is passing the callStack to each instance and dropping it on the floor for the cases where you ignore the constraint.
>>>>
>>>> I’m starting to warm to the idea of putting HasCallStack constraints on the obviously partial combinators in base if we can demonstrate that the performance impact isn't bad in practice, and even really, to some extent if there is a somewhat middling impact on the performance of code that leans on these hard to debug combinators, so long as the performance of their more total siblings remains unaffected. The impact on the perceived debuggability of Haskell seems _likely_ to significantly outweigh the performance concerns.
>>>>
>>>> Heck, off of the HEAD of cabal, which we’re encouraging folks to build to play with the ghc 8.8.1 alpha, just today we ran into an issue where the very build system we are using spat out an oh so informative “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that didnt include any explicit bounds.
>>>>
>>>> It seems worth implementing and measuring quite how bad the impact would be before pulling the trigger for good. The impact should be reasonable if the constraints only really live right at the outside of base though, then users can choose to propagate the constraint further outwards, just like they get to do with “error” today.
>>>>
>>>> Slightly more controversially, MonadFail’s fail could be similarly modified with near zero user facing impact other than a bit of additional static info flowing to explicit callstacks to make it take a HasCallStack constraint. The code in existing instances would all still work. This one I think would need more benchmarking though, as recovering from a fail is a far “lighter” affair than catching an exception, but I’d be happy to raise it to the CLC as a question, especially if we can get benchmarks in hand.
>>>>
>>>> —Edward
>>>>
>>>>
>>>> On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:
>>>>
>>>> I have just confirmed that we can specify HasCallStack separately for different instance of same typeclass, and here is the test code.
>>>>
>>>>
>>>> Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
>>>>>
>>>>> Surely then the call stack is only foldr1, and not foldr1's callsite?
>>>>>
>>>>> On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>> On Sun, 16 Jun 2019, Simon Jakobi wrote:
>>>>>>
>>>>>> > >  I think only the partial implementations should get that constraint.
>>>>>>
>>>>>> > Oops, I had the misconception that one couldn't add a HasCallStack
>>>>>> > constraint to method implementations.
>>>>>>
>>>>>> I think you have to declare:
>>>>>>
>>>>>> instance Foldable [] where
>>>>>>     foldl1 = foldl1List
>>>>>>
>>>>>> foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
>>>>>> foldl1List = ...
>>>>>>
>>>>>> Would this work?_______________________________________________
>>>>>> Libraries mailing list
>>>>>> [hidden email]
>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>>>
>>>> _______________________________________________
>>>> Libraries mailing list
>>>> [hidden email]
>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>>>
>>>> _______________________________________________
>>>> Libraries mailing list
>>>> [hidden email]
>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Eric Seidel-3
I’m not sure I understand your objection. You seem to be ok with libraries adding HasCallstack constraints for partial functions they control, but the proposal here is to do precisely that for base. The performance concern I completely understand, but there seems to be something else going on too.

Sent from my iPhone

> On Jun 19, 2019, at 11:11, Matthew Pickering <[hidden email]> wrote:
>
> I find this thread a bit concerning. In my opinion the current best
> way to get call stacks on exceptions is with `-xc`. Adding runtime
> overhead to every user program is not acceptable.
>
> `HasCallStack` is useful in libraries such as shake or hspec which use
> them to good effect to provide callstacks to users on exceptions
> controlled by the libraries.
>
> There is an argument that it would be good to disentangle `-xc` from
> `prof` but you would still have to compile `base` in this way which
> instruments these partial functions (as a user can not do it herself).
> I'm not too sure on the details but I don't think -xc uses any
> information from the profiling header so the fact it's connected with
> -prof seems more like convenience than necessity.
>
> Matt
>
>> On Wed, Jun 19, 2019 at 3:48 PM LuoChen <[hidden email]> wrote:
>>
>> @eric Thanks, this withFrozenCallstack  works as expected! Maybe we can provide a compiler option to automatically insert `withFrozenCallstack` somewhere when importing other packages, so that we can focus on current working package's call stack, just like david said.
>>
>> Eric Seidel <[hidden email]> 于2019年6月19日周三 下午7:25写道:
>>>
>>> You can use withFrozenCallstack to avoid adding new entries to the stacktrace (but you still need HasCallstack constraints all the way down to error, of course).
>>>
>>> http://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-Stack.html#v:withFrozenCallStack
>>>
>>> Sent from my iPhone
>>>
>>> On Jun 19, 2019, at 00:41, LuoChen <[hidden email]> wrote:
>>>
>>> @david It seem that we doesn't have such a mechanism yet. AFAIK with `HasCallStack` we can only build a call stack from the deepest bottom.
>>>
>>> Is it possible to build a call stack from middle of the call chain to the top, without containing the deepest parts?
>>> e.g. f1 -> f2 -> f3 -> error  (f -> g means f call g) , is it possible to build a call stack only contains f1 and f2, without f3 and error?
>>>
>>> David Feuer <[hidden email]> 于2019年6月18日周二 上午3:57写道:
>>>>
>>>> As I recall, the main things to watch out for, performance-wise, are recursive definitions. When we throw an error from a library function, we *typically* mean that the caller made a mistake. We don't want to build up the call stacks we'd need to debug the library function itself, unless we're actually doing so (in which case we can edit it in, of course).
>>>>
>>>>> On Mon, Jun 17, 2019, 1:09 PM Edward Kmett <[hidden email]> wrote:
>>>>>
>>>>> Re: your code. This is passing the callStack to each instance and dropping it on the floor for the cases where you ignore the constraint.
>>>>>
>>>>> I’m starting to warm to the idea of putting HasCallStack constraints on the obviously partial combinators in base if we can demonstrate that the performance impact isn't bad in practice, and even really, to some extent if there is a somewhat middling impact on the performance of code that leans on these hard to debug combinators, so long as the performance of their more total siblings remains unaffected. The impact on the perceived debuggability of Haskell seems _likely_ to significantly outweigh the performance concerns.
>>>>>
>>>>> Heck, off of the HEAD of cabal, which we’re encouraging folks to build to play with the ghc 8.8.1 alpha, just today we ran into an issue where the very build system we are using spat out an oh so informative “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that didnt include any explicit bounds.
>>>>>
>>>>> It seems worth implementing and measuring quite how bad the impact would be before pulling the trigger for good. The impact should be reasonable if the constraints only really live right at the outside of base though, then users can choose to propagate the constraint further outwards, just like they get to do with “error” today.
>>>>>
>>>>> Slightly more controversially, MonadFail’s fail could be similarly modified with near zero user facing impact other than a bit of additional static info flowing to explicit callstacks to make it take a HasCallStack constraint. The code in existing instances would all still work. This one I think would need more benchmarking though, as recovering from a fail is a far “lighter” affair than catching an exception, but I’d be happy to raise it to the CLC as a question, especially if we can get benchmarks in hand.
>>>>>
>>>>> —Edward
>>>>>
>>>>>
>>>>> On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:
>>>>>
>>>>> I have just confirmed that we can specify HasCallStack separately for different instance of same typeclass, and here is the test code.
>>>>>
>>>>>
>>>>> Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
>>>>>>
>>>>>> Surely then the call stack is only foldr1, and not foldr1's callsite?
>>>>>>
>>>>>>> On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:
>>>>>>>
>>>>>>>
>>>>>>> On Sun, 16 Jun 2019, Simon Jakobi wrote:
>>>>>>>
>>>>>>>>> I think only the partial implementations should get that constraint.
>>>>>>>
>>>>>>>> Oops, I had the misconception that one couldn't add a HasCallStack
>>>>>>>> constraint to method implementations.
>>>>>>>
>>>>>>> I think you have to declare:
>>>>>>>
>>>>>>> instance Foldable [] where
>>>>>>>    foldl1 = foldl1List
>>>>>>>
>>>>>>> foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
>>>>>>> foldl1List = ...
>>>>>>>
>>>>>>> Would this work?_______________________________________________
>>>>>>> Libraries mailing list
>>>>>>> [hidden email]
>>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>>>>
>>>>> _______________________________________________
>>>>> Libraries mailing list
>>>>> [hidden email]
>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>>>>
>>>>> _______________________________________________
>>>>> Libraries mailing list
>>>>> [hidden email]
>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>>
>>> _______________________________________________
>>> Libraries mailing list
>>> [hidden email]
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

Carter Schonwald
In reply to this post by Matthew Pickering
agreed, the -xc flag is an underused debugging tool, and making that easier to work with / front and center is probably a high impact approach

On Wed, Jun 19, 2019 at 11:11 AM Matthew Pickering <[hidden email]> wrote:
I find this thread a bit concerning. In my opinion the current best
way to get call stacks on exceptions is with `-xc`. Adding runtime
overhead to every user program is not acceptable.

`HasCallStack` is useful in libraries such as shake or hspec which use
them to good effect to provide callstacks to users on exceptions
controlled by the libraries.

There is an argument that it would be good to disentangle `-xc` from
`prof` but you would still have to compile `base` in this way which
instruments these partial functions (as a user can not do it herself).
I'm not too sure on the details but I don't think -xc uses any
information from the profiling header so the fact it's connected with
-prof seems more like convenience than necessity.

Matt

On Wed, Jun 19, 2019 at 3:48 PM LuoChen <[hidden email]> wrote:
>
> @eric Thanks, this withFrozenCallstack  works as expected! Maybe we can provide a compiler option to automatically insert `withFrozenCallstack` somewhere when importing other packages, so that we can focus on current working package's call stack, just like david said.
>
> Eric Seidel <[hidden email]> 于2019年6月19日周三 下午7:25写道:
>>
>> You can use withFrozenCallstack to avoid adding new entries to the stacktrace (but you still need HasCallstack constraints all the way down to error, of course).
>>
>> http://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-Stack.html#v:withFrozenCallStack
>>
>> Sent from my iPhone
>>
>> On Jun 19, 2019, at 00:41, LuoChen <[hidden email]> wrote:
>>
>> @david It seem that we doesn't have such a mechanism yet. AFAIK with `HasCallStack` we can only build a call stack from the deepest bottom.
>>
>> Is it possible to build a call stack from middle of the call chain to the top, without containing the deepest parts?
>> e.g. f1 -> f2 -> f3 -> error  (f -> g means f call g) , is it possible to build a call stack only contains f1 and f2, without f3 and error?
>>
>> David Feuer <[hidden email]> 于2019年6月18日周二 上午3:57写道:
>>>
>>> As I recall, the main things to watch out for, performance-wise, are recursive definitions. When we throw an error from a library function, we *typically* mean that the caller made a mistake. We don't want to build up the call stacks we'd need to debug the library function itself, unless we're actually doing so (in which case we can edit it in, of course).
>>>
>>> On Mon, Jun 17, 2019, 1:09 PM Edward Kmett <[hidden email]> wrote:
>>>>
>>>> Re: your code. This is passing the callStack to each instance and dropping it on the floor for the cases where you ignore the constraint.
>>>>
>>>> I’m starting to warm to the idea of putting HasCallStack constraints on the obviously partial combinators in base if we can demonstrate that the performance impact isn't bad in practice, and even really, to some extent if there is a somewhat middling impact on the performance of code that leans on these hard to debug combinators, so long as the performance of their more total siblings remains unaffected. The impact on the perceived debuggability of Haskell seems _likely_ to significantly outweigh the performance concerns.
>>>>
>>>> Heck, off of the HEAD of cabal, which we’re encouraging folks to build to play with the ghc 8.8.1 alpha, just today we ran into an issue where the very build system we are using spat out an oh so informative “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that didnt include any explicit bounds.
>>>>
>>>> It seems worth implementing and measuring quite how bad the impact would be before pulling the trigger for good. The impact should be reasonable if the constraints only really live right at the outside of base though, then users can choose to propagate the constraint further outwards, just like they get to do with “error” today.
>>>>
>>>> Slightly more controversially, MonadFail’s fail could be similarly modified with near zero user facing impact other than a bit of additional static info flowing to explicit callstacks to make it take a HasCallStack constraint. The code in existing instances would all still work. This one I think would need more benchmarking though, as recovering from a fail is a far “lighter” affair than catching an exception, but I’d be happy to raise it to the CLC as a question, especially if we can get benchmarks in hand.
>>>>
>>>> —Edward
>>>>
>>>>
>>>> On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:
>>>>
>>>> I have just confirmed that we can specify HasCallStack separately for different instance of same typeclass, and here is the test code.
>>>>
>>>>
>>>> Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
>>>>>
>>>>> Surely then the call stack is only foldr1, and not foldr1's callsite?
>>>>>
>>>>> On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>> On Sun, 16 Jun 2019, Simon Jakobi wrote:
>>>>>>
>>>>>> > >  I think only the partial implementations should get that constraint.
>>>>>>
>>>>>> > Oops, I had the misconception that one couldn't add a HasCallStack
>>>>>> > constraint to method implementations.
>>>>>>
>>>>>> I think you have to declare:
>>>>>>
>>>>>> instance Foldable [] where
>>>>>>     foldl1 = foldl1List
>>>>>>
>>>>>> foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
>>>>>> foldl1List = ...
>>>>>>
>>>>>> Would this work?_______________________________________________
>>>>>> Libraries mailing list
>>>>>> [hidden email]
>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>>>
>>>> _______________________________________________
>>>> Libraries mailing list
>>>> [hidden email]
>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>>>
>>>> _______________________________________________
>>>> Libraries mailing list
>>>> [hidden email]
>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: add HasCallStack for all partial functions in base

LuoChen
Thanks for all your response very much, I greatly appreciate that my immature idea was discussed seriously. I like this community and hope to do more, but now I have several questions and I cannot answer them by myself since my knowledge is limited.

1. Is `-xc` as good as `HasCallStack` in debugging experience, without considering performance issue?
2. Will `withFrozenCallStack` cancel the adverse effect of performance caused by `HasCallStack`?
3. Is there a test set for performance so we can measure the performance lost after the change?

For the 1st question, AFAIK there is indeed some differences between the two stack tracing solutions:
First, the stack trace generated by `-xc` has no line number and column number of the call site.
Second, when we run a long running program, such as a web service, it is not easy to reproduce the issue, most of the time, you have no chance to recompile or restart your program to debug, all you can rely on is the printed logs, this makes `-xc` not as useful as `HasCallStack`. 
Third, the `HasCallStack` solution is much more friendly for learners, you don't need to know any tricks to get a stack trace printed, this reason seems ridiculous but IMHO it indeed affects the learning curve of Haskell.

Carter Schonwald <[hidden email]> 于2019年6月22日周六 上午2:42写道:
agreed, the -xc flag is an underused debugging tool, and making that easier to work with / front and center is probably a high impact approach

On Wed, Jun 19, 2019 at 11:11 AM Matthew Pickering <[hidden email]> wrote:
I find this thread a bit concerning. In my opinion the current best
way to get call stacks on exceptions is with `-xc`. Adding runtime
overhead to every user program is not acceptable.

`HasCallStack` is useful in libraries such as shake or hspec which use
them to good effect to provide callstacks to users on exceptions
controlled by the libraries.

There is an argument that it would be good to disentangle `-xc` from
`prof` but you would still have to compile `base` in this way which
instruments these partial functions (as a user can not do it herself).
I'm not too sure on the details but I don't think -xc uses any
information from the profiling header so the fact it's connected with
-prof seems more like convenience than necessity.

Matt

On Wed, Jun 19, 2019 at 3:48 PM LuoChen <[hidden email]> wrote:
>
> @eric Thanks, this withFrozenCallstack  works as expected! Maybe we can provide a compiler option to automatically insert `withFrozenCallstack` somewhere when importing other packages, so that we can focus on current working package's call stack, just like david said.
>
> Eric Seidel <[hidden email]> 于2019年6月19日周三 下午7:25写道:
>>
>> You can use withFrozenCallstack to avoid adding new entries to the stacktrace (but you still need HasCallstack constraints all the way down to error, of course).
>>
>> http://hackage.haskell.org/package/base-4.12.0.0/docs/GHC-Stack.html#v:withFrozenCallStack
>>
>> Sent from my iPhone
>>
>> On Jun 19, 2019, at 00:41, LuoChen <[hidden email]> wrote:
>>
>> @david It seem that we doesn't have such a mechanism yet. AFAIK with `HasCallStack` we can only build a call stack from the deepest bottom.
>>
>> Is it possible to build a call stack from middle of the call chain to the top, without containing the deepest parts?
>> e.g. f1 -> f2 -> f3 -> error  (f -> g means f call g) , is it possible to build a call stack only contains f1 and f2, without f3 and error?
>>
>> David Feuer <[hidden email]> 于2019年6月18日周二 上午3:57写道:
>>>
>>> As I recall, the main things to watch out for, performance-wise, are recursive definitions. When we throw an error from a library function, we *typically* mean that the caller made a mistake. We don't want to build up the call stacks we'd need to debug the library function itself, unless we're actually doing so (in which case we can edit it in, of course).
>>>
>>> On Mon, Jun 17, 2019, 1:09 PM Edward Kmett <[hidden email]> wrote:
>>>>
>>>> Re: your code. This is passing the callStack to each instance and dropping it on the floor for the cases where you ignore the constraint.
>>>>
>>>> I’m starting to warm to the idea of putting HasCallStack constraints on the obviously partial combinators in base if we can demonstrate that the performance impact isn't bad in practice, and even really, to some extent if there is a somewhat middling impact on the performance of code that leans on these hard to debug combinators, so long as the performance of their more total siblings remains unaffected. The impact on the perceived debuggability of Haskell seems _likely_ to significantly outweigh the performance concerns.
>>>>
>>>> Heck, off of the HEAD of cabal, which we’re encouraging folks to build to play with the ghc 8.8.1 alpha, just today we ran into an issue where the very build system we are using spat out an oh so informative “Prelude.foldr1: Empty list” when using a pkgconfig-depends stanza that didnt include any explicit bounds.
>>>>
>>>> It seems worth implementing and measuring quite how bad the impact would be before pulling the trigger for good. The impact should be reasonable if the constraints only really live right at the outside of base though, then users can choose to propagate the constraint further outwards, just like they get to do with “error” today.
>>>>
>>>> Slightly more controversially, MonadFail’s fail could be similarly modified with near zero user facing impact other than a bit of additional static info flowing to explicit callstacks to make it take a HasCallStack constraint. The code in existing instances would all still work. This one I think would need more benchmarking though, as recovering from a fail is a far “lighter” affair than catching an exception, but I’d be happy to raise it to the CLC as a question, especially if we can get benchmarks in hand.
>>>>
>>>> —Edward
>>>>
>>>>
>>>> On Jun 17, 2019, at 4:06 PM, LuoChen <[hidden email]> wrote:
>>>>
>>>> I have just confirmed that we can specify HasCallStack separately for different instance of same typeclass, and here is the test code.
>>>>
>>>>
>>>> Oliver Charles <[hidden email]> 于2019年6月17日周一 上午3:02写道:
>>>>>
>>>>> Surely then the call stack is only foldr1, and not foldr1's callsite?
>>>>>
>>>>> On Sun, 16 Jun 2019, 8:27 pm Henning Thielemann, <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>> On Sun, 16 Jun 2019, Simon Jakobi wrote:
>>>>>>
>>>>>> > >  I think only the partial implementations should get that constraint.
>>>>>>
>>>>>> > Oops, I had the misconception that one couldn't add a HasCallStack
>>>>>> > constraint to method implementations.
>>>>>>
>>>>>> I think you have to declare:
>>>>>>
>>>>>> instance Foldable [] where
>>>>>>     foldl1 = foldl1List
>>>>>>
>>>>>> foldl1List :: HasCallStack => (a->a->a) -> [a] -> a
>>>>>> foldl1List = ...
>>>>>>
>>>>>> Would this work?_______________________________________________
>>>>>> Libraries mailing list
>>>>>> [hidden email]
>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>>>
>>>> _______________________________________________
>>>> Libraries mailing list
>>>> [hidden email]
>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>>>
>>>> _______________________________________________
>>>> Libraries mailing list
>>>> [hidden email]
>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries