comment on this debugging trick?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

comment on this debugging trick?

Dennis Raddle
When I design my code, I am aware of the properties I expect my internal data representations to have, or I want input data to conform to my expectations. For example, right now I'm writing code that reads MusicXML. MusicXML is a crazy language, way too complicated for what it does, and the music typesetters that export MusicXML all do their own idiosyncratic things with it. I'm only reading the output of one typesetter, Sibelius, and although I don't know the internals of Sibelius, I can make some assumptions about what it's going to produce by a few examples, plus my application doesn't use the full range of MusicXML. I don't need to handle every case, is what I'm getting at.

However, if I should have been wrong about my assumptions when I wrote the code, I don't want my program to behave erratically. I want to find out exactly what went wrong as soon as possible.

Previously, I was including a lot of exception throwing, with each exception having a unique message describing what had happened, and in particular describing where in the code it is located so I can find the problem spot

If I throw generic error messages like "Error: problem parsing" and I have many of those located in the code, I need the debugger to find it. Same if I use things like "fromJust" or "head" 

I haven't had good experiences using the debugger. Maybe that's my fault, but I like that I can locate my unique exceptions and that they are descriptive.

But a month ago I decided this system is pretty ugly. It bloats the code a lot, and requires writing a lot of messages for situations that will probably never occur.

I am trying a different method now. I write the code so that surprising behavior will result in a case or pattern exhaustion, which produces a run time message with a file name and line number. I'm not sure why ghc gives the location for a case exhaustion, but not something like "head".

For example, instead of using head xs I can write

x = case xs of {y:_ -> y}

This is a little bloat but not too bad. Most of the time I'm actually structuring cases and patterns, and it actually helps me to simplify code to think of how to write it with the fewest cases and so that a case exhaustion will indicate something pretty specific.

Any comments welcome.


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: comment on this debugging trick?

Jeffrey Brown
Elliot Cameron, very smart guy, once advised me to make illegal state invalid. Jake Brownson, another, recommended trying to use Maybes and Eithers instead of throwing exceptions. I'm having trouble coming up with examples but I think they're both helpful.

That said, your trick of replacing, for instance, head with a case statement seems good to me ...

On Fri, Nov 27, 2015 at 5:56 AM, Dennis Raddle <[hidden email]> wrote:
When I design my code, I am aware of the properties I expect my internal data representations to have, or I want input data to conform to my expectations. For example, right now I'm writing code that reads MusicXML. MusicXML is a crazy language, way too complicated for what it does, and the music typesetters that export MusicXML all do their own idiosyncratic things with it. I'm only reading the output of one typesetter, Sibelius, and although I don't know the internals of Sibelius, I can make some assumptions about what it's going to produce by a few examples, plus my application doesn't use the full range of MusicXML. I don't need to handle every case, is what I'm getting at.

However, if I should have been wrong about my assumptions when I wrote the code, I don't want my program to behave erratically. I want to find out exactly what went wrong as soon as possible.

Previously, I was including a lot of exception throwing, with each exception having a unique message describing what had happened, and in particular describing where in the code it is located so I can find the problem spot

If I throw generic error messages like "Error: problem parsing" and I have many of those located in the code, I need the debugger to find it. Same if I use things like "fromJust" or "head" 

I haven't had good experiences using the debugger. Maybe that's my fault, but I like that I can locate my unique exceptions and that they are descriptive.

But a month ago I decided this system is pretty ugly. It bloats the code a lot, and requires writing a lot of messages for situations that will probably never occur.

I am trying a different method now. I write the code so that surprising behavior will result in a case or pattern exhaustion, which produces a run time message with a file name and line number. I'm not sure why ghc gives the location for a case exhaustion, but not something like "head".

For example, instead of using head xs I can write

x = case xs of {y:_ -> y}

This is a little bloat but not too bad. Most of the time I'm actually structuring cases and patterns, and it actually helps me to simplify code to think of how to write it with the fewest cases and so that a case exhaustion will indicate something pretty specific.

Any comments welcome.


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners




--
Jeffrey Benjamin Brown

_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: comment on this debugging trick?

David McBride
In reply to this post by Dennis Raddle
If you compile with -prof you can use the traceStack function from Debug.Trace.  That's something.

On Fri, Nov 27, 2015 at 8:56 AM, Dennis Raddle <[hidden email]> wrote:
When I design my code, I am aware of the properties I expect my internal data representations to have, or I want input data to conform to my expectations. For example, right now I'm writing code that reads MusicXML. MusicXML is a crazy language, way too complicated for what it does, and the music typesetters that export MusicXML all do their own idiosyncratic things with it. I'm only reading the output of one typesetter, Sibelius, and although I don't know the internals of Sibelius, I can make some assumptions about what it's going to produce by a few examples, plus my application doesn't use the full range of MusicXML. I don't need to handle every case, is what I'm getting at.

However, if I should have been wrong about my assumptions when I wrote the code, I don't want my program to behave erratically. I want to find out exactly what went wrong as soon as possible.

Previously, I was including a lot of exception throwing, with each exception having a unique message describing what had happened, and in particular describing where in the code it is located so I can find the problem spot

If I throw generic error messages like "Error: problem parsing" and I have many of those located in the code, I need the debugger to find it. Same if I use things like "fromJust" or "head" 

I haven't had good experiences using the debugger. Maybe that's my fault, but I like that I can locate my unique exceptions and that they are descriptive.

But a month ago I decided this system is pretty ugly. It bloats the code a lot, and requires writing a lot of messages for situations that will probably never occur.

I am trying a different method now. I write the code so that surprising behavior will result in a case or pattern exhaustion, which produces a run time message with a file name and line number. I'm not sure why ghc gives the location for a case exhaustion, but not something like "head".

For example, instead of using head xs I can write

x = case xs of {y:_ -> y}

This is a little bloat but not too bad. Most of the time I'm actually structuring cases and patterns, and it actually helps me to simplify code to think of how to write it with the fewest cases and so that a case exhaustion will indicate something pretty specific.

Any comments welcome.


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: comment on this debugging trick?

Dennis Raddle
In reply to this post by Jeffrey Brown


On Fri, Nov 27, 2015 at 10:31 AM, Jeffrey Brown <[hidden email]> wrote:
Elliot Cameron, very smart guy, once advised me to make illegal state invalid. Jake Brownson, another, recommended trying to use Maybes and Eithers instead of throwing exceptions. I'm having trouble coming up with examples but I think they're both helpful.


I think I know roughly what you are talking about, but there are places where unexpected input might result in something like an empty list. Suppose I need the head of that list under normal conditions. Maybe the list is produced by library code so I can't alter the data structures. That's the kind of situation I want to identify precisely and immediately.

D



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: comment on this debugging trick?

Jeffrey Brown
In parsing libraries for Haskell the "parse" function (or its equivalent) typically returns an Either. For instance there's

parse :: Stream s Identity t => Parsec s () a -> SourceName-> s -> Either ParseError a

in Text.Parsec.Prim. This is an example of making invalid state impossible. parse could return a list of all possible parses, with the empty list signifying that parsing failed. But by using an Either, the case of failure can be distinguished as a Left. Haskell will know to treat the Left differently, and if you need an error report, that left can bubble (through multiple Either-returning functions, with some handy do-notation) up to the user without ever invoking an exception to the ordinary control flow.

On Fri, Nov 27, 2015 at 12:51 PM, Dennis Raddle <[hidden email]> wrote:


On Fri, Nov 27, 2015 at 10:31 AM, Jeffrey Brown <[hidden email]> wrote:
Elliot Cameron, very smart guy, once advised me to make illegal state invalid. Jake Brownson, another, recommended trying to use Maybes and Eithers instead of throwing exceptions. I'm having trouble coming up with examples but I think they're both helpful.


I think I know roughly what you are talking about, but there are places where unexpected input might result in something like an empty list. Suppose I need the head of that list under normal conditions. Maybe the list is produced by library code so I can't alter the data structures. That's the kind of situation I want to identify precisely and immediately.

D



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners




--
Jeffrey Benjamin Brown

_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: comment on this debugging trick?

Dennis Raddle


On Fri, Nov 27, 2015 at 2:51 PM, Jeffrey Brown <[hidden email]> wrote:
In parsing libraries for Haskell the "parse" function (or its equivalent) typically returns an Either. For instance there's

parse :: Stream s Identity t => Parsec s () a -> SourceName-> s -> Either ParseError a

in Text.Parsec.Prim. This is an example of making invalid state impossible. parse could return a list of all possible parses, with the empty list signifying that parsing failed. But by using an Either, the case of failure can be distinguished as a Left. Haskell will know to treat the Left differently, and if you need an error report, that left can bubble (through multiple Either-returning functions, with some handy do-notation) up to the user without ever invoking an exception to the ordinary control flow.


I've used the Either monad for exception handling. Yes, I was using do notation. This current project is just for myself. Exceptions represent either bugs or malformed input. I can catch some of them in the IO monad so my program prints an error but keeps running and lets me try again. If the program crashes out, no big problem. 

I'm parsing MusicXML with Text.XML.Light. The XML is always well-formed. However *MusicXML* is not a well-defined language. Like, does every <note> element have a child element called <voice>? No guarantee I can find, yet it has been true in the examples I've tried with the only typesetter I'm using to generate MusicXML.  I can get my program running quickly without bothering with the case that <voice> is absent. But if that case someday occurs, I want to know precisely and immediately. Yet I don't want to write a detailed error message for every violated assumption, of which there are dozens necessary. So my solution is to find these things with case exhaustions. The program crashes and I have to inspect the code, but no problem.


D




_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: comment on this debugging trick?

Stephen Tetley-2
Hi Dennis

For this use case I would make a small, general parser combinator
library / monad on top of XML.Light, then use the new parser
combinators to make a specialized parser for your subset MusicXML.

The common parser combinator libraries (Parsec, Attoparsec, ...) can't
be used with XML.Light because their mechanics are consuming an input
stream whereas processing XML (or JSON) is moving a cursor through a
tree, but the common API provided by parser combinator libraries can
be re-used productively for a tree (cursor) parser. As well as moving
the cursor, the custom parser monad can handle errors and backtracking
if needed.

Best wishes

Stephen

On 27 November 2015 at 23:55, Dennis Raddle <[hidden email]> wrote:
>
>
[snip]

>
>
> I've used the Either monad for exception handling. Yes, I was using do
> notation. This current project is just for myself. Exceptions represent
> either bugs or malformed input. I can catch some of them in the IO monad so
> my program prints an error but keeps running and lets me try again. If the
> program crashes out, no big problem.
>
> I'm parsing MusicXML with Text.XML.Light. The XML is always well-formed.
> However *MusicXML* is not a well-defined language. Like, does every <note>
> element have a child element called <voice>? No guarantee I can find, yet it
> has been true in the examples I've tried with the only typesetter I'm using
> to generate MusicXML.  I can get my program running quickly without
> bothering with the case that <voice> is absent. But if that case someday
> occurs, I want to know precisely and immediately. Yet I don't want to write
> a detailed error message for every violated assumption, of which there are
> dozens necessary. So my solution is to find these things with case
> exhaustions. The program crashes and I have to inspect the code, but no
> problem.
>
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: comment on this debugging trick?

Jeffrey Brown
 (Parsec, Attoparsec, ...) can't
> be used with XML.Light because their mechanics are consuming 
> an input stream whereas processing XML (or JSON) is 
> moving a cursor through a tree

Stephen, the "consuming an input vs. moving a cursor through a tree" distinction you're drawing is unclear to me. Can you elaborate, or provide references? 

Also, Text.JSON.Parsec [1] would appear to be a counterexample to your claim.


On Sat, Nov 28, 2015 at 2:14 AM, Stephen Tetley <[hidden email]> wrote:
Hi Dennis

For this use case I would make a small, general parser combinator
library / monad on top of XML.Light, then use the new parser
combinators to make a specialized parser for your subset MusicXML.

The common parser combinator libraries (Parsec, Attoparsec, ...) can't
be used with XML.Light because their mechanics are consuming an input
stream whereas processing XML (or JSON) is moving a cursor through a
tree, but the common API provided by parser combinator libraries can
be re-used productively for a tree (cursor) parser. As well as moving
the cursor, the custom parser monad can handle errors and backtracking
if needed.

Best wishes

Stephen

On 27 November 2015 at 23:55, Dennis Raddle <[hidden email]> wrote:
>
>
[snip]
>
>
> I've used the Either monad for exception handling. Yes, I was using do
> notation. This current project is just for myself. Exceptions represent
> either bugs or malformed input. I can catch some of them in the IO monad so
> my program prints an error but keeps running and lets me try again. If the
> program crashes out, no big problem.
>
> I'm parsing MusicXML with Text.XML.Light. The XML is always well-formed.
> However *MusicXML* is not a well-defined language. Like, does every <note>
> element have a child element called <voice>? No guarantee I can find, yet it
> has been true in the examples I've tried with the only typesetter I'm using
> to generate MusicXML.  I can get my program running quickly without
> bothering with the case that <voice> is absent. But if that case someday
> occurs, I want to know precisely and immediately. Yet I don't want to write
> a detailed error message for every violated assumption, of which there are
> dozens necessary. So my solution is to find these things with case
> exhaustions. The program crashes and I have to inspect the code, but no
> problem.
>
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



--
Jeffrey Benjamin Brown

_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: comment on this debugging trick?

Stephen Tetley-2
Hi Jeffrey

The example you linked to is a JSON parser written with Parsec - it
takes an input stream of characters and returns a JSON tree. To go
from "uni-typed" JSON to Haskell data represented by your domain
datatypes you have to do further conversion. This conversion is from
"tree to tree" rather than "from input stream to tree".

If the correspondence between the JSON (or XML) is one-to-one you can
do a fairly mechanical deserialization (I think some of the libraries
already have Template Haskell processors for this). In Dennis's case
he wants to go from a very large XML representation to deal with a
more manageable subset in Haskell and believes that the input he works
with always belongs to the manageable subset. To get from MusicXML's
large tree - represented again by a uni-typed tree in XML.Light -
parser combinators provide a fine API for the job but the internal
machinary needs to be able to traverse a tree (descend child nodes,
climb back out on failure etc.) rather than consume an input stream.

On 28 November 2015 at 17:58, Jeffrey Brown <[hidden email]> wrote:

>>  (Parsec, Attoparsec, ...) can't
>> be used with XML.Light because their mechanics are consuming
>> an input stream whereas processing XML (or JSON) is
>> moving a cursor through a tree
>
> Stephen, the "consuming an input vs. moving a cursor through a tree"
> distinction you're drawing is unclear to me. Can you elaborate, or provide
> references?
>
> Also, Text.JSON.Parsec [1] would appear to be a counterexample to your
> claim.
>
> [1]
> https://hackage.haskell.org/package/json-0.9.1/docs/Text-JSON-Parsec.html
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: comment on this debugging trick?

Dennis Raddle
In reply to this post by Stephen Tetley-2


On Sat, Nov 28, 2015 at 2:14 AM, Stephen Tetley <[hidden email]> wrote:
Hi Dennis

For this use case I would make a small, general parser combinator
library / monad on top of XML.Light, then use the new parser
combinators to make a specialized parser for your subset MusicXML.

The common parser combinator libraries (Parsec, Attoparsec, ...) can't
be used with XML.Light because their mechanics are consuming an input
stream whereas processing XML (or JSON) is moving a cursor through a
tree, but the common API provided by parser combinator libraries can
be re-used productively for a tree (cursor) parser. As well as moving
the cursor, the custom parser monad can handle errors and backtracking
if needed.


Thanks for the pointers, Stephen. I will look into it. I don't know if I'll have time to learn how to do this though. I'll keep it in mind for the future.

This is a small, ongoing, no-pressure project, which is even lower pressure when you consider I'm using what I call "just in time" debugging and design. When I look at my use cases, I don't implement any that seem unlikely. If they ever do occur, I can deal with them then. But I don't just ignore cases willy-nilly -- I always consider if they do occur, that the resulting bug will be obvious right away and lead me directly to the answer.

This is interesting when dealing with music playback. Some unexpected cases won't give an error message but will affect the playback sound -- so I always ask myself, "Will this bug be obvious, really screw up the sound in a major way, so I know it's happening?" Because what I don't want are bugs that create small effects that I might miss for a while so I miss the critical moment they were introduced.

I don't know the definition of the different processes but does "Just In Time" design/debugging have something to do with Agile?

D



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners