Deprecating fromIntegral

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
44 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Deprecating fromIntegral

Niklas Hambüchen
This is not a joke.

I just found a bug in base (https://ghc.haskell.org/trac/ghc/ticket/14262):

  import System.IO
  hWaitForInput stdin 4294968296

This code should wait for 49.something days, but it waits for 1 second.

It is caused by a quick use of `fromIntegral` to "convert" `Int -> CInt`
(where CInt = Int32), causing an overflow.

I argue that `fromIntegral` causes unnoticed data corruption without any
warning, and thus are worse than partial functions.

In this case, this data corruption occurs in the most fundamental code
that we all use and have to rely upon (`base`).

So I propose that we add a deprecation pragma to `fromIntegral`
(however, keeping it forever, like Java does, as there's no benefit to
removing it, we just want that people use safer functions over time),
and instead provide a `safeFromIntegral` that can only widen ranges, and
one or more `unsafeFromIntegral` or appropriately named functions that
can error, wrap, return Maybe, or have whatever desired behaviour when
the input value doesn't fit into the output type.

For some inspiration, `foundation` provides ideas in that direction:

http://hackage.haskell.org/package/basement-0.0.2/docs/Basement-From.html
http://hackage.haskell.org/package/basement-0.0.2/docs/Basement-IntegralConv.html

It is a bit funny how we argue that Haskell is good for high assurance
programming, and we build all sorts of fancy constructs to eliminate
more run-time errors, when in fact we do worse than C in detecting
errors in the most basic types a computer can use, such as converting
between 64-bit and 32-bit integers.

Let's fix that.

It avoids real problems in real production systems that are difficult to
debug when they happen (who writes a unit test checking that a timeout
of 50 days still works? This is the kind of stuff where you have to rely
on -- very basic -- types).

I'd like to probe support for such a change before doing any work on it.

Niklas
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

amindfv
I'm -1 on this (there are lots of places where you need to know the bounds of the numeric type you're using), but would be fine with adding more documentation to hWaitForInput and/or fromIntegral.

Tom


> El 21 sept 2017, a las 08:24, Niklas Hambüchen <[hidden email]> escribió:
>
> This is not a joke.
>
> I just found a bug in base (https://ghc.haskell.org/trac/ghc/ticket/14262):
>
>  import System.IO
>  hWaitForInput stdin 4294968296
>
> This code should wait for 49.something days, but it waits for 1 second.
>
> It is caused by a quick use of `fromIntegral` to "convert" `Int -> CInt`
> (where CInt = Int32), causing an overflow.
>
> I argue that `fromIntegral` causes unnoticed data corruption without any
> warning, and thus are worse than partial functions.
>
> In this case, this data corruption occurs in the most fundamental code
> that we all use and have to rely upon (`base`).
>
> So I propose that we add a deprecation pragma to `fromIntegral`
> (however, keeping it forever, like Java does, as there's no benefit to
> removing it, we just want that people use safer functions over time),
> and instead provide a `safeFromIntegral` that can only widen ranges, and
> one or more `unsafeFromIntegral` or appropriately named functions that
> can error, wrap, return Maybe, or have whatever desired behaviour when
> the input value doesn't fit into the output type.
>
> For some inspiration, `foundation` provides ideas in that direction:
>
> http://hackage.haskell.org/package/basement-0.0.2/docs/Basement-From.html
> http://hackage.haskell.org/package/basement-0.0.2/docs/Basement-IntegralConv.html
>
> It is a bit funny how we argue that Haskell is good for high assurance
> programming, and we build all sorts of fancy constructs to eliminate
> more run-time errors, when in fact we do worse than C in detecting
> errors in the most basic types a computer can use, such as converting
> between 64-bit and 32-bit integers.
>
> Let's fix that.
>
> It avoids real problems in real production systems that are difficult to
> debug when they happen (who writes a unit test checking that a timeout
> of 50 days still works? This is the kind of stuff where you have to rely
> on -- very basic -- types).
>
> I'd like to probe support for such a change before doing any work on it.
>
> Niklas
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Niklas Hambüchen
On 21/09/17 14:35, [hidden email] wrote:
> I'm -1 on this (there are lots of places where you need to know the bounds of the numeric type you're using)

I don't quite understand what you mean with "you need to know the
bounds", or what role `fromIntegral` has in that, can you elaborate?
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Niklas Hambüchen
Also, just to make that clear, the purpose here is not to to remove a
function that does something useful without providing a replacement.

It is to make sure that functions have appropriate names for what they
do, so that you don't use a function accidentally that has different
behaviour from what you expect.

For example, in the use of `ready` in GHC's Handle code


https://github.com/ghc/ghc/blob/ghc-8.2.1-release/libraries/base/GHC/IO/FD.hs#L399

the author certainly did not intend to have any wrapping behaviour, but
nothing in that code looks wrong unless you think really, really hard.

Because it looks correct.

(And has looked correct for 17 years:
https://github.com/ghc/ghc/blame/b83cfb91b4d36d148ebe171d31e2676c8f10f371/libraries/base/GHC/IO.hsc#L61)

If you wanted wrapping behaviour, a `wrapIntegral` would be your friend.

If a `wrapIntegral` is used in a place that shouldn't have it, that gets
noticed immediately in review.

On 21/09/17 14:53, Niklas Hambüchen wrote:
> On 21/09/17 14:35, [hidden email] wrote:
>> I'm -1 on this (there are lots of places where you need to know the bounds of the numeric type you're using)
>
> I don't quite understand what you mean with "you need to know the
> bounds", or what role `fromIntegral` has in that, can you elaborate?
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Henning Thielemann
In reply to this post by Niklas Hambüchen

On Thu, 21 Sep 2017, Niklas Hambüchen wrote:

> It is a bit funny how we argue that Haskell is good for high assurance
> programming, and we build all sorts of fancy constructs to eliminate
> more run-time errors, when in fact we do worse than C in detecting
> errors in the most basic types a computer can use, such as converting
> between 64-bit and 32-bit integers.

That's all very true! Other safety oriented languages like Modula-3 at
least perform a runtime check on integer conversion.

However, I'd like to see practical solutions first before adding
deprecations.

fromInteger will always be partial when converting to a fixed width
integer. fromInteger is used for converting number literals to the wanted
type. Thus every number literal will issue a warning and I cannot see how
to circumvent this.

Another problem I see is that integralUpsize and integralDownsize are not
Haskell 98 and multi-parameter type classes are not really satisfying here
because you have to define n^2 instances for n types. A convenient
conversion function would certainly need even more advanced types, e.g.
type-level comparison of bit widths.

If we have fixed fromIntegral then the next natural question would arise:
How to protect arithmetic operations plus, minus, times against overflow?
Aren't overflows as dangerous as losing bits on integer conversion? Is
there a solution that works both for conversion and arithmetics?

For now the least invasive solution would be to provide a partial integer
conversion function.

> So I propose that we add a deprecation pragma to `fromIntegral`
> (however, keeping it forever, like Java does, as there's no benefit to
> removing it, we just want that people use safer functions over time),
> and instead provide a `safeFromIntegral` that can only widen ranges, and
> one or more `unsafeFromIntegral` or appropriately named functions that
> can error, wrap, return Maybe, or have whatever desired behaviour when
> the input value doesn't fit into the output type.

As always, I object to use the terms 'safe' and 'unsafe' when actually
'total' and 'partial' are meant. LLVM uses 'ext' and 'trunc'. I'd prefer
names along these lines.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Herbert Valerio Riedel-3
In reply to this post by Niklas Hambüchen
On 2017-09-21 at 14:24:14 +0200, Niklas Hambüchen wrote:

[...]

> I argue that `fromIntegral` causes unnoticed data corruption without any
> warning, and thus are worse than partial functions.

[...]

> So I propose that we add a deprecation pragma to `fromIntegral`

Tbh, given how ubiquitous the use of `fromIntegral` is throughout the
Haskell ecosystem, having 'fromIntegral' suddely emit warnings when
`-Wall` is active is not realistic IMHO.

You'd have to introduce a separate (opt-in) category of WARNING pragmas
for that (something some of us would also like to see for the category
of partial functions) which isn't implied by -Wall.

> (however, keeping it forever, like Java does, as there's no benefit to
> removing it, we just want that people use safer functions over time),
> and instead provide a `safeFromIntegral` that can only widen ranges, and
> one or more `unsafeFromIntegral` or appropriately named functions that
> can error, wrap, return Maybe, or have whatever desired behaviour when
> the input value doesn't fit into the output type.

[...]

> It avoids real problems in real production systems that are difficult to
> debug when they happen (who writes a unit test checking that a timeout
> of 50 days still works? This is the kind of stuff where you have to rely
> on -- very basic -- types).

I ran into this very problem early on. I'll use the opportunity to
shamelessly plug an older package of mine probably only few know about
which solved this very problem in mission critical systems for me while
avoiding to have to enumerate O(n^2) transitions explicitly:

  http://hackage.haskell.org/package/int-cast-0.1.2.0/docs/Data-IntCast.html

But it isn't plain Haskell 2010...

Another related package which may be useful in this context is
http://hackage.haskell.org/package/safeint which helps avoid silent
integer overflows during arithemtic operations.


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Edward Kmett-2
In reply to this post by Niklas Hambüchen
I'm -1 on this proposal.

fromInteger and fromIntegral are a huge part of our ecosystem. They comprise a large part of the difference between how we handle numeric literals and how literally every other language on the planet handles numeric literals. fromIntegral shows up all over the place in any code that is even remotely mathematical to deal with converting between integer indices and other numeric types for computation.

If we pretend Num is about working with rings fromInteger provides the canonical ring homomorphism from Z into every ring, so that half of the equation has to remain in place. toInteger is Integral reason for existence, so you're just asking the existing users to replace fromIntegral with 'fromInteger . toInteger' do dodge a warning that doesn't come with any other clear explanation about how to fix.

Any properly replacement requires language extensions such as multi parameter type classes that we've failed to demonstrate an ability to standardize as well as n^2 instances and makes it difficult to convert between integral types and numbers supplied by different packages.

Now, if you wanted to write a package that provided widening and shrinking conversions and this gradually was picked up by a larger and larger cross section of the community and more organically worked its way into base that'd be one thing, but these nicer safe conversions haven't yet been adopted by any significant fraction of the community at this point, so any form of deprecation is very much premature.

On the other hand I'm definitely +1 on adding documentation to hWaitForInput or on fromIntegral about how you should be careful when switching between numeric types of different sizes.

-Edward

On Thu, Sep 21, 2017 at 8:24 AM, Niklas Hambüchen <[hidden email]> wrote:
This is not a joke.

I just found a bug in base (https://ghc.haskell.org/trac/ghc/ticket/14262):

  import System.IO
  hWaitForInput stdin 4294968296

This code should wait for 49.something days, but it waits for 1 second.

It is caused by a quick use of `fromIntegral` to "convert" `Int -> CInt`
(where CInt = Int32), causing an overflow.

I argue that `fromIntegral` causes unnoticed data corruption without any
warning, and thus are worse than partial functions.

In this case, this data corruption occurs in the most fundamental code
that we all use and have to rely upon (`base`).

So I propose that we add a deprecation pragma to `fromIntegral`
(however, keeping it forever, like Java does, as there's no benefit to
removing it, we just want that people use safer functions over time),
and instead provide a `safeFromIntegral` that can only widen ranges, and
one or more `unsafeFromIntegral` or appropriately named functions that
can error, wrap, return Maybe, or have whatever desired behaviour when
the input value doesn't fit into the output type.

For some inspiration, `foundation` provides ideas in that direction:

http://hackage.haskell.org/package/basement-0.0.2/docs/Basement-From.html
http://hackage.haskell.org/package/basement-0.0.2/docs/Basement-IntegralConv.html

It is a bit funny how we argue that Haskell is good for high assurance
programming, and we build all sorts of fancy constructs to eliminate
more run-time errors, when in fact we do worse than C in detecting
errors in the most basic types a computer can use, such as converting
between 64-bit and 32-bit integers.

Let's fix that.

It avoids real problems in real production systems that are difficult to
debug when they happen (who writes a unit test checking that a timeout
of 50 days still works? This is the kind of stuff where you have to rely
on -- very basic -- types).

I'd like to probe support for such a change before doing any work on it.

Niklas
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Niklas Hambüchen
In reply to this post by Henning Thielemann
Hey Henning,

On 21/09/17 15:21, Henning Thielemann wrote:
> That's all very true! Other safety oriented languages like Modula-3 at
> least perform a runtime check on integer conversion.
>
> However, I'd like to see practical solutions first before adding
> deprecations.

I agree, deprecating before finding a solution doesn't make sense.

I intend this thread to be mainly about discussing possible solutions
and problems with them, hopefully we'll find something good.

> fromInteger will always be partial when converting to a fixed width
> integer. fromInteger is used for converting number literals to the
> wanted type. Thus every number literal will issue a warning and I cannot
> see how to circumvent this.

Yes. I would say though that `fromInteger` being partial is a less
severe problem than `fromIntegral`'s silent overflow; if a `fromInteger`
fails partially, you'd at least get a nice run-time error, which would
be better.

(Though, in practice the current `fromInteger is not partial, e.g.
`(fromInteger 256 :: Word8) == 0` without error, and I suspect it would
be good if we added a partial function in addition to the silently
wrapping one here too.)

> Another problem I see is that integralUpsize and integralDownsize are
> not Haskell 98 and multi-parameter type classes are not really
> satisfying here because you have to define n^2 instances for n types. A
> convenient conversion function would certainly need even more advanced
> types, e.g. type-level comparison of bit widths.

That's a good point, and that's why I think that...

> For now the least invasive solution would be to provide a partial
> integer conversion function.

... this might be a good first step.

It would turn

 "You ship code into production and 50 days later it corrupts your data"

into

 "You ship code into production and 50 days later it crashes"

which is already much better, and, I believe, easy to implement.

> If we have fixed fromIntegral then the next natural question would
> arise: How to protect arithmetic operations plus, minus, times against
> overflow? Aren't overflows as dangerous as losing bits on integer
> conversion? Is there a solution that works both for conversion and
> arithmetics?

Yes, all types of overflows can be problematic and dangerous.

You could say though that overflows "within the type" are somewhat less
problematic: Arithmetic overflows are more well-known and in people's
minds, it is relatively easy for a programmer to pick a type that is
large enough so that "+" doesn't overflow, and that only stops working
once you need to convert to another type, when the `fromIntegral`
problem occurs.
In "their own types", people work confidently, it's when interfacing
with others' code where the big bugs occur (like in the example I gave).

Haskell provides easy methods to prevent overflow *within* one's own
numeric types if one desires (like implementing "+" with `error` if it
doesn't fit); but currently our conversion functions *between* types
don't give us good explicit means to do so.

Also, how to deal with in-type overflows is an unsolved problem in
general, while C already solves much of the conversion problem.

I think solving the parts that C has solved would already avoid a good
amount of bugs.

> As always, I object to use the terms 'safe' and 'unsafe' when actually
> 'total' and 'partial' are meant. LLVM uses 'ext' and 'trunc'. I'd prefer
> names along these lines.

Good point.

I used "safe" and "unsafe" to express the idea that one kind would
always succeed in the naturally expected fashion and one would not, but
the more precisely the naming is, the better.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Henning Thielemann

On Thu, 21 Sep 2017, Niklas Hambüchen wrote:

> It would turn
>
> "You ship code into production and 50 days later it corrupts your data"
>
> into
>
> "You ship code into production and 50 days later it crashes"
>
> which is already much better, and, I believe, easy to implement.

I thought that it would abort immediately, wouldn't it?
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Niklas Hambüchen
In reply to this post by Herbert Valerio Riedel-3
Hey Herbert,

> I ran into this very problem early on. I'll use the opportunity to
> shamelessly plug an older package of mine probably only few know about
> which solved this very problem in mission critical systems for me while
> avoiding to have to enumerate O(n^2) transitions explicitly:
>
>
http://hackage.haskell.org/package/int-cast-0.1.2.0/docs/Data-IntCast.html

This is very useful, thanks for that.

> Tbh, given how ubiquitous the use of `fromIntegral` is throughout the
> Haskell ecosystem, having 'fromIntegral' suddely emit warnings when
> `-Wall` is active is not realistic IMHO.

Here I disagree.

It is "realistic" because we can just do it, if we want.

The questions more if we want to.

I'd say yes, because:

Deprecation-marking does not break any code, it is the softest way to
make a transition. The Java community has this figured out: Deprecations
are common, they don't break the build, they don't change behaviour,
downstream libraries adapt quickly to them, and the whole ecosystem can
advance to a more solid foundation gracefully.

The way you phrase it sounds as if you'd expect a large backlash from
your users if you emitted these extra warnings.

But I think the opposite can be expected:
Haskell users would love if -Wall pointed out *more* ways in which their
code may break.
We use -Wall because we *want* to be warned of slumbering bugs.

I think your users would cheer at you, not complain.

(Even the C people regularly cheer as new warnings make it into -Wall of
gcc and clang.)
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Niklas Hambüchen
In reply to this post by Henning Thielemann
On 21/09/17 17:42, Henning Thielemann wrote:
> I thought that it would abort immediately, wouldn't it?

Yes, I did a bit of a simplification here; certainly an overflow would
either create "corrupt"ed data in memory immediately or (better) abort
immediately.

I was thinking about the case where you deploy code and after it running
for a while (e.g. tracking elapsed milliseconds over 50 days of uptime)
something overflows and either corrupts or aborts.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Niklas Hambüchen
In reply to this post by Edward Kmett-2
Hey Edward,

I'm not going after Num here.

What I care about is that base provide conversion functions with clear
names that indicate what they do, so that the intent of the author
becomes obvious.

Currently we cram all number conversion functionality into
`fromIntegral`, and offer no standard way for talking about code that
has no intent to wrap.

This code:

  fdReady ... (fromIntegral msecs)

does not look obviously incorrect, but this code would:

  fdReady ... (fromIntegralWrap msecs)

Of course it would be great if a name change came along with a set of
type classes that make it compile-time-impossible to do an unintended
cast, but in absence of one agreed way to do that, more functions with
better names and appropriate error behaviour would already go a long way
for avoiding bugs like this.

To throw out some concrete examples, there could be:

* maybeFromInteger / maybeFromIntegral
  - that return Nothing when the input doesn't fit
* runtimeCheckedFromInteger / runtimeCheckedFromIntegral
  - `error` on Nothing
* fromIntegerWrap / fromIntegralWrap
  - what the current functions do
* fromInteger / fromIntegral
  - unchanged semantics, but emitting a deprecation warning

Deprecation seems the best method by which we can systematically
highlight all use sites where a replacement with a function of a better
name, or missing edge case handling, should be inserted, without
breaking code.

Niklas
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Evan Laforge
In reply to this post by Niklas Hambüchen
On Thu, Sep 21, 2017 at 8:57 AM, Niklas Hambüchen <[hidden email]> wrote:
> Deprecation-marking does not break any code, it is the softest way to
> make a transition. The Java community has this figured out: Deprecations
> are common, they don't break the build, they don't change behaviour,
> downstream libraries adapt quickly to them, and the whole ecosystem can
> advance to a more solid foundation gracefully.

As I remember it, where I worked there was a distinction between
deprecations and warnings.  Deprecations had no visible output outside
the documentation, unless you used an IDE and turned on the
dotted-underline feature, while warnings would be printed during the
compile.  You were not supposed to deprecate anything without a
clearly documented "do this instead" but you didn't need a plan to fix
all the callers.  But to emit a warning, a bounded-time plan was
required, which probably involved a global fix via automatic
refactoring.

Otherwise, warnings accumulate, and once they get past a certain
amount everyone ignores all of them equally, which is a loss as we
lose the high value ones too.  Most of them come out of compiling
stuff you don't own and don't understand and don't have time to go
fix.

If I were to suddenly get 10,000 lines of warnings... well, if it can
be fixed with a global search and replace then it's ok if they only
indicate a few real problems.  But if the fix is more complicated and
requires individually looking at each place, then they had better have
a pretty good positive rate.  Otherwise I wind up turning off the
warning and it might as well not exist.

You could say there's a place for low value lint type warnings which
you could turn on just for hlint style suggestions, but it needs to be
integrated with the source control so it can warn only about the bits
that you just changed.  Otherwise no one can repeatedly comb through
the 10,000 suggestions in case some have been added, so in practice it
just stays off forever.

Also in my experience the bit about "downstream libraries adapt
quickly to them" is wildly optimistic :)
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

John Wiegley-2
In reply to this post by Niklas Hambüchen
>>>>> "NH" == Niklas Hambüchen <[hidden email]> writes:

NH> Here I disagree.
NH> It is "realistic" because we can just do it, if we want.
NH> The questions more if we want to.
NH> I'd say yes, because:

Why is it 'fromIntegral' that should be promoted into a warning? What about
all the other partial functions in base that don't generate warnings either?

--
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Edward Kmett-2
In reply to this post by Niklas Hambüchen
Note:

hWaitForInput stdin 4294968296


is using fromInteger, not fromIntegral. fromInteger is the member of Num. fromIntegral is defined in terms of it and toInteger.

Deprecating fromIntegral would do nothing to the example you offered at all as it isn't being called.

Like it or not the existing numeric types in the Haskell ecosystem wrap, and changing this behavior would have non-trivial ramifications for almost all Haskell source code written to date.

This isn't a small change you are proposing.

If your goal is to address this micro-issue, it would be a much easier target to aim for hWaitForInput taking an Integer rather than an Int, removing the issue at the source. There are already packages providing safer delays, e.g.


It has gradually become clear that using Int in delay and hWaitForInput and delay the like was a mistake. After all if you're going to set up to block anyways the overhead of allocating a single Integer constructor is trivial.

For comparison, I have zero objection to going after the type signature of hWaitForInput!

Changing fromInteger on the other hand is a Prelude affecting change that bubbles all the way down into Num. I'm very much against breaking every single Num instance, and making every Haskell developer audit every application of a very common function to fix one overflow in one naively designed system call.

-Edward

On Thu, Sep 21, 2017 at 1:26 PM, Niklas Hambüchen <[hidden email]> wrote:
Hey Edward,

I'm not going after Num here.

What I care about is that base provide conversion functions with clear
names that indicate what they do, so that the intent of the author
becomes obvious.

Currently we cram all number conversion functionality into
`fromIntegral`, and offer no standard way for talking about code that
has no intent to wrap.

This code:

  fdReady ... (fromIntegral msecs)

does not look obviously incorrect, but this code would:

  fdReady ... (fromIntegralWrap msecs)

Of course it would be great if a name change came along with a set of
type classes that make it compile-time-impossible to do an unintended
cast, but in absence of one agreed way to do that, more functions with
better names and appropriate error behaviour would already go a long way
for avoiding bugs like this.

To throw out some concrete examples, there could be:

* maybeFromInteger / maybeFromIntegral
  - that return Nothing when the input doesn't fit
* runtimeCheckedFromInteger / runtimeCheckedFromIntegral
  - `error` on Nothing
* fromIntegerWrap / fromIntegralWrap
  - what the current functions do
* fromInteger / fromIntegral
  - unchanged semantics, but emitting a deprecation warning

Deprecation seems the best method by which we can systematically
highlight all use sites where a replacement with a function of a better
name, or missing edge case handling, should be inserted, without
breaking code.

Niklas


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Niklas Hambüchen
In reply to this post by John Wiegley-2
On 21/09/17 19:47, John Wiegley wrote:
> Why is it 'fromIntegral' that should be promoted into a warning? What about
> all the other partial functions in base that don't generate warnings either?

Because `fromIntegral` is not a partial function, it fails silently.

Also because fixing all partial functions in base is way out of scope of
what I intended to propose.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

John Wiegley-2
>>>>> "NH" == Niklas Hambüchen <[hidden email]> writes:

NH> Because `fromIntegral` is not a partial function, it fails silently.

Ah, yes, I didn't mean "partial" per se, but more in the sense of "not having
a sane answer for all inputs".

I'm -1 to changing the behavior of fromIntegral now, but +1 on adding other
functions to make the behavior more explicit.

--
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Niklas Hambüchen
In reply to this post by Edward Kmett-2
On 21/09/17 19:57, Edward Kmett wrote:
> hWaitForInput stdin 4294968296
>
> is using fromInteger, not fromIntegral.fromInteger is the member of Num.
> fromIntegral is defined in terms of it and toInteger.
>
> Deprecating fromIntegral would do nothing to the example you offered at
> all as it isn't being called.

Ah, here is the misunderstanding.

It is not the call to to hWaitForInput that is overflowing.

hWaitForInput takes an `Int`, and so passing 4294968296 to it is
perfectly fine (on 64-bit platforms).

As described in the linked bug
https://ghc.haskell.org/trac/ghc/ticket/14262, the overflow is in the
code that implements hWaitForInput,

  ready :: FD -> Bool -> Int -> IO Bool
  ready fd write msecs = do
    r <- throwErrnoIfMinus1Retry "GHC.IO.FD.ready" $
            fdReady (fdFD fd) (fromIntegral $ fromEnum $ write)
                              (fromIntegral msecs)

in the `fromIntegral msecs`.

I provided the `hWaitForInput 4294968296` bit only to give an easily
reproducible example of how `fromIntegral` results in a significant bug.

I realise now that the `4294968296` is why you brought up the topic of
integer literals (which confused me a bit).

> Like it or not the existing numeric types in the Haskell ecosystem wrap,
> and changing this behavior would have non-trivial ramifications for
> almost all Haskell source code written to date.

I did not propose to change the behaviour of any existing function.

I said:

"
* fromInteger / fromIntegral
  - unchanged semantics, but emitting a deprecation warning
"

Niklas
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

Niklas Hambüchen
In reply to this post by John Wiegley-2
> I'm -1 to changing the behavior of fromIntegral now

I never suggested to change the behaviour of fromIntegral.

I even argued against that twice:

* once in my original email (saying we should deprecate the current
fromIntegral but keep it forever for backwards compatibility)
* once in my reply to Edward ("unchanged semantics, but emitting a
deprecation warning")

> but +1 on adding other
> functions to make the behavior more explicit.

Then you are +1 on my proposal, because that is exactly what I wrote.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating fromIntegral

John Wiegley-2
>>>>> "NH" == Niklas Hambüchen <[hidden email]> writes:

NH> Then you are +1 on my proposal, because that is exactly what I wrote.

OK, thanks for clarifying then, and apologies for the added confusion.

--
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
123