withFile defeats dead handle closure

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

withFile defeats dead handle closure

David Feuer
withFile keeps the handle it creates alive until the user action completes. If we want to avoid this, we can. All that's required is to capture the weak reference that holds the handle's finalizer. Then instead of closing the handle in the end (or on exception) using hClose, withFile can run the handle's finalizer. Should we do this? Or does it run too much against the idea that handles should be managed as explicitly as possible?

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

Oleg Grenrus
I would be afraid of making this change. File descriptors are a scarce
resource, and relying on non-deterministic GC behavior to free them is
big -1.

Also finalizers are not free in current GC. (Though, you probably cannot
won't that many open files that it would matter).

- Oleg

On 30.12.2020 21.23, David Feuer wrote:

> withFile keeps the handle it creates alive until the user action
> completes. If we want to avoid this, we can. All that's required is to
> capture the weak reference that holds the handle's finalizer. Then
> instead of closing the handle in the end (or on exception) using
> hClose, withFile can run the handle's finalizer. Should we do this? Or
> does it run too much against the idea that handles should be managed
> as explicitly as possible?
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

David Feuer
Finalizers aren't free, but every handle already has a finalizer, so that's not an extra cost.

On Wed, Dec 30, 2020, 2:50 PM Oleg Grenrus <[hidden email]> wrote:
I would be afraid of making this change. File descriptors are a scarce
resource, and relying on non-deterministic GC behavior to free them is
big -1.

Also finalizers are not free in current GC. (Though, you probably cannot
won't that many open files that it would matter).

- Oleg

On 30.12.2020 21.23, David Feuer wrote:
> withFile keeps the handle it creates alive until the user action
> completes. If we want to avoid this, we can. All that's required is to
> capture the weak reference that holds the handle's finalizer. Then
> instead of closing the handle in the end (or on exception) using
> hClose, withFile can run the handle's finalizer. Should we do this? Or
> does it run too much against the idea that handles should be managed
> as explicitly as possible?
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

Henning Thielemann
In reply to this post by David Feuer

On Wed, 30 Dec 2020, David Feuer wrote:

> withFile keeps the handle it creates alive until the user action
> completes. If we want to avoid this, we can.

Why should we?
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

Haskell - Libraries mailing list
Although it might be nice to have a weak-reference-based alternative to
withFile, I think it would be best to keep the behavior of withFile as
predictable as possible (i.e. the current behavior), since closing a
file can be semantically significant.  For instance, if someone is using
/dev/watchdog to make their system automatically reboot when their
userspace daemon fails, closing the file handle early would cause the
watchdog functionality to be disabled, which is almost certainly not
what someone would mean by `withFile "/dev/watchdog" $ \_ -> blah`.

On 12/30/20 3:47 PM, Henning Thielemann wrote:

>
> On Wed, 30 Dec 2020, David Feuer wrote:
>
>> withFile keeps the handle it creates alive until the user action
>> completes. If we want to avoid this, we can.
>
> Why should we?
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

Henning Thielemann

On Wed, 30 Dec 2020, Ryan Trinkle via Libraries wrote:

> Although it might be nice to have a weak-reference-based alternative to
> withFile, I think it would be best to keep the behavior of withFile as
> predictable as possible (i.e. the current behavior), since closing a
> file can be semantically significant.

Right, if I open and close and re-open a file, it must be closed before I
can re-open it.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

Haskell - Libraries mailing list
I think David's suggestion may have been that withFile would still close
the file at the end of the block, but would *also* close it if the
handle died.

On 12/30/20 4:05 PM, Henning Thielemann wrote:

>
> On Wed, 30 Dec 2020, Ryan Trinkle via Libraries wrote:
>
>> Although it might be nice to have a weak-reference-based alternative to
>> withFile, I think it would be best to keep the behavior of withFile as
>> predictable as possible (i.e. the current behavior), since closing a
>> file can be semantically significant.
>
> Right, if I open and close and re-open a file, it must be closed
> before I can re-open it.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

Oliver Charles-3
While maybe not directly related to this, for many small changes suggested to base the answer is often: go write a library and see how it works out. This is often for fairly stylistic changes to "fringe" components of base. Here we're talking about a full blown operational change to something incredibly common. If anything, I think it would have to subject to the same process - try it out, really use it anger, see what breaks and - hopefully! - what gets better.

On Wed, 30 Dec 2020, at 9:18 PM, Ryan Trinkle via Libraries wrote:
I think David's suggestion may have been that withFile would still close
the file at the end of the block, but would *also* close it if the
handle died.

On 12/30/20 4:05 PM, Henning Thielemann wrote:
>
> On Wed, 30 Dec 2020, Ryan Trinkle via Libraries wrote:
>
>> Although it might be nice to have a weak-reference-based alternative to
>> withFile, I think it would be best to keep the behavior of withFile as
>> predictable as possible (i.e. the current behavior), since closing a
>> file can be semantically significant.
>
> Right, if I open and close and re-open a file, it must be closed
> before I can re-open it.
_______________________________________________
Libraries mailing list



_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

David Feuer
To be clear, I wasn't formally proposing a change. I was trying to get
a sense of what people thought of the notion before going down too
many rabbit holes. I think Ryan Trinkle's concern is quite strong
enough to kill this idea as a change to `withFile` itself.

On Wed, Dec 30, 2020 at 4:26 PM Oliver Charles <[hidden email]> wrote:

>
> While maybe not directly related to this, for many small changes suggested to base the answer is often: go write a library and see how it works out. This is often for fairly stylistic changes to "fringe" components of base. Here we're talking about a full blown operational change to something incredibly common. If anything, I think it would have to subject to the same process - try it out, really use it anger, see what breaks and - hopefully! - what gets better.
>
> On Wed, 30 Dec 2020, at 9:18 PM, Ryan Trinkle via Libraries wrote:
>
> I think David's suggestion may have been that withFile would still close
> the file at the end of the block, but would *also* close it if the
> handle died.
>
> On 12/30/20 4:05 PM, Henning Thielemann wrote:
> >
> > On Wed, 30 Dec 2020, Ryan Trinkle via Libraries wrote:
> >
> >> Although it might be nice to have a weak-reference-based alternative to
> >> withFile, I think it would be best to keep the behavior of withFile as
> >> predictable as possible (i.e. the current behavior), since closing a
> >> file can be semantically significant.
> >
> > Right, if I open and close and re-open a file, it must be closed
> > before I can re-open it.
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

Viktor Dukhovni
In reply to this post by David Feuer
On Wed, Dec 30, 2020 at 02:23:20PM -0500, David Feuer wrote:

> withFile keeps the handle it creates alive until the user action completes.
> If we want to avoid this, we can. All that's required is to capture the
> weak reference that holds the handle's finalizer. Then instead of closing
> the handle in the end (or on exception) using hClose, withFile can run the
> handle's finalizer. Should we do this? Or does it run too much against the
> idea that handles should be managed as explicitly as possible?

I'm not sure what you mean by "keeps alive", it passes the handle to the
action, and closes it when the action returns, but the action, can (if
it so chooses) also close the handle in a more timely manner, because
"hClose" is idempotent.  So it does not seem like any change is needed
here...  Am I missing something?

The below compiles and runs, with no errors:

    module Main (main) where
    import System.IO

    main :: IO ()
    main = do
        withFile "/etc/passwd" ReadMode $ \f -> hClose f >> hClose f
        withFile "/etc/passwd" ReadMode $ \f -> hClose f >> hClose f

--
    Viktor.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

David Feuer
The handle will not be closed early if it becomes unreachable within
the user action (without the user closing it). It sounds like the
consensus is that that's how it should be, so I'll leave that well
enough alone.

On Wed, Dec 30, 2020 at 4:32 PM Viktor Dukhovni <[hidden email]> wrote:

>
> On Wed, Dec 30, 2020 at 02:23:20PM -0500, David Feuer wrote:
>
> > withFile keeps the handle it creates alive until the user action completes.
> > If we want to avoid this, we can. All that's required is to capture the
> > weak reference that holds the handle's finalizer. Then instead of closing
> > the handle in the end (or on exception) using hClose, withFile can run the
> > handle's finalizer. Should we do this? Or does it run too much against the
> > idea that handles should be managed as explicitly as possible?
>
> I'm not sure what you mean by "keeps alive", it passes the handle to the
> action, and closes it when the action returns, but the action, can (if
> it so chooses) also close the handle in a more timely manner, because
> "hClose" is idempotent.  So it does not seem like any change is needed
> here...  Am I missing something?
>
> The below compiles and runs, with no errors:
>
>     module Main (main) where
>     import System.IO
>
>     main :: IO ()
>     main = do
>         withFile "/etc/passwd" ReadMode $ \f -> hClose f >> hClose f
>         withFile "/etc/passwd" ReadMode $ \f -> hClose f >> hClose f
>
> --
>     Viktor.
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

Viktor Dukhovni
On Wed, Dec 30, 2020 at 04:39:52PM -0500, David Feuer wrote:

> The handle will not be closed early if it becomes unreachable within
> the user action (without the user closing it). It sounds like the
> consensus is that that's how it should be, so I'll leave that well
> enough alone.

I see, the converse of what I guessed you were saying.  I agree this is
not worth changing.  It would amount to relying on GC to close the
handle perhaps a bit earlier, and my take is that any such reliance is a
bug.

If the action keeps executing long after the file is no longer needed
then the programmer used "withFile" in too broad a context and needs to
narrow its scope to just the portion of the code that is actually using
the file.

--
    Viktor.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: withFile defeats dead handle closure

Sven Panne-2
Am Mi., 30. Dez. 2020 um 22:47 Uhr schrieb Viktor Dukhovni <[hidden email]>:
[...]  It would amount to relying on GC to close the handle perhaps a bit earlier, and my take is that any such reliance is a bug.[...]

+1

As a general rule of thumb: Automatic memory management is good for, well, memory management, but nothing else. Quite the opposite: Most of the time it's horrible for other tasks. There are tons of points in the design space of automatic memory management with vastly different trade-offs regarding throughput, latency, concurrency etc., so tying other "heavy" resources like file descriptors to automatic memory management is a very, very bad idea. People had cunning ideas about how to (ab-)use finalizers etc. for decades, but basically all of them turned out to be bad. Hans-J. Boehm has written extensively about this, and his papers are still worth reading today.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries