File operations in base and fork+exec

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

File operations in base and fork+exec

Merijn Verstraaten
Hi!

So I realise proposing backwards incompatible changes to base is tilting at windmills, but I just noticed that most (all?) file operations in base don't open files with O_CLOEXEC, which basically means that every open Handle/file descriptor gets leaked to every single subprocess you fork+exec from Haskell. This seems highly undesirable. For POSIX there appears to be an operation in base that disables this, but it doesn't seem to easily exported.

I would like to propose that all operations in base open files with O_CLOEXEC by default and exposing an easy method to mark specific file descriptors to be preserved across exec (This can be achieved by using fcntl to remove the CLOEXEC bit from a specific fd).

I'm not sure to what extent Windows behaviour is similar to POSIX, but in principle it should behave the same, I think.

Cheers,
Merijn
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: File operations in base and fork+exec

John Lato-2

Big +1. I got bitten by this a few years ago and was going to do some work on the Unix package to make it easier to do this properly but never got around to it.


On Fri, Jun 23, 2017, 10:46 Merijn Verstraaten <[hidden email]> wrote:
Hi!

So I realise proposing backwards incompatible changes to base is tilting at windmills, but I just noticed that most (all?) file operations in base don't open files with O_CLOEXEC, which basically means that every open Handle/file descriptor gets leaked to every single subprocess you fork+exec from Haskell. This seems highly undesirable. For POSIX there appears to be an operation in base that disables this, but it doesn't seem to easily exported.

I would like to propose that all operations in base open files with O_CLOEXEC by default and exposing an easy method to mark specific file descriptors to be preserved across exec (This can be achieved by using fcntl to remove the CLOEXEC bit from a specific fd).

I'm not sure to what extent Windows behaviour is similar to POSIX, but in principle it should behave the same, I think.

Cheers,
Merijn
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: File operations in base and fork+exec

Niklas Hambüchen
In reply to this post by Merijn Verstraaten
Hi Merijn,

I have proposed the same 2 years ago:
https://mail.haskell.org/pipermail/haskell-cafe/2015-July/120523.html

I am very much in support of that.

As mentioned, Python shows that one can pull that kind of change cleanly.

Niklas

On 23/06/17 10:46, Merijn Verstraaten wrote:
> I would like to propose that all operations in base open files with O_CLOEXEC by default and exposing an easy method to mark specific file descriptors to be preserved across exec
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: File operations in base and fork+exec

Erik Hesselink
+1. Would be good to get this default in 'network' as well, I ran into it with sockets in the past. Relevant ticket: https://github.com/haskell/network/issues/119

Erik

On 23 June 2017 at 13:33, Niklas Hambüchen <[hidden email]> wrote:
Hi Merijn,

I have proposed the same 2 years ago:
https://mail.haskell.org/pipermail/haskell-cafe/2015-July/120523.html

I am very much in support of that.

As mentioned, Python shows that one can pull that kind of change cleanly.

Niklas

On 23/06/17 10:46, Merijn Verstraaten wrote:
> I would like to propose that all operations in base open files with O_CLOEXEC by default and exposing an easy method to mark specific file descriptors to be preserved across exec
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: File operations in base and fork+exec

Bardur Arantsson-2
In reply to this post by Merijn Verstraaten
On 2017-06-23 10:46, Merijn Verstraaten wrote:
> Hi!
>
> So I realise proposing backwards incompatible changes to base is tilting at windmills, but I just noticed that most (all?) file operations in base don't open files with O_CLOEXEC, which basically means that every open Handle/file descriptor gets leaked to every single subprocess you fork+exec from Haskell. This seems highly undesirable. For POSIX there appears to be an operation in base that disables this, but it doesn't seem to easily exported.
>
> I would like to propose that all operations in base open files with O_CLOEXEC by default and exposing an easy method to mark specific file descriptors to be preserved across exec (This can be achieved by using fcntl to remove the CLOEXEC bit from a specific fd).
>
> I'm not sure to what extent Windows behaviour is similar to POSIX, but in principle it should behave the same, I think.
>

+1. Current behavior is suboptimal for *almost* every use case and also
highly surprising.

Regards,

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: File operations in base and fork+exec

Edward Kmett-2
I'd like to jump on the "this has bitten me before" bandwagon. +1

-Edward

On Fri, Jun 23, 2017 at 1:49 PM, Bardur Arantsson <[hidden email]> wrote:
On 2017-06-23 10:46, Merijn Verstraaten wrote:
> Hi!
>
> So I realise proposing backwards incompatible changes to base is tilting at windmills, but I just noticed that most (all?) file operations in base don't open files with O_CLOEXEC, which basically means that every open Handle/file descriptor gets leaked to every single subprocess you fork+exec from Haskell. This seems highly undesirable. For POSIX there appears to be an operation in base that disables this, but it doesn't seem to easily exported.
>
> I would like to propose that all operations in base open files with O_CLOEXEC by default and exposing an easy method to mark specific file descriptors to be preserved across exec (This can be achieved by using fcntl to remove the CLOEXEC bit from a specific fd).
>
> I'm not sure to what extent Windows behaviour is similar to POSIX, but in principle it should behave the same, I think.
>

+1. Current behavior is suboptimal for *almost* every use case and also
highly surprising.

Regards,

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: File operations in base and fork+exec

Evan Laforge
Same here.  Very confusing bug and hard to track down.

On Fri, Jun 23, 2017 at 5:41 AM, Edward Kmett <[hidden email]> wrote:

> I'd like to jump on the "this has bitten me before" bandwagon. +1
>
> -Edward
>
> On Fri, Jun 23, 2017 at 1:49 PM, Bardur Arantsson <[hidden email]>
> wrote:
>>
>> On 2017-06-23 10:46, Merijn Verstraaten wrote:
>> > Hi!
>> >
>> > So I realise proposing backwards incompatible changes to base is tilting
>> > at windmills, but I just noticed that most (all?) file operations in base
>> > don't open files with O_CLOEXEC, which basically means that every open
>> > Handle/file descriptor gets leaked to every single subprocess you fork+exec
>> > from Haskell. This seems highly undesirable. For POSIX there appears to be
>> > an operation in base that disables this, but it doesn't seem to easily
>> > exported.
>> >
>> > I would like to propose that all operations in base open files with
>> > O_CLOEXEC by default and exposing an easy method to mark specific file
>> > descriptors to be preserved across exec (This can be achieved by using fcntl
>> > to remove the CLOEXEC bit from a specific fd).
>> >
>> > I'm not sure to what extent Windows behaviour is similar to POSIX, but
>> > in principle it should behave the same, I think.
>> >
>>
>> +1. Current behavior is suboptimal for *almost* every use case and also
>> highly surprising.
>>
>> Regards,
>>
>> _______________________________________________
>> Libraries mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>
>
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Loading...