about openTempFile

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

about openTempFile

Manlio Perillo-3
Hi.

After reading the chapter about IO in the "Real Word Haskell" book, I
have noted that there is no support for opening a temporary file that
will be automatically removed at program termination.

The Python tempfile module, as an example, implements a wrapper around
mkstemp function that does exactly this, and the code is portable; on
Windows it uses O_TEMPORARY_FILE flag, on POSIX systems the file is
unlink-ed as soon as it is created (but note that the code is not signal
safe - well, many functions in the Python standard library are not
signal safe).

There are reasons why GHC library does not implement this?


The Python version also set the FD_CLOEXEC, O_NOINHERIT and O_NOFOLLOW
flags (where available).
The GHC version, instead, set the O_NOCTTY flag.



Manlio Perillo
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: about openTempFile

Dougal Stanton-2
On Wed, Sep 17, 2008 at 1:17 PM, Manlio Perillo
<[hidden email]> wrote:
> The Python tempfile module, as an example, implements a wrapper around
> mkstemp function that does exactly this, and the code is portable; on
> Windows it uses O_TEMPORARY_FILE flag, on POSIX systems the file is
> unlink-ed as soon as it is created (but note that the code is not signal
> safe - well, many functions in the Python standard library are not signal
> safe).

Something like:

> withTempFile :: String -> ((FilePath, Handle) -> IO b) -> IO b
> withTempFile name action = do
>     tmpdir <- getTemporaryDirectory
>     bracket
>         (openTempFile tmpdir name)
>         (\(fp,h) -> hClose h >> removeFile fp)
>         action

I don't know if this has the safety requirements you mean?


--
Dougal Stanton
[hidden email] // http://www.dougalstanton.net
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: about openTempFile

David Roundy-2
On Wed, Sep 17, 2008 at 02:10:56PM +0100, Dougal Stanton wrote:

> On Wed, Sep 17, 2008 at 1:17 PM, Manlio Perillo
> <[hidden email]> wrote:
> > The Python tempfile module, as an example, implements a wrapper around
> > mkstemp function that does exactly this, and the code is portable; on
> > Windows it uses O_TEMPORARY_FILE flag, on POSIX systems the file is
> > unlink-ed as soon as it is created (but note that the code is not signal
> > safe - well, many functions in the Python standard library are not signal
> > safe).
>
> Something like:
>
> > withTempFile :: String -> ((FilePath, Handle) -> IO b) -> IO b
> > withTempFile name action = do
> >     tmpdir <- getTemporaryDirectory
> >     bracket
> >         (openTempFile tmpdir name)
> >         (\(fp,h) -> hClose h >> removeFile fp)
> >         action
>
> I don't know if this has the safety requirements you mean?

You need to be sure to use the bracket from Control.Exception, not the
one from System.IO or IO (which won't work).  And you also need
special work to make the code safe from signals.  But basically, this
is the right idea.

David
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: about openTempFile

Manlio Perillo-3
In reply to this post by Dougal Stanton-2
Dougal Stanton ha scritto:

> On Wed, Sep 17, 2008 at 1:17 PM, Manlio Perillo
> <[hidden email]> wrote:
>> The Python tempfile module, as an example, implements a wrapper around
>> mkstemp function that does exactly this, and the code is portable; on
>> Windows it uses O_TEMPORARY_FILE flag, on POSIX systems the file is
>> unlink-ed as soon as it is created (but note that the code is not signal
>> safe - well, many functions in the Python standard library are not signal
>> safe).
>
> Something like:
>
>> withTempFile :: String -> ((FilePath, Handle) -> IO b) -> IO b
>> withTempFile name action = do
>>     tmpdir <- getTemporaryDirectory
>>     bracket
>>         (openTempFile tmpdir name)
>>         (\(fp,h) -> hClose h >> removeFile fp)
>>         action
>
> I don't know if this has the safety requirements you mean?
>

Yes, this is the ideal solution for Haskell.

But since GHC does not implement such a function, I was curious to know
why it don't even implement a function that make sure the temporary file
is removed.

>


Thanks  Manlio Perillo
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: about openTempFile

Don Stewart-2
In reply to this post by Manlio Perillo-3
manlio_perillo:

> Hi.
>
> After reading the chapter about IO in the "Real Word Haskell" book, I
> have noted that there is no support for opening a temporary file that
> will be automatically removed at program termination.
>
> The Python tempfile module, as an example, implements a wrapper around
> mkstemp function that does exactly this, and the code is portable; on
> Windows it uses O_TEMPORARY_FILE flag, on POSIX systems the file is
> unlink-ed as soon as it is created (but note that the code is not signal
> safe - well, many functions in the Python standard library are not
> signal safe).
>
> There are reasons why GHC library does not implement this?
>
>
> The Python version also set the FD_CLOEXEC, O_NOINHERIT and O_NOFOLLOW
> flags (where available).
> The GHC version, instead, set the O_NOCTTY flag.

It would be something like:

    withTempFile :: FilePath -> String -> ((FilePath,Handle) -> IO r) -> IO r
    withTempFile name tmp = bracket (openTempFile name tmp)
                                    (\(f,h) -> hClose h >> removeFile f)

-- Don
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: about openTempFile

Don Stewart-2
In reply to this post by Manlio Perillo-3
manlio_perillo:

> Dougal Stanton ha scritto:
> >On Wed, Sep 17, 2008 at 1:17 PM, Manlio Perillo
> ><[hidden email]> wrote:
> >>The Python tempfile module, as an example, implements a wrapper around
> >>mkstemp function that does exactly this, and the code is portable; on
> >>Windows it uses O_TEMPORARY_FILE flag, on POSIX systems the file is
> >>unlink-ed as soon as it is created (but note that the code is not signal
> >>safe - well, many functions in the Python standard library are not signal
> >>safe).
> >
> >Something like:
> >
> >>withTempFile :: String -> ((FilePath, Handle) -> IO b) -> IO b
> >>withTempFile name action = do
> >>    tmpdir <- getTemporaryDirectory
> >>    bracket
> >>        (openTempFile tmpdir name)
> >>        (\(fp,h) -> hClose h >> removeFile fp)
> >>        action
> >
> >I don't know if this has the safety requirements you mean?
> >
>
> Yes, this is the ideal solution for Haskell.
>
> But since GHC does not implement such a function, I was curious to know
> why it don't even implement a function that make sure the temporary file
> is removed.
>

Because it is two lines to write, so no one has yet proposed it for the
base library.

-- Don
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

RE: about openTempFile

Mitchell, Neil
> > But since GHC does not implement such a function, I was curious to
> > know why it don't even implement a function that make sure the
> > temporary file is removed.
> >
>
> Because it is two lines to write, so no one has yet proposed
> it for the base library.

Map is 2 lines, but we have that as a library (1 line in Hugs).
Otherwise is a whole one lexeme long, but we still provide a library
function.

This one is tricky and has lots of edge cases! If its useful, it should
be in the libraries.

Thanks

Neil

==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:

http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: about openTempFile

David Roundy-2
On Wed, Sep 17, 2008 at 04:51:07PM +0100, Mitchell, Neil wrote:

> > > But since GHC does not implement such a function, I was curious to
> > > know why it don't even implement a function that make sure the
> > > temporary file is removed.
> > >
> >
> > Because it is two lines to write, so no one has yet proposed
> > it for the base library.
>
> Map is 2 lines, but we have that as a library (1 line in Hugs).
> Otherwise is a whole one lexeme long, but we still provide a library
> function.
>
> This one is tricky and has lots of edge cases! If its useful, it should
> be in the libraries.

In particular, the code Don quoted is incorrect depending on which
import statements are used.  If we assume that the default is the
bracket available in Haskell 98, then it is definitely incorrect.

It also doesn't behave properly in the presence of signals or keyboard
interrupts (i.e. posix or windows systems).  Of course, a larger
change in the standard libraries is needed in order to solve this
particular problem, so I'd want to agree on a solution for that
problem first.  Which is perhaps a better reason why it's not yet in
the standard libraries: it's perhaps not a good idea to put a function
with such serious issues into the standard libraries.

David
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: about openTempFile

Brandon S Allbery KF8NH
In reply to this post by Manlio Perillo-3
On 2008 Sep 17, at 8:17, Manlio Perillo wrote:
> The Python tempfile module, as an example, implements a wrapper  
> around mkstemp function that does exactly this, and the code is  
> portable; on Windows it uses O_TEMPORARY_FILE flag, on POSIX systems  
> the file is unlink-ed as soon as it is created (but note that the  
> code is not signal safe - well, many functions in the Python  
> standard library are not signal safe).
>
> There are reasons why GHC library does not implement this?

POSIX doesn't guaranteed that open-and-unlink works; HP-UX is a  
"POSIX" platform on which it doesn't.

--
brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [hidden email]
system administrator [openafs,heimdal,too many hats] [hidden email]
electrical and computer engineering, carnegie mellon university    KF8NH


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: about openTempFile

Manlio Perillo-3
In reply to this post by David Roundy-2
David Roundy ha scritto:
> [...]
> In particular, the code Don quoted is incorrect depending on which
> import statements are used.  If we assume that the default is the
> bracket available in Haskell 98, then it is definitely incorrect.
>
> It also doesn't behave properly in the presence of signals or keyboard
> interrupts (i.e. posix or windows systems).  

Maybe a withSignalsMasked function, at least on Posix systems (it is not
defined in System.Posix.Signals)?

I have no idea how signals works under Windows.

 > [...]


Manlio Perillo
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: about openTempFile

David Roundy-2
On Wed, Sep 17, 2008 at 10:11:28PM +0200, Manlio Perillo wrote:

> David Roundy ha scritto:
>> [...]
>> In particular, the code Don quoted is incorrect depending on which
>> import statements are used.  If we assume that the default is the
>> bracket available in Haskell 98, then it is definitely incorrect.
>>
>> It also doesn't behave properly in the presence of signals or keyboard
>> interrupts (i.e. posix or windows systems).  
>
> Maybe a withSignalsMasked function, at least on Posix systems (it is not  
> defined in System.Posix.Signals)?

Darcs does something like this, and I obviously think it should be
moved into the standard libraries.  In my opinion signals ought to
throw asynchronous exceptions by default, then we'd have unified
handling of exceptional cases.

> I have no idea how signals works under Windows.

Windows doesn't have signals per se, but does have something similar
for keyboard interrupts, which in darcs we also convert into an
asynchronous exception.  I'm not clear as to how well it works.  Under
cygwin shells, ctrl-C does something other than send a windows
keyboard interrupt, so darcs doesn't work so well in that
environment.

David
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: about openTempFile

Aaron Denney
In reply to this post by Brandon S Allbery KF8NH
On 2008-09-17, Brandon S. Allbery KF8NH <[hidden email]> wrote:

> On 2008 Sep 17, at 8:17, Manlio Perillo wrote:
>> The Python tempfile module, as an example, implements a wrapper  
>> around mkstemp function that does exactly this, and the code is  
>> portable; on Windows it uses O_TEMPORARY_FILE flag, on POSIX systems  
>> the file is unlink-ed as soon as it is created (but note that the  
>> code is not signal safe - well, many functions in the Python  
>> standard library are not signal safe).
>>
>> There are reasons why GHC library does not implement this?
>
> POSIX doesn't guaranteed that open-and-unlink works; HP-UX is a  
> "POSIX" platform on which it doesn't.

Huh.  SuS does indeed allow EBUSY for "The file named by the path
argument cannot be unlinked because it is being used by the system or
another process and the implementation considers this an error."

Did HPUX's behavior change at some point?  This is a standard idiom,
and I don't remember having any trouble with it, but I haven't used
anything earlier than 9.  The manpages for 11 only document being a
mount point as cause for EBUSY.

--
Aaron Denney
-><-

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: about openTempFile

Brandon S Allbery KF8NH
On 2008 Sep 17, at 18:20, Aaron Denney wrote:

> On 2008-09-17, Brandon S. Allbery KF8NH <[hidden email]> wrote:
>> On 2008 Sep 17, at 8:17, Manlio Perillo wrote:
>>> The Python tempfile module, as an example, implements a wrapper
>>> around mkstemp function that does exactly this, and the code is
>>> portable; on Windows it uses O_TEMPORARY_FILE flag, on POSIX systems
>>> the file is unlink-ed as soon as it is created (but note that the
>>> code is not signal safe - well, many functions in the Python
>>> standard library are not signal safe).
>>>
>>> There are reasons why GHC library does not implement this?
>>
>> POSIX doesn't guaranteed that open-and-unlink works; HP-UX is a
>> "POSIX" platform on which it doesn't.
>
> Did HPUX's behavior change at some point?  This is a standard idiom,
> and I don't remember having any trouble with it, but I haven't used
> anything earlier than 9.  The manpages for 11 only document being a
> mount point as cause for EBUSY.

It may have but I recall it being cited as an issue with HP/UX 9 and 10.

--
brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [hidden email]
system administrator [openafs,heimdal,too many hats] [hidden email]
electrical and computer engineering, carnegie mellon university    KF8NH


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe