Windows: openFile gives permission denied when file in use

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Windows: openFile gives permission denied when file in use

Michael Snoyman
Hi all,

I just received a bug report from a client that, when an input file is
open in FrameMaker, my program gives a "permission denied error". This
bug is reproducible with a simple Haskell program:

import System.IO

main = do
    putStrLn "here1"
    h <- openFile "filename.txt" ReadMode
    putStrLn "here2"

I tried writing a simple C program using fopen, and it ran just fine.
Does anyone have experience with this issue, and know of a workaround?

Thanks,
Michael

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied when file in use

Antoine Latter-2
On Wed, Dec 28, 2011 at 3:52 PM, Michael Snoyman <[hidden email]> wrote:

> Hi all,
>
> I just received a bug report from a client that, when an input file is
> open in FrameMaker, my program gives a "permission denied error". This
> bug is reproducible with a simple Haskell program:
>
> import System.IO
>
> main = do
>    putStrLn "here1"
>    h <- openFile "filename.txt" ReadMode
>    putStrLn "here2"
>
> I tried writing a simple C program using fopen, and it ran just fine.
> Does anyone have experience with this issue, and know of a workaround?
>

When GHC opens files for reading, it asks windows to disallow write
access to the file. I'm guessing that Framemaker has the file open for
writing, so GHC can't get that permission.

I imagine that the Haskell runtime does this to make lazy-io less crazy.

Here's the call GHC makes:
https://github.com/ghc/packages-base/blob/0e1a02b96cfd03b8488e3ff4ce232466d6d5ca77/include/HsBase.h#L580

To open a file for reading in your C demo in a similar way you could
do something like:

fd = _sopen("file_name", _O_RDONLY | _O_NOINHERIT,_SH_DENYWR, 0);

Here "_SH_DENYWR" is telling windows to deny others from writing to this file.

Here's the msdn link for _sopen and _wsopen:
http://msdn.microsoft.com/en-us/library/w7sa2b22%28VS.80%29.aspx

I haven't tested any of that, but that should help you in reproducing
how GHC opens files for read on windows.

You should be able to use System.Win32.File.createFile with a share
mode of (fILE_SHARE_READ .|. fILE_SHARE_WRITE) and then wrangling a
Haskell Handle out of that, but I haven't tried it.

Or you could modify the call to _wsopen and FFI call that - it takes
fewer parameters and might be less confusing.

Antoine

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied when file in use

Chris Wong
On Thu, Dec 29, 2011 at 2:45 PM, Antoine Latter <[hidden email]> wrote:
> [...]
>
> When GHC opens files for reading, it asks windows to disallow write
> access to the file. I'm guessing that Framemaker has the file open for
> writing, so GHC can't get that permission.

In fact, this is required behavior according to the Haskell Report:

> Implementations should enforce as far as possible, at least locally to the Haskell process, multiple-reader single-writer locking on files. That is, there may either be many handles on the same file which manage input, or just one handle on the file which manages output.

I guess on Windows, "as far as possible" means locking it across the
whole system.

(See http://www.haskell.org/onlinereport/haskell2010/haskellch41.html#x49-32800041.3.4
for the gory details)

-- Chris

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied when file in use

Steve Schafer
On Thu, 29 Dec 2011 16:10:03 +1300, [hidden email] wrote:

>In fact, this is required behavior according to the Haskell Report:
>
>> Implementations should enforce as far as possible, at least locally
>> to the Haskell process, multiple-reader single-writer locking on
>> files. That is, there may either be many handles on the same file
>> which manage input, or just one handle on the file which manages
>> output.

The second sentence somewhat contradicts the first.

The first sentence says, "multiple-reader single-writer" which implies
multiple readers AND at most one writer (i.e., at the same time). This
is pretty typical file-locking behavior, and, from the symptoms, appears
to be the way that Framemaker opens the file.

The second sentence, on the other hand, implies that there can be
multiple readers OR one writer, but not both (which appears to be the
way that GHC operates).

>I guess on Windows, "as far as possible" means locking it across the
>whole system.

Windows does allow finer-grained control (including byte-range locking),
but most applications don't bother.

-Steve Schafer

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied when file in use

Antoine Latter-2
In reply to this post by Michael Snoyman
On Wed, Dec 28, 2011 at 3:52 PM, Michael Snoyman <[hidden email]> wrote:
> Hi all,
>
> I just received a bug report from a client that, when an input file is
> open in FrameMaker, my program gives a "permission denied error". This
> bug is reproducible with a simple Haskell program:
>

This bug and its discussion is similar, but not identical:
http://hackage.haskell.org/trac/ghc/ticket/4363

> import System.IO
>
> main = do
>    putStrLn "here1"
>    h <- openFile "filename.txt" ReadMode
>    putStrLn "here2"
>
> I tried writing a simple C program using fopen, and it ran just fine.
> Does anyone have experience with this issue, and know of a workaround?
>
> Thanks,
> Michael
>
> _______________________________________________
> Haskell-Cafe mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied when file in use

Michael Snoyman
In reply to this post by Antoine Latter-2
On Thu, Dec 29, 2011 at 3:45 AM, Antoine Latter <[hidden email]> wrote:

> On Wed, Dec 28, 2011 at 3:52 PM, Michael Snoyman <[hidden email]> wrote:
>> Hi all,
>>
>> I just received a bug report from a client that, when an input file is
>> open in FrameMaker, my program gives a "permission denied error". This
>> bug is reproducible with a simple Haskell program:
>>
>> import System.IO
>>
>> main = do
>>    putStrLn "here1"
>>    h <- openFile "filename.txt" ReadMode
>>    putStrLn "here2"
>>
>> I tried writing a simple C program using fopen, and it ran just fine.
>> Does anyone have experience with this issue, and know of a workaround?
>>
>
> When GHC opens files for reading, it asks windows to disallow write
> access to the file. I'm guessing that Framemaker has the file open for
> writing, so GHC can't get that permission.
>
> I imagine that the Haskell runtime does this to make lazy-io less crazy.
>
> Here's the call GHC makes:
> https://github.com/ghc/packages-base/blob/0e1a02b96cfd03b8488e3ff4ce232466d6d5ca77/include/HsBase.h#L580
>
> To open a file for reading in your C demo in a similar way you could
> do something like:
>
> fd = _sopen("file_name", _O_RDONLY | _O_NOINHERIT,_SH_DENYWR, 0);
>
> Here "_SH_DENYWR" is telling windows to deny others from writing to this file.
>
> Here's the msdn link for _sopen and _wsopen:
> http://msdn.microsoft.com/en-us/library/w7sa2b22%28VS.80%29.aspx
>
> I haven't tested any of that, but that should help you in reproducing
> how GHC opens files for read on windows.
>
> You should be able to use System.Win32.File.createFile with a share
> mode of (fILE_SHARE_READ .|. fILE_SHARE_WRITE) and then wrangling a
> Haskell Handle out of that, but I haven't tried it.
>
> Or you could modify the call to _wsopen and FFI call that - it takes
> fewer parameters and might be less confusing.
>
> Antoine

Thanks for the advice Antoine, it was spot on. I modified my
uri-conduit package with the following commit:

https://github.com/snoyberg/xml/commit/a4763739093c525d8f509b11e72a2d17894afaac

Since conduits re-implement buffering themselves, I don't think
there's any advantage to wrapping up the FD in a Handle again, except
perhaps for better integration with the async calls of the
multi-threaded runtime. But since I'm not using that for my Windows
code, and I don't think the multi-threaded runtime supports Windows
particularly well in the first place, this seems like an acceptable
trade-off.

Does anyone see any issues with the code? Would it be useful for me to
expose this code elsewhere, such as in conduit itself?

Michael

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied when file in use

Andrew Coppin
In reply to this post by Antoine Latter-2
On 29/12/2011 04:29 AM, Antoine Latter wrote:

> On Wed, Dec 28, 2011 at 3:52 PM, Michael Snoyman<[hidden email]>  wrote:
>> Hi all,
>>
>> I just received a bug report from a client that, when an input file is
>> open in FrameMaker, my program gives a "permission denied error". This
>> bug is reproducible with a simple Haskell program:
>>
>
> This bug and its discussion is similar, but not identical:
> http://hackage.haskell.org/trac/ghc/ticket/4363

This one has been rumbling on for ages. As others have said, the Report
demands that locking occur, which is probably a mistake. The daft thing
is, apparently Linux doesn't really support locking, so on that platform
these types of thing all work fine, and only on Windows, which does and
always has supported propper locking, people get these errors. And yet,
many people seem surprised to hear that Windows can actually turn off
the locking; they seem completely unaware of the extensive and highly
flexible locking facilities that Windows provides. Every time I hear
"oh, I don't think Windows can handle that", I sigh with resignation.

Whatever we change the default to, it would be useful if we could expose
locking mechanisms in a reasonably portable way so that people who care
about such things can alter the default if it is not what they want.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied when file in use

Iustin Pop-2
On Thu, Dec 29, 2011 at 11:49:11AM +0000, Andrew Coppin wrote:

> On 29/12/2011 04:29 AM, Antoine Latter wrote:
> >On Wed, Dec 28, 2011 at 3:52 PM, Michael Snoyman<[hidden email]>  wrote:
> >>Hi all,
> >>
> >>I just received a bug report from a client that, when an input file is
> >>open in FrameMaker, my program gives a "permission denied error". This
> >>bug is reproducible with a simple Haskell program:
> >>
> >
> >This bug and its discussion is similar, but not identical:
> >http://hackage.haskell.org/trac/ghc/ticket/4363
>
> This one has been rumbling on for ages. As others have said, the
> Report demands that locking occur, which is probably a mistake. The
> daft thing is, apparently Linux doesn't really support locking, so
> on that platform these types of thing all work fine, and only on
> Windows, which does and always has supported propper locking, people
> get these errors. And yet, many people seem surprised to hear that
> Windows can actually turn off the locking; they seem completely
> unaware of the extensive and highly flexible locking facilities that
> Windows provides. Every time I hear "oh, I don't think Windows can
> handle that", I sigh with resignation.

Sorry to say, but it seems you yourself are unaware of the "extensive
and highly flexible" locking facilities on Linux :) The defaults on
Linux are advisory locking, not mandatory, but claiming Linux doesn't
support locking is plain wrong.

regards,
iustin

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied when file in use

Andrew Coppin
>> Every time I hear "oh, I don't think Windows can
>> handle that", I sigh with resignation.
>
> Sorry to say, but it seems you yourself are unaware of the "extensive
> and highly flexible" locking facilities on Linux :) The defaults on
> Linux are advisory locking, not mandatory, but claiming Linux doesn't
> support locking is plain wrong.

I would suggest that advisory locking isn't particularly useful. I
gather that Linux does now support real locking though. (And file update
notifications, and ACLs, and lots of other things that Windows has had
for far longer.)

Either way, I have no interest in starting a Windows vs Linux flamewar.
I'm just saying it would be nice if Haskell could support more of what
these two OSes have to offer.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied when file in use

Iustin Pop-2
On Thu, Dec 29, 2011 at 12:20:18PM +0000, Andrew Coppin wrote:
> >>Every time I hear "oh, I don't think Windows can
> >>handle that", I sigh with resignation.
> >
> >Sorry to say, but it seems you yourself are unaware of the "extensive
> >and highly flexible" locking facilities on Linux :) The defaults on
> >Linux are advisory locking, not mandatory, but claiming Linux doesn't
> >support locking is plain wrong.
>
> I would suggest that advisory locking isn't particularly useful.

In my experience (as an application writer) it is very useful; it's just
a different paradigm, not a weaker one. Off-hand I don't remember a case
where having mandatory locking would have improved things.

> I
> gather that Linux does now support real locking though. (And file
> update notifications, and ACLs, and lots of other things that
> Windows has had for far longer.)

Hrmm: "Mandatory File Locking For The Linux Operating System, 15 April
1996" :)

> Either way, I have no interest in starting a Windows vs Linux
> flamewar.

Me neither - I just wanted to point out that your email sounded _too_
eager to blame the current state of affairs on the fact that Linux
doesn't support proper locking. Whereas, the problem is just one of
taking into account platform differences better.

> I'm just saying it would be nice if Haskell could support
> more of what these two OSes have to offer.

Totally agreed!

all the best,
iustin

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied when file in use

Andrew Coppin
>> I
>> gather that Linux does now support real locking though. (And file
>> update notifications, and ACLs, and lots of other things that
>> Windows has had for far longer.)
>
> Hrmm: "Mandatory File Locking For The Linux Operating System, 15 April
> 1996" :)

Have a reference for when it was actually implemented?

I tried to look this up, and got this:

http://0pointer.de/blog/projects/locking.html

Sounds like it's not so "solved" after all. (Or perhaps the author just
doesn't know enough about it, it's hard to say...) The Wikipedia page on
file locking also points out a list of Unix-related locking problems.
(But again, that might just be editor bias.)

>> I'm just saying it would be nice if Haskell could support
>> more of what these two OSes have to offer.
>
> Totally agreed!

Good, that was my main point. Now, who's volunteering to implement all
this? :-}

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied whenfile in use

Donn Cave-4
In reply to this post by Andrew Coppin
Quoth Andrew Coppin <[hidden email]>,
> On 29/12/2011 04:29 AM, Antoine Latter wrote:
...
>> This bug and its discussion is similar, but not identical:
>> http://hackage.haskell.org/trac/ghc/ticket/4363
>
> This one has been rumbling on for ages. As others have said, the Report
> demands that locking occur, which is probably a mistake.

The rationale that followed may have been a little sketchy, but
apparently everyone agrees with with the point.  I see the ticket
led to a discussion on the libraries list -
http://www.haskell.org/pipermail/libraries/2011-October/016978.html

... wherein Ian Lynagh proposed to remove this feature and let the
programmer enforce locking or not, as in other programming languages'
base I/O libraries.  This met with enthusiastic universal support,
so whatever the Report may say, it looks to me like the GHC libraries
will eventually not do this.

        Donn

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied whenfile in use

Antoine Latter-2
On Thu, Dec 29, 2011 at 10:53 AM, Donn Cave <[hidden email]> wrote:

> Quoth Andrew Coppin <[hidden email]>,
>> On 29/12/2011 04:29 AM, Antoine Latter wrote:
> ...
>>> This bug and its discussion is similar, but not identical:
>>> http://hackage.haskell.org/trac/ghc/ticket/4363
>>
>> This one has been rumbling on for ages. As others have said, the Report
>> demands that locking occur, which is probably a mistake.
>
> The rationale that followed may have been a little sketchy, but
> apparently everyone agrees with with the point.  I see the ticket
> led to a discussion on the libraries list -
> http://www.haskell.org/pipermail/libraries/2011-October/016978.html
>
> ... wherein Ian Lynagh proposed to remove this feature and let the
> programmer enforce locking or not, as in other programming languages'
> base I/O libraries.  This met with enthusiastic universal support,
> so whatever the Report may say, it looks to me like the GHC libraries
> will eventually not do this.
>

Wouldn't this lead to 'getContents' and friends being much less safe
than they already are? Or would we have to do something at the GHC
runtime level of things to add locking?

Antoine

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied whenfilein use

Donn Cave-4
Quoth Antoine Latter <[hidden email]>,
...

>> http://www.haskell.org/pipermail/libraries/2011-October/016978.html
>>
>> ... wherein Ian Lynagh proposed to remove this feature and let the
>> programmer enforce locking or not, as in other programming languages'
>> base I/O libraries.  This met with enthusiastic universal support,
>> so whatever the Report may say, it looks to me like the GHC libraries
>> will eventually not do this.
>>
>
> Wouldn't this lead to 'getContents' and friends being much less safe
> than they already are? Or would we have to do something at the GHC
> runtime level of things to add locking?

Interesting question.  I tend to steer clear of that function, rather
than try to keep track of all the things that can go wrong with it!
but I would guess, the effect is the same on any I/O strategy that
executes the same way: e.g. if you were to read pieces of the file
iteratively you would be similarly exposed to the possibility of
concurrent modifications.

That risk may be less obvious with getContents if you take the naive
view that it returns the contents of the file in the same way that normal
I/O operations return data, but then you will eventually be punished for
that naivety anyway!

Are you aware of some possible special issue with getContents?

        Donn

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission denied whenfilein use

Antoine Latter-2
On Thu, Dec 29, 2011 at 11:49 AM, Donn Cave <[hidden email]> wrote:

> Quoth Antoine Latter <[hidden email]>,
> ...
>>> http://www.haskell.org/pipermail/libraries/2011-October/016978.html
>>>
>>> ... wherein Ian Lynagh proposed to remove this feature and let the
>>> programmer enforce locking or not, as in other programming languages'
>>> base I/O libraries.  This met with enthusiastic universal support,
>>> so whatever the Report may say, it looks to me like the GHC libraries
>>> will eventually not do this.
>>>
>>
>> Wouldn't this lead to 'getContents' and friends being much less safe
>> than they already are? Or would we have to do something at the GHC
>> runtime level of things to add locking?
>
> Interesting question.  I tend to steer clear of that function, rather
> than try to keep track of all the things that can go wrong with it!
> but I would guess, the effect is the same on any I/O strategy that
> executes the same way: e.g. if you were to read pieces of the file
> iteratively you would be similarly exposed to the possibility of
> concurrent modifications.
>
> That risk may be less obvious with getContents if you take the naive
> view that it returns the contents of the file in the same way that normal
> I/O operations return data, but then you will eventually be punished for
> that naivety anyway!
>
> Are you aware of some possible special issue with getContents?
>

Well, the issue would more be with 'hGetContents'.

Would this program then loop:

append fromFilePath toFilePath = do
  str <- readFile fromFile
  writeFile toFile str

if 'from' and 'to' where the same file?

Currently the locking prevents this.

Antoine

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission deniedwhenfilein use

Donn Cave-4
Quoth Antoine Latter <[hidden email]>,
...
> Would this program then loop:
>
> append fromFilePath toFilePath = do
>   str <- readFile fromFile
>   writeFile toFile str
>
> if 'from' and 'to' where the same file?
>
> Currently the locking prevents this.

Do you mean, locking makes that work, or just makes it fail in a
different way?  Anyway, I think that's an example that would have
the same issues without hGetContents - that is, any way you set
out to overwrite a file by modifying its contents, you have the
same problem.

        Donn

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Windows: openFile gives permission deniedwhenfilein use

Antoine Latter-2
On Thu, Dec 29, 2011 at 12:28 PM, Donn Cave <[hidden email]> wrote:

> Quoth Antoine Latter <[hidden email]>,
> ...
>> Would this program then loop:
>>
>> append fromFilePath toFilePath = do
>>   str <- readFile fromFile
>>   writeFile toFile str
>>
>> if 'from' and 'to' where the same file?
>>
>> Currently the locking prevents this.
>
> Do you mean, locking makes that work, or just makes it fail in a
> different way?  Anyway, I think that's an example that would have
> the same issues without hGetContents - that is, any way you set
> out to overwrite a file by modifying its contents, you have the
> same problem.
>

Locking makes this fail with an exception, I think. Not locking would
make this mis-behave.

And lets pretend I wrote the function that was in my head:

append fromFilePath toFilePath = do
   str <- readFile fromFile
   appendFile toFile str

Here, without locking, I would expect to fill my disk up, whereas
previously my program would have crashed before writing out any data.

Maybe these are pathological examples, but they are (I think!) the
justification for having the locking in the first place.

Antoine

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe