Adding System.FilePath

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
90 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Adding System.FilePath

Neil Mitchell
Hi,

When it comes to adding System.FilePath, we seem to have reached a
concensus of indecision, so I am going to attempt to set out the
various options, and see if we can pick one.

1) Do not let programmers type "import System.FilePath", i.e. do not
include the module

No one seems to be advocating this.

2) Let programmers type "import System.FilePath" and have it work in
Cabal Setup.hs files, and in all Cabal built programs without
requiring the user specify that they are using the filepath package.

This is what everyone wants.  Within this there are two approaches
being advocated:

a) Add to base.

b) Add as a library which we demand all Haskell compilers ship, and
tweak Cabal to automatically use this library at all times.

Option a involves absolutely no additional work for me - the patch has
been posted to this mailing list. Option b involves Cabal hacking,
establishing new procedures, coming up with a definition of libraries
which are always available etc. It is likely that option b will be
required once (if ever) base is split up. I certainly don't have time
to do option b.

So can I propose we add filepath, with this agreement that once
(someone else) gets around to doing option b, we can split it out
afterwards?

The particular argument for upgrading the filepath library I think is
not important - I never intend to do more filepath hacking. Someone
might add one or two functions, but thats about the most I can
invision. It's not a particularly pleasant job, its ugly - the only
reason I started this was because the alternative is having everything
broken on Windows - something I do care about.

Thanks

Neil
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Sven Panne
On Thursday 15 March 2007 17:25, Neil Mitchell wrote:

> [...]
> So can I propose we add filepath, with this agreement that once
> (someone else) gets around to doing option b, we can split it out
> afterwards?
>
> The particular argument for upgrading the filepath library I think is
> not important - I never intend to do more filepath hacking. Someone
> might add one or two functions, but thats about the most I can
> invision. It's not a particularly pleasant job, its ugly - the only
> reason I started this was because the alternative is having everything
> broken on Windows - something I do care about.

I would prefer b), too, mainly because I think that splitting up base in a
sensible way most people can agree on is a Herculean task.

Although this has almost been discussed to death, I still have one issue: File
paths on  *nix are *not* strings, they are simply a bunch of bytes, hopefully
Haskell' will get this right. Adding a String-based module to base would give
a conceptually wrong thing an "official touch". :-( Don't get me wrong: The
operations in FilePath are OK, just the concrete representation is not
correct. *sigh*

What is the general strategy for Haskell' regarding this topic: Is I/O out of
the scope of Haskell' itself? If not, what are the plans/the strategy for
transitioning to something more correct? To be honest: I don't see an easy
way out without breaking lots of existing programs.

Cheers,
   S.
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re[2]: Adding System.FilePath

Bulat Ziganshin-2
Hello Sven,

Thursday, March 15, 2007, 8:10:18 PM, you wrote:

> What is the general strategy for Haskell' regarding this topic: Is I/O out of
> the scope of Haskell' itself? If not, what are the plans/the strategy for
> transitioning to something more correct? To be honest: I don't see an easy
> way out without breaking lots of existing programs.

i see: just providing operations in libraries and allow one to import
libraries he need. so, one can import FilePath 1.0 with String-based
path representation or FilePath 2.0 with ByteString-based one


--
Best regards,
 Bulat                            mailto:[hidden email]

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Sven Panne
On Thursday 15 March 2007 18:57, Bulat Ziganshin wrote:
> Thursday, March 15, 2007, 8:10:18 PM, you wrote:
> > What is the general strategy for Haskell' regarding this topic: Is I/O
> > out of the scope of Haskell' itself? If not, what are the plans/the
> > strategy for transitioning to something more correct? To be honest: I
> > don't see an easy way out without breaking lots of existing programs.
>
> i see: just providing operations in libraries and allow one to import
> libraries he need. so, one can import FilePath 1.0 with String-based
> path representation or FilePath 2.0 with ByteString-based one

Alas, this is no solution at all. The point is: Currently there is no explicit
conversion FilePath <-> String, although this is really needed. Users e.g.
enter FilePaths somehow through a GUI, on the command line, in a
configuration file etc., so you often start with a String. But this has to be
converted somehow to a bunch of bytes for accessing the file, and this can't
be handled without an API change (where does the encoding come from?). The
same holds for reading e.g. a directory which should be presented to the
user, only the other way round: You get a bunch of bytes and have to decode
these for presentation purposes.

Specifying a "default" encoding is not really the right way to go, because
this leads to sloppy programs which only work under special circumstances and
suddenly break on other platforms with different defaults.

Cheers,
   S.

P.S.: The command line arguments are not really strings in *nix, either, so
the current API is wrong, too.
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Neil Mitchell
Hi

> Alas, this is no solution at all. The point is: Currently there is no explicit
> conversion FilePath <-> String, although this is really needed. Users e.g.
> enter FilePaths somehow through a GUI, on the command line, in a
> configuration file etc., so you often start with a String.

Surely both FilePath's and command line arguments are sufficiently
string like that a reasonable interpretation of them is String? We
don't want to provide an interface to Unix, we want to provide a
proper abstraction over the underlying details.

It would be a shame if on my Windows box I had to jump through
artificial hoops because Posix is a bit broken...

Thanks

Neil
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Sven Panne
On Thursday 15 March 2007 19:18, Neil Mitchell wrote:
> Surely both FilePath's and command line arguments are sufficiently
> string like that a reasonable interpretation of them is String? We
> don't want to provide an interface to Unix, we want to provide a
> proper abstraction over the underlying details.
>
> It would be a shame if on my Windows box I had to jump through
> artificial hoops because Posix is a bit broken...

I'm lacking the time to list all those artificial hoops Windows has brought
us, so I only urge everybody again: Let's not oversimplify things and handle
things the Windows way. Although I would love to have a world where POSIX &
friends used Unicode in tons of places, this is simply not the case. FilePath
is meant to be an *abstraction*, hiding platform differences, and FilePath =
String miserably fails to do this. Haskell got this wrong initially, but I'm
hoping that Haskell' addresses this issue.

As a compromise, we can leave the current FilePath as it is until things are
settled, but I would really like to have a big, fat warning in the
documentation then...

Cheers,
   S.
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Neil Mitchell
Hi

> As a compromise, we can leave the current FilePath as it is until things are
> settled, but I would really like to have a big, fat warning in the
> documentation then...

What would the warning say?

"Haskell currently does not work when reading files or accessing
command line arguments on Unix. Until such time as this is fixed
please avoid opening files, or allowing the user to pass command line
flags. The workaround is to either use Windows or Perl." :-)

Do you have a concrete proposal for what FilePath's should look like
in Haskell'? "FilePath = String" is currently the least broken design,
since no one has put forward a better one.

Thanks

Neil
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re[2]: Adding System.FilePath

Bulat Ziganshin-2
Hello Neil,

Thursday, March 15, 2007, 9:43:48 PM, you wrote:

> Do you have a concrete proposal for what FilePath's should look like
> in Haskell'? "FilePath = String" is currently the least broken design,
> since no one has put forward a better one.

does http://haskell.org/haskellwiki/Library/IO not count?


--
Best regards,
 Bulat                            mailto:[hidden email]

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Sven Panne
In reply to this post by Neil Mitchell
On Thursday 15 March 2007 19:43, Neil Mitchell wrote:
> What would the warning say?
>
> "Haskell currently does not work when reading files or accessing
> command line arguments on Unix. Until such time as this is fixed
> please avoid opening files, or allowing the user to pass command line
> flags. The workaround is to either use Windows or Perl." :-)

At least this would be a starting point... ;-)

> Do you have a concrete proposal for what FilePath's should look like
> in Haskell'? "FilePath = String" is currently the least broken design,
> since no one has put forward a better one.

Of course I can't come up with an elaborate proposal in 5 minutes, but I think
the main points should be:

   * FilePath is abstract, perhaps even constructed from FilePathElements
(there are pros and cons for introducing the latter, I'm not totally sure
about all implications).

   * Explicit operations for constructing and accessing a FilePath are
provided.

   * Operations like the ones in your proposal are provided.

Before one can make a concrete proposal, a proposal for a general encoding
framework is needed and should be standardized, because we need a "fit" here.

Another option would be handling things exactly the other way round: FilePath
could be a synonym for ByteString, so things could look like the following
(assuming functions from our imaginary encoding framework) :

   ...
   entries1 <- getDirectoryContents (encode utf8 "blah")
   mapM_ (putStrLn . decode utf8) entries1
   entries2 <- getDirectoryContents myByteString
   ...

But in general, I have a bad feeling about type synonyms for such cases...

Cheers,
   S.
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Andres Loeh-2
In reply to this post by Neil Mitchell
Hi there.

> 2) Let programmers type "import System.FilePath" and have it work in
> Cabal Setup.hs files, and in all Cabal built programs without
> requiring the user specify that they are using the filepath package.
>
> This is what everyone wants.  Within this there are two approaches
> being advocated:
>
> a) Add to base.
>
> b) Add as a library which we demand all Haskell compilers ship, and
> tweak Cabal to automatically use this library at all times.
>
> Option a involves absolutely no additional work for me - the patch has
> been posted to this mailing list. Option b involves Cabal hacking,
> establishing new procedures, coming up with a definition of libraries
> which are always available etc. It is likely that option b will be
> required once (if ever) base is split up. I certainly don't have time
> to do option b.
>
> So can I propose we add filepath, with this agreement that once
> (someone else) gets around to doing option b, we can split it out
> afterwards?

I think option 2a would be a huge improvement over the situation we have
right now, so I'd be strongly in favor of adding FilePath to base for
the time being.

It's certainly not a perfect solution, I am aware of that, but none of
the arguments given could convince me that it'll actually make things
worse compared to now, so we should not prevent progress by trying to
make everything perfect in a single step.

Cheers,
  Andres
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Neil Mitchell
In reply to this post by Sven Panne
Hi

> Of course I can't come up with an elaborate proposal in 5 minutes, but I think
> the main points should be:
>
>    * FilePath is abstract, perhaps even constructed from FilePathElements
> (there are pros and cons for introducing the latter, I'm not totally sure
> about all implications).
>
>    * Explicit operations for constructing and accessing a FilePath are
> provided.
>
>    * Operations like the ones in your proposal are provided.

In general, I don't see anything wrong with this proposal, but I'm
perfectly happy with the status at the moment. I suspect that people
who do things on FilePath's using the fact that they are String tend
to get it wrong anyway, other than showing them.

Of course, the reverse compatability situation in this case would be a
complete nightmare - possibly bad enough to make it just plain
impossible.

The upside to adding my System.FilePath module as soon as possible
would be that if people can be slowly weaned off hacking at FilePath's
as String's, then your more drastic change is much more likely to be
acceptable. In addition, if people move to entirely using
System.FilePath then their code would already be portable to your
scheme.

Thanks

Neil
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: Adding System.FilePath

Neil Mitchell
In reply to this post by Bulat Ziganshin-2
Hi Bulat,

> > Do you have a concrete proposal for what FilePath's should look like
> > in Haskell'? "FilePath = String" is currently the least broken design,
> > since no one has put forward a better one.
>
> does http://haskell.org/haskellwiki/Library/IO not count?

It is a start, but not the whole answer. What is the type of readFile?
Stringable a => a -> IO String ? What about getContents? Can all this
be implemented without breaking existing code? Does this stop people
from chopping and changing at strings? Is that desirable?

There are also lots of other questions, does this code want to go into
base? Do we want to replace the existing code in base? Is breaking
nhc/yhc for something as fundamental as open files acceptable?

Plenty of open questions, I suspect solving them all will take months
at the very least...

Thanks

Neil
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re[2]: Adding System.FilePath

Bulat Ziganshin-2
In reply to this post by Andres Loeh-2
Hello Andres,

Thursday, March 15, 2007, 10:31:56 PM, you wrote:

>> a) Add to base.
>>
>> b) Add as a library which we demand all Haskell compilers ship, and
>> tweak Cabal to automatically use this library at all times.
> I think option 2a would be a huge improvement over the situation we have
> right now, so I'd be strongly in favor of adding FilePath to base for
> the time being.

i think that it will be degradation because new base version will be
not available for users before 6.8 arrives and then it will be hard to
change this interface without breaking compatibility with existing
code. otoh, providing it as separate library solves these problems. if
developer is too lazy to cabalize his code, its not the cause to add
his module to the base. let's wait for someone who will be more eager


--
Best regards,
 Bulat                            mailto:[hidden email]

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re[4]: Adding System.FilePath

Bulat Ziganshin-2
In reply to this post by Neil Mitchell
Hello Neil,

Thursday, March 15, 2007, 10:54:03 PM, you wrote:

>> > Do you have a concrete proposal for what FilePath's should look like
>> > in Haskell'? "FilePath = String" is currently the least broken design,
>> > since no one has put forward a better one.
>>
>> does http://haskell.org/haskellwiki/Library/IO not count?

> It is a start, but not the whole answer. What is the type of readFile?
> Stringable a =>> a -> IO String ?

yes, if we say only about Filepath part and not about type for representing
file contents. another variant - provide some FilePath class. but
providing your module as separate library will allow to improve this
while keeping backward compatibility while including it into base
makes this scenario very problematic

> Stringable a =>> What about getContents? Can all this
> be implemented without breaking existing code? Does this stop people
> from chopping and changing at strings? Is that desirable?

the beauty of this proposal is that we can issue 3rd, 4th and any
other version based on real users feedback. including some module into
base means stopping of its active development, you can see the
situation with ByteString

> There are also lots of other questions, does this code want to go into
> base? Do we want to replace the existing code in base?

> Plenty of open questions, I suspect solving them all will take months
> at the very least...

i think that existing base should be conserved for keeping
compatibility with already developed apps/libs and new functionality
should be provided in form of cabal packages that can be easily
upgraded and even several versions can co-exist in one installation

then, functionality should be moved step-by-step from base to these
new packages. new functionality should go immediately into packages.
otherwise, for example, your shining new version of Cabal which uses
System.FilePath module, will be not compatible with ghc 6.6, and the
same will be true for all other apps/libs

then these packages will evolve. backward compatibility will be
provided by supplying old versions of libraries (via hackage or
preinstalled)

>Is breaking nhc/yhc for something as fundamental as open files acceptable?

i don't understood this question


--
Best regards,
 Bulat                            mailto:[hidden email]

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Ian Lynagh
In reply to this post by Neil Mitchell
On Thu, Mar 15, 2007 at 06:43:48PM +0000, Neil Mitchell wrote:
>
> Do you have a concrete proposal for what FilePath's should look like
> in Haskell'? "FilePath = String" is currently the least broken design,
> since no one has put forward a better one.

I proposed something in
http://www.haskell.org/pipermail/libraries/2005-July/004189.html
although I never made code for it.

I agree that we don't need to wait for this problem to be solved before
doing something with your FilePath module, though.


Thanks
Ian

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Ian Lynagh
In reply to this post by Sven Panne
On Thu, Mar 15, 2007 at 06:10:18PM +0100, Sven Panne wrote:
>
> What is the general strategy for Haskell' regarding this topic: Is I/O out of
> the scope of Haskell' itself?

When I last asked Isaac it (well, the fate of the standard libraries in
general rather than IO in particular) hadn't been decided, and I haven't
heard anything since.

I recently opened a haskell-prime ticket for this issue:
http://hackage.haskell.org/trac/haskell-prime/ticket/118

It would be good to resolve this ASAP as a number of other issues are
blocked on it.


Thanks
Ian

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Ian Lynagh
In reply to this post by Neil Mitchell
On Thu, Mar 15, 2007 at 04:25:55PM +0000, Neil Mitchell wrote:
>
> Option b involves Cabal hacking,

No Cabal hacking is necessary. We might have to add "filepath" to a list
in the cabal-setup source somewhere (I'm not sure if it already has a
list or just uses --make at the moment), but that is hardly a large
task.


Thanks
Ian

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Re[4]: Adding System.FilePath

Duncan Coutts
In reply to this post by Bulat Ziganshin-2
On Thu, 2007-03-15 at 23:25 +0300, Bulat Ziganshin wrote:

> the beauty of this proposal is that we can issue 3rd, 4th and any
> other version based on real users feedback. including some module into
> base means stopping of its active development, you can see the
> situation with ByteString

You keep saying this but I really do not think it is true.

There's nothing stopping people from adding new ByteString style
functions. We export all the internals to make that possible. For
example you'll notice that people have been looking at Unicode stuff,
binary encoding, compression etc. There's been no lack of progress.

We're not changing the main exported api because it already matches the
Data.List api which was the original intention. That does not stop you
from making extensions and exporting them from different modules.

We're also planning to replace the current ByteString implementation
with the version described in our paper that does the stream fusion
rather than the older style functional array fusion. We can change this
in base any time since it does not break any external APIs. We probably
will not get around to doing that before 6.6.1 since we're working on
other things at the moment but the point is that we could.

You say that things added to base never change, have you considered that
this might be because things get added to base when they are mature
rather than because adding things to base imposes a code freeze?

Of course if a module is still maturing and in a state of flux then
adding it to base wouldn't be a good idea as one could not promise API
stability. System.FilePath doesn't seem to be in that situation however,
in my opinion it's ok to go into base (and come out again if/when base
gets split up).

Duncan

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Adding System.FilePath

Wolfgang Thaller
In reply to this post by Neil Mitchell
On 15-Mar-07, at 7:18 PM, Neil Mitchell wrote:

>> Alas, this is no solution at all. The point is: Currently there is  
>> no explicit
>> conversion FilePath <-> String, although this is really needed.  
>> Users e.g.
>> enter FilePaths somehow through a GUI, on the command line, in a
>> configuration file etc., so you often start with a String.
>
> Surely both FilePath's and command line arguments are sufficiently
> string like that a reasonable interpretation of them is String? We
> don't want to provide an interface to Unix, we want to provide a
> proper abstraction over the underlying details.

Indeed, paths and command-line arguments are becoming very string-
like on Unix systems, too.
On Mac OS X, the locale for file names is pretty much hardcoded to  
UTF-8. Mac OS X's native file system stores file names in UTF-16, but  
the POSIX layer sees it as UTF-8.
For the last several years, all Linux distros I have used myself had  
all their filenames stored in UTF-8. And even before that, there was  
usually a single filename encoding used for the entire system. I have  
also heard that Java, Qt and GTK all can't handle file names that are  
invalid in the current system-wide encoding. For all intents and  
purposes, filenames that don't match my system encoding on my linux  
box are unusable. I only need to be able to access them from 'mv' and  
'rm', and if 'fsck' automatically renamed them for me, I'd be even  
happier. If people want to write Haskell replacements for these basic  
tools, they can be expected to use platform-specific lower-level  
functions imported from System.Posix.

> It would be a shame if on my Windows box I had to jump through
> artificial hoops because Posix is a bit broken...

Repeat the above statement with s/Windows/Mac OS X/ and again with s/
Windows/Single-User Desktop Linux/.

Cheers,

Wolfgang

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Re[4]: Adding System.FilePath

Ian Lynagh
In reply to this post by Duncan Coutts
On Fri, Mar 16, 2007 at 09:44:26AM +1100, Duncan Coutts wrote:
>
> Of course if a module is still maturing and in a state of flux then
> adding it to base wouldn't be a good idea as one could not promise API
> stability. System.FilePath doesn't seem to be in that situation however,
> in my opinion it's ok to go into base (and come out again if/when base
> gets split up).

I think that's roughly the argument you made for FPS, but looking at the
repo history there are at least these API changes since the GHC 6.6
release:

* export the right 'cycle'
* Add a unit tests file, a test that append is lazy in the tail, and make it so.
* "HEADS UP: Change CString api"
* Make the lazy cons lazy, and add a cons' for when you want it to be strict
* Make fromForeignPtr take the start so it is truly the inverse of toForeignPtr
* Define headTail :: ByteString -> Maybe (Word8, ByteString)

Sure, all of these can be worked around by importing internal modules
and copying the corrected definitions (as I have had to do), but it
would be much simpler if I could just depend on bytestring >= 1.1.


Thanks
Ian

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
12345