Abstract FilePath Proposal

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
53 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Abstract FilePath Proposal

Herbert Valerio Riedel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello *,

What?
=====

We (see From: & CC: headers) propose, plain and simple, to turn the
currently defined type-synonym

  type FilePath = String

into an abstract/opaque data type instead.

Why/How/When?
=============

For details (including motivation and a suggested transition scheme)
please consult

  https://ghc.haskell.org/trac/ghc/wiki/Proposal/AbstractFilePath



Suggested discussion period: 4 weeks
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJVjXkZAAoJELo8uj/+IrV0WXUP/0romoKazwLbQpaMAKgCNZon
BsY8Di44w6rkbdBXoky0xZooII8LJJyQfexH0BLRYEVLZFy0+LB8XzpPt8Ekg526
YlY4x0qFm9oiJbJDMqHUnb6z6Lr2KxzBcV37drTPbltUA+HB49DUVkkPbvHimpL2
28SIyhAr4fN6fLpGcFAkv6Rcs0mkvnTp7vsC0HNyshmGi6qQ+C+eB4mklQzWOPcn
koHZ2wtI8AJmyTdHKcXKAIFM0r+xl4MJ5445IvDjvIuGXZCzybXMw9Ss/4wSG3VN
qSIJVEDGZXrBCc12fPxPEB0Bqx9MIVytjplXKIo8rFrk93h3at9t9kDM26z+9PZ5
KYnEdjRKF4KL4j+3xqJDOEJT15GVRbGRRzb9A8xH0YIQ0S3Q3pt1PAfla1Hss75+
NRQgfowZYryL9dfCkAj2XNfdQ+pUk25N3bNig11se+zjk2JO77QRM0u3GOYZ9+CU
tSlwhtIMF32xnjgQyWE5yBBiEg3/Y+S+809tVaPseUEzkQJXMGq5TFxBrN6bj1Vm
awr6QghThKjeoRwky5bmFn/gept/lbYN6VV5B6gNznGP5xgFrmvVtmjbQJBRMYCv
aEUnrYqxkkbIddJjD5gl771/LWH4M2F1yBgJjfiZw2paEVAXKxEr327LsbOQaPdb
HjIPRrJbVK9AABo4AZ/Y
=lg0o
-----END PGP SIGNATURE-----
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Henning Thielemann

On Fri, 26 Jun 2015, Herbert Valerio Riedel wrote:

> What?
> =====
>
> We (see From: & CC: headers) propose, plain and simple, to turn the
> currently defined type-synonym
>
>  type FilePath = String
>
> into an abstract/opaque data type instead.

Has someone else tried the pathtype package?
    http://hackage.haskell.org/package/pathtype

I found its idea great, but in practice I could not use it because it
always tries to canonicalize paths on initialization.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Henning Thielemann

On Fri, 26 Jun 2015, Henning Thielemann wrote:

> On Fri, 26 Jun 2015, Herbert Valerio Riedel wrote:
>
>> What?
>> =====
>>
>> We (see From: & CC: headers) propose, plain and simple, to turn the
>> currently defined type-synonym
>>
>>  type FilePath = String
>>
>> into an abstract/opaque data type instead.
>
> Has someone else tried the pathtype package?
>   http://hackage.haskell.org/package/pathtype

Hm, your last point "Decisions assumed by this Proposal" seems to mean,
that you want to leave out more specialised types from this proposal. That
is, dir/file distinction might be defined on top of the new FilePath type.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Herbert Valerio Riedel
On 2015-06-26 at 18:22:16 +0200, Henning Thielemann wrote:
[...]

>>>  type FilePath = String
>>>
>>> into an abstract/opaque data type instead.
>>
>> Has someone else tried the pathtype package?
>>   http://hackage.haskell.org/package/pathtype
>
> Hm, your last point "Decisions assumed by this Proposal" seems to
> mean, that you want to leave out more specialised types from this
> proposal. That is, dir/file distinction might be defined on top of the
> new FilePath type.

Yes, because the proposal is meant to only make the smallest incremental
change needed (i.e. change FilePath datatype, provide conversion
functions) to to achieve the primary goals (reduce space/time overhead &
make it a distinct type from String) in a way suitable for a future
Haskell Report, while trying to stay close enough that you can still
write code that works with both the H2010 and a AFPP definition of
`FilePath`

Trying to redesign the FilePath type to also include dir/file
distinction seemed too daunting, as there's quite some additional
design-space area to explore (do drive-letters deserve a separate type?
do we use DataKinds? What invariants can/shall be represented at the
type-level? what errors are caught at the type-level, which are caught
at runtime? etc...), parts of which may require type-system extensions,
while just having a KISS-style opaque FilePath evades this.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Herbert Valerio Riedel
In reply to this post by Herbert Valerio Riedel
On 2015-06-27 at 03:30:56 +0200, Bardur Arantsson wrote:

[...]

> I am *entirely* behind this in priciple and if it doesn't break too much
> of Hackage, also in practice, but...
>
> ... how much of Hackage *does* this break?

This won't be for free: I expect most packages which currently do more
than just opaquely pass around FilePaths to require fixes.

Some examples:

 - `writeFile "doo/foo.bar" ...`
   `_ <- readFile ("doo" </> "foo" <.> "bar")`

   This will break unless -XOverloadedStrings happens to be enabled

 - Unless we generalise (++) to (<>), all cases where `FilePath`s are
   concatenated via (++) will break.

 - Code that uses Data.List rather than the `filepath` package for
   FilePath manipulation will need fixups (simplest fix: explicitly
   convert to/from String for the manipulation)

 - Some code, like e.g.

     fnames <- System.Environment.getArgs
     forM fnames $ \fn -> print =<< readFile fn

   will inevitably need to insert explicit conversions to/from FilePaths

I tried to simulate the effect on Hackage, but this turned out to be
more time-demanding than I hoped for and I had to abort. But the above
is what I encountered in my attempt.

> The reason that I'm in favor in principle is that paths really *are*
> opaque things -- platforms have entirely different conventions. AFAICT
> the only thing that they seem to agree on is that there is a "hierarchy"
> of some sort. (And not much else, including such things as case
> (in-)sensivity or character sets.).

> For example, in POSIX they're just strings of bytes without any
> specified encoding, and I'd love if they could be make to work like
> that when dealing with files in Haskell.

Yes, if you look e.g. at

  http://hackage.haskell.org/package/unix

you see a lot of API duplication, which wouldn't have been needed if
FilePath was an opaque type w/ lossless conversion to/from
ByteString.
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Joachim Breitner-2
In reply to this post by Herbert Valerio Riedel
Hi,

Am Freitag, den 26.06.2015, 18:08 +0200 schrieb Herbert Valerio Riedel:
> Why/How/When?
> =============
>
> For details (including motivation and a suggested transition scheme)
> please consult
>
>   https://ghc.haskell.org/trac/ghc/wiki/Proposal/AbstractFilePath
>

unless proven wrong, I assume that this breaks lots of code. So by
default, I am doubtful towards that proposal.

Also, the argument about inefficiency does not convince me. In a
Prelude where "type String = [Char]", this is – as bad as it may be –
consistent and expected. Just like you manually have to reach out for
dedicated string types if you want efficient Haskell, you can manually
reach out for dedicated FilePath libraries. I doubt that it is a good
idea to fix the String problem in small, isolated chunks.

However, the argument that, depending on the system, "type FilePath =
String" is simply wrong (because the semantics of file paths are not
unicode points), is one that I buy in. On such systems, the Prelude is,
in a moral sense, wrongly typed, and the situation is comparable to a
hypothetical "type String = [Word8]" – which is just wrong in a world
of unicode. So for me, the faithful representation of system filepaths
makes this proposal interesting. I like it when Haskell “gets it
right”.

So if the transition can be made somewhat smooth, I’m leaning towards
supporting it.

Greetings,
Joachim




--
Joachim “nomeata” Breitner
  [hidden email]http://www.joachim-breitner.de/
  Jabber: [hidden email]  • GPG-Key: 0xF0FBF51F
  Debian Developer: [hidden email]

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

SV: Abstract FilePath Proposal

Niklas Larsson
In reply to this post by Herbert Valerio Riedel
Hi!

Instead of trying to minimally patch the existing API and still breaking loads of code, why not make a new API that doesn't have to compromise and depreciate the old one?

Niklas

Från: [hidden email]
Skickat: ‎2015-‎06-‎26 18:09
Till: [hidden email]; [hidden email]
Ämne: Abstract FilePath Proposal

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello *,

What?
=====

We (see From: & CC: headers) propose, plain and simple, to turn the
currently defined type-synonym

  type FilePath = String

into an abstract/opaque data type instead.

Why/How/When?
=============

For details (including motivation and a suggested transition scheme)
please consult

  https://ghc.haskell.org/trac/ghc/wiki/Proposal/AbstractFilePath



Suggested discussion period: 4 weeks
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJVjXkZAAoJELo8uj/+IrV0WXUP/0romoKazwLbQpaMAKgCNZon
BsY8Di44w6rkbdBXoky0xZooII8LJJyQfexH0BLRYEVLZFy0+LB8XzpPt8Ekg526
YlY4x0qFm9oiJbJDMqHUnb6z6Lr2KxzBcV37drTPbltUA+HB49DUVkkPbvHimpL2
28SIyhAr4fN6fLpGcFAkv6Rcs0mkvnTp7vsC0HNyshmGi6qQ+C+eB4mklQzWOPcn
koHZ2wtI8AJmyTdHKcXKAIFM0r+xl4MJ5445IvDjvIuGXZCzybXMw9Ss/4wSG3VN
qSIJVEDGZXrBCc12fPxPEB0Bqx9MIVytjplXKIo8rFrk93h3at9t9kDM26z+9PZ5
KYnEdjRKF4KL4j+3xqJDOEJT15GVRbGRRzb9A8xH0YIQ0S3Q3pt1PAfla1Hss75+
NRQgfowZYryL9dfCkAj2XNfdQ+pUk25N3bNig11se+zjk2JO77QRM0u3GOYZ9+CU
tSlwhtIMF32xnjgQyWE5yBBiEg3/Y+S+809tVaPseUEzkQJXMGq5TFxBrN6bj1Vm
awr6QghThKjeoRwky5bmFn/gept/lbYN6VV5B6gNznGP5xgFrmvVtmjbQJBRMYCv
aEUnrYqxkkbIddJjD5gl771/LWH4M2F1yBgJjfiZw2paEVAXKxEr327LsbOQaPdb
HjIPRrJbVK9AABo4AZ/Y
=lg0o
-----END PGP SIGNATURE-----
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: SV: Abstract FilePath Proposal

Kostiantyn Rybnikov

Because new api already exists in libraries, but FilePath from base is still being used, which makes things worse (now your programs have all those conversions all over).

I like the idea with gradual deprecation warning, but it's not clear if it's feasible to implement.

27 черв. 2015 12:33 "Niklas Larsson" <[hidden email]> пише:
Hi!

Instead of trying to minimally patch the existing API and still breaking loads of code, why not make a new API that doesn't have to compromise and depreciate the old one?

Niklas

Från: [hidden email]
Skickat: ‎2015-‎06-‎26 18:09
Till: [hidden email]; [hidden email]
Ämne: Abstract FilePath Proposal

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello *,

What?
=====

We (see From: & CC: headers) propose, plain and simple, to turn the
currently defined type-synonym

  type FilePath = String

into an abstract/opaque data type instead.

Why/How/When?
=============

For details (including motivation and a suggested transition scheme)
please consult

  https://ghc.haskell.org/trac/ghc/wiki/Proposal/AbstractFilePath



Suggested discussion period: 4 weeks
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJVjXkZAAoJELo8uj/+IrV0WXUP/0romoKazwLbQpaMAKgCNZon
BsY8Di44w6rkbdBXoky0xZooII8LJJyQfexH0BLRYEVLZFy0+LB8XzpPt8Ekg526
YlY4x0qFm9oiJbJDMqHUnb6z6Lr2KxzBcV37drTPbltUA+HB49DUVkkPbvHimpL2
28SIyhAr4fN6fLpGcFAkv6Rcs0mkvnTp7vsC0HNyshmGi6qQ+C+eB4mklQzWOPcn
koHZ2wtI8AJmyTdHKcXKAIFM0r+xl4MJ5445IvDjvIuGXZCzybXMw9Ss/4wSG3VN
qSIJVEDGZXrBCc12fPxPEB0Bqx9MIVytjplXKIo8rFrk93h3at9t9kDM26z+9PZ5
KYnEdjRKF4KL4j+3xqJDOEJT15GVRbGRRzb9A8xH0YIQ0S3Q3pt1PAfla1Hss75+
NRQgfowZYryL9dfCkAj2XNfdQ+pUk25N3bNig11se+zjk2JO77QRM0u3GOYZ9+CU
tSlwhtIMF32xnjgQyWE5yBBiEg3/Y+S+809tVaPseUEzkQJXMGq5TFxBrN6bj1Vm
awr6QghThKjeoRwky5bmFn/gept/lbYN6VV5B6gNznGP5xgFrmvVtmjbQJBRMYCv
aEUnrYqxkkbIddJjD5gl771/LWH4M2F1yBgJjfiZw2paEVAXKxEr327LsbOQaPdb
HjIPRrJbVK9AABo4AZ/Y
=lg0o
-----END PGP SIGNATURE-----
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Abstract FilePath Proposal

Neil Mitchell
Hi Niklas,

The function writeFile takes a FilePath. We could fork base or tell everyone to use writeFile2, but in practice everyone will keep using writeFile, and this String for FilePath. This approach is the only thing we could figure that made sense.

Henning: we do not propose normalisation on initialisation. For ASCII strings fromFilePath . toFilePath will be id. It might also be for unicode on some/all platforms. Of course, you can write your own FilePath creator that does normalisation on construction. 

Thanks, Neil 

On Saturday, 27 June 2015, Niklas Larsson <<a href="javascript:_e(%7B%7D,&#39;cvml&#39;,&#39;metaniklas@gmail.com&#39;);" target="_blank">metaniklas@...> wrote:
Hi!

Instead of trying to minimally patch the existing API and still breaking loads of code, why not make a new API that doesn't have to compromise and depreciate the old one?

Niklas

Från: Herbert Valerio Riedel
Skickat: ‎2015-‎06-‎26 18:09
Till: [hidden email]; [hidden email]
Ämne: Abstract FilePath Proposal

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello *,

What?
=====

We (see From: & CC: headers) propose, plain and simple, to turn the
currently defined type-synonym

  type FilePath = String

into an abstract/opaque data type instead.

Why/How/When?
=============

For details (including motivation and a suggested transition scheme)
please consult

  https://ghc.haskell.org/trac/ghc/wiki/Proposal/AbstractFilePath



Suggested discussion period: 4 weeks
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJVjXkZAAoJELo8uj/+IrV0WXUP/0romoKazwLbQpaMAKgCNZon
BsY8Di44w6rkbdBXoky0xZooII8LJJyQfexH0BLRYEVLZFy0+LB8XzpPt8Ekg526
YlY4x0qFm9oiJbJDMqHUnb6z6Lr2KxzBcV37drTPbltUA+HB49DUVkkPbvHimpL2
28SIyhAr4fN6fLpGcFAkv6Rcs0mkvnTp7vsC0HNyshmGi6qQ+C+eB4mklQzWOPcn
koHZ2wtI8AJmyTdHKcXKAIFM0r+xl4MJ5445IvDjvIuGXZCzybXMw9Ss/4wSG3VN
qSIJVEDGZXrBCc12fPxPEB0Bqx9MIVytjplXKIo8rFrk93h3at9t9kDM26z+9PZ5
KYnEdjRKF4KL4j+3xqJDOEJT15GVRbGRRzb9A8xH0YIQ0S3Q3pt1PAfla1Hss75+
NRQgfowZYryL9dfCkAj2XNfdQ+pUk25N3bNig11se+zjk2JO77QRM0u3GOYZ9+CU
tSlwhtIMF32xnjgQyWE5yBBiEg3/Y+S+809tVaPseUEzkQJXMGq5TFxBrN6bj1Vm
awr6QghThKjeoRwky5bmFn/gept/lbYN6VV5B6gNznGP5xgFrmvVtmjbQJBRMYCv
aEUnrYqxkkbIddJjD5gl771/LWH4M2F1yBgJjfiZw2paEVAXKxEr327LsbOQaPdb
HjIPRrJbVK9AABo4AZ/Y
=lg0o
-----END PGP SIGNATURE-----
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Bardur Arantsson-2
In reply to this post by Herbert Valerio Riedel
On 06/27/2015 09:54 AM, Herbert Valerio Riedel wrote:

> This won't be for free: I expect most packages which currently do more
> than just opaquely pass around FilePaths to require fixes.
>
> Some examples:
>
>  - `writeFile "doo/foo.bar" ...`
>    `_ <- readFile ("doo" </> "foo" <.> "bar")`
>
>    This will break unless -XOverloadedStrings happens to be enabled
>
>  - Unless we generalise (++) to (<>), all cases where `FilePath`s are
>    concatenated via (++) will break.
>
>  - Code that uses Data.List rather than the `filepath` package for
>    FilePath manipulation will need fixups (simplest fix: explicitly
>    convert to/from String for the manipulation)
>
>  - Some code, like e.g.
>
>      fnames <- System.Environment.getArgs
>      forM fnames $ \fn -> print =<< readFile fn
>
>    will inevitably need to insert explicit conversions to/from FilePaths
>
> I tried to simulate the effect on Hackage, but this turned out to be
> more time-demanding than I hoped for and I had to abort. But the above
> is what I encountered in my attempt.

It could, of course, be argued that some of this is "deserved breakage"
since blind concatenation (etc.) is probably already wrong in many cases.

(If someone is going to help fix up the breakage, then I'm still +1 :))

Regards,

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

amindfv
El Jun 27, 2015, a las 7:39, Bardur Arantsson <[hidden email]> escribió:

> On 06/27/2015 09:54 AM, Herbert Valerio Riedel wrote:
>> This won't be for free: I expect most packages which currently do more
>> than just opaquely pass around FilePaths to require fixes.
>>
>> Some examples:
>>
>> - `writeFile "doo/foo.bar" ...`
>>  `_ <- readFile ("doo" </> "foo" <.> "bar")`
>>
>>  This will break unless -XOverloadedStrings happens to be enabled
>>
>> - Unless we generalise (++) to (<>), all cases where `FilePath`s are
>>  concatenated via (++) will break.
>>
>> - Code that uses Data.List rather than the `filepath` package for
>>  FilePath manipulation will need fixups (simplest fix: explicitly
>>  convert to/from String for the manipulation)
>>
>> - Some code, like e.g.
>>
>>    fnames <- System.Environment.getArgs
>>    forM fnames $ \fn -> print =<< readFile fn
>>
>>  will inevitably need to insert explicit conversions to/from FilePaths
>>
>> I tried to simulate the effect on Hackage, but this turned out to be
>> more time-demanding than I hoped for and I had to abort. But the above
>> is what I encountered in my attempt.
>
> It could, of course, be argued that some of this is "deserved breakage"
> since blind concatenation (etc.) is probably already wrong in many cases.
>
> (If someone is going to help fix up the breakage, then I'm still +1 :))

Just to be clear, every existing "creation" of a FilePath (without OverloadedStrings) will be broken, right?

So e.g. 'writeFile "test.txt" "test"' will be just as broken as 'writeFile ("foo" </> "test.txt") "test"' (or the '++' version)

I'm not +1 or -1 yet, just clarifying.

Tom



>
> Regards,
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

David Fox-12
In reply to this post by Herbert Valerio Riedel


On Fri, Jun 26, 2015 at 9:08 AM, Herbert Valerio Riedel <[hidden email]> wrote:

We (see From: & CC: headers) propose, plain and simple, to turn the
currently defined type-synonym

  type FilePath = String

into an abstract/opaque data type instead.

Why/How/When?

​I've had success with a slightly different "How":

Phase 1: Replace FilePath with a type class, with instances for the old FilePath (i.e. String) and the new implementation.

Phase 2:  Wait until a suitable amount of hackage builds without the string instance.

Phase 3: Deprecate the String instance - move it to an old-filepath package.

Phase 4: Replace the type class with the new implementation
This way the new implementation is available immediately, packages can begin converting at once, benefits can be assessed.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Herbert Valerio Riedel-3
On 2015-06-27 at 14:56:33 +0200, David Fox wrote:

[...]

> ​I've had success with a slightly different "How":

What was your concrete use-case btw?

> Phase 1: Replace FilePath with a type class, with instances for the old
> FilePath (i.e. String) and the new implementation.

what would that comprise in the FilePath case?

I assume adding a transitional class whose methods are not exposed (and
whose typeclass name is exported from some GHC-specific internal-marked
module)? i.e.

  class IsFilePath a where
    privateToFilePath :: a -> FilePath
    privateFromFilePath :: FilePath -> a
   
  instance IsFilePath FilePath where
    privateToFilePath   = id
    privateFromFilePath = id
   
  instance IsFilePath [Char] where
    privateToFilePath   = System.IO.toFilePath
    privateFromFilePath = System.IO.fromFilePath

?

as well as changing a lot of type-sigs in base & filepath from
e.g.

  writeFile :: FilePath -> String -> IO ()
  openTempFile :: FilePath -> String -> IO (FilePath, Handle)

to

  writeFile    :: IsFilePath a => a -> String -> IO ()
  openTempFile :: IsFilePath a => a -> String -> IO (a, Handle)


?

> Phase 2:  Wait until a suitable amount of hackage builds without the string
> instance.

I can see Stackage helping with that by using a custom GHC which lacks
the legacy `IsFilePath [Char]`-instance. So I'd be optimistic that Phase2 could be
accomplished within one year for the Stackage-subset of Hackage.

> Phase 3: Deprecate the String instance - move it to an old-filepath package.
>
> Phase 4: Replace the type class with the new implementation

I assume this means getting rid again of the typeclass, and changing the
type-sigs back to i.e.

  writeFile :: FilePath -> String -> IO ()
  openTempFile :: FilePath -> String -> IO (FilePath, Handle)

(but now with with the new opaque `FilePath`)?

> This way the new implementation is available immediately, packages can
> begin converting at once, benefits can be assessed.

This scheme seems feasible at first glance, as long as the typeclass
doesn't start spreading across packages and find its way into type-sigs
(in which case it'd become more disruptive to get rid of it
again). Otoh, I'm not sure (assuming I understood how your scheme works)
it can be avoided to have the typeclass spread, since if not every API
that now has `FilePath` arguments in their type-sigs gets generalised to
have `IsFilePath a => a` arguments instead, we can't reach the goal of
"Phase 2".

But I suspect that I didn't fully understand how your proposed
transition scheme works exactly... so please correct me where I got it
wrong!


Cheers,
  hvr
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Herbert Valerio Riedel-3
In reply to this post by amindfv
On 2015-06-27 at 14:11:52 +0200, [hidden email] wrote:

[...]

>>> - `writeFile "doo/foo.bar" ...`
>>>  `_ <- readFile ("doo" </> "foo" <.> "bar")`
>>>
>>>  This will break unless -XOverloadedStrings happens to be enabled

[...]

> Just to be clear, every existing "creation" of a FilePath (without
> OverloadedStrings) will be broken, right?
>
> So e.g. 'writeFile "test.txt" "test"' will be just as broken as
> 'writeFile ("foo" </> "test.txt") "test"' (or the '++' version)

That's right; w/o OverloadedStrings, you'd have to write e.g.

  writeFile (toFilePath "test.txt") "test"

(which will be possible to write as soon as `filepath` starts exposing
the forward-compat `{to,from}FilePath` conversion functions)

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

David Fox-12
In reply to this post by Herbert Valerio Riedel-3


On Sat, Jun 27, 2015 at 6:37 AM, Herbert Valerio Riedel <[hidden email]> wrote:
On 2015-06-27 at 14:56:33 +0200, David Fox wrote:

[...]

> ​I've had success with a slightly different "How":

What was your concrete use-case btw?

> Phase 1: Replace FilePath with a type class, with instances for the old
> FilePath (i.e. String) and the new implementation.

what would that comprise in the FilePath case?

I assume adding a transitional class whose methods are not exposed (and
whose typeclass name is exported from some GHC-specific internal-marked
module)? i.e.

  class IsFilePath a where
    privateToFilePath :: a -> FilePath
    privateFromFilePath :: FilePath -> a

  instance IsFilePath FilePath where
    privateToFilePath   = id
    privateFromFilePath = id

  instance IsFilePath [Char] where
    privateToFilePath   = System.IO.toFilePath
    privateFromFilePath = System.IO.fromFilePath

?

as well as changing a lot of type-sigs in base & filepath from
e.g.

  writeFile :: FilePath -> String -> IO ()
  openTempFile :: FilePath -> String -> IO (FilePath, Handle)

to

  writeFile    :: IsFilePath a => a -> String -> IO ()
  openTempFile :: IsFilePath a => a -> String -> IO (a, Handle)


?

> Phase 2:  Wait until a suitable amount of hackage builds without the string
> instance.

I can see Stackage helping with that by using a custom GHC which lacks
the legacy `IsFilePath [Char]`-instance. So I'd be optimistic that Phase2 could be
accomplished within one year for the Stackage-subset of Hackage.

> Phase 3: Deprecate the String instance - move it to an old-filepath package.
>
> Phase 4: Replace the type class with the new implementation

I assume this means getting rid again of the typeclass, and changing the
type-sigs back to i.e.

  writeFile :: FilePath -> String -> IO ()
  openTempFile :: FilePath -> String -> IO (FilePath, Handle)

(but now with with the new opaque `FilePath`)?

> This way the new implementation is available immediately, packages can
> begin converting at once, benefits can be assessed.

This scheme seems feasible at first glance, as long as the typeclass
doesn't start spreading across packages and find its way into type-sigs
(in which case it'd become more disruptive to get rid of it
again). Otoh, I'm not sure (assuming I understood how your scheme works)
it can be avoided to have the typeclass spread, since if not every API
that now has `FilePath` arguments in their type-sigs gets generalised to
have `IsFilePath a => a` arguments instead, we can't reach the goal of
"Phase 2".

But I suspect that I didn't fully understand how your proposed
transition scheme works exactly... so please correct me where I got it
wrong!

​You are right, your approach is more appropriate for use by a community.​  I missed some of the problems that would arise.


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Yitzchak Gale
In reply to this post by Joachim Breitner-2
I am strong -1 on this proposal, due to the arguments
of Joachim, unless we are getting the semantics right.
Are we?

By semantics, I mean just the lowest-level definition
of what a path is made of.

On POSIX, it's a raw string of bytes, with interpretation
in any human-readable way delegated to the application.

On Mac OS X, it's normalized Unicode. The important
point is *normalized* - if you create a FilePath from two
different Unicode strings that have the same normalized
form, the result FilePaths must be equal on Mac OS X.

On Windows, it's UTF-16. Basically, modulo the quirks.

Correct semantics means:

1. There are different FilePath types for each platform,
because the semantics of a path are vastly different for each.

2. By default, the FilePath for the current platform is used.
But the FilePaths for other platforms are also available,
and there is a simple API for best-effort conversion
between them.

Will this proposal provide that? If not, don't do this to us.
Please.

Joachim Breitner wrote:
unless proven wrong, I assume that this breaks lots of code. So by
default, I am doubtful towards that proposal.

Also, the argument about inefficiency does not convince me. In a
Prelude where "type String = [Char]", this is – as bad as it may be –
consistent and expected. Just like you manually have to reach out for
dedicated string types if you want efficient Haskell, you can manually
reach out for dedicated FilePath libraries. I doubt that it is a good
idea to fix the String problem in small, isolated chunks.

However, the argument that, depending on the system, "type FilePath =
String" is simply wrong (because the semantics of file paths are not
unicode points), is one that I buy in. On such systems, the Prelude is,
in a moral sense, wrongly typed, and the situation is comparable to a
hypothetical "type String = [Word8]" – which is just wrong in a world
of unicode. So for me, the faithful representation of system filepaths
makes this proposal interesting. I like it when Haskell “gets it
right”.

The problem is that I'm not sure this proposal "gets it right".
And if not, I agree with your arguments that this proposal
would not be worth the pain.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Brandon Allbery
On Sat, Jun 27, 2015 at 4:50 PM, Yitzchak Gale <[hidden email]> wrote:
On Mac OS X, it's normalized Unicode. The important
point is *normalized* - if you create a FilePath from two
different Unicode strings that have the same normalized
form, the result FilePaths must be equal on Mac OS X.

This is only true for higher level OS X APIs. ghc normally operates in the BSD layer, which mostly follows POSIX semantics; in particular, filesystem paths are bytestrings in the BSD layer, and only normalized in Cocoa APIs. (Which, among other things, means you can make a GUI application dump core by trying to use a file dialog in a directory containing a filename created using the BSD API which does not use a UTF8 encoding.)

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

David Turner-2
Hi,

I'm +1 on the general idea of this proposal. Using String for filenames has caused me all sorts of trouble, particularly when I've had to deal with a bunch of files whose names don't all use the same encoding.

However, be careful about the exact semantics of filenames on Windows. Quoting MSDN:

There is no need to perform any Unicode normalization on path and file name strings for use by the Windows file I/O API functions because the file system treats path and file names as an opaque sequence of WCHARs. Any normalization that your application requires should be performed with this in mind, external of any calls to related Windows file I/O API functions.


Thus FilePath = String (or Text) doesn't really seem correct on Windows either (although it'll be pretty close as long as you stay within the BMP).

By my reckoning, when you get down to brass tacks, all filesystems on all platforms name files with sequences of bytes. There are various interesting ways to represent these bytes to human beings as sequences of characters, but aiming for FilePath = ByteString everywhere and dealing with the conversion to characters elsewhere seems more correct.

Cheers,

David



On 27 June 2015 at 22:02, Brandon Allbery <[hidden email]> wrote:

> On Sat, Jun 27, 2015 at 4:50 PM, Yitzchak Gale <[hidden email]> wrote:
>>
>> On Mac OS X, it's normalized Unicode. The important
>> point is *normalized* - if you create a FilePath from two
>> different Unicode strings that have the same normalized
>> form, the result FilePaths must be equal on Mac OS X.
>
>
> This is only true for higher level OS X APIs. ghc normally operates in the
> BSD layer, which mostly follows POSIX semantics; in particular, filesystem
> paths are bytestrings in the BSD layer, and only normalized in Cocoa APIs.
> (Which, among other things, means you can make a GUI application dump core
> by trying to use a file dialog in a directory containing a filename created
> using the BSD API which does not use a UTF8 encoding.)
>
> --
> brandon s allbery kf8nh                               sine nomine associates
> [hidden email]                                  [hidden email]
> unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Yitzchak Gale
In reply to this post by Brandon Allbery
I wrote:
> On Mac OS X, it's normalized Unicode.

Brandon Allbery wrote:
This is only true for higher level OS X APIs. ghc normally operates in the BSD layer, which mostly follows POSIX semantics; in particular, filesystem paths are bytestrings in the BSD layer, and only normalized in Cocoa APIs.

If I happen to be writing a program that only interacts with the
BSD layer, then I can use the POSIX-style FilePath.
But as an application writer, most of the time I'll be dealing
with Mac OS X paths that need to work correctly with Cocoa.


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Abstract FilePath Proposal

Yitzchak Gale
In reply to this post by David Turner-2
OK, based on what David and Brandon wrote, I guess
that representing paths as bytestrings does make
some low-level sense on all platforms. Although
for Windows we would still need some way to deal
with the requirement that the bytestring have an even
length.

We will need platform-dependent coercions of
paths to and from String/Text. Those might sometimes
be partial functions. We need a notion of the coercions
for the current platform, and we also need it to be
possible to access the coercions for all platforms.

On Sun, Jun 28, 2015 at 12:28 AM David Turner <[hidden email]> wrote:
Hi,

I'm +1 on the general idea of this proposal. Using String for filenames has caused me all sorts of trouble, particularly when I've had to deal with a bunch of files whose names don't all use the same encoding.

However, be careful about the exact semantics of filenames on Windows. Quoting MSDN:

There is no need to perform any Unicode normalization on path and file name strings for use by the Windows file I/O API functions because the file system treats path and file names as an opaque sequence of WCHARs. Any normalization that your application requires should be performed with this in mind, external of any calls to related Windows file I/O API functions.


Thus FilePath = String (or Text) doesn't really seem correct on Windows either (although it'll be pretty close as long as you stay within the BMP).

By my reckoning, when you get down to brass tacks, all filesystems on all platforms name files with sequences of bytes. There are various interesting ways to represent these bytes to human beings as sequences of characters, but aiming for FilePath = ByteString everywhere and dealing with the conversion to characters elsewhere seems more correct.

Cheers,

David



On 27 June 2015 at 22:02, Brandon Allbery <[hidden email]> wrote:

> On Sat, Jun 27, 2015 at 4:50 PM, Yitzchak Gale <[hidden email]> wrote:
>>
>> On Mac OS X, it's normalized Unicode. The important
>> point is *normalized* - if you create a FilePath from two
>> different Unicode strings that have the same normalized
>> form, the result FilePaths must be equal on Mac OS X.
>
>
> This is only true for higher level OS X APIs. ghc normally operates in the
> BSD layer, which mostly follows POSIX semantics; in particular, filesystem
> paths are bytestrings in the BSD layer, and only normalized in Cocoa APIs.
> (Which, among other things, means you can make a GUI application dump core
> by trying to use a file dialog in a directory containing a filename created
> using the BSD API which does not use a UTF8 encoding.)
>
> --
> brandon s allbery kf8nh                               sine nomine associates
> [hidden email]                                  [hidden email]
> unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
123