On the state of 'tar'

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

On the state of 'tar'

Julian Ospald
Friends,

'tar' [0][1] is an excellent native haskell implementation of the tar
format. However, there are currently a lot of unattended issues:

* unicode filepaths broken due to Char8 use [2]
* executable bits handled improperly [3]
* no support for long filepath extension [4][5]
* doesn't handle hardlinks properly [6]
* handles symbolic links too strictly [7]

Most of these issues are 2 to 4 years old, some of them have PRs that
have never been reviewed.

'tar' fails to unpack our own ghc bindists even [4]. I consider it, in
this state, too unreliable for production use. Since it is in the
'haskell' namespace on github and probably the first hit on hackage,
this needs to be improved quickly, IMO.

Users currently can use tar-bytestring [8] (no windows support) or
libarchive [9] (not a native implementation), but 'tar' should
ultimately be fixed and maintained properly.

--
[0] https://hackage.haskell.org/package/tar
[1] https://github.com/haskell/tar
[2] https://github.com/haskell/tar/issues/6
[3] https://github.com/haskell/tar/issues/25
[4] https://github.com/haskell/tar/issues/49
[5] https://github.com/haskell/tar/issues/27
[6] https://github.com/haskell/tar/issues/51
[7] https://github.com/haskell/tar/issues/32
[8] https://github.com/hasufell/tar-bytestring
[9] https://hackage.haskell.org/package/libarchive
--


Cheers,
Julian
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: On the state of 'tar'

Vanessa McHale-2
FWIW, the tar format is a mess, which is part of why I chose to bind to libarchive (with all the difficulties) instead of writing my own library. There’s a nice post here: https://www.cyphar.com/blog/post/20190121-ociv2-images-i-tar

There’s also archive-sig (http://hackage.haskell.org/package/archive-sig) which tries to be a “common interface” via backpack and has implementations such as archive-tar-bytestring (http://hackage.haskell.org/package/archive-tar-bytestring) that can be swapped out with libarchive/tar proper so you don’t need to commit to any one library. 

Cheers,
Vanessa McHale

On Apr 7, 2020, at 11:50 AM, hasufell <[hidden email]> wrote:

Friends,

'tar' [0][1] is an excellent native haskell implementation of the tar
format. However, there are currently a lot of unattended issues:

* unicode filepaths broken due to Char8 use [2]
* executable bits handled improperly [3]
* no support for long filepath extension [4][5]
* doesn't handle hardlinks properly [6]
* handles symbolic links too strictly [7]

Most of these issues are 2 to 4 years old, some of them have PRs that
have never been reviewed.

'tar' fails to unpack our own ghc bindists even [4]. I consider it, in
this state, too unreliable for production use. Since it is in the
'haskell' namespace on github and probably the first hit on hackage,
this needs to be improved quickly, IMO.

Users currently can use tar-bytestring [8] (no windows support) or
libarchive [9] (not a native implementation), but 'tar' should
ultimately be fixed and maintained properly.

--
[0] https://hackage.haskell.org/package/tar
[1] https://github.com/haskell/tar
[2] https://github.com/haskell/tar/issues/6
[3] https://github.com/haskell/tar/issues/25
[4] https://github.com/haskell/tar/issues/49
[5] https://github.com/haskell/tar/issues/27
[6] https://github.com/haskell/tar/issues/51
[7] https://github.com/haskell/tar/issues/32
[8] https://github.com/hasufell/tar-bytestring
[9] https://hackage.haskell.org/package/libarchive
--


Cheers,
Julian
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: On the state of 'tar'

Vanessa McHale-2
In reply to this post by Julian Ospald
Oh, and I think there’s tar-conduit (and ztar, which shells out to tar) as well! trời ơi

> On Apr 7, 2020, at 11:50 AM, hasufell <[hidden email]> wrote:
>
> Friends,
>
> 'tar' [0][1] is an excellent native haskell implementation of the tar
> format. However, there are currently a lot of unattended issues:
>
> * unicode filepaths broken due to Char8 use [2]
> * executable bits handled improperly [3]
> * no support for long filepath extension [4][5]
> * doesn't handle hardlinks properly [6]
> * handles symbolic links too strictly [7]
>
> Most of these issues are 2 to 4 years old, some of them have PRs that
> have never been reviewed.
>
> 'tar' fails to unpack our own ghc bindists even [4]. I consider it, in
> this state, too unreliable for production use. Since it is in the
> 'haskell' namespace on github and probably the first hit on hackage,
> this needs to be improved quickly, IMO.
>
> Users currently can use tar-bytestring [8] (no windows support) or
> libarchive [9] (not a native implementation), but 'tar' should
> ultimately be fixed and maintained properly.
>
> --
> [0] https://hackage.haskell.org/package/tar
> [1] https://github.com/haskell/tar
> [2] https://github.com/haskell/tar/issues/6
> [3] https://github.com/haskell/tar/issues/25
> [4] https://github.com/haskell/tar/issues/49
> [5] https://github.com/haskell/tar/issues/27
> [6] https://github.com/haskell/tar/issues/51
> [7] https://github.com/haskell/tar/issues/32
> [8] https://github.com/hasufell/tar-bytestring
> [9] https://hackage.haskell.org/package/libarchive
> --
>
>
> Cheers,
> Julian
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: On the state of 'tar'

Julian Ospald
In reply to this post by Vanessa McHale-2
The problem with libarchive and bindings in general is portability.

A half way out of this would be to allow cabal/ghc to do partially
static linking. But that's probably not gonna happen anytime soon.

Alternatively... have a +static cabal flag that builds from bundled
source? Ideas...

But I think a native 'tar' implementation is still worthwhile.


On 07/04/2020 21:35, Vanessa McHale wrote:

> FWIW, the tar format is a mess, which is part of why I chose to bind to
> libarchive (with all the difficulties) instead of writing my own
> library. There’s a nice post
> here: https://www.cyphar.com/blog/post/20190121-ociv2-images-i-tar
>
> There’s also archive-sig
> (http://hackage.haskell.org/package/archive-sig) which tries to be a
> “common interface” via backpack and has implementations such as
> archive-tar-bytestring
> (http://hackage.haskell.org/package/archive-tar-bytestring) that can be
> swapped out with libarchive/tar proper so you don’t need to commit to
> any one library. 
>
> Cheers,
> Vanessa McHale
>
>> On Apr 7, 2020, at 11:50 AM, hasufell <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>> Friends,
>>
>> 'tar' [0][1] is an excellent native haskell implementation of the tar
>> format. However, there are currently a lot of unattended issues:
>>
>> * unicode filepaths broken due to Char8 use [2]
>> * executable bits handled improperly [3]
>> * no support for long filepath extension [4][5]
>> * doesn't handle hardlinks properly [6]
>> * handles symbolic links too strictly [7]
>>
>> Most of these issues are 2 to 4 years old, some of them have PRs that
>> have never been reviewed.
>>
>> 'tar' fails to unpack our own ghc bindists even [4]. I consider it, in
>> this state, too unreliable for production use. Since it is in the
>> 'haskell' namespace on github and probably the first hit on hackage,
>> this needs to be improved quickly, IMO.
>>
>> Users currently can use tar-bytestring [8] (no windows support) or
>> libarchive [9] (not a native implementation), but 'tar' should
>> ultimately be fixed and maintained properly.
>>
>> --
>> [0] https://hackage.haskell.org/package/tar
>> [1] https://github.com/haskell/tar
>> [2] https://github.com/haskell/tar/issues/6
>> [3] https://github.com/haskell/tar/issues/25
>> [4] https://github.com/haskell/tar/issues/49
>> [5] https://github.com/haskell/tar/issues/27
>> [6] https://github.com/haskell/tar/issues/51
>> [7] https://github.com/haskell/tar/issues/32
>> [8] https://github.com/hasufell/tar-bytestring
>> [9] https://hackage.haskell.org/package/libarchive
>> --
>>
>>
>> Cheers,
>> Julian
>> _______________________________________________
>> Libraries mailing list
>> [hidden email] <mailto:[hidden email]>
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: On the state of 'tar'

Vanessa McHale-2
Oh good point. I’ve had problems with static linking. The reason I didn’t bundle libarchive is that it requires a config.h with macros that are generated with ./configure…

It would certainly be nice to have partially static linking via cabal!

And yes I agree, I’d use it if it were broad enough

On Apr 7, 2020, at 2:56 PM, hasufell <[hidden email]> wrote:

The problem with libarchive and bindings in general is portability.

A half way out of this would be to allow cabal/ghc to do partially
static linking. But that's probably not gonna happen anytime soon.

Alternatively... have a +static cabal flag that builds from bundled
source? Ideas...

But I think a native 'tar' implementation is still worthwhile.


On 07/04/2020 21:35, Vanessa McHale wrote:
FWIW, the tar format is a mess, which is part of why I chose to bind to
libarchive (with all the difficulties) instead of writing my own
library. There’s a nice post
here: https://www.cyphar.com/blog/post/20190121-ociv2-images-i-tar

There’s also archive-sig
(http://hackage.haskell.org/package/archive-sig) which tries to be a
“common interface” via backpack and has implementations such as
archive-tar-bytestring
(http://hackage.haskell.org/package/archive-tar-bytestring) that can be
swapped out with libarchive/tar proper so you don’t need to commit to
any one library. 

Cheers,
Vanessa McHale

On Apr 7, 2020, at 11:50 AM, hasufell <[hidden email]
<[hidden email]>> wrote:

Friends,

'tar' [0][1] is an excellent native haskell implementation of the tar
format. However, there are currently a lot of unattended issues:

* unicode filepaths broken due to Char8 use [2]
* executable bits handled improperly [3]
* no support for long filepath extension [4][5]
* doesn't handle hardlinks properly [6]
* handles symbolic links too strictly [7]

Most of these issues are 2 to 4 years old, some of them have PRs that
have never been reviewed.

'tar' fails to unpack our own ghc bindists even [4]. I consider it, in
this state, too unreliable for production use. Since it is in the
'haskell' namespace on github and probably the first hit on hackage,
this needs to be improved quickly, IMO.

Users currently can use tar-bytestring [8] (no windows support) or
libarchive [9] (not a native implementation), but 'tar' should
ultimately be fixed and maintained properly.

--
[0] https://hackage.haskell.org/package/tar
[1] https://github.com/haskell/tar
[2] https://github.com/haskell/tar/issues/6
[3] https://github.com/haskell/tar/issues/25
[4] https://github.com/haskell/tar/issues/49
[5] https://github.com/haskell/tar/issues/27
[6] https://github.com/haskell/tar/issues/51
[7] https://github.com/haskell/tar/issues/32
[8] https://github.com/hasufell/tar-bytestring
[9] https://hackage.haskell.org/package/libarchive
--


Cheers,
Julian
_______________________________________________
Libraries mailing list
[hidden email] <[hidden email]>
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: On the state of 'tar'

Oleg Grenrus
Check `build-type: Configure` which eg `network` uses.

On 7. Apr 2020, at 23.18, Vanessa McHale <[hidden email]> wrote:

Oh good point. I’ve had problems with static linking. The reason I didn’t bundle libarchive is that it requires a config.h with macros that are generated with ./configure…

It would certainly be nice to have partially static linking via cabal!

And yes I agree, I’d use it if it were broad enough

On Apr 7, 2020, at 2:56 PM, hasufell <[hidden email]> wrote:

The problem with libarchive and bindings in general is portability.

A half way out of this would be to allow cabal/ghc to do partially
static linking. But that's probably not gonna happen anytime soon.

Alternatively... have a +static cabal flag that builds from bundled
source? Ideas...

But I think a native 'tar' implementation is still worthwhile.


On 07/04/2020 21:35, Vanessa McHale wrote:
FWIW, the tar format is a mess, which is part of why I chose to bind to
libarchive (with all the difficulties) instead of writing my own
library. There’s a nice post
here: https://www.cyphar.com/blog/post/20190121-ociv2-images-i-tar

There’s also archive-sig
(http://hackage.haskell.org/package/archive-sig) which tries to be a
“common interface” via backpack and has implementations such as
archive-tar-bytestring
(http://hackage.haskell.org/package/archive-tar-bytestring) that can be
swapped out with libarchive/tar proper so you don’t need to commit to
any one library. 

Cheers,
Vanessa McHale

On Apr 7, 2020, at 11:50 AM, hasufell <[hidden email]
<[hidden email]>> wrote:

Friends,

'tar' [0][1] is an excellent native haskell implementation of the tar
format. However, there are currently a lot of unattended issues:

* unicode filepaths broken due to Char8 use [2]
* executable bits handled improperly [3]
* no support for long filepath extension [4][5]
* doesn't handle hardlinks properly [6]
* handles symbolic links too strictly [7]

Most of these issues are 2 to 4 years old, some of them have PRs that
have never been reviewed.

'tar' fails to unpack our own ghc bindists even [4]. I consider it, in
this state, too unreliable for production use. Since it is in the
'haskell' namespace on github and probably the first hit on hackage,
this needs to be improved quickly, IMO.

Users currently can use tar-bytestring [8] (no windows support) or
libarchive [9] (not a native implementation), but 'tar' should
ultimately be fixed and maintained properly.

--
[0] https://hackage.haskell.org/package/tar
[1] https://github.com/haskell/tar
[2] https://github.com/haskell/tar/issues/6
[3] https://github.com/haskell/tar/issues/25
[4] https://github.com/haskell/tar/issues/49
[5] https://github.com/haskell/tar/issues/27
[6] https://github.com/haskell/tar/issues/51
[7] https://github.com/haskell/tar/issues/32
[8] https://github.com/hasufell/tar-bytestring
[9] https://hackage.haskell.org/package/libarchive
--


Cheers,
Julian
_______________________________________________
Libraries mailing list
[hidden email] <[hidden email]>
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: On the state of 'tar'

Carter Schonwald
https://github.com/haskell/network/blob/master/configure.ac and such? 
configure/autoconf is amazing (and terrible :) )

On Tue, Apr 7, 2020 at 6:01 PM Oleg Grenrus <[hidden email]> wrote:
Check `build-type: Configure` which eg `network` uses.

On 7. Apr 2020, at 23.18, Vanessa McHale <[hidden email]> wrote:

Oh good point. I’ve had problems with static linking. The reason I didn’t bundle libarchive is that it requires a config.h with macros that are generated with ./configure…

It would certainly be nice to have partially static linking via cabal!

And yes I agree, I’d use it if it were broad enough

On Apr 7, 2020, at 2:56 PM, hasufell <[hidden email]> wrote:

The problem with libarchive and bindings in general is portability.

A half way out of this would be to allow cabal/ghc to do partially
static linking. But that's probably not gonna happen anytime soon.

Alternatively... have a +static cabal flag that builds from bundled
source? Ideas...

But I think a native 'tar' implementation is still worthwhile.


On 07/04/2020 21:35, Vanessa McHale wrote:
FWIW, the tar format is a mess, which is part of why I chose to bind to
libarchive (with all the difficulties) instead of writing my own
library. There’s a nice post
here: https://www.cyphar.com/blog/post/20190121-ociv2-images-i-tar

There’s also archive-sig
(http://hackage.haskell.org/package/archive-sig) which tries to be a
“common interface” via backpack and has implementations such as
archive-tar-bytestring
(http://hackage.haskell.org/package/archive-tar-bytestring) that can be
swapped out with libarchive/tar proper so you don’t need to commit to
any one library. 

Cheers,
Vanessa McHale

On Apr 7, 2020, at 11:50 AM, hasufell <[hidden email]
<[hidden email]>> wrote:

Friends,

'tar' [0][1] is an excellent native haskell implementation of the tar
format. However, there are currently a lot of unattended issues:

* unicode filepaths broken due to Char8 use [2]
* executable bits handled improperly [3]
* no support for long filepath extension [4][5]
* doesn't handle hardlinks properly [6]
* handles symbolic links too strictly [7]

Most of these issues are 2 to 4 years old, some of them have PRs that
have never been reviewed.

'tar' fails to unpack our own ghc bindists even [4]. I consider it, in
this state, too unreliable for production use. Since it is in the
'haskell' namespace on github and probably the first hit on hackage,
this needs to be improved quickly, IMO.

Users currently can use tar-bytestring [8] (no windows support) or
libarchive [9] (not a native implementation), but 'tar' should
ultimately be fixed and maintained properly.

--
[0] https://hackage.haskell.org/package/tar
[1] https://github.com/haskell/tar
[2] https://github.com/haskell/tar/issues/6
[3] https://github.com/haskell/tar/issues/25
[4] https://github.com/haskell/tar/issues/49
[5] https://github.com/haskell/tar/issues/27
[6] https://github.com/haskell/tar/issues/51
[7] https://github.com/haskell/tar/issues/32
[8] https://github.com/hasufell/tar-bytestring
[9] https://hackage.haskell.org/package/libarchive
--


Cheers,
Julian
_______________________________________________
Libraries mailing list
[hidden email] <[hidden email]>
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries