Hackage inconsistent? (pandoc.cabal in 1.13.1)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Hackage inconsistent? (pandoc.cabal in 1.13.1)

Magnus Therning
It seems pandoc 1.13.1 package on hackage has gotten into an
insonsistent state.  I just noticed this strange situation with the dependency
on http-client:

pandoc.cabal at [1] has

  if flag(https)
     Build-Depends: http-client >= 0.3.2 && < 0.5,
                    http-client-tls >= 0.2 && < 0.3,
                    http-types >= 0.8 && < 0.9
     cpp-options:   -DHTTP_CLIENT

pandoc.cabal found in [2] has

    if flag(https)
     Build-Depends: http-client >= 0.3.2 && < 0.4,
                    http-client-tls >= 0.2 && < 0.3,
                    http-types >= 0.8 && < 0.9
     cpp-options:   -DHTTP_CLIENT

How can something like this even happen?

/M

[1]: http://hackage.haskell.org/package/pandoc-1.13.1/pandoc.cabal
[2]: http://hackage.haskell.org/package/pandoc-1.13.1/src/pandoc.cabal

--
Magnus Therning                      OpenPGP: 0xAB4DFBA4
email: [hidden email]   jabber: [hidden email]
twitter: magthe               http://therning.org/magnus

Any fool can write code that a computer can understand.  Good programmers
write code that humans can understand.
     -- Martin Fowler

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe

attachment0 (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

mantkiew
On Hackage you can adjust bounds of dependencies without having to upload a new package version and that was done in consistently with github repo.

Michal



  Original Message  
From: Magnus Therning
Sent: Monday, October 27, 2014 2:25 PM
To: Haskell Cafe
Subject: [Haskell-cafe] Hackage inconsistent? (pandoc.cabal in 1.13.1)

It seems pandoc 1.13.1 package on hackage has gotten into an
insonsistent state. I just noticed this strange situation with the dependency
on http-client:

pandoc.cabal at [1] has

if flag(https)
Build-Depends: http-client >= 0.3.2 && < 0.5,
http-client-tls >= 0.2 && < 0.3,
http-types >= 0.8 && < 0.9
cpp-options: -DHTTP_CLIENT

pandoc.cabal found in [2] has

if flag(https)
Build-Depends: http-client >= 0.3.2 && < 0.4,
http-client-tls >= 0.2 && < 0.3,
http-types >= 0.8 && < 0.9
cpp-options: -DHTTP_CLIENT

How can something like this even happen?

/M

[1]: http://hackage.haskell.org/package/pandoc-1.13.1/pandoc.cabal
[2]: http://hackage.haskell.org/package/pandoc-1.13.1/src/pandoc.cabal

--
Magnus Therning OpenPGP: 0xAB4DFBA4
email: [hidden email] jabber: [hidden email]
twitter: magthe http://therning.org/magnus

Any fool can write code that a computer can understand. Good programmers
write code that humans can understand.
-- Martin Fowler
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Magnus Therning
On Mon, Oct 27, 2014 at 02:30:29PM -0400, [hidden email] wrote:
> On Hackage you can adjust bounds of dependencies without having to
> upload a new package version and that was done in consistently with
> github repo.

I notice there already is a discussion on the danger of allowing
modifications like this, in particular related to reproducibility of
builds [1].

I have to say I really do hope that a solution is found where I can
go back to trusting version numbers that still allows minor tweaks.

The current situation is particularly irritating.  Up until now it was
safe to use the index to inspect Cabal files, because the Cabal file
found in the index was the same as the one found in the package tar
ball.  Removing this relationship is a *big* change!  Is there any
chance of it being restored?

/M

[1]: https://github.com/haskell/hackage-server/issues/52

--
Magnus Therning                      OpenPGP: 0xAB4DFBA4
email: [hidden email]   jabber: [hidden email]
twitter: magthe               http://therning.org/magnus

Most software today is very much like an Egyptian pyramid with
millions of bricks piled on top of each other, with no structural
integrity, but just done by brute force and thousands of slaves.
     -- Alan Kay

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe

attachment0 (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Erik Hesselink
In reply to this post by Magnus Therning
This is possible because there's a feature on the new hackage to allow
(partial) editing of the cabal file. See e.g. here [1]. This can be
useful for relaxing or tightening version bounds.

Erik

[1] https://github.com/haskell/hackage-server/issues/52

On Mon, Oct 27, 2014 at 7:25 PM, Magnus Therning <[hidden email]> wrote:

> It seems pandoc 1.13.1 package on hackage has gotten into an
> insonsistent state.  I just noticed this strange situation with the dependency
> on http-client:
>
> pandoc.cabal at [1] has
>
>   if flag(https)
>      Build-Depends: http-client >= 0.3.2 && < 0.5,
>                     http-client-tls >= 0.2 && < 0.3,
>                     http-types >= 0.8 && < 0.9
>      cpp-options:   -DHTTP_CLIENT
>
> pandoc.cabal found in [2] has
>
>     if flag(https)
>      Build-Depends: http-client >= 0.3.2 && < 0.4,
>                     http-client-tls >= 0.2 && < 0.3,
>                     http-types >= 0.8 && < 0.9
>      cpp-options:   -DHTTP_CLIENT
>
> How can something like this even happen?
>
> /M
>
> [1]: http://hackage.haskell.org/package/pandoc-1.13.1/pandoc.cabal
> [2]: http://hackage.haskell.org/package/pandoc-1.13.1/src/pandoc.cabal
>
> --
> Magnus Therning                      OpenPGP: 0xAB4DFBA4
> email: [hidden email]   jabber: [hidden email]
> twitter: magthe               http://therning.org/magnus
>
> Any fool can write code that a computer can understand.  Good programmers
> write code that humans can understand.
>      -- Martin Fowler
>
> _______________________________________________
> Haskell-Cafe mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

John MacFarlane
In reply to this post by Magnus Therning
I must say, I did pause a bit before making this change to
pandoc on Hackage, precisely because I realized that the
cabal file in the package would not match the one on
Hackage.

But I went ahead, assuming that if the means were provided
to make this kind of change, it should be okay.

It is time-consuming for me to make a proper pandoc
release.  I need to write up release notes, rebuild the
website, generate and test the binary packages on various
different platforms, upload to github releases and Hackage,
send announcement emails, etc.  Even with automation this
all takes work.  I'd rather not have to do this every time a
version bound changes on a dependent package, so I like this
way of making minor tweaks to version bounds; I hope I
can be persuaded that it's not a bad idea.

John

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Erik Hesselink
I was a bit hesitant about using this feature as well. We've had some
confusion when we used it [1][2][3]. The Nix guys also seem to be
having some trouble fitting it into their workflow [4]. But in [1] I
did a quick grep to see how many packages used the feature, and there
are already quite a lot. There is an 'x-revision' property added to
the cabal file in the index, so in theory I guess tools could learn to
deal with it.

In general I think the biggest win for this feature is not even in
relaxing bounds, but in tightening them when it turns out they're too
loose. Previously cabal was always free to pick an old, unconstrained
version and would often do so, leading to type errors during builds.

Erik

[1] https://github.com/silkapp/rest/issues/57
[2] https://github.com/silkapp/aeson-utils/issues/1
[3] https://github.com/silkapp/json-schema/issues/12
[4] https://github.com/NixOS/cabal2nix/issues/84

On Mon, Oct 27, 2014 at 8:30 PM, John MacFarlane <[hidden email]> wrote:

> I must say, I did pause a bit before making this change to
> pandoc on Hackage, precisely because I realized that the
> cabal file in the package would not match the one on
> Hackage.
>
> But I went ahead, assuming that if the means were provided
> to make this kind of change, it should be okay.
>
> It is time-consuming for me to make a proper pandoc
> release.  I need to write up release notes, rebuild the
> website, generate and test the binary packages on various
> different platforms, upload to github releases and Hackage,
> send announcement emails, etc.  Even with automation this
> all takes work.  I'd rather not have to do this every time a
> version bound changes on a dependent package, so I like this
> way of making minor tweaks to version bounds; I hope I
> can be persuaded that it's not a bad idea.
>
> John
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Magnus Therning
On Mon, Oct 27, 2014 at 08:48:16PM +0100, Erik Hesselink wrote:
> I was a bit hesitant about using this feature as well. We've had
> some confusion when we used it [1][2][3]. The Nix guys also seem to
> be having some trouble fitting it into their workflow [4]. But in
> [1] I did a quick grep to see how many packages used the feature,
> and there are already quite a lot. There is an 'x-revision' property
> added to the cabal file in the index, so in theory I guess tools
> could learn to deal with it.

Personally I would have liked the feature more if a change to tools
had been required in order to take advantage of the modified .cabal.

> In general I think the biggest win for this feature is not even in
> relaxing bounds, but in tightening them when it turns out they're
> too loose. Previously cabal was always free to pick an old,
> unconstrained version and would often do so, leading to type errors
> during builds.

I gather this means that cabal-install has been modified to (or maybe it
always has) replace the in-tar-ball .cabal with the .cabal from the
00-index.  Is that correct?

/M

--
Magnus Therning                      OpenPGP: 0xAB4DFBA4
email: [hidden email]   jabber: [hidden email]
twitter: magthe               http://therning.org/magnus

In a hierarchy, every employee tends to rise to his level of incompetence.
     -- The Peter Principle

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe

attachment0 (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Magnus Therning
In reply to this post by John MacFarlane
On Mon, Oct 27, 2014 at 12:30:03PM -0700, John MacFarlane wrote:

> I must say, I did pause a bit before making this change to pandoc on
> Hackage, precisely because I realized that the cabal file in the
> package would not match the one on Hackage.
>
> But I went ahead, assuming that if the means were provided to make
> this kind of change, it should be okay.
>
> It is time-consuming for me to make a proper pandoc release.  I need
> to write up release notes, rebuild the website, generate and test
> the binary packages on various different platforms, upload to github
> releases and Hackage, send announcement emails, etc.  Even with
> automation this all takes work.  I'd rather not have to do this
> every time a version bound changes on a dependent package, so I like
> this way of making minor tweaks to version bounds; I hope I can be
> persuaded that it's not a bad idea.
I can fully appreciate the usefulness of the feature; most of the
patches I make to package Haskell packages for Archlinux is of the
kind that modifies dependency restrictions.

I've read through the discussion at
https://github.com/haskell/hackage-server/issues/52 but I see no one
arguing for making this change a bit less intrusive and more opt-in
for tools, e.g. by keeping 00-index as it's always been and then
putting the Hackage edits of .cabals in an overlay (an 01-index).
Has this been up for proposed/discussed somewhere else?

/M

--
Magnus Therning                      OpenPGP: 0xAB4DFBA4
email: [hidden email]   jabber: [hidden email]
twitter: magthe               http://therning.org/magnus

As long as there are ill-defined goals, bizarre bugs, and unrealistic
schedules, there will be Real Programmers willing to jump in and Solve The
Problem, saving the documentation for later.  Long live Fortran!
     -- Ed Post

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe

attachment0 (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Mikhail Glushenkov-2
In reply to this post by Magnus Therning
Hi,

On 27 October 2014 22:22, Magnus Therning <[hidden email]> wrote:
> I gather this means that cabal-install has been modified to (or maybe it
> always has) replace the in-tar-ball .cabal with the .cabal from the
> 00-index.  Is that correct?

Yes, 'cabal get' (formerly known as 'unpack') now does this.
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Mateusz Kowalczyk
In reply to this post by Erik Hesselink
On 10/27/2014 07:48 PM, Erik Hesselink wrote:

> I was a bit hesitant about using this feature as well. We've had some
> confusion when we used it [1][2][3]. The Nix guys also seem to be
> having some trouble fitting it into their workflow [4]. But in [1] I
> did a quick grep to see how many packages used the feature, and there
> are already quite a lot. There is an 'x-revision' property added to
> the cabal file in the index, so in theory I guess tools could learn to
> deal with it.
>
> In general I think the biggest win for this feature is not even in
> relaxing bounds, but in tightening them when it turns out they're too
> loose. Previously cabal was always free to pick an old, unconstrained
> version and would often do so, leading to type errors during builds.
>
> Erik
>
> [1] https://github.com/silkapp/rest/issues/57
> [2] https://github.com/silkapp/aeson-utils/issues/1
> [3] https://github.com/silkapp/json-schema/issues/12
> [4] https://github.com/NixOS/cabal2nix/issues/84
>
> On Mon, Oct 27, 2014 at 8:30 PM, John MacFarlane <[hidden email]> wrote:
>> I must say, I did pause a bit before making this change to
>> pandoc on Hackage, precisely because I realized that the
>> cabal file in the package would not match the one on
>> Hackage.
>>
>> But I went ahead, assuming that if the means were provided
>> to make this kind of change, it should be okay.
>>
>> It is time-consuming for me to make a proper pandoc
>> release.  I need to write up release notes, rebuild the
>> website, generate and test the binary packages on various
>> different platforms, upload to github releases and Hackage,
>> send announcement emails, etc.  Even with automation this
>> all takes work.  I'd rather not have to do this every time a
>> version bound changes on a dependent package, so I like this
>> way of making minor tweaks to version bounds; I hope I
>> can be persuaded that it's not a bad idea.
>>
>> John
>>

TL;DR version of below: I think Hackage should generate a new package
version for each info change rather than the change being ‘silent’
making it pretty exclusive to tools which don't care about what they
actually build and what hash it has (cabal). This brings us back to ‘all
version are explicit’ from the ‘we have versions but there are some
updates you need to pull in your tools too’.

Long version:

As Erik points out at his [4], this really messes things up for us in
the nix world, and really anywhere that

1. Wants to grab from Hackage
2. Wants a reproducible build

The common justification for this is that it doesn't change the tarball
hash. Well then, this ‘update info after upload’ feature then
automatically excludes every single system which uses this tarball. Why?

1. Maintainer uploads a package
2. Distros package it and note down the hash
3. Something happens and maintainer bumps the bounds on Hackage only
4. Distros still grab the very same package as they did before (tarball
doesn't change, remember?). Inside is the cabal file with old
dependencies, so the build fails if the distro updated other packages
(almost certainly).

So even though the hash doesn't change, builds suddenly start failing.
If you're using cabal, you don't get a reproducible build at all because
which versions it picks depends on whatever info is on Hackage at build
time (well, last cabal update time) and not what's in the tarball. It is
currently perfectly possible for two separate users to have ‘cabal
update’ and know about exactly the same package versions available but
end up with different version numbers used because one ‘cabal update’d
before the info on Hackage was changed and the other did not.

But John is right that sometimes making releases is tiresome. For
version bumps it can be as easy as checking out the last tag, bumping
deps and uploading, but it can be much more involved too. I admit that I
used this myself, although only for the stuff that hope to release
properly soon and I happen to maintain on NixOS too which AFAICT is the
only distro which packages Haskell fast enough to notice breakage within
few days (in huge part thanks to Peter Simons ;) )

So clearly there is a problem here: only cabal users use the information
from Hackage info bumps. Everyone else who looks into the tarball just
fails to build. cabal users who happen to cabal update at unfortunate
time also fail to build.

I can only think of one ‘solution’ which fixes this and doesn't require
considerable effort and co-ordination between maintainers and packagers.
Here is how situation currently looks like

1. I upload awesomepackage (ap) to Hackage under ap-1.0.0.0
2. Time passes, I have commits on top
3. One of those packager type of folk who like to demand things come
onto my issue tracker and say I should update my dependencies but I
don't want to release right now
4. I go onto Hackage and use that new cool feature, changing the cabal
file on ap-1.0.0.0.
5. Packager is unhappy because the problem is not fixed, ap-1.0.0.0
gives them the same thing. The only thing they can do is monkeypatch
deps on their side.

Instead, I think at steps 4 and 5 the following should happen:

4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1
package with identical content but new info without any extra effort on
my side.
5. Packager simply now points to ap-1.0.0.0-fix1 which has the new hash
and builds great again. cabal does The Right Thing™ here and resolves
ap-1.0.0.0 to ap-1.0.0.0-fix1 so everything keeps working there as it
does now.

It does seem a bit wasteful space-wise but with clever programming on
Hackage side that could probably be avoided. There is no longer a
problem with these ‘hidden’ changes happening. Everyone can point at the
*exact* version they want used like they do now and they get what
exactly what they asked for. A less wasteful way would be for the
packaging distros to update their tools to look at the Hackage-only
cabal files as well. Personally I think that seems like the wrong place
to solve this though.

Lastly, Hackage should announce with big fat red letters on the front
page of the package that the source tarball contains different info than
what's shown on Hackage. It's really not possible to tell as a user at
the moment unless we specifically look for it.

Sorry for the long-winded message, hopefully you heeded the TL;DR at the
top and skipped this.

--
Mateusz K.
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Andreas Abel
On 28.10.2014 01:38, Mateusz Kowalczyk wrote:
 > 4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1
 > package with identical content but new info without any extra effort on
 > my side.

Basically you suggest that there should be a last digit in the version
number that is only used for dependency fixes and is handled
automatically by cabal and hackage.  We could as well say that Hackage
generates

   ap-1.0.0.0.1
   ap-1.0.0.0.2  -- for the next dependency fix

but you cannot specify the 5th digit in cabal commands, so when you say

   cabal install ap-1.0.0.0

you get the latest, i.e. ap-1.0.0.0.2.

--Andreas

On 28.10.2014 01:38, Mateusz Kowalczyk wrote:

> On 10/27/2014 07:48 PM, Erik Hesselink wrote:
>> I was a bit hesitant about using this feature as well. We've had some
>> confusion when we used it [1][2][3]. The Nix guys also seem to be
>> having some trouble fitting it into their workflow [4]. But in [1] I
>> did a quick grep to see how many packages used the feature, and there
>> are already quite a lot. There is an 'x-revision' property added to
>> the cabal file in the index, so in theory I guess tools could learn to
>> deal with it.
>>
>> In general I think the biggest win for this feature is not even in
>> relaxing bounds, but in tightening them when it turns out they're too
>> loose. Previously cabal was always free to pick an old, unconstrained
>> version and would often do so, leading to type errors during builds.
>>
>> Erik
>>
>> [1] https://github.com/silkapp/rest/issues/57
>> [2] https://github.com/silkapp/aeson-utils/issues/1
>> [3] https://github.com/silkapp/json-schema/issues/12
>> [4] https://github.com/NixOS/cabal2nix/issues/84
>>
>> On Mon, Oct 27, 2014 at 8:30 PM, John MacFarlane <[hidden email]> wrote:
>>> I must say, I did pause a bit before making this change to
>>> pandoc on Hackage, precisely because I realized that the
>>> cabal file in the package would not match the one on
>>> Hackage.
>>>
>>> But I went ahead, assuming that if the means were provided
>>> to make this kind of change, it should be okay.
>>>
>>> It is time-consuming for me to make a proper pandoc
>>> release.  I need to write up release notes, rebuild the
>>> website, generate and test the binary packages on various
>>> different platforms, upload to github releases and Hackage,
>>> send announcement emails, etc.  Even with automation this
>>> all takes work.  I'd rather not have to do this every time a
>>> version bound changes on a dependent package, so I like this
>>> way of making minor tweaks to version bounds; I hope I
>>> can be persuaded that it's not a bad idea.
>>>
>>> John
>>>
>
> TL;DR version of below: I think Hackage should generate a new package
> version for each info change rather than the change being ‘silent’
> making it pretty exclusive to tools which don't care about what they
> actually build and what hash it has (cabal). This brings us back to ‘all
> version are explicit’ from the ‘we have versions but there are some
> updates you need to pull in your tools too’.
>
> Long version:
>
> As Erik points out at his [4], this really messes things up for us in
> the nix world, and really anywhere that
>
> 1. Wants to grab from Hackage
> 2. Wants a reproducible build
>
> The common justification for this is that it doesn't change the tarball
> hash. Well then, this ‘update info after upload’ feature then
> automatically excludes every single system which uses this tarball. Why?
>
> 1. Maintainer uploads a package
> 2. Distros package it and note down the hash
> 3. Something happens and maintainer bumps the bounds on Hackage only
> 4. Distros still grab the very same package as they did before (tarball
> doesn't change, remember?). Inside is the cabal file with old
> dependencies, so the build fails if the distro updated other packages
> (almost certainly).
>
> So even though the hash doesn't change, builds suddenly start failing.
> If you're using cabal, you don't get a reproducible build at all because
> which versions it picks depends on whatever info is on Hackage at build
> time (well, last cabal update time) and not what's in the tarball. It is
> currently perfectly possible for two separate users to have ‘cabal
> update’ and know about exactly the same package versions available but
> end up with different version numbers used because one ‘cabal update’d
> before the info on Hackage was changed and the other did not.
>
> But John is right that sometimes making releases is tiresome. For
> version bumps it can be as easy as checking out the last tag, bumping
> deps and uploading, but it can be much more involved too. I admit that I
> used this myself, although only for the stuff that hope to release
> properly soon and I happen to maintain on NixOS too which AFAICT is the
> only distro which packages Haskell fast enough to notice breakage within
> few days (in huge part thanks to Peter Simons ;) )
>
> So clearly there is a problem here: only cabal users use the information
> from Hackage info bumps. Everyone else who looks into the tarball just
> fails to build. cabal users who happen to cabal update at unfortunate
> time also fail to build.
>
> I can only think of one ‘solution’ which fixes this and doesn't require
> considerable effort and co-ordination between maintainers and packagers.
> Here is how situation currently looks like
>
> 1. I upload awesomepackage (ap) to Hackage under ap-1.0.0.0
> 2. Time passes, I have commits on top
> 3. One of those packager type of folk who like to demand things come
> onto my issue tracker and say I should update my dependencies but I
> don't want to release right now
> 4. I go onto Hackage and use that new cool feature, changing the cabal
> file on ap-1.0.0.0.
> 5. Packager is unhappy because the problem is not fixed, ap-1.0.0.0
> gives them the same thing. The only thing they can do is monkeypatch
> deps on their side.
>
> Instead, I think at steps 4 and 5 the following should happen:
>
> 4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1
> package with identical content but new info without any extra effort on
> my side.
> 5. Packager simply now points to ap-1.0.0.0-fix1 which has the new hash
> and builds great again. cabal does The Right Thing™ here and resolves
> ap-1.0.0.0 to ap-1.0.0.0-fix1 so everything keeps working there as it
> does now.
>
> It does seem a bit wasteful space-wise but with clever programming on
> Hackage side that could probably be avoided. There is no longer a
> problem with these ‘hidden’ changes happening. Everyone can point at the
> *exact* version they want used like they do now and they get what
> exactly what they asked for. A less wasteful way would be for the
> packaging distros to update their tools to look at the Hackage-only
> cabal files as well. Personally I think that seems like the wrong place
> to solve this though.
>
> Lastly, Hackage should announce with big fat red letters on the front
> page of the package that the source tarball contains different info than
> what's shown on Hackage. It's really not possible to tell as a user at
> the moment unless we specifically look for it.
>
> Sorry for the long-winded message, hopefully you heeded the TL;DR at the
> top and skipped this.
>


--
Andreas Abel  <><      Du bist der geliebte Mensch.

Department of Computer Science and Engineering
Chalmers and Gothenburg University, Sweden

[hidden email]
http://www2.tcs.ifi.lmu.de/~abel/
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Carter Schonwald
per se, thats what hackage DOES now. Though it is valid to want some sort of +patchlevel suffix to be visible for distro engineering purposes or such.

Speaking as a hackage trustee, this .cabal editing facility has been a VERY VERY good thing overall. A lot of problems have been prevented because of it, and that its creating new problems that haven't existed before should be considered a testament to its success! ;) 

On Tue, Oct 28, 2014 at 8:17 AM, Andreas Abel <[hidden email]> wrote:
On 28.10.2014 01:38, Mateusz Kowalczyk wrote:
> 4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1
> package with identical content but new info without any extra effort on
> my side.

Basically you suggest that there should be a last digit in the version number that is only used for dependency fixes and is handled automatically by cabal and hackage.  We could as well say that Hackage generates

  ap-1.0.0.0.1
  ap-1.0.0.0.2  -- for the next dependency fix

but you cannot specify the 5th digit in cabal commands, so when you say

  cabal install ap-1.0.0.0

you get the latest, i.e. ap-1.0.0.0.2.

--Andreas


On 28.10.2014 01:38, Mateusz Kowalczyk wrote:
On 10/27/2014 07:48 PM, Erik Hesselink wrote:
I was a bit hesitant about using this feature as well. We've had some
confusion when we used it [1][2][3]. The Nix guys also seem to be
having some trouble fitting it into their workflow [4]. But in [1] I
did a quick grep to see how many packages used the feature, and there
are already quite a lot. There is an 'x-revision' property added to
the cabal file in the index, so in theory I guess tools could learn to
deal with it.

In general I think the biggest win for this feature is not even in
relaxing bounds, but in tightening them when it turns out they're too
loose. Previously cabal was always free to pick an old, unconstrained
version and would often do so, leading to type errors during builds.

Erik

[1] https://github.com/silkapp/rest/issues/57
[2] https://github.com/silkapp/aeson-utils/issues/1
[3] https://github.com/silkapp/json-schema/issues/12
[4] https://github.com/NixOS/cabal2nix/issues/84

On Mon, Oct 27, 2014 at 8:30 PM, John MacFarlane <[hidden email]> wrote:
I must say, I did pause a bit before making this change to
pandoc on Hackage, precisely because I realized that the
cabal file in the package would not match the one on
Hackage.

But I went ahead, assuming that if the means were provided
to make this kind of change, it should be okay.

It is time-consuming for me to make a proper pandoc
release.  I need to write up release notes, rebuild the
website, generate and test the binary packages on various
different platforms, upload to github releases and Hackage,
send announcement emails, etc.  Even with automation this
all takes work.  I'd rather not have to do this every time a
version bound changes on a dependent package, so I like this
way of making minor tweaks to version bounds; I hope I
can be persuaded that it's not a bad idea.

John


TL;DR version of below: I think Hackage should generate a new package
version for each info change rather than the change being ‘silent’
making it pretty exclusive to tools which don't care about what they
actually build and what hash it has (cabal). This brings us back to ‘all
version are explicit’ from the ‘we have versions but there are some
updates you need to pull in your tools too’.

Long version:

As Erik points out at his [4], this really messes things up for us in
the nix world, and really anywhere that

1. Wants to grab from Hackage
2. Wants a reproducible build

The common justification for this is that it doesn't change the tarball
hash. Well then, this ‘update info after upload’ feature then
automatically excludes every single system which uses this tarball. Why?

1. Maintainer uploads a package
2. Distros package it and note down the hash
3. Something happens and maintainer bumps the bounds on Hackage only
4. Distros still grab the very same package as they did before (tarball
doesn't change, remember?). Inside is the cabal file with old
dependencies, so the build fails if the distro updated other packages
(almost certainly).

So even though the hash doesn't change, builds suddenly start failing.
If you're using cabal, you don't get a reproducible build at all because
which versions it picks depends on whatever info is on Hackage at build
time (well, last cabal update time) and not what's in the tarball. It is
currently perfectly possible for two separate users to have ‘cabal
update’ and know about exactly the same package versions available but
end up with different version numbers used because one ‘cabal update’d
before the info on Hackage was changed and the other did not.

But John is right that sometimes making releases is tiresome. For
version bumps it can be as easy as checking out the last tag, bumping
deps and uploading, but it can be much more involved too. I admit that I
used this myself, although only for the stuff that hope to release
properly soon and I happen to maintain on NixOS too which AFAICT is the
only distro which packages Haskell fast enough to notice breakage within
few days (in huge part thanks to Peter Simons ;) )

So clearly there is a problem here: only cabal users use the information
from Hackage info bumps. Everyone else who looks into the tarball just
fails to build. cabal users who happen to cabal update at unfortunate
time also fail to build.

I can only think of one ‘solution’ which fixes this and doesn't require
considerable effort and co-ordination between maintainers and packagers.
Here is how situation currently looks like

1. I upload awesomepackage (ap) to Hackage under ap-1.0.0.0
2. Time passes, I have commits on top
3. One of those packager type of folk who like to demand things come
onto my issue tracker and say I should update my dependencies but I
don't want to release right now
4. I go onto Hackage and use that new cool feature, changing the cabal
file on ap-1.0.0.0.
5. Packager is unhappy because the problem is not fixed, ap-1.0.0.0
gives them the same thing. The only thing they can do is monkeypatch
deps on their side.

Instead, I think at steps 4 and 5 the following should happen:

4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1
package with identical content but new info without any extra effort on
my side.
5. Packager simply now points to ap-1.0.0.0-fix1 which has the new hash
and builds great again. cabal does The Right Thing™ here and resolves
ap-1.0.0.0 to ap-1.0.0.0-fix1 so everything keeps working there as it
does now.

It does seem a bit wasteful space-wise but with clever programming on
Hackage side that could probably be avoided. There is no longer a
problem with these ‘hidden’ changes happening. Everyone can point at the
*exact* version they want used like they do now and they get what
exactly what they asked for. A less wasteful way would be for the
packaging distros to update their tools to look at the Hackage-only
cabal files as well. Personally I think that seems like the wrong place
to solve this though.

Lastly, Hackage should announce with big fat red letters on the front
page of the package that the source tarball contains different info than
what's shown on Hackage. It's really not possible to tell as a user at
the moment unless we specifically look for it.

Sorry for the long-winded message, hopefully you heeded the TL;DR at the
top and skipped this.



--
Andreas Abel  <><      Du bist der geliebte Mensch.

Department of Computer Science and Engineering
Chalmers and Gothenburg University, Sweden

[hidden email]
http://www2.tcs.ifi.lmu.de/~abel/

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Magnus Therning
On Tue, Oct 28, 2014 at 05:22:26PM -0400, Carter Schonwald wrote:
> per se, thats what hackage DOES now. Though it is valid to want some
> sort of +patchlevel suffix to be visible for distro engineering
> purposes or such.
>
> Speaking as a hackage trustee, this .cabal editing facility has been
> a VERY VERY good thing overall. A lot of problems have been
> prevented because of it, and that its creating new problems that
> haven't existed before should be considered a testament to its
> success! ;)

If you define success as "swapping one set of problems for another",
then we have very different definitions of success ;)

Seriously though, I don't doubt that the feature has value *for some
users* it's just a bit unfortunate that it was implemented in a way
that detracts value for other users.  Especially when I suspect there
is a very easy way to implement it that wouldn't have impacted those
other users at all.

I'm not a cabal-install user, so I'm curious to hear how this change
works with cabal-install 1.18.0.5, which is what we package in
ArchHaskell.  Any problems with that?  If so, will support for this
feature be backported?

/M

--
Magnus Therning                      OpenPGP: 0xAB4DFBA4
email: [hidden email]   jabber: [hidden email]
twitter: magthe               http://therning.org/magnus

Goto labels should be left-aligned in all caps and should include the
programmer's name, home phone number, and credit card number.
     -- Abdul Nizar

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe

attachment0 (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Mikhail Glushenkov-2
Hi,

On 29 October 2014 08:40, Magnus Therning <[hidden email]> wrote:
> I'm not a cabal-install user, so I'm curious to hear how this change
> works with cabal-install 1.18.0.5, which is what we package in
> ArchHaskell.  Any problems with that?  If so, will support for this
> feature be backported?

This feature was added two years ago, in the 1.16 release [1].

[1] https://github.com/haskell/cabal/commit/22d3772222ebb5dd49d60882845915999ebc97e2
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Magnus Therning
On Wed, Oct 29, 2014 at 09:09:25AM +0100, Mikhail Glushenkov wrote:

> Hi,
>
> On 29 October 2014 08:40, Magnus Therning <[hidden email]> wrote:
> > I'm not a cabal-install user, so I'm curious to hear how this change
> > works with cabal-install 1.18.0.5, which is what we package in
> > ArchHaskell.  Any problems with that?  If so, will support for this
> > feature be backported?
>
> This feature was added two years ago, in the 1.16 release [1].
>
> [1] https://github.com/haskell/cabal/commit/22d3772222ebb5dd49d60882845915999ebc97e2
Excellent, then that's one less thing to worry about.

/M

--
Magnus Therning                      OpenPGP: 0xAB4DFBA4
email: [hidden email]   jabber: [hidden email]
twitter: magthe               http://therning.org/magnus

Goto labels should be left-aligned in all caps and should include the
programmer's name, home phone number, and credit card number.
     -- Abdul Nizar

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe

attachment0 (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Peter Simons
In reply to this post by Andreas Abel
Hi Andreas,

 > On 28.10.2014 01:38, Mateusz Kowalczyk wrote:
 >
 >> 4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1
 >> package with identical content but new info without any extra effort
 >> on my side.
 >
 > Basically you suggest that there should be a last digit in the
 > version number that is only used for dependency fixes and is handled
 > automatically by cabal and hackage.

the important part of Mateusz suggestion is that the version number
ought to *change* after an edit. The exact mechanics of that change --
i.e. whether Hackage bumps some digit or appends a tag or whatever --
aren't very important. What is important is that it's possible to
distinguish the edited version from the original one. As of now, users
cannot tell which variant of the package they've installed.


 > We could as well say that Hackage generates
 >
 >   ap-1.0.0.0.1
 >   ap-1.0.0.0.2  -- for the next dependency fix

That would be one possible solution, yes. Personally, I think these kind
of version numbers are a bit silly, so I would prefer to append a tag,
but I don't feel strongly about that.


 > You cannot specify the 5th digit in cabal commands, so when you say
 >
 >   cabal install ap-1.0.0.0
 >
 > you get the latest, i.e. ap-1.0.0.0.2.

Hackage edits are destructive: the original version is gone afterwards,
and it's no longer possible to install it. As such, this point doesn't
seem relevant. What is relevant, however, that people who have installed
version ap-1.0.0.0.1 prior to the edit can actually tell that there is
an update available -- ap-1.0.0.0.2 is *newer*. In the current state of
affairs, it's not possible to tell that a package has been updated
("edited") just by looking at its version number.

Best regards,
Peter

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Tom Ellis
On Thu, Oct 30, 2014 at 12:01:09PM +0100, Peter Simons wrote:
>  > We could as well say that Hackage generates
>  >
>  >   ap-1.0.0.0.1
>  >   ap-1.0.0.0.2  -- for the next dependency fix
>
> That would be one possible solution, yes. Personally, I think these kind
> of version numbers are a bit silly, so I would prefer to append a tag,
> but I don't feel strongly about that.

How would this be any different from just uploading a new version with an
updated cabal file?
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Erik Hesselink
In reply to this post by Peter Simons
On Thu, Oct 30, 2014 at 12:01 PM, Peter Simons <[hidden email]> wrote:
> aren't very important. What is important is that it's possible to
> distinguish the edited version from the original one. As of now, users
> cannot tell which variant of the package they've installed.

Why is this important? I believe there is no difference between two
revisions after they're installed, at least not currently, since
you're only allowed to edit dependency version ranges, and perhaps
some informational fields.

Erik
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Ivan Lazar Miljenovic
On 30 October 2014 22:09, Erik Hesselink <[hidden email]> wrote:
> On Thu, Oct 30, 2014 at 12:01 PM, Peter Simons <[hidden email]> wrote:
>> aren't very important. What is important is that it's possible to
>> distinguish the edited version from the original one. As of now, users
>> cannot tell which variant of the package they've installed.
>
> Why is this important? I believe there is no difference between two
> revisions after they're installed, at least not currently, since
> you're only allowed to edit dependency version ranges, and perhaps
> some informational fields.

* For people doing cabal -> foo converters to know that there _is_ a
newer version

* "Oh, wait, I can't update the version of library foo because I've
got the latest available version of bar and it doesn't support the new
version of foo" without knowing there's a (pseudo-) newer version
which _does_ allow the newer version of foo.

>
> Erik
> _______________________________________________
> Haskell-Cafe mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell-cafe



--
Ivan Lazar Miljenovic
[hidden email]
http://IvanMiljenovic.wordpress.com
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Hackage inconsistent? (pandoc.cabal in 1.13.1)

Peter Simons
In reply to this post by Erik Hesselink
Hi Erik,

 >> What is important is that it's possible to distinguish the edited
 >> version from the original one. As of now, users cannot tell which
 >> variant of the package they've installed.
 >
 > Why is this important? I believe there is no difference between two
 > revisions after they're installed, at least not currently, since
 > you're only allowed to edit dependency version ranges [...].

suppose my application foo-1.0 depends on bar >= 1.0, but it turns out
that bar-1.0 has a bug that's fixed in version 1.0.1. So I edit foo-1.0
to depend on bar > 1.0 instead. Now, a user of foo-1.0 sends me an
e-mail and complains about crashes in my code.

At that point, I cannot know whether he's using bar >= 1.0 or bar > 1.0,
because the version number of foo didn't change after the edit.

Best regards,
Peter

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
12