Splitting Network.URI from the network package

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
41 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Splitting Network.URI from the network package

Michael Snoyman
This was brought up last year[1], and I'd like to bring it up again, based on a recent issue I was working through with a user[2]. I realize that this is a breaking change, but:

1. It's a minor breaking change: you simply need to add an extra package to your build-depends.
2. The problems caused by having a parsec dependency in network can be severe, especially for new users (I'll describe the details after the proposal).

Concretely, I believe we should do the following:

1. Create a new package, network-uri, version 2.5.0.0, which exposes no modules and has an upper bound `network < 2.6.
2. Create a second release of network-uri, version 3.0.0.0, which provides the Network.URI module verbatim as provided by the network package today, and has a lower bound `network >= 3.0`.
3. Release network version 3.0.0.0, with no changes from the currently released version, except that (a) no Network.URI module is provided, and (b) there is no parsec dependency.

I don't remember how the discussion went last time, but I seem to remember general consensus. I'd like to set a discussion period of two weeks (August 15).

## Motivation

To give a concrete example of why this problem is severe, consider the following data:

* The network package is a pain to install on Windows for most users (especially new users), since it requires msys.
* Most Windows users therefore install the Haskell Platform, avoiding the msys dependency.
* The current release of HP installs text version 0.11.3.1. text is a dependency of parsec, and parsec is a dependency of network. Therefore, you can't build against a new version of text *and* use the network package without recompiling network, which as I mentioned is difficult.
* A number of popular packages depend on newer versions of text. For example, since 0.8.0.0, aeson requires text version 1.1.0.0 or later. as does attoparsec since version 0.12.0.0.

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Daniel Díaz Casanueva
I have been affected by this problem in the past, so +1 from me. Plus, IMHO, it seems independent from the rest of the package.

Regards,
Daniel Díaz.

On Friday, August 1, 2014, Michael Snoyman <[hidden email]> wrote:
This was brought up last year[1], and I'd like to bring it up again, based on a recent issue I was working through with a user[2]. I realize that this is a breaking change, but:

1. It's a minor breaking change: you simply need to add an extra package to your build-depends.
2. The problems caused by having a parsec dependency in network can be severe, especially for new users (I'll describe the details after the proposal).

Concretely, I believe we should do the following:

1. Create a new package, network-uri, version 2.5.0.0, which exposes no modules and has an upper bound `network < 2.6.
2. Create a second release of network-uri, version 3.0.0.0, which provides the Network.URI module verbatim as provided by the network package today, and has a lower bound `network >= 3.0`.
3. Release network version 3.0.0.0, with no changes from the currently released version, except that (a) no Network.URI module is provided, and (b) there is no parsec dependency.

I don't remember how the discussion went last time, but I seem to remember general consensus. I'd like to set a discussion period of two weeks (August 15).

## Motivation

To give a concrete example of why this problem is severe, consider the following data:

* The network package is a pain to install on Windows for most users (especially new users), since it requires msys.
* Most Windows users therefore install the Haskell Platform, avoiding the msys dependency.
* The current release of HP installs text version 0.11.3.1. text is a dependency of parsec, and parsec is a dependency of network. Therefore, you can't build against a new version of text *and* use the network package without recompiling network, which as I mentioned is difficult.
* A number of popular packages depend on newer versions of text. For example, since 0.8.0.0, aeson requires text version 1.1.0.0 or later. as does attoparsec since version 0.12.0.0.

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Christian Maeder-2
In reply to this post by Michael Snoyman
+1 (although I still depend on HTTP)

Since HTTP is in the HP, network-uri must also go into the HP. Who is
going to maintain network-uri?

Christian

Am 01.08.2014 13:31, schrieb Michael Snoyman:

> This was brought up last year[1], and I'd like to bring it up again,
> based on a recent issue I was working through with a user[2]. I realize
> that this is a breaking change, but:
>
> 1. It's a minor breaking change: you simply need to add an extra package
> to your build-depends.
> 2. The problems caused by having a parsec dependency in network can be
> severe, especially for new users (I'll describe the details after the
> proposal).
>
> Concretely, I believe we should do the following:
>
> 1. Create a new package, network-uri, version 2.5.0.0, which exposes no
> modules and has an upper bound `network < 2.6.
> 2. Create a second release of network-uri, version 3.0.0.0, which
> provides the Network.URI module verbatim as provided by the network
> package today, and has a lower bound `network >= 3.0`.
> 3. Release network version 3.0.0.0, with no changes from the currently
> released version, except that (a) no Network.URI module is provided, and
> (b) there is no parsec dependency.
>
> I don't remember how the discussion went last time, but I seem to
> remember general consensus. I'd like to set a discussion period of two
> weeks (August 15).
>
> ## Motivation
>
> To give a concrete example of why this problem is severe, consider the
> following data:
>
> * The network package is a pain to install on Windows for most users
> (especially new users), since it requires msys.
> * Most Windows users therefore install the Haskell Platform, avoiding
> the msys dependency.
> * The current release of HP installs text version 0.11.3.1. text is a
> dependency of parsec, and parsec is a dependency of network. Therefore,
> you can't build against a new version of text *and* use the network
> package without recompiling network, which as I mentioned is difficult.
> * A number of popular packages depend on newer versions of text. For
> example, since 0.8.0.0, aeson requires text version 1.1.0.0 or later. as
> does attoparsec since version 0.12.0.0.
>
> [1] http://www.haskell.org/pipermail/libraries/2013-January/019234.html
> [2] https://groups.google.com/d/msg/yesodweb/auk2vByXgO8/lUZ9oanKyMwJ
>
>
> _______________________________________________
> Libraries mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/libraries
>

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Michael Snoyman
Given that there's (hopefully) very little changing in Network.URI, I'll step up to the plate on maintenance if no one else will (though I'd be quite happy if someone else wants to take it instead).


On Fri, Aug 1, 2014 at 8:17 PM, Christian Maeder <[hidden email]> wrote:
+1 (although I still depend on HTTP)

Since HTTP is in the HP, network-uri must also go into the HP. Who is going to maintain network-uri?

Christian

Am 01.08.2014 13:31, schrieb Michael Snoyman:
This was brought up last year[1], and I'd like to bring it up again,
based on a recent issue I was working through with a user[2]. I realize
that this is a breaking change, but:

1. It's a minor breaking change: you simply need to add an extra package
to your build-depends.
2. The problems caused by having a parsec dependency in network can be
severe, especially for new users (I'll describe the details after the
proposal).

Concretely, I believe we should do the following:

1. Create a new package, network-uri, version 2.5.0.0, which exposes no
modules and has an upper bound `network < 2.6.
2. Create a second release of network-uri, version 3.0.0.0, which
provides the Network.URI module verbatim as provided by the network
package today, and has a lower bound `network >= 3.0`.
3. Release network version 3.0.0.0, with no changes from the currently
released version, except that (a) no Network.URI module is provided, and
(b) there is no parsec dependency.

I don't remember how the discussion went last time, but I seem to
remember general consensus. I'd like to set a discussion period of two
weeks (August 15).

## Motivation

To give a concrete example of why this problem is severe, consider the
following data:

* The network package is a pain to install on Windows for most users
(especially new users), since it requires msys.
* Most Windows users therefore install the Haskell Platform, avoiding
the msys dependency.
* The current release of HP installs text version 0.11.3.1. text is a
dependency of parsec, and parsec is a dependency of network. Therefore,
you can't build against a new version of text *and* use the network
package without recompiling network, which as I mentioned is difficult.
* A number of popular packages depend on newer versions of text. For
example, since 0.8.0.0, aeson requires text version 1.1.0.0 or later. as
does attoparsec since version 0.12.0.0.

[1] http://www.haskell.org/pipermail/libraries/2013-January/019234.html
[2] https://groups.google.com/d/msg/yesodweb/auk2vByXgO8/lUZ9oanKyMwJ


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries




_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Kazu Yamamoto (山本和彦)
In reply to this post by Michael Snoyman
Hi,

> This was brought up last year[1], and I'd like to bring it up again, based
> on a recent issue I was working through with a user[2]. I realize that this
> is a breaking change, but:

+1

> 1. Create a new package, network-uri, version 2.5.0.0, which exposes no
> modules and has an upper bound `network < 2.6.

Yes, I think that we agreed with the package name of "network-uri".

> 2. Create a second release of network-uri, version 3.0.0.0, which provides
> the Network.URI module verbatim as provided by the network package today,
> and has a lower bound `network >= 3.0`.
> 3. Release network version 3.0.0.0, with no changes from the currently
> released version, except that (a) no Network.URI module is provided, and
> (b) there is no parsec dependency.
>
> I don't remember how the discussion went last time, but I seem to remember
> general consensus. I'd like to set a discussion period of two weeks (August
> 15).

The consensus was to separate the network-uri package from the network
package. We did not reach how to do it. But I think you should put it
forward because we should not repeat the plateau again.

--Kazu
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Carter Schonwald
while we're doing breaking changes, would adding bytestring.char and text support to a split out network-uri be in scope. OR should those be in their own wee packages?


On Sun, Aug 3, 2014 at 10:55 PM, Kazu Yamamoto <[hidden email]> wrote:
Hi,

> This was brought up last year[1], and I'd like to bring it up again, based
> on a recent issue I was working through with a user[2]. I realize that this
> is a breaking change, but:

+1

> 1. Create a new package, network-uri, version 2.5.0.0, which exposes no
> modules and has an upper bound `network < 2.6.

Yes, I think that we agreed with the package name of "network-uri".

> 2. Create a second release of network-uri, version 3.0.0.0, which provides
> the Network.URI module verbatim as provided by the network package today,
> and has a lower bound `network >= 3.0`.
> 3. Release network version 3.0.0.0, with no changes from the currently
> released version, except that (a) no Network.URI module is provided, and
> (b) there is no parsec dependency.
>
> I don't remember how the discussion went last time, but I seem to remember
> general consensus. I'd like to set a discussion period of two weeks (August
> 15).

The consensus was to separate the network-uri package from the network
package. We did not reach how to do it. But I think you should put it
forward because we should not repeat the plateau again.

--Kazu
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Michael Snoyman
I see no reason *not* to include those in network-uri itself (I've actually given some thought to that), especially given that that it wouldn't be adding any new dependencies (network-uri will depend on parsec, which already depends on bytestring and text). However, I'd recommend adding those in a post-3.0 release, for two reasons:

1. I think as a general policy, package splitting/merging should not be accompanied by any other API changes, as that can be confusing for users.
2. Adding a new module like Network.URI.Text would not be a breaking change, so it won't cause any new upgrade issues in the future.

As far as implementation of such a library, we have plenty of good questions that come up (should we introduces an attoparsec dep here? should one of the implementations be the canonical one and the others just wrappers around it?), but I'd rather address those after this proposal comes to an end and we have network-uri as its own entity.


On Mon, Aug 4, 2014 at 6:56 AM, Carter Schonwald <[hidden email]> wrote:
while we're doing breaking changes, would adding bytestring.char and text support to a split out network-uri be in scope. OR should those be in their own wee packages?


On Sun, Aug 3, 2014 at 10:55 PM, Kazu Yamamoto <[hidden email]> wrote:
Hi,

> This was brought up last year[1], and I'd like to bring it up again, based
> on a recent issue I was working through with a user[2]. I realize that this
> is a breaking change, but:

+1

> 1. Create a new package, network-uri, version 2.5.0.0, which exposes no
> modules and has an upper bound `network < 2.6.

Yes, I think that we agreed with the package name of "network-uri".

> 2. Create a second release of network-uri, version 3.0.0.0, which provides
> the Network.URI module verbatim as provided by the network package today,
> and has a lower bound `network >= 3.0`.
> 3. Release network version 3.0.0.0, with no changes from the currently
> released version, except that (a) no Network.URI module is provided, and
> (b) there is no parsec dependency.
>
> I don't remember how the discussion went last time, but I seem to remember
> general consensus. I'd like to set a discussion period of two weeks (August
> 15).

The consensus was to separate the network-uri package from the network
package. We did not reach how to do it. But I think you should put it
forward because we should not repeat the plateau again.

--Kazu
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries



_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Johan Tibell-2
Hi Michael,

I'm supportive of this (just as I've been in the past). We need to
have network-uri added to the HP, which I don't think should be a
problem as the API doesn't change with this split, assuming that the
new maintainer agrees to follow the HP rules (e.g. don't add non-HP
packages as dependencies, don't radical change the API without
discussion.)

I have a slight preference to use 2.6 as the network version number,
as I'm saving 3.0 for something more juicy.

> 1. Create a new package, network-uri, version 2.5.0.0, which exposes no modules and has an upper bound `network < 2.6.

I'm sure there's something clever behind this empty package, but I
don't quite understand it. How does this help. Note that depending on
network-uri but not network will not let you use Network.URI in your
code, as that module will be hidden.

> 2. Create a second release of network-uri, version 3.0.0.0, which provides the Network.URI module verbatim as provided by the network package today, and has a lower bound `network >= 3.0`.

(And < X+1, as the HP follows the PVP.)
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Michael Snoyman



On Mon, Aug 4, 2014 at 6:55 PM, Johan Tibell <[hidden email]> wrote:
Hi Michael,

I'm supportive of this (just as I've been in the past). We need to
have network-uri added to the HP, which I don't think should be a
problem as the API doesn't change with this split, assuming that the
new maintainer agrees to follow the HP rules (e.g. don't add non-HP
packages as dependencies, don't radical change the API without
discussion.)

I have a slight preference to use 2.6 as the network version number,
as I'm saving 3.0 for something more juicy.


I have no objection to 2.6 instead of 3.0. I didn't feel strongly one way or the other, I just wanted to make the proposal as concrete as possible. Consider it amended to use 2.6 instead.
 
> 1. Create a new package, network-uri, version 2.5.0.0, which exposes no modules and has an upper bound `network < 2.6.

I'm sure there's something clever behind this empty package, but I
don't quite understand it. How does this help. Note that depending on
network-uri but not network will not let you use Network.URI in your
code, as that module will be hidden.


The idea is that users of Network.URI will be able to add a dependency along the lines of:

    build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

and work with both the pre- and post-split versions of the package, without any conditional compilation or cabal flags. I'm sensitive to adding any requirements of cabal flags, since (1) it clutters cabal files quite a bit[1], and (2) we just went through some painful cabal-install dependency solver issues around flags.
 
> 2. Create a second release of network-uri, version 3.0.0.0, which provides the Network.URI module verbatim as provided by the network package today, and has a lower bound `network >= 3.0`.

(And < X+1, as the HP follows the PVP.)

I really don't want this to turn into another PVP discussion. And I'm not going to object to including an upper bound here if that's deemed "required" for some reason (I only left it out of the proposal since it's irrelevant to my point). All that said...

Can you name a single way in which an upper bound *actually* helps? network-uri will not depend on any functionality in network itself[2], so there's no way that a breaking change in network will possibly cause network-uri to stop compiling. The *only* possibility is that network could expose a module that conflicts with a module in network-uri. However:

1. That is not a protected situation by the PVP anyway; adding a new module to a package is a minor version bump, so an upper bound wouldn't provide protection.
2. I actually think we *should* have a guarantee that network never expose a module that conflicts with network-uri. Better yet, I think there should be an accepted standard that no two Haskell Platform packages export the same module name. (I'm not actually proposing that here, it's clearly separate than my main proposal.)

So if someone can articulate a good reason for an upper bound in network-uri on network, please let me know. Again, if this is going to end up blocking this proposal, I'd rather just include the upper bound.

Michael

[1] Two painful experiences for me were the network + network-bytestring merge, and the blaze-html/blaze-markdown split. I'd like to try to avoid that situation, and in my recent package merging, I've done so as much as possible.
[2] If there *is* some dependency from the Network.URI module to the rest of the network package, please let me know, I couldn't see one.

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Johan Tibell-2
On Mon, Aug 4, 2014 at 6:21 PM, Michael Snoyman <[hidden email]> wrote:
> The idea is that users of Network.URI will be able to add a dependency along
> the lines of:
>
>     build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7
>
> and work with both the pre- and post-split versions of the package, without
> any conditional compilation or cabal flags. I'm sensitive to adding any
> requirements of cabal flags, since (1) it clutters cabal files quite a
> bit[1], and (2) we just went through some painful cabal-install dependency
> solver issues around flags.

I agree that using flags to encode OR is a bit heavy weight. We should really support || in build-depends. For completeness, here's what a flag-based solution looks like. Assuming that network-uri-2.6 has the URI module and network-2.6 no longer has it we get this flag setup:

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6
  else
    build-depends: network < 2.6


If the user wants something else from network (e.g. Network.Socket) it would be

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6, network >= 2.6
  else
    build-depends: network < 2.6

Lets see if I understood your scheme correctly. We'd have

 * network-2.5: Has URI module.
 * network-uri-2.5: Doesn't have URI module. Depends on < network-2.6 (i.e. network-2.5 in this example).
 * network-2.6: Doesn't have URI module.
 * network-uri-2.6: Has URI module. Depends on network >= 2.6 (i.e. network 2.6 in this example).

Given

build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

legal combinations of the above packages are

 * network-2.5 and network-uri-2.5, which gives URI through network, and
 * network-2.6 and network-uri-2.6, which gives URI through network-uri.

There are two implications of this, which are slightly strange:

 * There's no point in anyone depending on only network-uri-2.5 (as it doesn't expose anything).
 * network-uri-2.6 and later will have to have a lower dependency on network forever, even though network-uri doesn't use anything from network (this is reflected when you discuss the upper bound in the context of the PVP later).

I don't know if there's any other implications. Can anyone think of any?

-- Johan


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Edward Z. Yang
In reply to this post by Michael Snoyman
Hello Michael,

I'd like to chime in that with module reexports (a new feature landing
in GHC 7.10, see https://ghc.haskell.org/trac/ghc/wiki/ModuleReexports),
network-uri can safely reexport the Network.URI module defined in old
versions of network (if both packages are included, GHC won't consider
the import ambiguous.)

If you wanted to do things completely without breaking BC, this would
suggest making a new package network-without-uri (but obviously a better
name), and users who don't use Network.URI can switch their
build-depends against that. network also defines Network.URI, but now it
reexports modules from network-without-uri.

Cheers,
Edward

Excerpts from Michael Snoyman's message of 2014-08-01 12:31:13 +0100:

> This was brought up last year[1], and I'd like to bring it up again, based
> on a recent issue I was working through with a user[2]. I realize that this
> is a breaking change, but:
>
> 1. It's a minor breaking change: you simply need to add an extra package to
> your build-depends.
> 2. The problems caused by having a parsec dependency in network can be
> severe, especially for new users (I'll describe the details after the
> proposal).
>
> Concretely, I believe we should do the following:
>
> 1. Create a new package, network-uri, version 2.5.0.0, which exposes no
> modules and has an upper bound `network < 2.6.
> 2. Create a second release of network-uri, version 3.0.0.0, which provides
> the Network.URI module verbatim as provided by the network package today,
> and has a lower bound `network >= 3.0`.
> 3. Release network version 3.0.0.0, with no changes from the currently
> released version, except that (a) no Network.URI module is provided, and
> (b) there is no parsec dependency.
>
> I don't remember how the discussion went last time, but I seem to remember
> general consensus. I'd like to set a discussion period of two weeks (August
> 15).
>
> ## Motivation
>
> To give a concrete example of why this problem is severe, consider the
> following data:
>
> * The network package is a pain to install on Windows for most users
> (especially new users), since it requires msys.
> * Most Windows users therefore install the Haskell Platform, avoiding the
> msys dependency.
> * The current release of HP installs text version 0.11.3.1. text is a
> dependency of parsec, and parsec is a dependency of network. Therefore, you
> can't build against a new version of text *and* use the network package
> without recompiling network, which as I mentioned is difficult.
> * A number of popular packages depend on newer versions of text. For
> example, since 0.8.0.0, aeson requires text version 1.1.0.0 or later. as
> does attoparsec since version 0.12.0.0.
>
> [1] http://www.haskell.org/pipermail/libraries/2013-January/019234.html
> [2] https://groups.google.com/d/msg/yesodweb/auk2vByXgO8/lUZ9oanKyMwJ
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Michael Snoyman
That's a nice feature, big +1 from me. But it's not something we can rely on until 7.10 has completely supplanted previous versions of GHC, but once that happens, I'll look forward to using it for these situations.

Without module reexports, I'm fairly certain the breakage from something like network-without-uri would be steeper than the proposal I'm making.


On Fri, Aug 8, 2014 at 3:11 PM, Edward Z. Yang <[hidden email]> wrote:
Hello Michael,

I'd like to chime in that with module reexports (a new feature landing
in GHC 7.10, see https://ghc.haskell.org/trac/ghc/wiki/ModuleReexports),
network-uri can safely reexport the Network.URI module defined in old
versions of network (if both packages are included, GHC won't consider
the import ambiguous.)

If you wanted to do things completely without breaking BC, this would
suggest making a new package network-without-uri (but obviously a better
name), and users who don't use Network.URI can switch their
build-depends against that. network also defines Network.URI, but now it
reexports modules from network-without-uri.

Cheers,
Edward

Excerpts from Michael Snoyman's message of 2014-08-01 12:31:13 +0100:
> This was brought up last year[1], and I'd like to bring it up again, based
> on a recent issue I was working through with a user[2]. I realize that this
> is a breaking change, but:
>
> 1. It's a minor breaking change: you simply need to add an extra package to
> your build-depends.
> 2. The problems caused by having a parsec dependency in network can be
> severe, especially for new users (I'll describe the details after the
> proposal).
>
> Concretely, I believe we should do the following:
>
> 1. Create a new package, network-uri, version 2.5.0.0, which exposes no
> modules and has an upper bound `network < 2.6.
> 2. Create a second release of network-uri, version 3.0.0.0, which provides
> the Network.URI module verbatim as provided by the network package today,
> and has a lower bound `network >= 3.0`.
> 3. Release network version 3.0.0.0, with no changes from the currently
> released version, except that (a) no Network.URI module is provided, and
> (b) there is no parsec dependency.
>
> I don't remember how the discussion went last time, but I seem to remember
> general consensus. I'd like to set a discussion period of two weeks (August
> 15).
>
> ## Motivation
>
> To give a concrete example of why this problem is severe, consider the
> following data:
>
> * The network package is a pain to install on Windows for most users
> (especially new users), since it requires msys.
> * Most Windows users therefore install the Haskell Platform, avoiding the
> msys dependency.
> * The current release of HP installs text version 0.11.3.1. text is a
> dependency of parsec, and parsec is a dependency of network. Therefore, you
> can't build against a new version of text *and* use the network package
> without recompiling network, which as I mentioned is difficult.
> * A number of popular packages depend on newer versions of text. For
> example, since 0.8.0.0, aeson requires text version 1.1.0.0 or later. as
> does attoparsec since version 0.12.0.0.
>
> [1] http://www.haskell.org/pipermail/libraries/2013-January/019234.html
> [2] https://groups.google.com/d/msg/yesodweb/auk2vByXgO8/lUZ9oanKyMwJ


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Edward Kmett-2
In reply to this post by Johan Tibell-2
The problem with a flag based solution is it puts extra pressure on the backtracker in cabal, and when you pile enough of those up then you get things where cabal refuses to find a solution. Not for you, but for packages 20 dependencies later in the chain.

What I do with transformers-compat is I have multiple minor versions of the same package extant to work around bugs in the backtracker and then have separate point release that can work with transformers 0.2, one for transformers 0.3 and one for transformers 0.4, then users can depend on both transformers >= 0.2 and transformers-compat >= 0.3.3  and their code 'just works' regardless of which package is supplying the module.






On Wed, Aug 6, 2014 at 6:09 AM, Johan Tibell <[hidden email]> wrote:
On Mon, Aug 4, 2014 at 6:21 PM, Michael Snoyman <[hidden email]> wrote:
> The idea is that users of Network.URI will be able to add a dependency along
> the lines of:
>
>     build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7
>
> and work with both the pre- and post-split versions of the package, without
> any conditional compilation or cabal flags. I'm sensitive to adding any
> requirements of cabal flags, since (1) it clutters cabal files quite a
> bit[1], and (2) we just went through some painful cabal-install dependency
> solver issues around flags.

I agree that using flags to encode OR is a bit heavy weight. We should really support || in build-depends. For completeness, here's what a flag-based solution looks like. Assuming that network-uri-2.6 has the URI module and network-2.6 no longer has it we get this flag setup:

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6
  else
    build-depends: network < 2.6


If the user wants something else from network (e.g. Network.Socket) it would be

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6, network >= 2.6
  else
    build-depends: network < 2.6

Lets see if I understood your scheme correctly. We'd have

 * network-2.5: Has URI module.
 * network-uri-2.5: Doesn't have URI module. Depends on < network-2.6 (i.e. network-2.5 in this example).
 * network-2.6: Doesn't have URI module.
 * network-uri-2.6: Has URI module. Depends on network >= 2.6 (i.e. network 2.6 in this example).

Given


build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

legal combinations of the above packages are

 * network-2.5 and network-uri-2.5, which gives URI through network, and
 * network-2.6 and network-uri-2.6, which gives URI through network-uri.

There are two implications of this, which are slightly strange:

 * There's no point in anyone depending on only network-uri-2.5 (as it doesn't expose anything).
 * network-uri-2.6 and later will have to have a lower dependency on network forever, even though network-uri doesn't use anything from network (this is reflected when you discuss the upper bound in the context of the PVP later).

I don't know if there's any other implications. Can anyone think of any?

-- Johan


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries



_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Johan Tibell-2
I don't think (but Andres can say more) that a simple non-nested flag should give the backtracker problems (bugs or increase the search space so much that it won't find a solution.)


On Sun, Aug 10, 2014 at 3:22 AM, Edward Kmett <[hidden email]> wrote:
The problem with a flag based solution is it puts extra pressure on the backtracker in cabal, and when you pile enough of those up then you get things where cabal refuses to find a solution. Not for you, but for packages 20 dependencies later in the chain.

What I do with transformers-compat is I have multiple minor versions of the same package extant to work around bugs in the backtracker and then have separate point release that can work with transformers 0.2, one for transformers 0.3 and one for transformers 0.4, then users can depend on both transformers >= 0.2 and transformers-compat >= 0.3.3  and their code 'just works' regardless of which package is supplying the module.






On Wed, Aug 6, 2014 at 6:09 AM, Johan Tibell <[hidden email]> wrote:
On Mon, Aug 4, 2014 at 6:21 PM, Michael Snoyman <[hidden email]> wrote:
> The idea is that users of Network.URI will be able to add a dependency along
> the lines of:
>
>     build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7
>
> and work with both the pre- and post-split versions of the package, without
> any conditional compilation or cabal flags. I'm sensitive to adding any
> requirements of cabal flags, since (1) it clutters cabal files quite a
> bit[1], and (2) we just went through some painful cabal-install dependency
> solver issues around flags.

I agree that using flags to encode OR is a bit heavy weight. We should really support || in build-depends. For completeness, here's what a flag-based solution looks like. Assuming that network-uri-2.6 has the URI module and network-2.6 no longer has it we get this flag setup:

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6
  else
    build-depends: network < 2.6


If the user wants something else from network (e.g. Network.Socket) it would be

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6, network >= 2.6
  else
    build-depends: network < 2.6

Lets see if I understood your scheme correctly. We'd have

 * network-2.5: Has URI module.
 * network-uri-2.5: Doesn't have URI module. Depends on < network-2.6 (i.e. network-2.5 in this example).
 * network-2.6: Doesn't have URI module.
 * network-uri-2.6: Has URI module. Depends on network >= 2.6 (i.e. network 2.6 in this example).

Given


build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

legal combinations of the above packages are

 * network-2.5 and network-uri-2.5, which gives URI through network, and
 * network-2.6 and network-uri-2.6, which gives URI through network-uri.

There are two implications of this, which are slightly strange:

 * There's no point in anyone depending on only network-uri-2.5 (as it doesn't expose anything).
 * network-uri-2.6 and later will have to have a lower dependency on network forever, even though network-uri doesn't use anything from network (this is reflected when you discuss the upper bound in the context of the PVP later).

I don't know if there's any other implications. Can anyone think of any?

-- Johan


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries




_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Edward Kmett-2
The problem is more that there seem to be now packages that are within a factor of 2 of the total amount of backtracking they can do, so it seems pretty much every flag way down at the bottom of the package hierarchy pushes something out past the limit.

I hear something from someone every time I add a flag. Take that for what it's worth.

-Edward


On Sun, Aug 10, 2014 at 5:48 AM, Johan Tibell <[hidden email]> wrote:
I don't think (but Andres can say more) that a simple non-nested flag should give the backtracker problems (bugs or increase the search space so much that it won't find a solution.)


On Sun, Aug 10, 2014 at 3:22 AM, Edward Kmett <[hidden email]> wrote:
The problem with a flag based solution is it puts extra pressure on the backtracker in cabal, and when you pile enough of those up then you get things where cabal refuses to find a solution. Not for you, but for packages 20 dependencies later in the chain.

What I do with transformers-compat is I have multiple minor versions of the same package extant to work around bugs in the backtracker and then have separate point release that can work with transformers 0.2, one for transformers 0.3 and one for transformers 0.4, then users can depend on both transformers >= 0.2 and transformers-compat >= 0.3.3  and their code 'just works' regardless of which package is supplying the module.






On Wed, Aug 6, 2014 at 6:09 AM, Johan Tibell <[hidden email]> wrote:
On Mon, Aug 4, 2014 at 6:21 PM, Michael Snoyman <[hidden email]> wrote:
> The idea is that users of Network.URI will be able to add a dependency along
> the lines of:
>
>     build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7
>
> and work with both the pre- and post-split versions of the package, without
> any conditional compilation or cabal flags. I'm sensitive to adding any
> requirements of cabal flags, since (1) it clutters cabal files quite a
> bit[1], and (2) we just went through some painful cabal-install dependency
> solver issues around flags.

I agree that using flags to encode OR is a bit heavy weight. We should really support || in build-depends. For completeness, here's what a flag-based solution looks like. Assuming that network-uri-2.6 has the URI module and network-2.6 no longer has it we get this flag setup:

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6
  else
    build-depends: network < 2.6


If the user wants something else from network (e.g. Network.Socket) it would be

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6, network >= 2.6
  else
    build-depends: network < 2.6

Lets see if I understood your scheme correctly. We'd have

 * network-2.5: Has URI module.
 * network-uri-2.5: Doesn't have URI module. Depends on < network-2.6 (i.e. network-2.5 in this example).
 * network-2.6: Doesn't have URI module.
 * network-uri-2.6: Has URI module. Depends on network >= 2.6 (i.e. network 2.6 in this example).

Given


build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

legal combinations of the above packages are

 * network-2.5 and network-uri-2.5, which gives URI through network, and
 * network-2.6 and network-uri-2.6, which gives URI through network-uri.

There are two implications of this, which are slightly strange:

 * There's no point in anyone depending on only network-uri-2.5 (as it doesn't expose anything).
 * network-uri-2.6 and later will have to have a lower dependency on network forever, even though network-uri doesn't use anything from network (this is reflected when you discuss the upper bound in the context of the PVP later).

I don't know if there's any other implications. Can anyone think of any?

-- Johan


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries





_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Johan Tibell-2
Is this still the case with cabal-install 1.20.0.3/1.18.0.5? We recently bumped the max backtracking limit by about 10x. My current understanding of the state of the solver is that as of the cabal-install versions I gave

 * there are no known bugs that cause the solver to find a plan if there is one (if the backtracking limit isn't reached) and
 * the backtracking limit is high enough for the packages we currently have on Hackage.

If this isn't true please let me know and we'll look into it.

If there's a problem I want to fix it at its source (i.e. in the solver) rather than trying too many workarounds in .cabal files. The latter leave residues that we have to live with for quite a long time. I'm happy to backport any solver fixes if it means we can have cleaner .cabal files.


On Sun, Aug 10, 2014 at 4:30 PM, Edward Kmett <[hidden email]> wrote:
The problem is more that there seem to be now packages that are within a factor of 2 of the total amount of backtracking they can do, so it seems pretty much every flag way down at the bottom of the package hierarchy pushes something out past the limit.

I hear something from someone every time I add a flag. Take that for what it's worth.

-Edward


On Sun, Aug 10, 2014 at 5:48 AM, Johan Tibell <[hidden email]> wrote:
I don't think (but Andres can say more) that a simple non-nested flag should give the backtracker problems (bugs or increase the search space so much that it won't find a solution.)


On Sun, Aug 10, 2014 at 3:22 AM, Edward Kmett <[hidden email]> wrote:
The problem with a flag based solution is it puts extra pressure on the backtracker in cabal, and when you pile enough of those up then you get things where cabal refuses to find a solution. Not for you, but for packages 20 dependencies later in the chain.

What I do with transformers-compat is I have multiple minor versions of the same package extant to work around bugs in the backtracker and then have separate point release that can work with transformers 0.2, one for transformers 0.3 and one for transformers 0.4, then users can depend on both transformers >= 0.2 and transformers-compat >= 0.3.3  and their code 'just works' regardless of which package is supplying the module.






On Wed, Aug 6, 2014 at 6:09 AM, Johan Tibell <[hidden email]> wrote:
On Mon, Aug 4, 2014 at 6:21 PM, Michael Snoyman <[hidden email]> wrote:
> The idea is that users of Network.URI will be able to add a dependency along
> the lines of:
>
>     build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7
>
> and work with both the pre- and post-split versions of the package, without
> any conditional compilation or cabal flags. I'm sensitive to adding any
> requirements of cabal flags, since (1) it clutters cabal files quite a
> bit[1], and (2) we just went through some painful cabal-install dependency
> solver issues around flags.

I agree that using flags to encode OR is a bit heavy weight. We should really support || in build-depends. For completeness, here's what a flag-based solution looks like. Assuming that network-uri-2.6 has the URI module and network-2.6 no longer has it we get this flag setup:

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6
  else
    build-depends: network < 2.6


If the user wants something else from network (e.g. Network.Socket) it would be

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6, network >= 2.6
  else
    build-depends: network < 2.6

Lets see if I understood your scheme correctly. We'd have

 * network-2.5: Has URI module.
 * network-uri-2.5: Doesn't have URI module. Depends on < network-2.6 (i.e. network-2.5 in this example).
 * network-2.6: Doesn't have URI module.
 * network-uri-2.6: Has URI module. Depends on network >= 2.6 (i.e. network 2.6 in this example).

Given


build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

legal combinations of the above packages are

 * network-2.5 and network-uri-2.5, which gives URI through network, and
 * network-2.6 and network-uri-2.6, which gives URI through network-uri.

There are two implications of this, which are slightly strange:

 * There's no point in anyone depending on only network-uri-2.5 (as it doesn't expose anything).
 * network-uri-2.6 and later will have to have a lower dependency on network forever, even though network-uri doesn't use anything from network (this is reflected when you discuss the upper bound in the context of the PVP later).

I don't know if there's any other implications. Can anyone think of any?

-- Johan


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries






_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Michael Snoyman
Just to be clear from my side: even if we were 100% certain that all issues we've experienced in the past from the solver with flags would not exist with the newest versions of cabal-install, I would still want to follow the same proposal as I gave at the beginning of this thread, because forcing every user of Network.URI to add conditionals to their cabal file is IMO more painful than simply having two versions of the network-uri package released.

The only modification to the proposal that I think would make sense would be to have a single version of network-uri, and the exposing of the Network.URI module be conditional on the version of network. But I'd rather stick with the original proposal than encounter possibly unknown solver bugs.


On Tue, Aug 12, 2014 at 3:44 PM, Johan Tibell <[hidden email]> wrote:
Is this still the case with cabal-install 1.20.0.3/1.18.0.5? We recently bumped the max backtracking limit by about 10x. My current understanding of the state of the solver is that as of the cabal-install versions I gave

 * there are no known bugs that cause the solver to find a plan if there is one (if the backtracking limit isn't reached) and
 * the backtracking limit is high enough for the packages we currently have on Hackage.

If this isn't true please let me know and we'll look into it.

If there's a problem I want to fix it at its source (i.e. in the solver) rather than trying too many workarounds in .cabal files. The latter leave residues that we have to live with for quite a long time. I'm happy to backport any solver fixes if it means we can have cleaner .cabal files.


On Sun, Aug 10, 2014 at 4:30 PM, Edward Kmett <[hidden email]> wrote:
The problem is more that there seem to be now packages that are within a factor of 2 of the total amount of backtracking they can do, so it seems pretty much every flag way down at the bottom of the package hierarchy pushes something out past the limit.

I hear something from someone every time I add a flag. Take that for what it's worth.

-Edward


On Sun, Aug 10, 2014 at 5:48 AM, Johan Tibell <[hidden email]> wrote:
I don't think (but Andres can say more) that a simple non-nested flag should give the backtracker problems (bugs or increase the search space so much that it won't find a solution.)


On Sun, Aug 10, 2014 at 3:22 AM, Edward Kmett <[hidden email]> wrote:
The problem with a flag based solution is it puts extra pressure on the backtracker in cabal, and when you pile enough of those up then you get things where cabal refuses to find a solution. Not for you, but for packages 20 dependencies later in the chain.

What I do with transformers-compat is I have multiple minor versions of the same package extant to work around bugs in the backtracker and then have separate point release that can work with transformers 0.2, one for transformers 0.3 and one for transformers 0.4, then users can depend on both transformers >= 0.2 and transformers-compat >= 0.3.3  and their code 'just works' regardless of which package is supplying the module.






On Wed, Aug 6, 2014 at 6:09 AM, Johan Tibell <[hidden email]> wrote:
On Mon, Aug 4, 2014 at 6:21 PM, Michael Snoyman <[hidden email]> wrote:
> The idea is that users of Network.URI will be able to add a dependency along
> the lines of:
>
>     build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7
>
> and work with both the pre- and post-split versions of the package, without
> any conditional compilation or cabal flags. I'm sensitive to adding any
> requirements of cabal flags, since (1) it clutters cabal files quite a
> bit[1], and (2) we just went through some painful cabal-install dependency
> solver issues around flags.

I agree that using flags to encode OR is a bit heavy weight. We should really support || in build-depends. For completeness, here's what a flag-based solution looks like. Assuming that network-uri-2.6 has the URI module and network-2.6 no longer has it we get this flag setup:

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6
  else
    build-depends: network < 2.6


If the user wants something else from network (e.g. Network.Socket) it would be

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6, network >= 2.6
  else
    build-depends: network < 2.6

Lets see if I understood your scheme correctly. We'd have

 * network-2.5: Has URI module.
 * network-uri-2.5: Doesn't have URI module. Depends on < network-2.6 (i.e. network-2.5 in this example).
 * network-2.6: Doesn't have URI module.
 * network-uri-2.6: Has URI module. Depends on network >= 2.6 (i.e. network 2.6 in this example).

Given


build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

legal combinations of the above packages are

 * network-2.5 and network-uri-2.5, which gives URI through network, and
 * network-2.6 and network-uri-2.6, which gives URI through network-uri.

There are two implications of this, which are slightly strange:

 * There's no point in anyone depending on only network-uri-2.5 (as it doesn't expose anything).
 * network-uri-2.6 and later will have to have a lower dependency on network forever, even though network-uri doesn't use anything from network (this is reflected when you discuss the upper bound in the context of the PVP later).

I don't know if there's any other implications. Can anyone think of any?

-- Johan


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries







_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Johan Tibell-2
I asked Duncan for some feedback, so here's a summary for his benefit:

First Michael's (and our) problem statement:
  • The network package is a pain to install on Windows for most users (especially new users), since it requires msys.
  • Most Windows users therefore install the Haskell Platform, avoiding the msys dependency.
  • The current release of HP installs text version 0.11.3.1. text is a dependency of parsec, and parsec is a dependency of network. Therefore, you can't build against a new version of text *and* use the network package without recompiling network, which as I mentioned is difficult.
  • A number of popular packages depend on newer versions of text. For example, since 0.8.0.0, aeson requires text version 1.1.0.0 or later. as does attoparsec since version 0.12.0.0.
The two options:

Option 1: Users of Network.URI use a flag
  • network-2.6 no longer exposes Network.URI.
  • network-uri-2.6 exposes Network.URI.
User who want only network-uri writes:

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6
  else
    build-depends: network < 2.6

(If the user also wants network he/she adds it to the first build-depends line.)

Cons:
  • More typing (an extra flag declaration, an if statement, and an extra build-depends line).
  • Could trigger possible unknown solver bugs (but there are no confirmed such bugs).
Options 2: Users of Network.URI depend on both network and a specially crafted network-uri
  • network-2.5 exposes Network.URI.
  • network-2.6 no longer exposes Network.URI.
  • network-uri-2.5 Exposes no modules. Depends on < network-2.6.
  • network-uri-2.6 exposes Network.URI. Depends on network >= 2.6.
User who want network-uri (only) writes:

build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

Cons:
  • network-uri has a false dependency on network (i.e. it doesn't actually need that package).
    • You can't build against a new version of text *and* use the network-uri package (this is the current problem we have with network in the problem description).
  • We need to teach users about the above special build-depends construction, just for use with the network and network-uri packages. Users who don't know of this special construction will still use a flag (and thus get a mix of cons from both approaches.)
  • network-uri-2.5 is an empty package, which might lead to slight confusion for some users.
Cheers,
  Johan


On Tue, Aug 12, 2014 at 2:59 PM, Michael Snoyman <[hidden email]> wrote:
Just to be clear from my side: even if we were 100% certain that all issues we've experienced in the past from the solver with flags would not exist with the newest versions of cabal-install, I would still want to follow the same proposal as I gave at the beginning of this thread, because forcing every user of Network.URI to add conditionals to their cabal file is IMO more painful than simply having two versions of the network-uri package released.

The only modification to the proposal that I think would make sense would be to have a single version of network-uri, and the exposing of the Network.URI module be conditional on the version of network. But I'd rather stick with the original proposal than encounter possibly unknown solver bugs.


On Tue, Aug 12, 2014 at 3:44 PM, Johan Tibell <[hidden email]> wrote:
Is this still the case with cabal-install 1.20.0.3/1.18.0.5? We recently bumped the max backtracking limit by about 10x. My current understanding of the state of the solver is that as of the cabal-install versions I gave

 * there are no known bugs that cause the solver to find a plan if there is one (if the backtracking limit isn't reached) and
 * the backtracking limit is high enough for the packages we currently have on Hackage.

If this isn't true please let me know and we'll look into it.

If there's a problem I want to fix it at its source (i.e. in the solver) rather than trying too many workarounds in .cabal files. The latter leave residues that we have to live with for quite a long time. I'm happy to backport any solver fixes if it means we can have cleaner .cabal files.


On Sun, Aug 10, 2014 at 4:30 PM, Edward Kmett <[hidden email]> wrote:
The problem is more that there seem to be now packages that are within a factor of 2 of the total amount of backtracking they can do, so it seems pretty much every flag way down at the bottom of the package hierarchy pushes something out past the limit.

I hear something from someone every time I add a flag. Take that for what it's worth.

-Edward


On Sun, Aug 10, 2014 at 5:48 AM, Johan Tibell <[hidden email]> wrote:
I don't think (but Andres can say more) that a simple non-nested flag should give the backtracker problems (bugs or increase the search space so much that it won't find a solution.)


On Sun, Aug 10, 2014 at 3:22 AM, Edward Kmett <[hidden email]> wrote:
The problem with a flag based solution is it puts extra pressure on the backtracker in cabal, and when you pile enough of those up then you get things where cabal refuses to find a solution. Not for you, but for packages 20 dependencies later in the chain.

What I do with transformers-compat is I have multiple minor versions of the same package extant to work around bugs in the backtracker and then have separate point release that can work with transformers 0.2, one for transformers 0.3 and one for transformers 0.4, then users can depend on both transformers >= 0.2 and transformers-compat >= 0.3.3  and their code 'just works' regardless of which package is supplying the module.






On Wed, Aug 6, 2014 at 6:09 AM, Johan Tibell <[hidden email]> wrote:
On Mon, Aug 4, 2014 at 6:21 PM, Michael Snoyman <[hidden email]> wrote:
> The idea is that users of Network.URI will be able to add a dependency along
> the lines of:
>
>     build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7
>
> and work with both the pre- and post-split versions of the package, without
> any conditional compilation or cabal flags. I'm sensitive to adding any
> requirements of cabal flags, since (1) it clutters cabal files quite a
> bit[1], and (2) we just went through some painful cabal-install dependency
> solver issues around flags.

I agree that using flags to encode OR is a bit heavy weight. We should really support || in build-depends. For completeness, here's what a flag-based solution looks like. Assuming that network-uri-2.6 has the URI module and network-2.6 no longer has it we get this flag setup:

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6
  else
    build-depends: network < 2.6


If the user wants something else from network (e.g. Network.Socket) it would be

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6, network >= 2.6
  else
    build-depends: network < 2.6

Lets see if I understood your scheme correctly. We'd have

 * network-2.5: Has URI module.
 * network-uri-2.5: Doesn't have URI module. Depends on < network-2.6 (i.e. network-2.5 in this example).
 * network-2.6: Doesn't have URI module.
 * network-uri-2.6: Has URI module. Depends on network >= 2.6 (i.e. network 2.6 in this example).

Given


build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

legal combinations of the above packages are

 * network-2.5 and network-uri-2.5, which gives URI through network, and
 * network-2.6 and network-uri-2.6, which gives URI through network-uri.

There are two implications of this, which are slightly strange:

 * There's no point in anyone depending on only network-uri-2.5 (as it doesn't expose anything).
 * network-uri-2.6 and later will have to have a lower dependency on network forever, even though network-uri doesn't use anything from network (this is reflected when you discuss the upper bound in the context of the PVP later).

I don't know if there's any other implications. Can anyone think of any?

-- Johan


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries








_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Michael Snoyman



On Wed, Aug 13, 2014 at 12:28 PM, Johan Tibell <[hidden email]> wrote:
I asked Duncan for some feedback, so here's a summary for his benefit:

First Michael's (and our) problem statement:
  • The network package is a pain to install on Windows for most users (especially new users), since it requires msys.
  • Most Windows users therefore install the Haskell Platform, avoiding the msys dependency.
  • The current release of HP installs text version 0.11.3.1. text is a dependency of parsec, and parsec is a dependency of network. Therefore, you can't build against a new version of text *and* use the network package without recompiling network, which as I mentioned is difficult.
  • A number of popular packages depend on newer versions of text. For example, since 0.8.0.0, aeson requires text version 1.1.0.0 or later. as does attoparsec since version 0.12.0.0.
The two options:

Option 1: Users of Network.URI use a flag
  • network-2.6 no longer exposes Network.URI.
  • network-uri-2.6 exposes Network.URI.
User who want only network-uri writes:

flag network-uri
  description: Get Network.URI from the network-uri package
  default: True

library
  if flag(network-uri)
    build-depends: network-uri >= 2.6
  else
    build-depends: network < 2.6

(If the user also wants network he/she adds it to the first build-depends line.)

Cons:
  • More typing (an extra flag declaration, an if statement, and an extra build-depends line).
  • Could trigger possible unknown solver bugs (but there are no confirmed such bugs).

I'd add in a very big "users will very likely not know the right way to put in the build-depends correctly." This contrasts with your second con under option 2. I think it's highly unlikely that users will think that using a flag to indicate a conditional dependency is the obvious solution. Instead, I'd imagine we'd end up in the situation where users put in a dependency on both network and network-uri, it works on their systems, and then we get lots of downstream breakage when someone with an older version of network tries to build the package.
 
Options 2: Users of Network.URI depend on both network and a specially crafted network-uri
  • network-2.5 exposes Network.URI.
  • network-2.6 no longer exposes Network.URI.
  • network-uri-2.5 Exposes no modules. Depends on < network-2.6.
  • network-uri-2.6 exposes Network.URI. Depends on network >= 2.6.
User who want network-uri (only) writes:

build-depends: network >= 2.5 && < 2.7, network-uri >= 2.5 && < 2.7

 
Cons:
  • network-uri has a false dependency on network (i.e. it doesn't actually need that package).
    • You can't build against a new version of text *and* use the network-uri package (this is the current problem we have with network in the problem description).
Can you clarify this point? I would imagine that network will no longer have *any* dependency on text, so I don't see where this comes from.
  • We need to teach users about the above special build-depends construction, just for use with the network and network-uri packages. Users who don't know of this special construction will still use a flag (and thus get a mix of cons from both approaches.)
I think this is a bigger con with option 1. Teaching users about a strange flag conditional is far more difficult IMO than saying "just depend on both network and network-uri." Yes, both approaches have the potential create downstream breakage, but I think this one is easier to educate about.
 
  • network-uri-2.5 is an empty package, which might lead to slight confusion for some users.

I'd consider that a *very* minor point. network-uri-2.5 would simply have a note in its description field explaining why it exists, likely with a link to this thread.

Michael

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Network.URI from the network package

Duncan Coutts-4
In reply to this post by Johan Tibell-2
So it's true that using flags here is the standard solution (e.g. as
was used with the base split for a few years), so many (but not all)
users know this already.

I should also note (Andres: please correct me if I'm wrong) that as of
cabal-install 1.20, the cost in terms of backtracking for flags like
these is now quite small because choosing them is deferred as late as
possible. Edward: this should solve your problem with flags with
transformers, at least once 1.20 is widely used. It'd be great if you
could check that this is the case.

So overall it looks like both options work ok, and that there are not
very strong reasons either way.

If we pick option 2, it is inevitable that some people will use option
1 anyway, but that's probably ok. We think option 1 does not cause
problems for the solver (at least with cabal-install 1.20+, and works
with earlier but increases backtracking pressure).

Option 2 should solve the stated problem so long as users have
network-2.6 pre-installed from the HP (to avoid difficulties with
building it on windows). The false dependency in option 2 is a bit
unfortunate because it means you pull in network even if you only need
network-uri. Obviously in this specific case, that's not a big
practical problem because both should be pre-installed for HP users.
But it does point out that option 2 is not a general solution to
package splits.

Overall, on balance I think I'd go with option 1. Option 1 has the
benefit that it is our current standard solution to these problems
that at least some people know about. The main downside with option 1
is the rather verbose and non-intuitive syntax. We can focus our
development efforts on extending the dependency syntax to give us a
nicer general solution to package splits and merges. Option 2 while
superficially simpler would not help us longer term with educating
users on how to cope with these situations, since it's not a solution
that will work in general (the false dep will bite in other
situations). User education for option 2 would only help them with
this specific situation and not with other similar package splits.

So I think I'd recommend we continue to tell users about the current
(slightly ugly) standard solution, and work on adding better syntax.
An advantage of that approach is that we could have `cabal check`
suggest the use of the new syntax when it sees the existing (ugly)
pattern.

For reference, a bit more detail on the problem with the false dep in option 2:

> network-2.5 exposes Network.URI.
> network-2.6 no longer exposes Network.URI.
> network-uri-2.5 Exposes no modules. Depends on < network-2.6.
> network-uri-2.6 exposes Network.URI. Depends on network >= 2.6.

The problem is with network-uri-2.6 depending on network >= 2.6. This
is supposed to be the "new" state that we'd be in from then on, but
we're left with network-uri-2.6 depending on network, even though it
doesn't use it.

That dependency needs to be there because network-uri-2.6 conflicts
with network-2.5.

As I said, in this specific case it's not a major problem, but it
indicates we cannot take this approach in general for package splits
because it does not let us split to reduce dependencies (which is the
typical reason to want to split).

Thanks to Johan and Michael, Edward and others for setting out the
problem and a thoughtful discussion.

Duncan
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
123