Do something about Cabal?
Locked
12
12
Re: Do something about Cabal?
 
|
I always wondered: do we need a package management system at all? Putting aside for a second the problem of interfacing with C or other external dependencies. What about a system where every source file is automatically annotated with a unique hash and all the required LANGUAGE extensions. For example: -- State explicitly any non-standard language extension {-# LANGUAGE MultiParamTypeClasses ,PatternSynonyms ,RankNTypes #-} -- Add to the module name its hash (calculated on its source contents, ignoring comments) module App.Main.K306f1981b41c -- Import a module, specifying the exact version import Data.Maybe.K306f1981823d .... main: ... Finally, we publish it on a shared content addressable system (hackage like). This module is in fact now a stand-alone mini-package that can be easily built as it states all its dependencies in a non-ambiguous way: lookMaNoPackages run App.Main.K306f1981b41c The nice thing is that it could be done in a completely backwards-compatible way, as it doesn't require any change to the language itself. And a modern tool like haskell-language-server could automate the hashing/annotations making it convenient to use. Would this work? titto Il giorno ven 11 dic 2020 alle ore 12:40 Jack Kelly via Haskell-Cafe <[hidden email]> ha scritto:
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
|
|
In reply to this post by Immanuel Litzroth-2
On Fri, Dec 11, 2020 at 10:31:23AM +0100, Immanuel Litzroth wrote:
> I found the whole cabal experience confusing and not well documented. > I kept finding blogs online > that were not working anymore in my version of cabal If you find outdated Haskell-related documentation please report it at https://github.com/tomjaguarpaw/tilapia/issues/new and I shall try to do something about it. Tom _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
|
|
On Fri, Dec 11, 2020 at 01:54:57PM +0000, Tom Ellis wrote:
> On Fri, Dec 11, 2020 at 10:31:23AM +0100, Immanuel Litzroth wrote: > > I found the whole cabal experience confusing and not well documented. > > I kept finding blogs online > > that were not working anymore in my version of cabal > > If you find outdated Haskell-related documentation please report it at > > https://github.com/tomjaguarpaw/tilapia/issues/new And on that point, I am tracking your particular issue with https://www.haskell.org/cabal/ as https://github.com/tomjaguarpaw/tilapia/issues/44 Tom _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: [HF-discuss] Do something about Cabal?
|
|
In reply to this post by Bardur Arantsson-2
Am 11.12.20 um 10:50 schrieb Bardur Arantsson:
> FWIW, I do try to do this for my packages, but I'm very confident that > all of my packages probably have misleading lower bounds. > > It seems to me that some automated tooling is needed here, e.g. to try > building/testing with e.g. all deps at the lower bound, all deps at the > highest bound, etc. With indirect dependencies, you can run into combinatorial explosion problems (number of combinations is roughly exponential with dependency chain length, give or take a few orders of magnitude). Just something to keep in mind when discussing design alternatives. (I don't know what designs would even make sense, or what kinds of metadata would be available to help guide version selection.) Regards, Jo _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: [HF-discuss] Do something about Cabal?
|
|
On 11/12/2020 16.24, Joachim Durchholz wrote:
> Am 11.12.20 um 10:50 schrieb Bardur Arantsson: >> FWIW, I do try to do this for my packages, but I'm very confident that >> all of my packages probably have misleading lower bounds. >> >> It seems to me that some automated tooling is needed here, e.g. to try >> building/testing with e.g. all deps at the lower bound, all deps at the >> highest bound, etc. > > With indirect dependencies, you can run into combinatorial explosion > problems (number of combinations is roughly exponential with dependency > chain length, give or take a few orders of magnitude). > > Just something to keep in mind when discussing design alternatives. > (I don't know what designs would even make sense, or what kinds of > metadata would be available to help guide version selection.) > Indeed, it's a difficult problem with no obvious solution, but given the combinatorial explosion issue, it's probably better to (somehow) leave it up to a computer to explore that space rather than humans. :) Regards, _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: Do something about Cabal?
|
|
In reply to this post by Pasqualino "Titto" Assini
Am 11.12.20 um 13:48 schrieb Pasqualino "Titto" Assini:
> I always wondered: do we need a package management system at all? The main reason package management became a thing is indirect dependencies. Without package management: Pick up a dependency, manually hunt down everything that it depends on, then get the dependencies' dependencies, and you never know how many are there to come. Lather, rinse, repeat with every update. Sometimes you'll need a completely new set of indirect dependencies after an update. With package management, you specify the package you need and the package manager will do the rest. It should also tell you what it pulled in. If it's worth anything, you can tell it to use different dependencies; it's not a silver bullet that will solve your dependency incompatibilities for you, but it will give you a reasonable first approximation. > What about a system where every source file is automatically annotated > with a unique hash and all the required LANGUAGE extensions. Hashes are... difficult. What you really want is a hash of the API you're using, possibly including nonfunctional properties. Problem is that it's really hard to even define the section of the API that you need - probably an unsolved problem, actually. That's why people find semantic versioning so important. Usually, you have some global namespace where legal entities reside (Java did something that turned out to be pretty smart: domain names are per-entity anyway), with each entity assigning module names within their namespace. E.g. the owner of com.company manages com.company.module1, com.company.module2, etc. A module is identified by (sub)domain name and version number, throw in some recommendations about how to apply semantic versioning, and people will stop using those modules that don't have a good versioning policy anyway... and you're done. It's not perfect, but it works. (Hashes are overspecific, and they don't help with the relationship between version 1.2 and 1.2.2 of a module.) (Disclaimer: This is from the perspective of the Java ecosystem, slightly generalized. Semantic versioning may be less of an issue for functional languages than it has been and will be for imperative ones, so I don't expect this to carry over 1:1.) _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: [HF-discuss] Do something about Cabal?
|
|
In reply to this post by Immanuel Litzroth-2
The correct URL to the up-to-date cabal user guide, since a couple of
years ago or more, is: cabal.readthedocs.io In principle, someone could and should make a copy at www.haskell.org/cabal, but I don't know why it hasn't happened. Would be nice. cabal sandboxing is being phased out. Every blog on cabal sandboxing is out of date by definition. Don't bother. If your version of cabal-install is >= 3.0, when you issue "build" "run" etc they already mean v2-build, v2-run, etc. I refer you to the correct up-to-date cabal user guide, chapter 5 "nix-style local builds", for its model. I would not recommend spending time on the v1 model, unless you're a historian. "new-" was a transitional alias to "v2-" during a past transitional period that can be safely ignored (unless you're a historian). (Historians will remind me that the plan was, one day in the future, when the cabal devs introduce a 3rd model, the "new-" prefix will be revived again, this time aliasing to the 3rd model. I say that people are so happy with the current 2nd model that it won't happen.) On 2020-12-11 4:31 a.m., Immanuel Litzroth wrote: > I found the whole cabal experience confusing and not well documented. > I kept finding blogs online > that were not working anymore in my version of cabal ('cabal sandbox > init', 'cabal init --sandbox'...) when looking > for advice. I still don't understand the whole v1-build, v2-build, > new-build and build... > The manual doesn't seem to have a decent conceptual overview of what > the tool should do (e.g. an explanation > of what sandboxing is supposed to achieve, or the 4 build command versions). > Several of the links on this page https://www.haskell.org/cabal/ are dead > https://www.haskell.org/cabal/release/cabal-latest/doc/API/Cabal/ > or refer to old versions: > https://wiki.haskell.org/Upgrading_packages > > This should not be construed as a critique of what has been achieved, > but as honest feedback > of my experience. >> Innovation and advancing comes after competition, why shouldn't we embrace that wrt the tooling of Haskell? Duplication in efforts is a relative smaller problem IMHO. > Well the competition has "go build" and cargo... > And duplication of effort is a problem when there's not enough > resources for even one decent build tool > Immanuel > _______________________________________________ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. > Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: [HF-discuss] Do something about Cabal?
|
|
In reply to this post by Bardur Arantsson-2
On 12/11/20 10:52 AM, Bardur Arantsson wrote:
> > Indeed, it's a difficult problem with no obvious solution, but given the > combinatorial explosion issue, it's probably better to (somehow) leave > it up to a computer to explore that space rather than humans. :) > I'm reminded of the headline from this Onion article: https://www.theonion.com/no-way-to-prevent-this-says-only-nation-where-this-r-1819576527 To fix dependency hell, you need only not cause it in the first place. You have to standardize the language and a useful bit of code that you dub the "standard library." Then anyone writing serious software in your language has to be careful not to break API compatibility, and to use semantic/libtool versioning when doing so is unavoidable. That's an O(n = number of packages) algorithm for detecting breakage because instead of detecting it in every consumer, you're _indicating_ it once. This is what separates languages that a few people use to write software for themselves from the languages that a lot of people use to write software for others. If you don't cause yourself these problems in the first place, then you don't need a not-invented-here package manager, because you can pick up Nix or Gentoo prefix or Anaconda or whatever and simply plug it in. The same is true of build systems. On the other hand, if you want to do something that you shouldn't be doing in the first place (living in Antarctica, climbing Mt. Everest, using a language that breaks itself on Tuesdays, etc.) then specialized tools are often necessary to bring the level of difficulty down from "impossible" to merely "really stupid." _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: [HF-discuss] Do something about Cabal?
|
|
On Fri, 11 Dec 2020, Michael Orlitzky wrote: > To fix dependency hell, you need only not cause it in the first place. > You have to standardize the language and a useful bit of code that you > dub the "standard library." Then anyone writing serious software in your > language has to be careful not to break API compatibility, and to use > semantic/libtool versioning when doing so is unavoidable. That's an O(n > = number of packages) algorithm for detecting breakage because instead > of detecting it in every consumer, you're _indicating_ it once. Isn't this, what we are doing currently? When developing a library we try to stay within a major version range as long as possible, but if we need a breaking change, we increase the major version and have to adapt depending packages. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: [HF-discuss] Do something about Cabal?
|
|
On 12/11/20 2:48 PM, Henning Thielemann wrote:
> > On Fri, 11 Dec 2020, Michael Orlitzky wrote: > >> To fix dependency hell, you need only not cause it in the first place. >> You have to standardize the language and a useful bit of code that you >> dub the "standard library." Then anyone writing serious software in your >> language has to be careful not to break API compatibility, and to use >> semantic/libtool versioning when doing so is unavoidable. That's an O(n >> = number of packages) algorithm for detecting breakage because instead >> of detecting it in every consumer, you're _indicating_ it once. > > Isn't this, what we are doing currently? When developing a library we try > to stay within a major version range as long as possible, but if we need a > breaking change, we increase the major version and have to adapt depending > packages. > Some Haskell packages are pretty good in that regard, but my list was in order of importance. There is no such thing as "Haskell" nowadays, only "what GHC is doing this week," and the same goes for the simulacrum of a standard library that we call "base". The fact that, say, Cryptonite, is maintained intelligently is not as helpful when you consider that the words the author has written can change their meanings at any time. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
|
|
Thank you. That's exactly how I feel lately, but I couldn't phrase with that level of precision.
> On 11 Dec 2020, at 21:10, Michael Orlitzky <[hidden email]> wrote: > > There is no such thing as "Haskell" nowadays, only "what GHC is doing this week," _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: Do something about Cabal?
|
|
In reply to this post by Joachim Durchholz
I don't actually see the problem in producing a hash of the API you use. It is simply the hash of the list of your imports. Is this overspecific? In the sense that it is more restrictive than it could be? Yes, but at least you get reproducibility and a permanently working build. Semantic versioning sounds good in theory and often works in practice, but you are relying on the goodwill and the precision of the library maintainer when both of these qualities are in short supply. The advantage of using module-level hashes is that you: * Avoid mistakes due to incorrect semantic versioning * Get maximum flexibility, as you are exploding the 'package cage' and accessing every file individually * Avoid a lot of 'false incompatibilities', when for example you would like to access a new version of a module in an upgraded package but you can't upgrade because you depend on another package that declares to be compatible only with the previous version of the package you want to upgrade to even when it's not actually using the new module you are interested in at all. The inconvenience of hashes is that they are cumbersome to manage by hand, but this can be automated. In fact, the question is not really semantic versioning vs hash-versioning. Both can be used at the module level as well as to the package level. The question is if we can simplify the dependency management problem by "going smaller", moving from a bunch of related modules (a package) as a unit of code exchange to a single module. In principle one could even go smaller, considering every function or data type [1] as an independently sharable entity. So, what is the optimal 'size' for code sharing: package, module, or entity? Best, titto [1] This actually make a lot of sense if you are interested in cross-language interoperability, see https://github.com/Quid2/zm for an example. Il giorno ven 11 dic 2020 alle ore 18:02 Joachim Durchholz <[hidden email]> ha scritto: Am 11.12.20 um 13:48 schrieb Pasqualino "Titto" Assini: _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: Do something about Cabal?
|
|
On Sat, 12 Dec 2020, Pasqualino "Titto" Assini wrote: > I don't actually see the problem in producing a hash of the API you use. > It is simply the hash of the list of your imports. > > Is this overspecific? In the sense that it is more restrictive than it could be? How do you address with a hash that added identifiers in an imported module do not harm if you import qualified or explicitly? How do you address changed type of an imported variable? _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: Do something about Cabal?
|
|
Il giorno sab 12 dic 2020 alle ore 13:20 Henning Thielemann <[hidden email]> ha scritto:
The hash is obviously calculated on a normalised version of the module. As part of this normalisation step, all references to external definitions are fully qualified. And it is impossible to import a variable with a changed type, because if the type had changed so would have its definition and therefore the hash of the imported module. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: Do something about Cabal?
|
|
> The hash is obviously calculated on a normalised version of the module.
> > As part of this normalisation step, all references to external > definitions are fully qualified. > > And it is impossible to import a variable with a changed type, > because if the type had changed so would have its definition and > therefore the hash of the imported module. The normalization is actually the important point here. It's easy to come up with something that works reasonably, for some definition of "reasonably". Normalization can be too narrow (throw away relevant information) or too wide (leave in irrelevant information). Problem is, it's the library user that defines what parts of a library are of interest to the use case. Or, more specifically: What parts of a library's *semantics*. Most users care only about specific properties of the library functions they use. E.g. for a function that returns a list, some care about order and some don't. Some stuff may not even be properly expressible, like side-channel data leakage in a crypto hash function, or legal constraints you don't want to care about but have to. More on the programming sides, type bounds may become tighter or less tight. And even a less tight type bound can cause trouble (because your code might have to deal with less tight type bounds on results, for example - or you might be in a situation where you want to allow your own callers to pass in looser-constrained data - or maybe you do not want that because your own code depends on these bounds). And sometimes the type bounds need to be explicit because type inference isn't good enough, so you can't fully leave that to automation. So... hashes are good for checking for equality, but you want to express inequalities, otherwise your clients will have to recheck the full API with every upgrade. (Sure, with Haskell, this is much less of a problem than usual. But it won't go away.) _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
Re: [HF-discuss] Do something about Cabal?
|
|
In reply to this post by Albert Y. C. Lai
December 12, 2020 4:37 AM, "Albert Y. C. Lai" <[hidden email]> wrote:
(Re: cabal-install commands) > I would not recommend spending time on the v1 model, unless you're a > historian. There is one edge case that I think still works better under cabal v1- commands: https://github.com/haskell/cabal/issues/6515 This is where you create a shared, read-only set of packages (e.g., in a computer lab image) so that students don't fill their $HOME with compiled packages. Other than that, the v2-commands are a major step forward. -- Jack _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
|
|
In reply to this post by Ignat Insarov
Am 10.12.20 um 12:23 schrieb Ignat Insarov:
> Cabal is the second most used tool in Haskell after GHC. It has many > problems. It may be noticed that there is one and a half developers working on > it. This is clearly not enough to address these problems. I propose that this is > a good place to invest in. Cabal has *dramatically* improved over the years and since version 3.2 is working very smoothly for me. I generally favour cabal over stack because I like the extra flexibility in chosing package versions that cabal offers. Stack is also a very fine tool and has a slightly more polished UI. > It is ordinary to receive output like this: > > ``` > % cabal run example-executable > Warning: The package list for 'hackage.haskell.org' is 84 days old. > Run 'cabal update' to get the latest list of available packages. > Resolving dependencies... > cabal: Could not resolve dependencies: > [__0] trying: example-0.1.0.6 (user goal) > [__1] next goal: opaleye (dependency of example) > [__1] rejecting: opaleye-0.7.1.0, opaleye-0.7.0.0 (constraint from project > config TODO requires ==0.6.7006.1) > [__1] rejecting: opaleye-0.6.7006.1 (conflict: example => opaleye^>=0.7) > [__1] skipping: opaleye-0.6.7006.0, opaleye-0.6.7005.0, opaleye-0.6.7004.2, > opaleye-0.6.7004.1, opaleye-0.6.7004.0, opaleye-0.6.7003.1, > opaleye-0.6.7003.0, opaleye-0.6.1.0, opaleye-0.6.0.0, opaleye-0.5.4.0, > opaleye-0.5.3.1, opaleye-0.5.3.0, opaleye-0.5.2.2, opaleye-0.5.2.0, > opaleye-0.5.1.1, opaleye-0.5.1.0, opaleye-0.5.0.0, opaleye-0.4.2.0, > opaleye-0.4.1.0, opaleye-0.4.0.0, opaleye-0.3.1.2, opaleye-0.3.1, opaleye-0.3, > opaleye-0.2, opaleye-0.6.7002.0, opaleye-0.6.7001.0, opaleye-0.6.7000.0, > opaleye-0.5.2.1, opaleye-0.3.1.1 (has the same characteristics that caused the > previous version to fail: excluded by constraint '^>=0.7' from example) > [__1] fail (backjumping, conflict set: example, opaleye) > After searching the rest of the dependency tree exhaustively, these were the > goals I've had most trouble fulfilling: opaleye, example > ``` > > There are so many things that are wrong here. The output is a bit verbose but the punch line at the end is pretty plain: > goals I've had most trouble fulfilling: opaleye, example It doesn't take a PHD to conclude that you'll have to experiment with (usually: relax) the version bounds for these two packages to find a valid build plan. And the detailed output, however intimidating at first, is acually helpful to find out what goes wrong. I don't know how much effort goes into stackage (curating packages into coherent sets) but my guess is: quite a bit. Without that extra effort stack wouldn't be as useful as it is. Cheers Ben _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. |
«
Return to Haskell - Haskell-Cafe
|
1 view|%1 views
| Free forum by Nabble | Edit this page |