Listing native package requirements based on cabal information

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Listing native package requirements based on cabal information

David Thomas
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Dan Burton
I have wished for this on multiple occasions. I don't believe such a thing exists as of yet.

-- Dan Burton


On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]> wrote:
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe



_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

David Thomas
Ok, well, if that's the case I'd like to see about remedying that.  Anyone have any thoughts as to how to best go about this?  I'm not clear on exactly what info lives where, especially across systems.  Entirely manual population would be a (barely) acceptable fallback.


On Tue, Mar 18, 2014 at 9:36 AM, Dan Burton <[hidden email]> wrote:
I have wished for this on multiple occasions. I don't believe such a thing exists as of yet.

-- Dan Burton


On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]> wrote:
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe




_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

mantkiew
Certainly NIX is an interesting approach. It already comes with a large base of dependencies, a format for specifying them. NIX can be installed in any Linux distro and serve as an environment for building packages. That might provide a cross-distribution solution to the native dependency problem.


Michal


On Tue, Mar 18, 2014 at 1:59 PM, David Thomas <[hidden email]> wrote:
Ok, well, if that's the case I'd like to see about remedying that.  Anyone have any thoughts as to how to best go about this?  I'm not clear on exactly what info lives where, especially across systems.  Entirely manual population would be a (barely) acceptable fallback.


On Tue, Mar 18, 2014 at 9:36 AM, Dan Burton <[hidden email]> wrote:
I have wished for this on multiple occasions. I don't believe such a thing exists as of yet.

-- Dan Burton


On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]> wrote:
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe




_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe



[hidden email]

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

mantkiew
This is not an immediate solution but I can imagine listing NIX packages as dependencies inside a cabal file, then NIX would create a sandbox with these dependencies and cabal would build in that sandbox. The advantages are that you're not messing around with the underlying operating system's packages, NIX handles all dependencies transitively, and everything is specified declaratively. Sounds like a nice GSoC project :-)  You could also do cross GHC version's builds as GHC itself can be sandboxed. Quite intriguing.

Michal


On Tue, Mar 18, 2014 at 2:16 PM, Michal Antkiewicz <[hidden email]> wrote:
Certainly NIX is an interesting approach. It already comes with a large base of dependencies, a format for specifying them. NIX can be installed in any Linux distro and serve as an environment for building packages. That might provide a cross-distribution solution to the native dependency problem.


Michal



On Tue, Mar 18, 2014 at 1:59 PM, David Thomas <[hidden email]> wrote:
Ok, well, if that's the case I'd like to see about remedying that.  Anyone have any thoughts as to how to best go about this?  I'm not clear on exactly what info lives where, especially across systems.  Entirely manual population would be a (barely) acceptable fallback.


On Tue, Mar 18, 2014 at 9:36 AM, Dan Burton <[hidden email]> wrote:
I have wished for this on multiple occasions. I don't believe such a thing exists as of yet.

-- Dan Burton


On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]> wrote:
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe




_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe



[hidden email]



_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

David Thomas
One option that just occurred to me would be to allow passing a script to cabal that could be passed the extra-libraries (if any), and could install if it knew the relevant OS packages (or NIX packages), or abort with a cleaner error message.

Actually, wrapping ghc might be sufficient (though not ideal).



On Tue, Mar 18, 2014 at 11:29 AM, Michal Antkiewicz <[hidden email]> wrote:
This is not an immediate solution but I can imagine listing NIX packages as dependencies inside a cabal file, then NIX would create a sandbox with these dependencies and cabal would build in that sandbox. The advantages are that you're not messing around with the underlying operating system's packages, NIX handles all dependencies transitively, and everything is specified declaratively. Sounds like a nice GSoC project :-)  You could also do cross GHC version's builds as GHC itself can be sandboxed. Quite intriguing.

Michal



On Tue, Mar 18, 2014 at 2:16 PM, Michal Antkiewicz <[hidden email]> wrote:
Certainly NIX is an interesting approach. It already comes with a large base of dependencies, a format for specifying them. NIX can be installed in any Linux distro and serve as an environment for building packages. That might provide a cross-distribution solution to the native dependency problem.


Michal



On Tue, Mar 18, 2014 at 1:59 PM, David Thomas <[hidden email]> wrote:
Ok, well, if that's the case I'd like to see about remedying that.  Anyone have any thoughts as to how to best go about this?  I'm not clear on exactly what info lives where, especially across systems.  Entirely manual population would be a (barely) acceptable fallback.


On Tue, Mar 18, 2014 at 9:36 AM, Dan Burton <[hidden email]> wrote:
I have wished for this on multiple occasions. I don't believe such a thing exists as of yet.

-- Dan Burton


On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]> wrote:
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe




_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe



[hidden email]




_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Tobias Dammers
Hmm, but then, not all package managers/repositories use the same names
for packages. So even if there were a way to extract the required
libraries from cabal, feeding that information to the package manager
isn't going to be trivial at all. (And frankly, looking at build systems
for other languages, I've never seen anything that does this - even
autotools doesn't really do a lot more than check whether a library is
available).

That said, even if we could just get a list of required libraries out of
cabal, in a somewhat human and machine readable format, that would be a
huge win in itself.

On Tue, Mar 18, 2014 at 11:48:33AM -0700, David Thomas wrote:

> One option that just occurred to me would be to allow passing a script to
> cabal that could be passed the extra-libraries (if any), and could install
> if it knew the relevant OS packages (or NIX packages), or abort with a
> cleaner error message.
>
> Actually, wrapping ghc might be sufficient (though not ideal).
>
>
>
> On Tue, Mar 18, 2014 at 11:29 AM, Michal Antkiewicz <
> [hidden email]> wrote:
>
> > This is not an immediate solution but I can imagine listing NIX packages
> > as dependencies inside a cabal file, then NIX would create a sandbox with
> > these dependencies and cabal would build in that sandbox. The advantages
> > are that you're not messing around with the underlying operating system's
> > packages, NIX handles all dependencies transitively, and everything is
> > specified declaratively. Sounds like a nice GSoC project :-)  You could
> > also do cross GHC version's builds as GHC itself can be sandboxed. Quite
> > intriguing.
> >
> > Michal
> >
> >
> >
> > On Tue, Mar 18, 2014 at 2:16 PM, Michal Antkiewicz <
> > [hidden email]> wrote:
> >
> >> Certainly NIX is an interesting approach. It already comes with a large
> >> base of dependencies, a format for specifying them. NIX can be installed in
> >> any Linux distro and serve as an environment for building packages. That
> >> might provide a cross-distribution solution to the native dependency
> >> problem.
> >>
> >> See, a nice post by Oliver Charles
> >>
> >> http://ocharles.org.uk/blog/posts/2014-02-04-how-i-develop-with-nixos.html
> >>
> >> Michal
> >>
> >>
> >>
> >> On Tue, Mar 18, 2014 at 1:59 PM, David Thomas <[hidden email]>wrote:
> >>
> >>> Ok, well, if that's the case I'd like to see about remedying that.
> >>> Anyone have any thoughts as to how to best go about this?  I'm not clear on
> >>> exactly what info lives where, especially across systems.  Entirely manual
> >>> population would be a (barely) acceptable fallback.
> >>>
> >>>
> >>> On Tue, Mar 18, 2014 at 9:36 AM, Dan Burton <[hidden email]>wrote:
> >>>
> >>>> I have wished for this on multiple occasions. I don't believe such a
> >>>> thing exists as of yet.
> >>>>
> >>>> -- Dan Burton
> >>>>
> >>>>
> >>>> On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]
> >>>> > wrote:
> >>>>
> >>>>> Is there a way to extract this?  I'm looking to make it easier for
> >>>>> newcomers to my project to get things building across different linux
> >>>>> distros.
> >>>>>
> >>>>> _______________________________________________
> >>>>> Haskell-Cafe mailing list
> >>>>> [hidden email]
> >>>>> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >>>>>
> >>>>>
> >>>>
> >>>
> >>> _______________________________________________
> >>> Haskell-Cafe mailing list
> >>> [hidden email]
> >>> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >>>
> >>>
> >>
> >> <[hidden email]>
> >>
> >
> >
> >

> _______________________________________________
> Haskell-Cafe mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Rogan Creswick
In reply to this post by David Thomas
On Tue, Mar 18, 2014 at 11:48 AM, David Thomas <[hidden email]> wrote:
One option that just occurred to me would be to allow passing a script to cabal that could be passed the extra-libraries (if any), and could install if it knew the relevant OS packages (or NIX packages), or abort with a cleaner error message.


You can do this sort of thing in a custom Setup.hs (using build-type: custom).  Depending on the hook you use, you can get access to the cabal file fields there directly.

--Rogan
 
Actually, wrapping ghc might be sufficient (though not ideal).



On Tue, Mar 18, 2014 at 11:29 AM, Michal Antkiewicz <[hidden email]> wrote:
This is not an immediate solution but I can imagine listing NIX packages as dependencies inside a cabal file, then NIX would create a sandbox with these dependencies and cabal would build in that sandbox. The advantages are that you're not messing around with the underlying operating system's packages, NIX handles all dependencies transitively, and everything is specified declaratively. Sounds like a nice GSoC project :-)  You could also do cross GHC version's builds as GHC itself can be sandboxed. Quite intriguing.

Michal



On Tue, Mar 18, 2014 at 2:16 PM, Michal Antkiewicz <[hidden email]> wrote:
Certainly NIX is an interesting approach. It already comes with a large base of dependencies, a format for specifying them. NIX can be installed in any Linux distro and serve as an environment for building packages. That might provide a cross-distribution solution to the native dependency problem.


Michal



On Tue, Mar 18, 2014 at 1:59 PM, David Thomas <[hidden email]> wrote:
Ok, well, if that's the case I'd like to see about remedying that.  Anyone have any thoughts as to how to best go about this?  I'm not clear on exactly what info lives where, especially across systems.  Entirely manual population would be a (barely) acceptable fallback.


On Tue, Mar 18, 2014 at 9:36 AM, Dan Burton <[hidden email]> wrote:
I have wished for this on multiple occasions. I don't believe such a thing exists as of yet.

-- Dan Burton


On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]> wrote:
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe




_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe



[hidden email]




_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe



_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

David Thomas
Is that cabal file fields for the current project, or cabal file fields for all dependencies?


On Tue, Mar 18, 2014 at 1:59 PM, Rogan Creswick <[hidden email]> wrote:
On Tue, Mar 18, 2014 at 11:48 AM, David Thomas <[hidden email]> wrote:
One option that just occurred to me would be to allow passing a script to cabal that could be passed the extra-libraries (if any), and could install if it knew the relevant OS packages (or NIX packages), or abort with a cleaner error message.


You can do this sort of thing in a custom Setup.hs (using build-type: custom).  Depending on the hook you use, you can get access to the cabal file fields there directly.

--Rogan
 
Actually, wrapping ghc might be sufficient (though not ideal).



On Tue, Mar 18, 2014 at 11:29 AM, Michal Antkiewicz <[hidden email]> wrote:
This is not an immediate solution but I can imagine listing NIX packages as dependencies inside a cabal file, then NIX would create a sandbox with these dependencies and cabal would build in that sandbox. The advantages are that you're not messing around with the underlying operating system's packages, NIX handles all dependencies transitively, and everything is specified declaratively. Sounds like a nice GSoC project :-)  You could also do cross GHC version's builds as GHC itself can be sandboxed. Quite intriguing.

Michal



On Tue, Mar 18, 2014 at 2:16 PM, Michal Antkiewicz <[hidden email]> wrote:
Certainly NIX is an interesting approach. It already comes with a large base of dependencies, a format for specifying them. NIX can be installed in any Linux distro and serve as an environment for building packages. That might provide a cross-distribution solution to the native dependency problem.


Michal



On Tue, Mar 18, 2014 at 1:59 PM, David Thomas <[hidden email]> wrote:
Ok, well, if that's the case I'd like to see about remedying that.  Anyone have any thoughts as to how to best go about this?  I'm not clear on exactly what info lives where, especially across systems.  Entirely manual population would be a (barely) acceptable fallback.


On Tue, Mar 18, 2014 at 9:36 AM, Dan Burton <[hidden email]> wrote:
I have wished for this on multiple occasions. I don't believe such a thing exists as of yet.

-- Dan Burton


On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]> wrote:
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe




_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe



[hidden email]




_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe




_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Tobias Dammers
In reply to this post by Tobias Dammers
On Wed, Mar 19, 2014 at 10:15:08AM +1300, Chris Wong wrote:

> On Wed, Mar 19, 2014 at 9:47 AM, Tobias Dammers <[hidden email]> wrote:
> > Hmm, but then, not all package managers/repositories use the same names
> > for packages. So even if there were a way to extract the required
> > libraries from cabal, feeding that information to the package manager
> > isn't going to be trivial at all.
>
> It's not trivial, but I don't think it's difficult.
>
> For example, Debian Haskell packages all take the form
> libghc-NAME-dev. Fedora uses haskell-NAME. Sure, there's a bit of hard
> coding involved, but the set of prominent distributions is finite.

I think installing *haskell* packages is not the problem; cabal already
does a decent job at that. The problem we're facing is that cabal can
*only* install haskell packages, but not any of the native libraries
they depend on (usually through FFI, but other examples exist, such as
the pg_config tool from PostgreSQL).

Besides naming convention issues (which could be compensated for with
lookup dictionaries; a brute-force solution, sure, but better than
nothing), a bigger problem is that sometimes packages don't match 1:1
between distros, so one distro might just provide one monolithic
foobar-dev package, while another might split it up into
foobar-client-dev and foobar-server-dev, while yet another might provide
more than one alternative.

TL;DR: I think if we could just get a list of required native packages,
we'd be a long way.

>
> One problem, though, would be versions. I know we often ask for the
> latest and greatest packages, which may conflict with what the
> distribution supplies. So the benefits may not be as high as they
> seem.
>
> By the way -- has anyone looked into 0install? It hashes things,
> similarly to Nix, but also tries to integrate with the existing
> package manager. For example, if the user wishes to install GHC 7.6,
> but the Debian repositories already supply it, it will invoke apt-get
> instead of installing on its own.
>
> > (And frankly, looking at build systems
> > for other languages, I've never seen anything that does this - even
> > autotools doesn't really do a lot more than check whether a library is
> > available).
> >
> > That said, even if we could just get a list of required libraries out of
> > cabal, in a somewhat human and machine readable format, that would be a
> > huge win in itself.
> >
> > On Tue, Mar 18, 2014 at 11:48:33AM -0700, David Thomas wrote:
> >> One option that just occurred to me would be to allow passing a script to
> >> cabal that could be passed the extra-libraries (if any), and could install
> >> if it knew the relevant OS packages (or NIX packages), or abort with a
> >> cleaner error message.
> >>
> >> Actually, wrapping ghc might be sufficient (though not ideal).
> >>
> >>
> >>
> >> On Tue, Mar 18, 2014 at 11:29 AM, Michal Antkiewicz <
> >> [hidden email]> wrote:
> >>
> >> > This is not an immediate solution but I can imagine listing NIX packages
> >> > as dependencies inside a cabal file, then NIX would create a sandbox with
> >> > these dependencies and cabal would build in that sandbox. The advantages
> >> > are that you're not messing around with the underlying operating system's
> >> > packages, NIX handles all dependencies transitively, and everything is
> >> > specified declaratively. Sounds like a nice GSoC project :-)  You could
> >> > also do cross GHC version's builds as GHC itself can be sandboxed. Quite
> >> > intriguing.
> >> >
> >> > Michal
> >> >
> >> >
> >> >
> >> > On Tue, Mar 18, 2014 at 2:16 PM, Michal Antkiewicz <
> >> > [hidden email]> wrote:
> >> >
> >> >> Certainly NIX is an interesting approach. It already comes with a large
> >> >> base of dependencies, a format for specifying them. NIX can be installed in
> >> >> any Linux distro and serve as an environment for building packages. That
> >> >> might provide a cross-distribution solution to the native dependency
> >> >> problem.
> >> >>
> >> >> See, a nice post by Oliver Charles
> >> >>
> >> >> http://ocharles.org.uk/blog/posts/2014-02-04-how-i-develop-with-nixos.html
> >> >>
> >> >> Michal
> >> >>
> >> >>
> >> >>
> >> >> On Tue, Mar 18, 2014 at 1:59 PM, David Thomas <[hidden email]>wrote:
> >> >>
> >> >>> Ok, well, if that's the case I'd like to see about remedying that.
> >> >>> Anyone have any thoughts as to how to best go about this?  I'm not clear on
> >> >>> exactly what info lives where, especially across systems.  Entirely manual
> >> >>> population would be a (barely) acceptable fallback.
> >> >>>
> >> >>>
> >> >>> On Tue, Mar 18, 2014 at 9:36 AM, Dan Burton <[hidden email]>wrote:
> >> >>>
> >> >>>> I have wished for this on multiple occasions. I don't believe such a
> >> >>>> thing exists as of yet.
> >> >>>>
> >> >>>> -- Dan Burton
> >> >>>>
> >> >>>>
> >> >>>> On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]
> >> >>>> > wrote:
> >> >>>>
> >> >>>>> Is there a way to extract this?  I'm looking to make it easier for
> >> >>>>> newcomers to my project to get things building across different linux
> >> >>>>> distros.
> >> >>>>>
> >> >>>>> _______________________________________________
> >> >>>>> Haskell-Cafe mailing list
> >> >>>>> [hidden email]
> >> >>>>> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >> >>>>>
> >> >>>>>
> >> >>>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> Haskell-Cafe mailing list
> >> >>> [hidden email]
> >> >>> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >> >>>
> >> >>>
> >> >>
> >> >> <[hidden email]>
> >> >>
> >> >
> >> >
> >> >
> >
> >> _______________________________________________
> >> Haskell-Cafe mailing list
> >> [hidden email]
> >> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >
> > _______________________________________________
> > Haskell-Cafe mailing list
> > [hidden email]
> > http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Dan Burton
I would imagine that the people working on Stackage may have already done some work in this area.


-- Dan Burton


On Tue, Mar 18, 2014 at 2:54 PM, Tobias Dammers <[hidden email]> wrote:
On Wed, Mar 19, 2014 at 10:15:08AM +1300, Chris Wong wrote:
> On Wed, Mar 19, 2014 at 9:47 AM, Tobias Dammers <[hidden email]> wrote:
> > Hmm, but then, not all package managers/repositories use the same names
> > for packages. So even if there were a way to extract the required
> > libraries from cabal, feeding that information to the package manager
> > isn't going to be trivial at all.
>
> It's not trivial, but I don't think it's difficult.
>
> For example, Debian Haskell packages all take the form
> libghc-NAME-dev. Fedora uses haskell-NAME. Sure, there's a bit of hard
> coding involved, but the set of prominent distributions is finite.

I think installing *haskell* packages is not the problem; cabal already
does a decent job at that. The problem we're facing is that cabal can
*only* install haskell packages, but not any of the native libraries
they depend on (usually through FFI, but other examples exist, such as
the pg_config tool from PostgreSQL).

Besides naming convention issues (which could be compensated for with
lookup dictionaries; a brute-force solution, sure, but better than
nothing), a bigger problem is that sometimes packages don't match 1:1
between distros, so one distro might just provide one monolithic
foobar-dev package, while another might split it up into
foobar-client-dev and foobar-server-dev, while yet another might provide
more than one alternative.

TL;DR: I think if we could just get a list of required native packages,
we'd be a long way.

>
> One problem, though, would be versions. I know we often ask for the
> latest and greatest packages, which may conflict with what the
> distribution supplies. So the benefits may not be as high as they
> seem.
>
> By the way -- has anyone looked into 0install? It hashes things,
> similarly to Nix, but also tries to integrate with the existing
> package manager. For example, if the user wishes to install GHC 7.6,
> but the Debian repositories already supply it, it will invoke apt-get
> instead of installing on its own.
>
> > (And frankly, looking at build systems
> > for other languages, I've never seen anything that does this - even
> > autotools doesn't really do a lot more than check whether a library is
> > available).
> >
> > That said, even if we could just get a list of required libraries out of
> > cabal, in a somewhat human and machine readable format, that would be a
> > huge win in itself.
> >
> > On Tue, Mar 18, 2014 at 11:48:33AM -0700, David Thomas wrote:
> >> One option that just occurred to me would be to allow passing a script to
> >> cabal that could be passed the extra-libraries (if any), and could install
> >> if it knew the relevant OS packages (or NIX packages), or abort with a
> >> cleaner error message.
> >>
> >> Actually, wrapping ghc might be sufficient (though not ideal).
> >>
> >>
> >>
> >> On Tue, Mar 18, 2014 at 11:29 AM, Michal Antkiewicz <
> >> [hidden email]> wrote:
> >>
> >> > This is not an immediate solution but I can imagine listing NIX packages
> >> > as dependencies inside a cabal file, then NIX would create a sandbox with
> >> > these dependencies and cabal would build in that sandbox. The advantages
> >> > are that you're not messing around with the underlying operating system's
> >> > packages, NIX handles all dependencies transitively, and everything is
> >> > specified declaratively. Sounds like a nice GSoC project :-)  You could
> >> > also do cross GHC version's builds as GHC itself can be sandboxed. Quite
> >> > intriguing.
> >> >
> >> > Michal
> >> >
> >> >
> >> >
> >> > On Tue, Mar 18, 2014 at 2:16 PM, Michal Antkiewicz <
> >> > [hidden email]> wrote:
> >> >
> >> >> Certainly NIX is an interesting approach. It already comes with a large
> >> >> base of dependencies, a format for specifying them. NIX can be installed in
> >> >> any Linux distro and serve as an environment for building packages. That
> >> >> might provide a cross-distribution solution to the native dependency
> >> >> problem.
> >> >>
> >> >> See, a nice post by Oliver Charles
> >> >>
> >> >> http://ocharles.org.uk/blog/posts/2014-02-04-how-i-develop-with-nixos.html
> >> >>
> >> >> Michal
> >> >>
> >> >>
> >> >>
> >> >> On Tue, Mar 18, 2014 at 1:59 PM, David Thomas <[hidden email]>wrote:
> >> >>
> >> >>> Ok, well, if that's the case I'd like to see about remedying that.
> >> >>> Anyone have any thoughts as to how to best go about this?  I'm not clear on
> >> >>> exactly what info lives where, especially across systems.  Entirely manual
> >> >>> population would be a (barely) acceptable fallback.
> >> >>>
> >> >>>
> >> >>> On Tue, Mar 18, 2014 at 9:36 AM, Dan Burton <[hidden email]>wrote:
> >> >>>
> >> >>>> I have wished for this on multiple occasions. I don't believe such a
> >> >>>> thing exists as of yet.
> >> >>>>
> >> >>>> -- Dan Burton
> >> >>>>
> >> >>>>
> >> >>>> On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]
> >> >>>> > wrote:
> >> >>>>
> >> >>>>> Is there a way to extract this?  I'm looking to make it easier for
> >> >>>>> newcomers to my project to get things building across different linux
> >> >>>>> distros.
> >> >>>>>
> >> >>>>> _______________________________________________
> >> >>>>> Haskell-Cafe mailing list
> >> >>>>> [hidden email]
> >> >>>>> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >> >>>>>
> >> >>>>>
> >> >>>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> Haskell-Cafe mailing list
> >> >>> [hidden email]
> >> >>> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >> >>>
> >> >>>
> >> >>
> >> >> <[hidden email]>
> >> >>
> >> >
> >> >
> >> >
> >
> >> _______________________________________________
> >> Haskell-Cafe mailing list
> >> [hidden email]
> >> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >
> > _______________________________________________
> > Haskell-Cafe mailing list
> > [hidden email]
> > http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

mantkiew
In reply to this post by Tobias Dammers
Well, if naming is the problem then you need a cross-distribution package repository. The only one like that across linuxes and MacOS that I know of is NIX packages. 

Building inside a NIX sandbox would shield Cabal from the OS packages as distros often come with older versions.

Nix will also install all the dependencies transitively, so a cabal package only has to list the direct deps.

Re: 0 install - haven't used it but might be an alternative if it is cross-distribution and for mac.

Michal

  Original Message  
From: Tobias Dammers
Sent: Tuesday, March 18, 2014 5:57 PM
To: Chris Wong
Cc: [hidden email]
Subject: Re: [Haskell-cafe] Listing native package requirements based on cabal information

On Wed, Mar 19, 2014 at 10:15:08AM +1300, Chris Wong wrote:

> On Wed, Mar 19, 2014 at 9:47 AM, Tobias Dammers <[hidden email]> wrote:
> > Hmm, but then, not all package managers/repositories use the same names
> > for packages. So even if there were a way to extract the required
> > libraries from cabal, feeding that information to the package manager
> > isn't going to be trivial at all.
>
> It's not trivial, but I don't think it's difficult.
>
> For example, Debian Haskell packages all take the form
> libghc-NAME-dev. Fedora uses haskell-NAME. Sure, there's a bit of hard
> coding involved, but the set of prominent distributions is finite.

I think installing *haskell* packages is not the problem; cabal already
does a decent job at that. The problem we're facing is that cabal can
*only* install haskell packages, but not any of the native libraries
they depend on (usually through FFI, but other examples exist, such as
the pg_config tool from PostgreSQL).

Besides naming convention issues (which could be compensated for with
lookup dictionaries; a brute-force solution, sure, but better than
nothing), a bigger problem is that sometimes packages don't match 1:1
between distros, so one distro might just provide one monolithic
foobar-dev package, while another might split it up into
foobar-client-dev and foobar-server-dev, while yet another might provide
more than one alternative.

TL;DR: I think if we could just get a list of required native packages,
we'd be a long way.

>
> One problem, though, would be versions. I know we often ask for the
> latest and greatest packages, which may conflict with what the
> distribution supplies. So the benefits may not be as high as they
> seem.
>
> By the way -- has anyone looked into 0install? It hashes things,
> similarly to Nix, but also tries to integrate with the existing
> package manager. For example, if the user wishes to install GHC 7.6,
> but the Debian repositories already supply it, it will invoke apt-get
> instead of installing on its own.
>
> > (And frankly, looking at build systems
> > for other languages, I've never seen anything that does this - even
> > autotools doesn't really do a lot more than check whether a library is
> > available).
> >
> > That said, even if we could just get a list of required libraries out of
> > cabal, in a somewhat human and machine readable format, that would be a
> > huge win in itself.
> >
> > On Tue, Mar 18, 2014 at 11:48:33AM -0700, David Thomas wrote:
> >> One option that just occurred to me would be to allow passing a script to
> >> cabal that could be passed the extra-libraries (if any), and could install
> >> if it knew the relevant OS packages (or NIX packages), or abort with a
> >> cleaner error message.
> >>
> >> Actually, wrapping ghc might be sufficient (though not ideal).
> >>
> >>
> >>
> >> On Tue, Mar 18, 2014 at 11:29 AM, Michal Antkiewicz <
> >> [hidden email]> wrote:
> >>
> >> > This is not an immediate solution but I can imagine listing NIX packages
> >> > as dependencies inside a cabal file, then NIX would create a sandbox with
> >> > these dependencies and cabal would build in that sandbox. The advantages
> >> > are that you're not messing around with the underlying operating system's
> >> > packages, NIX handles all dependencies transitively, and everything is
> >> > specified declaratively. Sounds like a nice GSoC project :-) You could
> >> > also do cross GHC version's builds as GHC itself can be sandboxed. Quite
> >> > intriguing.
> >> >
> >> > Michal
> >> >
> >> >
> >> >
> >> > On Tue, Mar 18, 2014 at 2:16 PM, Michal Antkiewicz <
> >> > [hidden email]> wrote:
> >> >
> >> >> Certainly NIX is an interesting approach. It already comes with a large
> >> >> base of dependencies, a format for specifying them. NIX can be installed in
> >> >> any Linux distro and serve as an environment for building packages. That
> >> >> might provide a cross-distribution solution to the native dependency
> >> >> problem.
> >> >>
> >> >> See, a nice post by Oliver Charles
> >> >>
> >> >> http://ocharles.org.uk/blog/posts/2014-02-04-how-i-develop-with-nixos.html
> >> >>
> >> >> Michal
> >> >>
> >> >>
> >> >>
> >> >> On Tue, Mar 18, 2014 at 1:59 PM, David Thomas <[hidden email]>wrote:
> >> >>
> >> >>> Ok, well, if that's the case I'd like to see about remedying that.
> >> >>> Anyone have any thoughts as to how to best go about this? I'm not clear on
> >> >>> exactly what info lives where, especially across systems. Entirely manual
> >> >>> population would be a (barely) acceptable fallback.
> >> >>>
> >> >>>
> >> >>> On Tue, Mar 18, 2014 at 9:36 AM, Dan Burton <[hidden email]>wrote:
> >> >>>
> >> >>>> I have wished for this on multiple occasions. I don't believe such a
> >> >>>> thing exists as of yet.
> >> >>>>
> >> >>>> -- Dan Burton
> >> >>>>
> >> >>>>
> >> >>>> On Tue, Mar 18, 2014 at 9:26 AM, David Thomas <[hidden email]
> >> >>>> > wrote:
> >> >>>>
> >> >>>>> Is there a way to extract this? I'm looking to make it easier for
> >> >>>>> newcomers to my project to get things building across different linux
> >> >>>>> distros.
> >> >>>>>
> >> >>>>> _______________________________________________
> >> >>>>> Haskell-Cafe mailing list
> >> >>>>> [hidden email]
> >> >>>>> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >> >>>>>
> >> >>>>>
> >> >>>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> Haskell-Cafe mailing list
> >> >>> [hidden email]
> >> >>> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >> >>>
> >> >>>
> >> >>
> >> >> <[hidden email]>
> >> >>
> >> >
> >> >
> >> >
> >
> >> _______________________________________________
> >> Haskell-Cafe mailing list
> >> [hidden email]
> >> http://www.haskell.org/mailman/listinfo/haskell-cafe
> >
> > _______________________________________________
> > Haskell-Cafe mailing list
> > [hidden email]
> > http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Chris Warburton
In reply to this post by Tobias Dammers
Tobias Dammers <[hidden email]> writes:

> Besides naming convention issues (which could be compensated for with
> lookup dictionaries; a brute-force solution, sure, but better than
> nothing), a bigger problem is that sometimes packages don't match 1:1
> between distros, so one distro might just provide one monolithic
> foobar-dev package, while another might split it up into
> foobar-client-dev and foobar-server-dev, while yet another might provide
> more than one alternative.
>
> TL;DR: I think if we could just get a list of required native packages,
> we'd be a long way.

Package naming/splitting/etc. is the one reason that working across
distros is difficult. RPM/deb/etc. are trivial to convert between and
most other incompatibilities have known solutions for working around
(eg. look at any standalone app which offers a tar.gz).

I've been down this route before, and it inevitably ends up checking
individual filenames. Many RPMs depend on filenames rather than other
RPMs. PackageKit tries to apply this in a more format-neutral way.

It was a few years ago that I last looked, but the "just" is unwarranted
since this is *the* difficult problem. Tools like alien and checkinstall
can do the rest.

Regards,
Chris
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Alexander Vershilov
In reply to this post by David Thomas
In Gentoo Linux we use a common ebuild format that allowes to add native dependencies to packages.
To create a ebuild from cabal file we use tool called hackport [1].

This approach allow to bump dependencies when some authors are to lazy to bump them themselves,
even in presence of pull requests, and add compatibility patches. Also it's possible to create a desktop-icons,
additional files, filter test/bench programs that should not be installed into system. Also it supports a dependency
on package flags and profiling.

However this approach doesn't scale on other distros in a easy way, one possibility is to create static
packages using emerge, but this area require some more research.

[1] http://hackage.haskell.org/package/hackport

--
  Alexander


On 18 March 2014 20:26, David Thomas <[hidden email]> wrote:
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe




--
Alexander

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

David Thomas
Yeah, it seems that there are a number of potentially good options that are going to vary by distro and situation, which is why I've been thinking "hook to run arbitrary script".  Sufficiently populating the lookup table for that script for any given (Project, Distro) shouldn't be hard, and collecting those we should be able to quickly 1) round the rough edges off of the more common cases without requiring too much manual intervention, and 2) have a reasonable path for the less common cases.


On Wed, Mar 19, 2014 at 5:48 AM, Alexander V Vershilov <[hidden email]> wrote:
In Gentoo Linux we use a common ebuild format that allowes to add native dependencies to packages.
To create a ebuild from cabal file we use tool called hackport [1].

This approach allow to bump dependencies when some authors are to lazy to bump them themselves,
even in presence of pull requests, and add compatibility patches. Also it's possible to create a desktop-icons,
additional files, filter test/bench programs that should not be installed into system. Also it supports a dependency
on package flags and profiling.

However this approach doesn't scale on other distros in a easy way, one possibility is to create static
packages using emerge, but this area require some more research.

[1] http://hackage.haskell.org/package/hackport

--
  Alexander


On 18 March 2014 20:26, David Thomas <[hidden email]> wrote:
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe




--
Alexander


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Alexander Vershilov
I personally not sure about the one right solution, as there are a plenty of different approaches and rules that are applied
system wide, and heavily depend on the underlying package system. Of cause implementation of the new nice features like running
arbitrary script, but I personally don't know of many good examples that can make life much easier without introducing
additional problems.

I'll try to follow up this thread and ready to share our experience and solutions.

--
Alexander




On 19 March 2014 18:42, David Thomas <[hidden email]> wrote:
Yeah, it seems that there are a number of potentially good options that are going to vary by distro and situation, which is why I've been thinking "hook to run arbitrary script".  Sufficiently populating the lookup table for that script for any given (Project, Distro) shouldn't be hard, and collecting those we should be able to quickly 1) round the rough edges off of the more common cases without requiring too much manual intervention, and 2) have a reasonable path for the less common cases.


On Wed, Mar 19, 2014 at 5:48 AM, Alexander V Vershilov <[hidden email]> wrote:
In Gentoo Linux we use a common ebuild format that allowes to add native dependencies to packages.
To create a ebuild from cabal file we use tool called hackport [1].

This approach allow to bump dependencies when some authors are to lazy to bump them themselves,
even in presence of pull requests, and add compatibility patches. Also it's possible to create a desktop-icons,
additional files, filter test/bench programs that should not be installed into system. Also it supports a dependency
on package flags and profiling.

However this approach doesn't scale on other distros in a easy way, one possibility is to create static
packages using emerge, but this area require some more research.

[1] http://hackage.haskell.org/package/hackport

--
  Alexander


On 18 March 2014 20:26, David Thomas <[hidden email]> wrote:
Is there a way to extract this?  I'm looking to make it easier for newcomers to my project to get things building across different linux distros.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe




--
Alexander




--
Alexander

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Peter Simons
In reply to this post by David Thomas
Hi David,

 > Is there a way to extract this? I'm looking to make it easier for
 > newcomers to my project to get things building across different linux
 > distros.

what exactly do you mean when by "package requirements"?

Cabal descriptions list the required 3rd party libraries, header files,
and pkgconfig snippets, so there is a lot of information readily
available.

What is missing in your opinion?

Take care,
Peter

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Alexander Vershilov
Some packages require existing executables, e.g. git-annex require lsof.
There were some more examples of packages that doesn't have pkg-config
but require additional programs to be installed.

--
Alexander.


On 19 March 2014 20:09, Peter Simons <[hidden email]> wrote:
Hi David,

 > Is there a way to extract this? I'm looking to make it easier for
 > newcomers to my project to get things building across different linux
 > distros.

what exactly do you mean when by "package requirements"?

Cabal descriptions list the required 3rd party libraries, header files,
and pkgconfig snippets, so there is a lot of information readily
available.

What is missing in your opinion?

Take care,
Peter

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe



--
Alexander

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

David Thomas
In reply to this post by Peter Simons
Well, the most important thing is to make sure people can successfully build things.  Next most important is to minimize the manual intervention required (and the degree to which that manual intervention is spread out through time).

Getting all the necessary 3rd party libraries *for this package and all dependencies* is a good start.  Including, somehow, required utilities would also be worthwhile.  Being able to automate all of that, appropriate to distro and project, would be optimal (but potentially more work than it's worth, depending).


On Wed, Mar 19, 2014 at 9:09 AM, Peter Simons <[hidden email]> wrote:
Hi David,

 > Is there a way to extract this? I'm looking to make it easier for
 > newcomers to my project to get things building across different linux
 > distros.

what exactly do you mean when by "package requirements"?

Cabal descriptions list the required 3rd party libraries, header files,
and pkgconfig snippets, so there is a lot of information readily
available.

What is missing in your opinion?

Take care,
Peter

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Listing native package requirements based on cabal information

Peter Simons
In reply to this post by Alexander Vershilov
Hi Alexander,

 > Some packages require existing executables, e.g. git-annex require
 > lsof. There were some more examples of packages that doesn't have
 > pkg-config but require additional programs to be installed.

if the package requires system resources that the Cabal file doesn't
declare, then this is a bug in the Cabal file, no?

Take care,
Peter

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
12