brainstorming ways to stabalize Cabal interface?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

brainstorming ways to stabalize Cabal interface?

Isaac Jones-2
Greetings.  I'm keen to try to stabalize more of the Cabal interface,
but I'm not sure the best way to go about that.  I started a wiki page
to start to document what is stable and what is not.  Eventually this
should move into the end-user documentation:

http://hackage.haskell.org/trac/hackage/wiki/StableInterfaces

Does anyone have clever ideas for how to go about pushing these
interfaces toward stability?  Do we just need to declare that certain
things are unstable for the near-term (and possibly implement the
eternal compatibility in theory idea[1]) and continue to experiment
with them until they seem done enough?  Do we stare at them very hard?

Below[2] are my thoughts about what we _know_ to be stable, and the
types for the new hooks interface.


peace,

  isaac



[1] http://www.haskell.org/tmrwiki/EternalCompatibilityInTheory 

[2] Here's a summary of the bits of interface that need to be
    stabalized, or are already stabalized.

Stable Commands

    * ./setup configure
          o --prefix
          o --user
          o --ghc, --hugs
          o --verbose
    * ./setup build
    * ./setup install

Unstable Commands
    * everything else.  well, they're really _pretty_ stable.

Stable functions

    * defaultMain
    * defaultMainWithHooks defaultUserHooks
          o but regular defaultMainWithHooks isn't stable since userHooks changes.

Unstable functions

    * Various utility functions
    * UserHooks

runTests :: Args -> Bool -> PackageDescription -> LocalBuildInfo -> IO ExitCode, -- ^Used for @.\/setup test@
readDesc :: IO (Maybe PackageDescription), -- ^Read the description file
hookedPreProcessors :: [ PPSuffixHandler ],
   -- ^Custom preprocessors in addition to and overriding 'knownSuffixHandlers'.
hookedPrograms :: [Program],
   -- ^These programs are detected at configure time.  Arguments for them are added to the configure command.
  -- |Hook to run before configure command
preConf  :: Args -> ConfigFlags -> IO HookedBuildInfo,
-- |Over-ride this hook to get different behavior during configure.
confHook :: PackageDescription -> ConfigFlags -> IO LocalBuildInfo,
 -- |Hook to run after configure command
postConf :: Args -> ConfigFlags -> PackageDescription -> LocalBuildInfo -> IO ExitCode,
...
   * Everything else

peace,

  isaac
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: brainstorming ways to stabalize Cabal interface?

Duncan Coutts
On Wed, 2006-01-18 at 22:57 -0800, Isaac Jones wrote:

> Greetings.  I'm keen to try to stabalize more of the Cabal interface,
> but I'm not sure the best way to go about that.  I started a wiki page
> to start to document what is stable and what is not.  Eventually this
> should move into the end-user documentation:
>
> http://hackage.haskell.org/trac/hackage/wiki/StableInterfaces
>
> Does anyone have clever ideas for how to go about pushing these
> interfaces toward stability?  Do we just need to declare that certain
> things are unstable for the near-term (and possibly implement the
> eternal compatibility in theory idea[1]) and continue to experiment
> with them until they seem done enough?  Do we stare at them very hard?

An idea I've suggested before:

Add another (thin) layer.

The problem as many people have noted is that Haskell source code in the
form of Setup.lhs is quite brittle in the face of changes to the Cabal
API and the .cabal file format.

Often the first sign of trouble is that Setup.lhs just doesn't compile
or if it does it complains that it cannot parse the .cabal file because
there is a new field being used that the current Cabal version being
used doesn't grok. Neither of these situations provide good error
messages or any suggestion to the user of how to proceed.

So my suggestion is that we add a thin layer over the current user
interface:
runhaskell Setup.lhs <command>

instead we should use a little *program*, lets call it "hs-pkg" for now:
hs-pkg <command>

These commands should be the same as for Cabal's Setup.lhs. This program
will basically deffer to the existing mechanisms to compile the package
but it will be able to do many more checks than the user can.

It can read the .cabal file and discover which API version of the Cabal
package is required. For the case of ghc it can then check that the
required version of Cabal is actually installed and invoke runghc
specifying the exact version required.

It can do something about the issue of multiple compilers/interpreters
and the runhaskell program (that John Mecham has pointed out before).

Existing systems already do similar things and are more reliable as a
consequence. For Gentoo distro-packages of Cabal packages it doesn't
just run runhaskell and hope that the right packages are currently
exposed. It invokes ghc with the right -package Cabal-1.1.x flag. This
allows it to deal with packages that require different versions of the
Cabal library.

As a side note, this also allows the Setup.lhs file to be left out
entirely when it just does the default stuff, since the hs-pkg program
can deal it.

Duncan

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: brainstorming ways to stabalize Cabal interface?

Simon Marlow-5
Duncan Coutts wrote:

> On Wed, 2006-01-18 at 22:57 -0800, Isaac Jones wrote:
>> Greetings.  I'm keen to try to stabalize more of the Cabal interface,
>> but I'm not sure the best way to go about that.  I started a wiki page
>> to start to document what is stable and what is not.  Eventually this
>> should move into the end-user documentation:
>>
>> http://hackage.haskell.org/trac/hackage/wiki/StableInterfaces
>>
>> Does anyone have clever ideas for how to go about pushing these
>> interfaces toward stability?  Do we just need to declare that certain
>> things are unstable for the near-term (and possibly implement the
>> eternal compatibility in theory idea[1]) and continue to experiment
>> with them until they seem done enough?  Do we stare at them very hard?
>
> An idea I've suggested before:
>
> Add another (thin) layer.

[ good idea deleted ]

So there are basically three approaches we can take to mitigating the
backwards compatibility problem with Cabal.

  (1) Just declare what we claim to be stable, and try to get
      people to update their packages as necessary

  (2) EternalCompatibilityInTheory for Cabal only

  (3) Provide multiple versions of the Cabal package, with
      a wrapper program to invoke Setup.lhs as Duncan suggests

I think we should do at least 2 or 3, because 1 is likely to cause too
much pain for end-users.  Many packages will need to use "unstable"
parts of the interface just to work properly, so it's unfair to make
them second-class citizens.  As far as possible, we should ensure that
packages keep working for as long as we can support them.

In the past I suggested 2, but I think I'd be equally (or perhaps more)
happy with 3.  To compare 2 and 3:

   - it seems a bit strange to use ETC just for Cabal, but I don't
     support doing it for everything (it's too heavyweight a solution)

   - 3 requires us to provide multiple versions of the Cabal package,
     rather than one version that supports multiple interfaces.  This
     is perhaps less work for the Cabal developers (keeping the old
     interfaces working is easier, we just compile the old code), but
     it means more bloat.

   - 3 lets us do away with Setup.hs when it is just boilerplate

   - 3 means an upheaval in the end-user's view of Cabal.  They have to
     run "cabal build" (or whatever) instead of "runhaskell Setup.hs
     build"

   - the wrapper program can transparently compile Setup.hs instead of
     interpreting it each time, which is a lot faster

I think I'm coming down on the side of Duncan's suggestion, despite the
fact that it's a pretty big interface change.  We could migrate slowly
by making 'runhaskell Setup.hs' emit a warning for the time being.

Cheers,
        Simon
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: brainstorming ways to stabalize Cabal interface?

John Meacham
In reply to this post by Duncan Coutts
yes, yes, a thousand times yes. cabal needs to be a program instead of
or in addition to a library. probably in addition to since the library
certainly has a lot of useful functionality.
        John

--
John Meacham - ⑆repetae.net⑆john⑈
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: brainstorming ways to stabalize Cabal interface?

Duncan Coutts
In reply to this post by Simon Marlow-5
On Thu, 2006-01-19 at 10:50 +0000, Simon Marlow wrote:

> In the past I suggested 2, but I think I'd be equally (or perhaps more)
> happy with 3.  To compare 2 and 3:
>
>    - it seems a bit strange to use ETC just for Cabal, but I don't
>      support doing it for everything (it's too heavyweight a solution)
>
>    - 3 requires us to provide multiple versions of the Cabal package,
>      rather than one version that supports multiple interfaces.  This
>      is perhaps less work for the Cabal developers (keeping the old
>      interfaces working is easier, we just compile the old code), but
>      it means more bloat.

Though people only need the versions that are needed by the packages
they install. These can be dragged in on demand by distro package
managers. For the manual case the "cabal" wrapper program tells the user
what they need to install.

>    - 3 lets us do away with Setup.hs when it is just boilerplate

>    - 3 means an upheaval in the end-user's view of Cabal.  They have to
>      run "cabal build" (or whatever) instead of "runhaskell Setup.hs
>      build"
>
>    - the wrapper program can transparently compile Setup.hs instead of
>      interpreting it each time, which is a lot faster

Yeah, the Gentoo Cabal system does this.

> I think I'm coming down on the side of Duncan's suggestion, despite the
> fact that it's a pretty big interface change.  We could migrate slowly
> by making 'runhaskell Setup.hs' emit a warning for the time being.

We've had a few interface changes and inconsistencies before and
generally in practise:
      * At one time people were supposed to use ./Setup.lhs and the
        Setup.lhs file was supposed to invoke runhaskell via the #!
        method.
      * Some packages supply Setup.hs rather than Setup.lhs.
      * Some packages supply no Setup.(l)hs at all.
      * Some instructions suggest to use runghc Setup.lhs rather than
        runhaskell Setup.lhs on the basis that one is more reliable.

By having package developers and users using the same "cabal" script
should help reduce these and other inconsistencies. For example some
packages rely on optional packages for their Setup.lhs script itself!
For example a Setup.lhs file that uses modules from the FilePath
package. This is something that happens to work on the developers
machine but will not work on the users machine if the user does not have
the package. This is true even if the developer lists the FilePath
package in their .cabal file, since we can't run the setup program to
know that we need the FilePath package! A "cabal" wrapper program could
stop the developer from making that kind of mistake by compiling the
Setup.lhs using --ignore-all-packages (or whatever the ghc flag is) so
that it does not depend on any packages other that base and haskell98.

One other change that would be good to support this is to specify in a
new field in the .cabal file the version of the Cabal API that is
required. It can sort-of be done at the moment by using the
build-depends, but this will cause the resulting library to depend on
Cabal.

If we take this route, I think it'd be good to address John Meacham's
query about who provides cabal and the wrapper script/prog. Is it the
compilers or something separate? For example, John asked what happens if
runhaskell points at jhc (assuming jhc can even provide such a thing)
will that work if the user is trying to install a package for ghc?
Perhaps this is something that the script can sort out. Perhaps it
should use the implementation being targeted to compile/interpret the
Setup.(l)hs file.

Duncan

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: brainstorming ways to stabalize Cabal interface?

Isaac Jones-2
As to Duncan's first mail, I am becomming more inclined to agree on an
official wrapper script (which is why I started the cabal-install
effort).

Duncan Coutts <[hidden email]> writes:

(snip)
> We've had a few interface changes and inconsistencies before and
> generally in practise:
>       * At one time people were supposed to use ./Setup.lhs and the
>         Setup.lhs file was supposed to invoke runhaskell via the #!
>         method.
>       * Some packages supply Setup.hs rather than Setup.lhs.
>       * Some packages supply no Setup.(l)hs at all.
>       * Some instructions suggest to use runghc Setup.lhs rather than
>         runhaskell Setup.lhs on the basis that one is more reliable.

I can undertand how these have been a problem, and it would be good if
we could enforce consistency or write tools to make consistency less
important.  BTW, these don't represent any interface changes, just
people failing to conform to the standards.  The story has always been
"Provide a Setup.hs or Setup.lhs that the end user compilers or
interprets somehow."

John M. Says:
> yes, yes, a thousand times yes. cabal needs to be a program instead
> of or in addition to a library. probably in addition to since the
> library certainly has a lot of useful functionality.

Some of the reasons that I still have resisted makeing cabal into a
program are that I've been hoping that cabal-get would make that less
necessary, but cabal-get is not yet ready.  I've also been hoping that
runhaskell would become ubiquitous, but it hasn't.

But if I agree to this, you have to get it going for JHC ;)

peace,

  isaac
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: brainstorming ways to stabalize Cabal interface?

John Meacham
On Thu, Jan 19, 2006 at 01:59:01PM -0800, Isaac Jones wrote:
> But if I agree to this, you have to get it going for JHC ;)

certainly, although, rather than make jhc another special case in cabal,
I'd rather work on making a general compiler framework for it so that
jhc can just drop a file describing its interface in
/usr/share/lib/cabal/compilers/jhc.cabal-compiler  or something and
cabal will automatically be able to use it. Ideally, one would not have
to upgrade their cabal just because they install (or write) a new
haskell compiler. I think all compilers conform to one of

hmake-like: ghc --make, jhc, nhc + hmake
interpreter-like: hugs
gcc-like: ghc, nhc98

so a compiler declaration file would not have to be much more
complicated than a string telling it how to invoke the compiler and a
mapping of various extensions/cabal options -> compiler flags.

I'd also like to do something like this for preprocessors, which would
be a much simpler project so will probably do first.


incidentally, could we get cabal to ignore any field starting with 'x-'
as a user defined extension?

I'd like to use locally things like

x-publish-site: /home/john/public_html/...
x-darcs-repo: http://repetae.net/repos/jhc

or experimental things like
x-jhc-namespace: 0x220

without cabal getting huffy about unknown fields.

obviously any popular and generally useful ones would eventually be
standardized and the x- can be discarded.

this is a fairly standard convention among file formats and is used in
mime types too.

        John

--
John Meacham - ⑆repetae.net⑆john⑈
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: brainstorming ways to stabalize Cabal interface?

Ross Paterson
In reply to this post by Duncan Coutts
On Thu, Jan 19, 2006 at 09:35:32AM +0000, Duncan Coutts wrote:
> On Wed, 2006-01-18 at 22:57 -0800, Isaac Jones wrote:
> > I'm keen to try to stabilize more of the Cabal interface,
> [...]
> The problem as many people have noted is that Haskell source code in the
> form of Setup.lhs is quite brittle in the face of changes to the Cabal
> API and the .cabal file format.

Indeed.  I think the best hope for stability is to make setup scripts
optional, and to seek to reduce the situations where they're needed.
The vast majority of scripts are

        main = defaultMain
or
        main = defaultMainWithHooks defaultUserHooks

(Incidentally hackage and packages repository contain very few
exceptions to this; it seems Gentoo has a much larger collection.)
If we were to rely on an external tool like Duncan's hs-pkg, we'd need
a field saying which of these (or future ones) to use.  We'd also need
version numbers for the file format, which might be expected to change
less often than Cabal package versions.

Reducing the need for setup scripts involves identifying the missing
bits people are working around and implementing general facilities.
Eliminating the boilerplate scripts will help, if only by flagging the
packages of interest.

If we're happy with the idea of a wrapper script running another
Haskell program, we could also have a different optional program for
each function, instead of always calling setup -- a bit like hooks at the
program level.  This would make it easier to see what special treatment a
package needs, and for the wrapper to do the simple thing in other cases.
For example, a package might need a hook for testing, so that testing
might be brittle, but everything else could be handled by the wrapper.

I imagine that major changes to the interface weren't what Isaac had
in mind when he asked for stabilization ideas, but I think downplaying
Setup.lhs is the best bet for longer term stability.

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: brainstorming ways to stabalize Cabal interface?

Isaac Jones-2
Ross Paterson <[hidden email]> writes:

> On Thu, Jan 19, 2006 at 09:35:32AM +0000, Duncan Coutts wrote:
>> On Wed, 2006-01-18 at 22:57 -0800, Isaac Jones wrote:
>> > I'm keen to try to stabilize more of the Cabal interface,
>> [...]
>> The problem as many people have noted is that Haskell source code in the
>> form of Setup.lhs is quite brittle in the face of changes to the Cabal
>> API and the .cabal file format.
>
> Indeed.  I think the best hope for stability is to make setup scripts
> optional, and to seek to reduce the situations where they're needed.
> The vast majority of scripts are
>
> main = defaultMain
> or
> main = defaultMainWithHooks defaultUserHooks

Of course, the setup scripts are only a problem in the situations
where they're needed, not in these two situations.

> (Incidentally hackage and packages repository contain very few
> exceptions to this; it seems Gentoo has a much larger collection.)
> If we were to rely on an external tool like Duncan's hs-pkg, we'd need
> a field saying which of these (or future ones) to use.  We'd also need
> version numbers for the file format, which might be expected to change
> less often than Cabal package versions.
>
> Reducing the need for setup scripts involves identifying the missing
> bits people are working around and implementing general facilities.
> Eliminating the boilerplate scripts will help, if only by flagging the
> packages of interest.

The missing bits of cabal?  I disagree.  I don't think we should
eliminate the setup scripts.  There will always be special things that
folks need in their build system, and they will want a general
facility for them, such as the setup script or the multiple scripts
you mentioned.

I don't think that continuing to make cabal more complex is the
answer.  There are definitely some things it should do that it doesn't
do now, but if we abandon the setup scripts, then we will have to
always put everything into cabal itself.

We should instead try to stabalize the cabal interface and put the
complexity into the layered tools.

Stable hooks should solve the problem of the Setup scripts not
compiling, and having a cabal-version field and a more flexible parser
should solve the problem of new or unknown fields.  We've done hardly
any work in these directions.  I think we should at least try these
approaches.


peace,

  isaac
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries