simplifying user hooks (non-backward-compatible)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

simplifying user hooks (non-backward-compatible)

Isaac Jones-2
Andres Loeh proposed simplifying the hooks interface now that they've
grown to include not only pre-and-post hooks, but also er, "during"
hooks too.  That is, all of the actions are hooks now (preConf,
confHooks, postConf).

http://hackage.haskell.org/cgi-bin/trac/trac.cgi/ticket/23

So we could now get rid of the pre and post hooks, as long as the user
could call to the default hooks thusly:

myHooks = defaultUserHooks { confHook = myConf }
myConf pd cf = do myPreConf
                  lbi <- confHook defaultUserHooks
                  myPostConf pd cf lbi

(for suitable myPreConf and myPostConf functions)

This would greatly simplify the Distribution.Simple.UserHooks
structure bringing it from 34 fields to 14 fields, or so.

Downsides: Making pre and post hooks would now be slightly harder, and
it would break existing hooks-using code.

I like it.  What are your feelings?

peace,

  isaac
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

About cabal and compatibility [Was: simplifying user hooks (non-backward-compatible)]

Bugzilla from robdockins@fastmail.fm
[snip]

> This would greatly simplify the Distribution.Simple.UserHooks
> structure bringing it from 34 fields to 14 fields, or so.
>
> Downsides: Making pre and post hooks would now be slightly harder, and
> it would break existing hooks-using code.

I just want to mention that these kind of changes represent represent a VERY
big problem. I discussed that issue at some length in an email I intended to
send to this list very recently, but it seems to have gotten lost.

So, to recap:

Suppose Alice writes a project A with a Cabal build system that relies on user
hooks?  Now the interface changes.  Bob writes a Cabal build system using the
new user hooks for project B.  Alice doesn't have time or inclination to
update her build system (it still works right? Besides, Alice wants to work
on project A; she doesn't want to rewrite her build system every time the
cabal folks have a new good idea).  Now Jow (Debian package maintainer/gentoo
haskell user/whatever) wants to build both projects A and B.  What version of
Cabal should Joe have? How does he know how to enable the correct versions
for which packages, and what versions of which packages?  Ugh! What a
nightmare.

The interesting thing is that cabal seems to solve the versioning problem for
most every library except itself!

The solution:

I suggest that the cabal team very seriously consider using the Eternal
Compatibility in Theory method to manage interface change.

http://www.haskell.org/tmrwiki/EternalCompatibilityInTheory

Rob Dockins
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: About cabal and compatibility

Isaac Jones-2
Robert Dockins <[hidden email]> writes:

> [snip]
>
>> This would greatly simplify the Distribution.Simple.UserHooks
>> structure bringing it from 34 fields to 14 fields, or so.
>>
>> Downsides: Making pre and post hooks would now be slightly harder, and
>> it would break existing hooks-using code.
>
> I just want to mention that these kind of changes represent represent a VERY
> big problem. I discussed that issue at some length in an email I intended to
> send to this list very recently, but it seems to have gotten lost.

Are you sure they represent a very big problem in practice and not
just in theory?  The hooks have been a "very experimental" part of
Cabal, as the user manual has always said:

  You can customize the simple build infrastructure using hooks. [...]
  See Distribution.Simple for the details, but note that this
  interface is experimental, and likely to change in future releases.

I only know of one or two packages that use the hooks.  Do you know of
backward compatibility problems that cabal has caused?

BTW, I just added a "cabal-version" field to cabal so that if a
package requires a particular version of cabal, it can say so.

Further, my grand plan is that once stuff gets into hackage, we can
try making modifications and seeing if it breaks any packages, and if
so, offer patches to those package authors (since it's probably eaiser
for cabal hackers to write such patches than most package authors).

(snip)
> I suggest that the cabal team very seriously consider using the Eternal
> Compatibility in Theory method to manage interface change.

I'll look at this.

peace,

  isaac
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: About cabal and compatibility

Bugzilla from robdockins@fastmail.fm
On Saturday 10 December 2005 07:14 pm, you wrote:

> Robert Dockins <[hidden email]> writes:
> > [snip]
> >
> >> This would greatly simplify the Distribution.Simple.UserHooks
> >> structure bringing it from 34 fields to 14 fields, or so.
> >>
> >> Downsides: Making pre and post hooks would now be slightly harder, and
> >> it would break existing hooks-using code.
> >
> > I just want to mention that these kind of changes represent represent a
> > VERY big problem. I discussed that issue at some length in an email I
> > intended to send to this list very recently, but it seems to have gotten
> > lost.
>
> Are you sure they represent a very big problem in practice and not
> just in theory?  
[snip]

Well, obviously it isn't a practical problem yet.  If it was we'd have people
screaming about it on the lists.

The thing that got me thinking about it was Happy.  Its recently been put in a  
darcs, and Simon Marlow mentioned to me that they are looking at moving to
Cabal for the build system.

The Happy build system now does two things that aren't your standard build
actions:

1) It creates module which contains the version string before compilation
2) After compilation it runs CPP on a number of template files with various
different "-D" options to genreate the parser templates.

I was messing around with doing these using user hooks and I ended up needing
to tie pretty deeply into cabal even for the simple task 1).

I have attached my initial attempts so you can see what's happening.

The user hooks change you are talking about would require a complete rework of
my Setup.hs file.  Its only ~130 lines, but then Happy is a pretty small
project.

> I only know of one or two packages that use the hooks.  Do you know of
> backward compatibility problems that cabal has caused?

No, this is all speculation currently.

> BTW, I just added a "cabal-version" field to cabal so that if a
> package requires a particular version of cabal, it can say so.

Well, cabal the library already has to be invoked before the package file is
even read!  If the Setup.hs doesn't typecheck because the cabal interfaces
have changed, you won't even get that far.  To make a "cabal-version" field
work you'd need some kind of bootstrapping step (which would itself need a
very stable interface).

> Further, my grand plan is that once stuff gets into hackage, we can
> try making modifications and seeing if it breaks any packages, and if
> so, offer patches to those package authors (since it's probably eaiser
> for cabal hackers to write such patches than most package authors).

That depends on the complexity of the build system.  In my experience, just
trying to figure out what a build system is doing can be pretty difficult.  I
can't imagine you want to undertake the maintaince of other people's build
systems.

OTOH, if Cabal is only aiming at pretty simple projects, this may never be an
issue.

> (snip)
>
> > I suggest that the cabal team very seriously consider using the Eternal
> > Compatibility in Theory method to manage interface change.
>
> I'll look at this.
>
> peace,
>
>   isaac

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries

Setup.hs (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: About cabal and compatibility

Isaac Jones-2
Robert Dockins <[hidden email]> writes:

> On Saturday 10 December 2005 07:14 pm, you wrote:
>> Robert Dockins <[hidden email]> writes:
(snip)
> The Happy build system now does two things that aren't your standard build
> actions:
>
> 1) It creates module which contains the version string before compilation
> 2) After compilation it runs CPP on a number of template files with various
> different "-D" options to genreate the parser templates.
>
> I was messing around with doing these using user hooks and I ended up needing
> to tie pretty deeply into cabal even for the simple task 1).

What I would like to do is to stabalize an interface one of these
days.  The problem is that, IMO, we need experience with what users
want & need in order to stabalize the interface.  When writing layered
tools like cabal-get, Lemmih had to modify cabal's interface, and when
writing cabal-install, I had to modify the interface.

But I'll definitely look at the Eternal Compatibility thing!

> I have attached my initial attempts so you can see what's happening.
>
> The user hooks change you are talking about would require a complete rework of
> my Setup.hs file.  Its only ~130 lines, but then Happy is a pretty small
> project.

Not at all.  See below.

>> I only know of one or two packages that use the hooks.  Do you know of
>> backward compatibility problems that cabal has caused?
>
> No, this is all speculation currently.
>
>> BTW, I just added a "cabal-version" field to cabal so that if a
>> package requires a particular version of cabal, it can say so.
>
> Well, cabal the library already has to be invoked before the package file is
> even read!  If the Setup.hs doesn't typecheck because the cabal interfaces
> have changed, you won't even get that far.
(snip)

True enough, It's mainly useful for parsing the .cabal file.  Setup
can't use this information for compiling, but humans can use it to
understand compile failures, and cabal-get can use it.

>> Further, my grand plan is that once stuff gets into hackage, we can
>> try making modifications and seeing if it breaks any packages, and if
>> so, offer patches to those package authors (since it's probably eaiser
>> for cabal hackers to write such patches than most package authors).
>
> That depends on the complexity of the build system.  In my experience, just
> trying to figure out what a build system is doing can be pretty difficult.  I
> can't imagine you want to undertake the maintaince of other people's build
> systems.

I'm certainly not talking about maintaining them, but usually if you
modify an interface, the kinds of changes you make are mechanical and
repetative.

> OTOH, if Cabal is only aiming at pretty simple projects, this may never be an
> issue.

The plan with cabal was to handle simple projects at first, and add
more complex projects as we go along.  That's basically happening, and
my hope is that we can fix the "simple" interface sooner, and the more
complex interface later.

> import Data.Version
> import Control.Exception
> import System.Cmd
> import System.IO
> import System.Exit
> import System.Directory
> import Distribution.Setup
> import Distribution.Simple
> import Distribution.Simple.Utils
> import Distribution.Simple.LocalBuildInfo
> import Distribution.PackageDescription

It would be nice if you used qualified imports so I could see exactly
what it is that you need from cabal.  Also you'll get better error
messages if things disappear.

> main = do
>   descFile <- defaultPackageDesc
>   desc <- readPackageDescription descFile
>   let ver = (pkgVersion (package desc))
>   defaultMainWithHooks (mkHooks desc ver)
>
> mkHooks :: PackageDescription
>         -> Version
>         -> UserHooks
> mkHooks desc ver =
>   defaultUserHooks
>   { readDesc  = return (Just desc)
>   , postConf  = doPostConf ver
>   , postBuild = createTemplates
>   , postClean = cleanDerivedFiles
>   }

With the proposed change, all you have to do is something basically
like this, not reworking the whole build system:

>   defaultUserHooks
>   { readDesc  = return (Just desc)
>   , confHook  = (confHook defaultUserHooks) >> doPostConf ver
>   , buildHook = (buildHook defaultUserHooks) >> createTemplates
>   , cleanHook = (cleanHook) >> cleanDerivedFiles
>   }

(snip)

> createVersionModule :: Version -> IO ()
> createVersionModule version = do
>   h <- openFile "src/Version.hs" WriteMode
>   hPutStr h "module Version where\n"
>   hPutStr h ("version = \""++(showVersion version)++"\"")
>   hClose h

That's interesting.  I was recently thinking of generating something
like CabalInfo.hs which has information like the PackageDescription
and the LocalBuildInfo.  You would turn on this feature w/ the .cabal
file.  I'm not sure if any of the information is useful except for the
version number, though.

We could also add -DPACKAGE_VERSION or something to the cpp line.
That would be pretty easy.

(snip)

>        perlRE = "s/^#\\s+(\\d+)\\s+(\\\"[^\\\"]*\\\")/{-# LINE \\1 \\2 #-}/g;s/\\$$(Id:.*)\\$$/\\1/g"

>   rawSystem "perl" ["-pi.hspp","-e",perlRE,file]

aaaaaaaaahhhhhhhhhhhhhhhhhhhhhhhhh!!!!!!!!!

/me runs away ;)


But seriously folks, you should check out the new (experimental!)
"Program" interface in the CVS / darcs HEAD.  The nice thing about it
is you can add perl as a Program and configure will find it, and users
can say: ./setup configure --with-perl=/my/special/perl.


peace,

  isaac
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: About cabal and compatibility

Bugzilla from robdockins@fastmail.fm
> What I would like to do is to stabalize an interface one of these
> days.  The problem is that, IMO, we need experience with what users
> want & need in order to stabalize the interface.  When writing layered
> tools like cabal-get, Lemmih had to modify cabal's interface, and when
> writing cabal-install, I had to modify the interface.
>
> But I'll definitely look at the Eternal Compatibility thing!

That's good to hear.  I think it is a good fit for this situation.

> > import Data.Version
> > import Control.Exception
> > import System.Cmd
> > import System.IO
> > import System.Exit
> > import System.Directory
> > import Distribution.Setup
> > import Distribution.Simple
> > import Distribution.Simple.Utils
> > import Distribution.Simple.LocalBuildInfo
> > import Distribution.PackageDescription
>
> It would be nice if you used qualified imports so I could see exactly
> what it is that you need from cabal.  Also you'll get better error
> messages if things disappear.

I'm not sure what you mean here.  Do you mean with explicit import lists?  
Qualified means something very specific in this context and I don't see how
it would help.

> >        perlRE = "s/^#\\s+(\\d+)\\s+(\\\"[^\\\"]*\\\")/{-# LINE \\1 \\2
> > #-}/g;s/\\$$(Id:.*)\\$$/\\1/g"
> >
> >   rawSystem "perl" ["-pi.hspp","-e",perlRE,file]
>
> aaaaaaaaahhhhhhhhhhhhhhhhhhhhhhhhh!!!!!!!!!
>
> /me runs away ;)

Just so you don't think I'm responsable for that perl RE mess, its taken
directly from the Makefile ;)  Yeah, expecting perl on the path is a little
ugly but *shrug* its a pretty safe assumption (also made in the Makefile
BTW).

> But seriously folks, you should check out the new (experimental!)
> "Program" interface in the CVS / darcs HEAD.  The nice thing about it
> is you can add perl as a Program and configure will find it, and users
> can say: ./setup configure --with-perl=/my/special/perl.

That sounds nice; I'll have to take a look.

Rob Dockins
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: About cabal and compatibility

Simon Marlow
In reply to this post by Bugzilla from robdockins@fastmail.fm
On 11 December 2005 00:38, Robert Dockins wrote:

> The thing that got me thinking about it was Happy.  Its recently been
> put in a darcs, and Simon Marlow mentioned to me that they are
> looking at moving to Cabal for the build system.
>
> The Happy build system now does two things that aren't your standard
> build actions:
>
> 1) It creates module which contains the version string before
> compilation 2) After compilation it runs CPP on a number of template
> files with various different "-D" options to genreate the parser
> templates.

BTW, I recently Cabalised Happy.  Get it from the darcs repo:

  darcs get http://darcs.haskell.org/happy

Sorry you duplicated this work, I should have announced something, but I
was holding off because it needed a very recent version of Cabal.

Cheers,
        Simon
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: About cabal and compatibility

Simon Marlow
In reply to this post by Bugzilla from robdockins@fastmail.fm
On 11 December 2005 01:46, Isaac Jones wrote:

> Robert Dockins <[hidden email]> writes:

>> createVersionModule :: Version -> IO ()
>> createVersionModule version = do
>>   h <- openFile "src/Version.hs" WriteMode
>>   hPutStr h "module Version where\n"
>>   hPutStr h ("version = \""++(showVersion version)++"\"")
>>   hClose h
>
> That's interesting.  I was recently thinking of generating something
> like CabalInfo.hs which has information like the PackageDescription
> and the LocalBuildInfo.  You would turn on this feature w/ the .cabal
> file.  I'm not sure if any of the information is useful except for the
> version number, though.

This isn't necessary: you can get the version number from Paths_<pkg>
(it's an undocumented feature :-).  See the Cabalised Happy for an
example.  I should probably document it.

Cheers,
        Simon
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: simplifying user hooks (non-backward-compatible)

Isaac Jones-2
In reply to this post by Isaac Jones-2
Following up on this thread...

I modified the hooks interface quite a bit, again.  There's good news
and bad news about this.  The good news is that it's cleaned up and
should be easier to maintain and to avoid future modifications.  The
bad news is that this change itself will break stuff, of course.

If you have any trouble building your Setup scripts, please let me
know.  I really think that it was best to bite the bullet right now in
one big go instead of down the road with lots of little changes.  I
have a lot more confidence in the hooks interface, and I don't
actually expect that it'll change as often.

I made the types more consistent, and made sure there are accessor
functions on each of the Flags types so that if the flags types change
in the future, it shouldn't break lots of code.

Another piece of good / bad news is that I decided not to get rid of
the pre & post hooks.  They are nice for convenience and it wouldn't
be nearly so easy to write hooks without them.

That's bad because the interface to hooks is still pretty big, which
means that there's more likelihood that it'll change in the future.

Another weakness in the Hooks interface is that with command hooks
(like sDistHook) it's tempting to add parameters to them; basically
the stuff that we compute between the preSDist and sDist hook.  I
removed such params and have their values computed elsewhere.

Cabal hackers, please avoid adding parameters to these command hooks
if at all possible in order to keep the interface steady.  If you need
to compute a value to pass to these functions, compute it in the
function and / or make it available as a function that someone
crafting hooks can use as well, or consider whether it belongs in one
of the parameters already being passed to the hooks,
PackageDescription, LocalBuildInfo, UserHooks, Flags.

throwing myself on your mercy,

  isaac
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries