ANNOUNCE: Blueprint 0.1 -- PREVIEW

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

ANNOUNCE: Blueprint 0.1 -- PREVIEW

Gregory Crosswhite-2
I earlier posted to this list asking for help in getting Cabal to play with a Fortran compiler.  When it turned out that there was no answer to this, I of course did the obvious thing --- I wrote my own build system.  :-)

It already boasts the following features:

        * Fully self-hosting --- can both build and install itself.

        * Implicitly detects and tracks module dependencies.

        * Builds in parallel as much as possible.

        * Can scan through a source directory to add all files, saving the trouble of adding them manually.

        * Supports compiling C and Fortran sources in addition to Haskell sources, and linking everything together.

        * Utilizes a “tool” system to make it easy to extend its functionality, such as by adding a new language.

        * Flexible configuration system:  tool options (e.g., path to the Fortran compiler) can be detected automatically, set in a configuration file, or specified on the command line.

        * Provides help messages both to list the targets and to explain the configuration command-line options.

        * Does not terminate at the first error, but instead tries to do as much as possible and then report all errors.  (This is a general philosophy that holds both for the build, the configuration, etc.)

        * Easily supports multiple targets that can depend on the results from other targets.

        * Automatically resolves package dependencies, as read from .cabal file.  Detects when a module is not in any known dependency, and reports an error that also lists all packages that export this module.

What separates this system apart from others is that it uses Haskell’s built-in laziness to handle dependency ordering and parallelism.  That is, one expresses the desired build result as a function of the input sources;  when the result is demanded, it causes Haskell to go through and execute on-demand all of the necessary build steps.  To get parallelism, just tell GHC to use multiple threads with the “+RTS -N# -RTS” option, where # is the number of threads.  (All that I had to do to get this to work was put parLists in a couple of places.)  This is a big win for me because it means that I get the correct behavior without having to write and test my own complicated graph data structure for tracking the dependencies between build steps.

The philosophy of building up a big calculation of which only parts need be lazily demanded can be extended further to the build phases, which I call “targets”.  For example, the first line of the code defining the build target is:

        build = configure >>= \configuration -> ...

This demands the configuration target, and then uses the result in the build.  So when you run “./Setup configure”, you only demand that Setup compute the value of “configure”, but when you run “./Setup build” you demand both “build” and also “configure”.

All of the errors that are reported are “composable” in the sense that they can be combined into a new error message that leaves nothing out and also has no duplicates.  As much as possible the system tries not to stop until it has nothing else that it can do, allowing it to uncover as many errors as possible and then compose them into a single message.

I have posted Blueprint to Hackage so that people can see what I have done and possibly play with it.  I hope to eventually document it and add examples, but I figured it was better to release early and get it out there sooner rather than later so that people can see what I’ve done.  Thus, at this point it should be considered a PREVIEW release, not even an alpha release.  Having said that, within the domain of the functionality it implements it works well enough that I use it myself as the build system for a Physics simulation code that is a mixture of C, Haskell, and Fortran.

To see what Blueprint looks like in action, take a look at Setup.hs.  For example, Setup can either build a library (“build” target) or it can build itself (“bootstrap” target);  it is smart enough to use different flags and directories in the two cases so that there ends up being in essence two parallel builds.  Setup also has a lot of targets:  configure, reconfigure (delete cached configuration and then configure), build, rebuild (clean then build), clean, distclean, install, bootstrap (build itself rather than a library), haddock (documentation generation that does not work particularly well yet).

Anyway, take a look if any of this sounds interesting to you, and let me know if you have any questions!

- Greg

PS:  You're best bet for browsing through the sources is to use Leo (leo.sourceforge.net), since this makes it easier for you to see how everything is organized._______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANNOUNCE: Blueprint 0.1 -- PREVIEW

Ben Franksen
Gregory Crosswhite wrote:
> I have posted Blueprint to Hackage so that people can see what I have done
> and possibly play with it.

Very interesting, this. However, I could not build it. I get

ben@sarun[2]: ~/tmp > cabal install blueprint    
Resolving dependencies...
cabal: There is no installed version of base

Needless to say this is wrong:

ben@sarun[2]: ~/tmp > ghc-pkg list base      
/usr/local/lib/ghc-6.10.4/./package.conf:
    base-3.0.3.1, base-4.1.0.0
/home/ben/.ghc/i386-linux-6.10.4/package.conf:

This has not happened to me with any other packages from hackage.

Cheers
Ben

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: ANNOUNCE: Blueprint 0.1 -- PREVIEW

Gregory Crosswhite-2
Yes, that is because at this time Blueprint is presently of a lower quality than other packages on Hackage.  ;-)  At the moment you need to execute the setup script manually:

        runhaskell Setup.hs bootstrap
        ./Setup configure
        ./Setup build +RTS -N4 -RTS
        ./Setup install

(The "+RTS -N4 -RTS" part is optional;  it just tells Setup to use up to 4 threads to do building in parallel where possible.)

I have known about the problem that you just mentioned --- that is, although running "cabal configure" and "cabal build" in the build directory works just fine, "cabal install" does not work ---  but because my .cabal file looked fine I assumed until now that it was just a quirk of the "cabal" program.  I only just now figured out that to make "cabal install" work I need to make sure that the build dependencies are listed in the Library section rather than only in the main section, as otherwise cabal refuses to simply run "Setup install" even though that actually takes care of everything for it!

Anyway, thank you for checking out Blueprint!  At this point it might be better to pull the sources directly from github (the "Home Page" link) since I have made many improvements since that release (though I haven't fixed the "cabal install" problem yet).  And I do plan on thoroughly polishing and documenting Blueprint one of these days.  :-)

Cheers,
Greg


On Dec 4, 2009, at 11:02 AM, Ben Franksen wrote:

> Gregory Crosswhite wrote:
>> I have posted Blueprint to Hackage so that people can see what I have done
>> and possibly play with it.
>
> Very interesting, this. However, I could not build it. I get
>
> ben@sarun[2]: ~/tmp > cabal install blueprint    
> Resolving dependencies...
> cabal: There is no installed version of base
>
> Needless to say this is wrong:
>
> ben@sarun[2]: ~/tmp > ghc-pkg list base      
> /usr/local/lib/ghc-6.10.4/./package.conf:
>    base-3.0.3.1, base-4.1.0.0
> /home/ben/.ghc/i386-linux-6.10.4/package.conf:
>
> This has not happened to me with any other packages from hackage.
>
> Cheers
> Ben
>
> _______________________________________________
> Haskell-Cafe mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: ANNOUNCE: Blueprint 0.1 -- PREVIEW

Ben Franksen
Gregory Crosswhite wrote:

> On Dec 4, 2009, at 11:02 AM, Ben Franksen wrote:
>> Gregory Crosswhite wrote:
>>> I have posted Blueprint to Hackage so that people can see what I have
>>> done and possibly play with it.
>> Very interesting, this. However, I could not build it. I get [...]
> At the moment you need to execute the setup script manually:
>
> runhaskell Setup.hs bootstrap
> ./Setup configure
> ./Setup build +RTS -N4 -RTS
> ./Setup install
>
> (The "+RTS -N4 -RTS" part is optional;  it just tells Setup to use up to 4
> threads to do building in parallel where possible.)

Thanks. I recommend adding a small readme.txt explaining stuff like that.

> Anyway, thank you for checking out Blueprint!  At this point it might be
> better to pull the sources directly from github (the "Home Page" link)

Will do.

> since I have made many improvements since that release (though I haven't
> fixed the "cabal install" problem yet).  And I do plan on thoroughly
> polishing and documenting Blueprint one of these days.  :-)

Thanks & Cheers
Ben

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe