1-to-many preprocessor in stack/Cabal builds

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

1-to-many preprocessor in stack/Cabal builds

Andrey Sverdlichenko
Hi,

I'm writing a package which uses custom preprocessor for creating some of the source files. It takes one IDL file as input and writes several .hs modules, depending on IDL contents. I need to compile this modules with a few normal ones and link them as a library, preferably with generated modules exposed. It doesn't look like a complex task and never was in with make/etc, but I run into some problems now.

Obviously, cabal supports 1-to-1 preprocessors to create one Haskell module from one source file (alex, happy, etc), but I can't find anything for 1-to-many preprocessors. So, I decided to use hookedPrograms and hooks and do it myself. The call to preprocessor in the postConf hook worked just fine and created source files and module list.

Cabal pre-* hooks seems to be right place to read this module list and use it, but are presenting two problems: can only modify other-modules, not exposed-modules, which is tolerable, and there is no access to build configuration. The latter is fatal, I need to know build dir to read module list. So I have to change action hooks.
As the additional annoyance, there is absolutely no way to modify configuration once and for all, I have to override every single hook with the same "read module list and change config" wrapper.

So, I have buildHook, copyHook and regHook replaced with wrappers which change PackageDescription parameter and call original function. Library is built and installed, but somewhere between build and copy actions there is still a message "Warning: modules not listed in foo.cabal for library component (add to other-modules)". I can't identify where it comes from.

I'm using stack 1.0.4 to build this library, but all the problems seems to be cabal-related. Here is minimal example of what I ended up with: https://github.com/rblaze/cabal-hooks-sample

Now I wonder if I missed an easier way to handle this code generation. If not, at least how to get rid of warning about missing modules between build and copy actions?

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: 1-to-many preprocessor in stack/Cabal builds

Edward Z. Yang
Hello Andrey,

I do not know what the right way to sovle your problem is, but if it is
indeed the case that this cannot be done with Cabal, speaking as a Cabal
developer, we definitely should try to fix this.  A bug report
would be much appreciated.

> So, I have buildHook, copyHook and regHook replaced with wrappers which
> change PackageDescription parameter and call original function. Library is
> built and installed, but somewhere between build and copy actions there is
> still a message "Warning: modules not listed in foo.cabal for library
> component (add to other-modules)". I can't identify where it comes from.

I'm pretty sure this message comes from Stack; src/Stack/Types/Package.hs.

Cheers,
Edward
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: 1-to-many preprocessor in stack/Cabal builds

Andrey Sverdlichenko
Hi Edward,

I can do with current Cabal, it is just a lot of boilerplate and inconvenience.
What I would like to have as an ideal solution for my problem is a preprocessor interface similar to PPSuffixHandler, but with extra capabilities.
It should receive one file as an input, but be able to generate multiple modules with the names of its own choosing. And I want the resulting modules to be added to all build steps automatically. So it should looks approximately like this:

data SmartPreProcessor = SmartPreProcessor {
    platformIndependent :: Bool,
    runPreProcessor :: (FilePath, FilePath) -- Location of the source file relative to a base dir
                    -> Verbosity            -- verbosity
                    -> FilePath             -- output base dir
                    -> IO [FilePath]        -- output sources files. Should exit if the preprocessor fails
}

There may be design objections to such preprocessor, but again, this is an ideal solution to my problem.

On Sun, Mar 6, 2016 at 11:55 PM Edward Z. Yang <[hidden email]> wrote:
Hello Andrey,

I do not know what the right way to sovle your problem is, but if it is
indeed the case that this cannot be done with Cabal, speaking as a Cabal
developer, we definitely should try to fix this.  A bug report
would be much appreciated.

> So, I have buildHook, copyHook and regHook replaced with wrappers which
> change PackageDescription parameter and call original function. Library is
> built and installed, but somewhere between build and copy actions there is
> still a message "Warning: modules not listed in foo.cabal for library
> component (add to other-modules)". I can't identify where it comes from.

I'm pretty sure this message comes from Stack; src/Stack/Types/Package.hs.

Cheers,
Edward

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: 1-to-many preprocessor in stack/Cabal builds

Edward Z. Yang
OK.

If there is a simple patch which can help support your use case better,
we'd be happy to take it. However, I don't think a design like this is
the right long term plan. The correct long term plan, I think, is to do
something like ToolCabal <https://github.com/TiborIntelSoft/ToolCabal>
and make it so that you can just plug in custom build rules of your own,
which can include preprocessor processing.  This will also fix some of
our other problems, such as the fact that we repeatedly preprocess
executables even when it's not necessary
<https://github.com/haskell/cabal/issues/2432>

I'm hoping to take a crack at this at some point; hopefully your
boilerplate will tide you until then.

Edward

Excerpts from Andrey Sverdlichenko's message of 2016-03-07 10:45:56 -0800:

> Hi Edward,
>
> I can do with current Cabal, it is just a lot of boilerplate and
> inconvenience.
> What I would like to have as an ideal solution for my problem is a
> preprocessor interface similar to PPSuffixHandler, but with extra
> capabilities.
> It should receive one file as an input, but be able to generate multiple
> modules with the names of its own choosing. And I want the resulting
> modules to be added to all build steps automatically. So it should looks
> approximately like this:
>
> data SmartPreProcessor = SmartPreProcessor {
>     platformIndependent :: Bool,
>     runPreProcessor :: (FilePath, FilePath) -- Location of the source file
> relative to a base dir
>                     -> Verbosity            -- verbosity
>                     -> FilePath             -- output base dir
>                     -> IO [FilePath]        -- output sources files. Should
> exit if the preprocessor fails
> }
>
> There may be design objections to such preprocessor, but again, this is an
> ideal solution to my problem.
>
> On Sun, Mar 6, 2016 at 11:55 PM Edward Z. Yang <[hidden email]> wrote:
>
> > Hello Andrey,
> >
> > I do not know what the right way to sovle your problem is, but if it is
> > indeed the case that this cannot be done with Cabal, speaking as a Cabal
> > developer, we definitely should try to fix this.  A bug report
> > would be much appreciated.
> >
> > > So, I have buildHook, copyHook and regHook replaced with wrappers which
> > > change PackageDescription parameter and call original function. Library
> > is
> > > built and installed, but somewhere between build and copy actions there
> > is
> > > still a message "Warning: modules not listed in foo.cabal for library
> > > component (add to other-modules)". I can't identify where it comes from.
> >
> > I'm pretty sure this message comes from Stack; src/Stack/Types/Package.hs.
> >
> > Cheers,
> > Edward
> >
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe