Cabal and autogenerated files

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Cabal and autogenerated files

Jeremy O'Donoghue
Hi all,

Another, probably simple, question regarding cabalization.

Part of wxcore, the low level abstraction in wxHaskell, consists of
haskell modules which are generated automatically by parsing C headers
using another tool, wxdirect.

When trying to create an sdist package, we run into the problem that
because we export modules which are automatically generated, after a
'clean', they do not exist, so the sdist package generation fails.

We have tried using 'extra-tmp-files' to list the modules which are
autogenerated, but this isn't working.

Is this because we are generating the autogen modules in an autogen
directory, or is this approach likely to fail wherever we put the
autogenerated files? Our use case seems a reasonable one, as it could
reasonably exist for any project which relies on automatically
generated code.

Thanks for any suggestions.
Jeremy
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Cabal and autogenerated files

Duncan Coutts-4
On Thu, 2009-11-12 at 17:54 +0000, Jeremy O'Donoghue wrote:

> Hi all,
>
> Another, probably simple, question regarding cabalization.
>
> Part of wxcore, the low level abstraction in wxHaskell, consists of
> haskell modules which are generated automatically by parsing C headers
> using another tool, wxdirect.
>
> When trying to create an sdist package, we run into the problem that
> because we export modules which are automatically generated, after a
> 'clean', they do not exist, so the sdist package generation fails.

So I take it that these modules are generated "from nothing" rather than
something like happy/alex pre-processors where the .hs files are
generated from .y/.x files. Cabal supports the latter fairly well and
you can add custom pre-processors in this style. It doesn't really
support generating modules out of thin-air in the build step (though it
does this internally for the Paths_pkgname module).

> We have tried using 'extra-tmp-files' to list the modules which are
> autogenerated, but this isn't working.

extra-tmp-files means rm these files too when cleaning. It's mostly
redundant since the better thing to do is to keep all temp build files
in the build directory.

> Is this because we are generating the autogen modules in an autogen
> directory, or is this approach likely to fail wherever we put the
> autogenerated files? Our use case seems a reasonable one, as it could
> reasonably exist for any project which relies on automatically
> generated code.

Tell me if I've misunderstood your problem. We'll have to think about
how to get sdist to do the right thing for modules that have no ultimate
corresponding source file. File a ticket with your description of your
use case and what you'd like to be able to do.

Duncan

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Cabal and autogenerated files

John Millikin
In reply to this post by Jeremy O'Donoghue
I've had some luck with two techniques for this:

1. Create "stub" files, associated with a custom preprocessor which
knows how to parse them and generate a Haskell module. For example,
you might have "Foo.wx-stub" contain:

    [headers]
    wx/foo.h
    wx/otherheader.h

and then parse it into parameters for wxdirect. That allows Cabal to
generate the appropriate files automatically.

Downside: I've not figured out how to get proper dependency resolution
for this, so occasionally a clean and rebuild is required.

2. Use a Makefile to generate the files, placing them in a
subdirectory added to Cabal's search path. This works well for pretty
much anything, since Make is a bit more mature than Cabal's build
system, but having to remember the pre-processing step is annoying.

On Thu, Nov 12, 2009 at 09:54, Jeremy O'Donoghue
<[hidden email]> wrote:

> Hi all,
>
> Another, probably simple, question regarding cabalization.
>
> Part of wxcore, the low level abstraction in wxHaskell, consists of
> haskell modules which are generated automatically by parsing C headers
> using another tool, wxdirect.
>
> When trying to create an sdist package, we run into the problem that
> because we export modules which are automatically generated, after a
> 'clean', they do not exist, so the sdist package generation fails.
>
> We have tried using 'extra-tmp-files' to list the modules which are
> autogenerated, but this isn't working.
>
> Is this because we are generating the autogen modules in an autogen
> directory, or is this approach likely to fail wherever we put the
> autogenerated files? Our use case seems a reasonable one, as it could
> reasonably exist for any project which relies on automatically
> generated code.
>
> Thanks for any suggestions.
> Jeremy
> _______________________________________________
> Haskell-Cafe mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Cabal and autogenerated files

Jeremy O'Donoghue
In reply to this post by Duncan Coutts-4


On Thu, 12 Nov 2009 18:16 +0000, "Duncan Coutts"
<[hidden email]> wrote:

> On Thu, 2009-11-12 at 17:54 +0000, Jeremy O'Donoghue wrote:
> > Hi all,
> >
> > Another, probably simple, question regarding cabalization.
> >
> > Part of wxcore, the low level abstraction in wxHaskell, consists of
> > haskell modules which are generated automatically by parsing C headers
> > using another tool, wxdirect.
> >
> > When trying to create an sdist package, we run into the problem that
> > because we export modules which are automatically generated, after a
> > 'clean', they do not exist, so the sdist package generation fails.
>
> So I take it that these modules are generated "from nothing" rather than
> something like happy/alex pre-processors where the .hs files are
> generated from .y/.x files. Cabal supports the latter fairly well and
> you can add custom pre-processors in this style. It doesn't really
> support generating modules out of thin-air in the build step (though it
> does this internally for the Paths_pkgname module).

They don't come from nothing as they are derived from a C header file
and an Eiffel interface (for historical reasons).

It sounds as though the custom preprocessor approach is closest to what
we have/need.

We have a custom config hook containing the following:

myConfHook :: (Either GenericPackageDescription PackageDescription,
HookedBuildInfo) -> ConfigFlags -> IO LocalBuildInfo
myConfHook (pkg0, pbi) flags = do
    createDirectoryIfMissing True wxcoreDirectory
    system $ "wxdirect -t --wxc " ++ sourceDirectory ++ " -o " ++
    wxcoreDirectory ++ " " ++ wxcoreIncludeFile
    system $ "wxdirect -i --wxc " ++ sourceDirectory ++ " -o " ++
    wxcoreDirectory ++ " " ++ wxcoreIncludeFile
    system $ "wxdirect -c --wxc " ++ sourceDirectory ++ " -o " ++
    wxcoreDirectory ++ " " ++ wxcoreIncludeFile
    system $ "wxdirect -d --wxc " ++ sourceDirectory ++ " -o " ++
    wxcoreDirectory ++ " " ++ intercalate " " eiffelFiles

In this case, wxcoreDirectory resolves to
"dist/build/autogen/Graphics/UI/WXCore".

The calls to wxdirect generate a number of haskell files as output
(currently in dist/build/autogen).

Regards
Jeremy
--
  Jeremy O'Donoghue
  [hidden email]

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Cabal and autogenerated files

Duncan Coutts-4
On Fri, 2009-11-13 at 15:24 +0000, Jeremy O'Donoghue wrote:

> > So I take it that these modules are generated "from nothing" rather than
> > something like happy/alex pre-processors where the .hs files are
> > generated from .y/.x files. Cabal supports the latter fairly well and
> > you can add custom pre-processors in this style. It doesn't really
> > support generating modules out of thin-air in the build step (though it
> > does this internally for the Paths_pkgname module).
>
> They don't come from nothing as they are derived from a C header file
> and an Eiffel interface (for historical reasons).

Ok, but not files in a 1:1 correspondence with the modules you generate.
It's more like a bunch of files used to generate another bunch of files,
with no simple naming convention to relate the two.

> It sounds as though the custom preprocessor approach is closest to what
> we have/need.

Cabal's simple notion of preprocessor really only covers the case where
you've got:

Foo/Bar.hs generated from Foo/Bar.some-other-extension

The pre-processors are registered by the extension and the Cabal
pre-processing step decides what pre-processors to run by looking for
files with known extensions.

> We have a custom config hook containing the following:
>
> myConfHook :: (Either GenericPackageDescription PackageDescription,
> HookedBuildInfo) -> ConfigFlags -> IO LocalBuildInfo
> myConfHook (pkg0, pbi) flags = do
>     createDirectoryIfMissing True wxcoreDirectory
>     system $ "wxdirect -t --wxc " ++ sourceDirectory ++ " -o " ++
>     wxcoreDirectory ++ " " ++ wxcoreIncludeFile
>     system $ "wxdirect -i --wxc " ++ sourceDirectory ++ " -o " ++
>     wxcoreDirectory ++ " " ++ wxcoreIncludeFile
>     system $ "wxdirect -c --wxc " ++ sourceDirectory ++ " -o " ++
>     wxcoreDirectory ++ " " ++ wxcoreIncludeFile
>     system $ "wxdirect -d --wxc " ++ sourceDirectory ++ " -o " ++
>     wxcoreDirectory ++ " " ++ intercalate " " eiffelFiles
>
> In this case, wxcoreDirectory resolves to
> "dist/build/autogen/Graphics/UI/WXCore".
>
> The calls to wxdirect generate a number of haskell files as output
> (currently in dist/build/autogen).
>
> Regards
> Jeremy

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe