Relocatable dist

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Relocatable dist

Moritz Angermann-2
Hi,

while trying to make the binary-distribution logic work for
cross compilers.  I've come wonder, how hard it could be to
make GHC relocatable. (e.g. just unpack the distribution,
and use the compiler from the `bin` folder within).

Right now this does not work due to the need for the path to
package.conf, and this is hardcoded in the wrapper script to
provide the proper libdir to ghc via -B[1]. Supposedly this
is not an issue on Windows, as a relative path is common on
windows and finding the location of the executable can be done
safely. Or that's at least how I understand[1].

For macOS there is the haskell-platform, and ghc-dot-app[2]

From [3], it sounds like automake is a build, and not a packaging
system, and the binary dist usecase with configure is not
really a standard use case.

So why do I bring this up NOW? Apart from me trying to make
cross compiler binary distributions working?  The reason is
that we are also trying to move towards hadrian, and by "starting
from scratch", maybe we have a chance to reconsider how we
do things?

I must admit, I'm quite happy to use packages like llvm, by
just downloading a package and adding the `bin` path to my
$PATH.

There is however one thing that the current configure appraoch
does, which I think is quite noteworthy (apart from setting
$prefix).  That is, it does check for compilers and sets them
accordingly, which might be desirable.

Cheers,
 Moritz

--
[1]: https://ghc.haskell.org/trac/ghc/wiki/Building/Installing
[2]: https://ghc.haskell.org/trac/ghc/wiki/Building/MacOSX/Installer
[3]: https://lists.gnu.org/archive/html/automake/2006-10/msg00005.html
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Joachim Breitner-2
Hi,

Am Montag, den 23.10.2017, 14:04 +0800 schrieb Moritz Angermann:
> I've come wonder, how hard it could be to
> make GHC relocatable. (e.g. just unpack the distribution,
> and use the compiler from the `bin` folder within).

Yes yes yes please!

(Sorry for not contributing anything but motivation.)

Joachim


--
Joachim Breitner
  [hidden email]
  http://www.joachim-breitner.de/

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Manuel M T Chakravarty-4
In reply to this post by Moritz Angermann-2
Hi Moritz,

Haskell for Mac is fully relocatable (as every Mac users expects this of a Mac app).

Unfortunately, it is not just the libdir in the wrapper scripts that’s in the way of a relocatable GHC. Additional problems are (1) the (at least, as of 8.0) still not macOS-friendly RPATHs in dynamic libraries and (2) the absolute paths in package .conf files.

In Haskell for Mac, I use two arguably hacky solutions to those two issues. I rewrite all RPATHs in .dylibs to be relative and I have got a little tool that relocates .conf files. I’d certainly prefer a cleaner solution!

Those aspects of Haskell for Mac are actually in a separate framework (GHC.framework) that wraps GHC and some other tools. This is all in a public repo if you like to have a look


I haven’t put a license on it, but I’d be happy to make it all BSD3. It is of course all highly Mac specific and requires Xcode to build for all the code signing, sandboxing, etc.

Cheers,
Manuel

Am 23.10.2017 um 17:04 schrieb Moritz Angermann <[hidden email]>:

Hi,

while trying to make the binary-distribution logic work for
cross compilers.  I've come wonder, how hard it could be to
make GHC relocatable. (e.g. just unpack the distribution,
and use the compiler from the `bin` folder within).

Right now this does not work due to the need for the path to
package.conf, and this is hardcoded in the wrapper script to
provide the proper libdir to ghc via -B[1]. Supposedly this
is not an issue on Windows, as a relative path is common on
windows and finding the location of the executable can be done
safely. Or that's at least how I understand[1].

For macOS there is the haskell-platform, and ghc-dot-app[2]

From [3], it sounds like automake is a build, and not a packaging
system, and the binary dist usecase with configure is not
really a standard use case.

So why do I bring this up NOW? Apart from me trying to make
cross compiler binary distributions working?  The reason is
that we are also trying to move towards hadrian, and by "starting
from scratch", maybe we have a chance to reconsider how we
do things?

I must admit, I'm quite happy to use packages like llvm, by
just downloading a package and adding the `bin` path to my
$PATH.

There is however one thing that the current configure appraoch
does, which I think is quite noteworthy (apart from setting
$prefix).  That is, it does check for compilers and sets them
accordingly, which might be desirable.

Cheers,
Moritz

--
[1]: https://ghc.haskell.org/trac/ghc/wiki/Building/Installing
[2]: https://ghc.haskell.org/trac/ghc/wiki/Building/MacOSX/Installer
[3]: https://lists.gnu.org/archive/html/automake/2006-10/msg00005.html
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Moritz Angermann-2
Hi Manuel,

thanks for the link!  Ahh yes lovely RPATH (this reminds me, I should look
to verify if I can make GHC link recursively.)  The conf files came up on
#ghc yesterday as well. Maybe we need some $home, $sysroot or similar marker
to make them relocatable.

I'll probably take a crack at this today.  Not sure how far I get though.
The primary motivation is that packaging the cross compilers in a neat
way looks like layering patched upon patches onto the configure script. And
in light of Hadrian, we might just want to package up a directory and not
deal with the binary-dist logic where we can?

Cheers,
 Moritz

> On Oct 24, 2017, at 9:45 AM, Manuel M T Chakravarty <[hidden email]> wrote:
>
> Hi Moritz,
>
> Haskell for Mac is fully relocatable (as every Mac users expects this of a Mac app).
>
> Unfortunately, it is not just the libdir in the wrapper scripts that’s in the way of a relocatable GHC. Additional problems are (1) the (at least, as of 8.0) still not macOS-friendly RPATHs in dynamic libraries and (2) the absolute paths in package .conf files.
>
> In Haskell for Mac, I use two arguably hacky solutions to those two issues. I rewrite all RPATHs in .dylibs to be relative and I have got a little tool that relocates .conf files. I’d certainly prefer a cleaner solution!
>
> Those aspects of Haskell for Mac are actually in a separate framework (GHC.framework) that wraps GHC and some other tools. This is all in a public repo if you like to have a look
>
>   https://github.com/haskellformac/GHCframework
>
> I haven’t put a license on it, but I’d be happy to make it all BSD3. It is of course all highly Mac specific and requires Xcode to build for all the code signing, sandboxing, etc.
>
> Cheers,
> Manuel
>
>> Am 23.10.2017 um 17:04 schrieb Moritz Angermann <[hidden email]>:
>>
>> Hi,
>>
>> while trying to make the binary-distribution logic work for
>> cross compilers.  I've come wonder, how hard it could be to
>> make GHC relocatable. (e.g. just unpack the distribution,
>> and use the compiler from the `bin` folder within).
>>
>> Right now this does not work due to the need for the path to
>> package.conf, and this is hardcoded in the wrapper script to
>> provide the proper libdir to ghc via -B[1]. Supposedly this
>> is not an issue on Windows, as a relative path is common on
>> windows and finding the location of the executable can be done
>> safely. Or that's at least how I understand[1].
>>
>> For macOS there is the haskell-platform, and ghc-dot-app[2]
>>
>> From [3], it sounds like automake is a build, and not a packaging
>> system, and the binary dist usecase with configure is not
>> really a standard use case.
>>
>> So why do I bring this up NOW? Apart from me trying to make
>> cross compiler binary distributions working?  The reason is
>> that we are also trying to move towards hadrian, and by "starting
>> from scratch", maybe we have a chance to reconsider how we
>> do things?
>>
>> I must admit, I'm quite happy to use packages like llvm, by
>> just downloading a package and adding the `bin` path to my
>> $PATH.
>>
>> There is however one thing that the current configure appraoch
>> does, which I think is quite noteworthy (apart from setting
>> $prefix).  That is, it does check for compilers and sets them
>> accordingly, which might be desirable.
>>
>> Cheers,
>> Moritz
>>
>> --
>> [1]: https://ghc.haskell.org/trac/ghc/wiki/Building/Installing
>> [2]: https://ghc.haskell.org/trac/ghc/wiki/Building/MacOSX/Installer
>> [3]: https://lists.gnu.org/archive/html/automake/2006-10/msg00005.html
>> _______________________________________________
>> ghc-devs mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Moritz Angermann-2
Hi *,

so I've given this a stab[1].  While getting the wrapper requirement,
and the .conf files sorted was rather easy. (We already have getExecutablePath,
and the relocatable logic for windows.  As well as $topdir and ${pkgroot}
support for paths in .conf files.)

Now as Manuel laid out the dynamic libraries situation is a bit annoying.
Specifically as the ghc-stage2 we build *in-tree*, and the libraries have
a completely different layout from the final install location.  And as such
setting @rpath, and @loader_path, is quite tricky, as essentially the
installation is not a relocation of the stage1 or stage2 compiler.

However, I am now again at the point where I start hacking on the build
system, while Hadrian is imminent.  And this is quite depressing.

Cheers,
 Moritz

--
[1]: https://phabricator.haskell.org/D4121

> On Oct 24, 2017, at 9:54 AM, Moritz Angermann <[hidden email]> wrote:
>
> Hi Manuel,
>
> thanks for the link!  Ahh yes lovely RPATH (this reminds me, I should look
> to verify if I can make GHC link recursively.)  The conf files came up on
> #ghc yesterday as well. Maybe we need some $home, $sysroot or similar marker
> to make them relocatable.
>
> I'll probably take a crack at this today.  Not sure how far I get though.
> The primary motivation is that packaging the cross compilers in a neat
> way looks like layering patched upon patches onto the configure script. And
> in light of Hadrian, we might just want to package up a directory and not
> deal with the binary-dist logic where we can?
>
> Cheers,
> Moritz
>
>> On Oct 24, 2017, at 9:45 AM, Manuel M T Chakravarty <[hidden email]> wrote:
>>
>> Hi Moritz,
>>
>> Haskell for Mac is fully relocatable (as every Mac users expects this of a Mac app).
>>
>> Unfortunately, it is not just the libdir in the wrapper scripts that’s in the way of a relocatable GHC. Additional problems are (1) the (at least, as of 8.0) still not macOS-friendly RPATHs in dynamic libraries and (2) the absolute paths in package .conf files.
>>
>> In Haskell for Mac, I use two arguably hacky solutions to those two issues. I rewrite all RPATHs in .dylibs to be relative and I have got a little tool that relocates .conf files. I’d certainly prefer a cleaner solution!
>>
>> Those aspects of Haskell for Mac are actually in a separate framework (GHC.framework) that wraps GHC and some other tools. This is all in a public repo if you like to have a look
>>
>>  https://github.com/haskellformac/GHCframework
>>
>> I haven’t put a license on it, but I’d be happy to make it all BSD3. It is of course all highly Mac specific and requires Xcode to build for all the code signing, sandboxing, etc.
>>
>> Cheers,
>> Manuel
>>
>>> Am 23.10.2017 um 17:04 schrieb Moritz Angermann <[hidden email]>:
>>>
>>> Hi,
>>>
>>> while trying to make the binary-distribution logic work for
>>> cross compilers.  I've come wonder, how hard it could be to
>>> make GHC relocatable. (e.g. just unpack the distribution,
>>> and use the compiler from the `bin` folder within).
>>>
>>> Right now this does not work due to the need for the path to
>>> package.conf, and this is hardcoded in the wrapper script to
>>> provide the proper libdir to ghc via -B[1]. Supposedly this
>>> is not an issue on Windows, as a relative path is common on
>>> windows and finding the location of the executable can be done
>>> safely. Or that's at least how I understand[1].
>>>
>>> For macOS there is the haskell-platform, and ghc-dot-app[2]
>>>
>>> From [3], it sounds like automake is a build, and not a packaging
>>> system, and the binary dist usecase with configure is not
>>> really a standard use case.
>>>
>>> So why do I bring this up NOW? Apart from me trying to make
>>> cross compiler binary distributions working?  The reason is
>>> that we are also trying to move towards hadrian, and by "starting
>>> from scratch", maybe we have a chance to reconsider how we
>>> do things?
>>>
>>> I must admit, I'm quite happy to use packages like llvm, by
>>> just downloading a package and adding the `bin` path to my
>>> $PATH.
>>>
>>> There is however one thing that the current configure appraoch
>>> does, which I think is quite noteworthy (apart from setting
>>> $prefix).  That is, it does check for compilers and sets them
>>> accordingly, which might be desirable.
>>>
>>> Cheers,
>>> Moritz
>>>
>>> --
>>> [1]: https://ghc.haskell.org/trac/ghc/wiki/Building/Installing
>>> [2]: https://ghc.haskell.org/trac/ghc/wiki/Building/MacOSX/Installer
>>> [3]: https://lists.gnu.org/archive/html/automake/2006-10/msg00005.html
>>> _______________________________________________
>>> ghc-devs mailing list
>>> [hidden email]
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Brandon Allbery
On Tue, Oct 24, 2017 at 5:02 AM, Moritz Angermann <[hidden email]> wrote:
However, I am now again at the point where I start hacking on the build
system, while Hadrian is imminent.  And this is quite depressing

Realistically, while Hadrian going into the tree may be imminent, as I understand it Hadrian becoming the primary build system --- or even a viable alternative build system --- is not. It's more of a repo logistics speed bump. Just let it go for now.

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Manuel M T Chakravarty-4
In reply to this post by Moritz Angermann-2
Am 24.10.2017 um 20:02 schrieb Moritz Angermann <[hidden email]>:

Hi *,

so I've given this a stab[1].  While getting the wrapper requirement,
and the .conf files sorted was rather easy. (We already have getExecutablePath,
and the relocatable logic for windows.  As well as $topdir and ${pkgroot}
support for paths in .conf files.)

Awesome!

Now as Manuel laid out the dynamic libraries situation is a bit annoying.
Specifically as the ghc-stage2 we build *in-tree*, and the libraries have
a completely different layout from the final install location.  And as such
setting @rpath, and @loader_path, is quite tricky, as essentially the
installation is not a relocation of the stage1 or stage2 compiler.

Yes, that’s why I took the hacky short-cut of running the normal install procedure and then patching the libs up in place with ’install_name_tool’. (BTW, a not too widely known tool that makes messing with dylibs, RPATHs, and signatures way nicer than otool is <http://newosxbook.com/tools/jtool.html> — the output is so much more readable.)

However, I am now again at the point where I start hacking on the build
system, while Hadrian is imminent.  And this is quite depressing.

I hear you.

Cheers,
Manuel

Cheers,
Moritz

--
[1]: https://phabricator.haskell.org/D4121
On Oct 24, 2017, at 9:54 AM, Moritz Angermann <[hidden email]> wrote:

Hi Manuel,

thanks for the link!  Ahh yes lovely RPATH (this reminds me, I should look
to verify if I can make GHC link recursively.)  The conf files came up on
#ghc yesterday as well. Maybe we need some $home, $sysroot or similar marker
to make them relocatable.

I'll probably take a crack at this today.  Not sure how far I get though.
The primary motivation is that packaging the cross compilers in a neat
way looks like layering patched upon patches onto the configure script. And
in light of Hadrian, we might just want to package up a directory and not
deal with the binary-dist logic where we can?

Cheers,
Moritz

On Oct 24, 2017, at 9:45 AM, Manuel M T Chakravarty <[hidden email]> wrote:

Hi Moritz,

Haskell for Mac is fully relocatable (as every Mac users expects this of a Mac app).

Unfortunately, it is not just the libdir in the wrapper scripts that’s in the way of a relocatable GHC. Additional problems are (1) the (at least, as of 8.0) still not macOS-friendly RPATHs in dynamic libraries and (2) the absolute paths in package .conf files.

In Haskell for Mac, I use two arguably hacky solutions to those two issues. I rewrite all RPATHs in .dylibs to be relative and I have got a little tool that relocates .conf files. I’d certainly prefer a cleaner solution!

Those aspects of Haskell for Mac are actually in a separate framework (GHC.framework) that wraps GHC and some other tools. This is all in a public repo if you like to have a look

https://github.com/haskellformac/GHCframework

I haven’t put a license on it, but I’d be happy to make it all BSD3. It is of course all highly Mac specific and requires Xcode to build for all the code signing, sandboxing, etc.

Cheers,
Manuel

Am 23.10.2017 um 17:04 schrieb Moritz Angermann <[hidden email]>:

Hi,

while trying to make the binary-distribution logic work for
cross compilers.  I've come wonder, how hard it could be to
make GHC relocatable. (e.g. just unpack the distribution,
and use the compiler from the `bin` folder within).

Right now this does not work due to the need for the path to
package.conf, and this is hardcoded in the wrapper script to
provide the proper libdir to ghc via -B[1]. Supposedly this
is not an issue on Windows, as a relative path is common on
windows and finding the location of the executable can be done
safely. Or that's at least how I understand[1].

For macOS there is the haskell-platform, and ghc-dot-app[2]

From [3], it sounds like automake is a build, and not a packaging
system, and the binary dist usecase with configure is not
really a standard use case.

So why do I bring this up NOW? Apart from me trying to make
cross compiler binary distributions working?  The reason is
that we are also trying to move towards hadrian, and by "starting
from scratch", maybe we have a chance to reconsider how we
do things?

I must admit, I'm quite happy to use packages like llvm, by
just downloading a package and adding the `bin` path to my
$PATH.

There is however one thing that the current configure appraoch
does, which I think is quite noteworthy (apart from setting
$prefix).  That is, it does check for compilers and sets them
accordingly, which might be desirable.

Cheers,
Moritz

--
[1]: https://ghc.haskell.org/trac/ghc/wiki/Building/Installing
[2]: https://ghc.haskell.org/trac/ghc/wiki/Building/MacOSX/Installer
[3]: https://lists.gnu.org/archive/html/automake/2006-10/msg00005.html
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs



_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Manuel M T Chakravarty-4
In reply to this post by Brandon Allbery
Why are you saying that? 

I think, Moritz is right. Hadrian is supposed to be the build system for 8.4. Adding new functionality for cross-compilation to the old build system is frustrating.

Manuel

Brandon Allbery <[hidden email]>:

On Tue, Oct 24, 2017 at 5:02 AM, Moritz Angermann <[hidden email]> wrote:
However, I am now again at the point where I start hacking on the build
system, while Hadrian is imminent.  And this is quite depressing

Realistically, while Hadrian going into the tree may be imminent, as I understand it Hadrian becoming the primary build system --- or even a viable alternative build system --- is not. It's more of a repo logistics speed bump. Just let it go for now.

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Brandon Allbery
Discussion I've been seeing is Hadrian will not be ready for production use for 8.4. Pulling it in tree is scheduled for 8.4 mostly to make it easier to keep up to date with other changes and to make it easier to work on and test the various missing pieces: as I understand it, Hadrian can't yet deal with all of the various platform special cases, etc. that an actual release requires. 

On Tue, Oct 24, 2017 at 8:02 PM, Manuel M T Chakravarty <[hidden email]> wrote:
Why are you saying that? 

I think, Moritz is right. Hadrian is supposed to be the build system for 8.4. Adding new functionality for cross-compilation to the old build system is frustrating.

Manuel

Brandon Allbery <[hidden email]>:

On Tue, Oct 24, 2017 at 5:02 AM, Moritz Angermann <[hidden email]> wrote:
However, I am now again at the point where I start hacking on the build
system, while Hadrian is imminent.  And this is quite depressing

Realistically, while Hadrian going into the tree may be imminent, as I understand it Hadrian becoming the primary build system --- or even a viable alternative build system --- is not. It's more of a repo logistics speed bump. Just let it go for now.

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs




--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Manuel M T Chakravarty-4
That doesn’t mean it can’t be used for cross-compilation once it is in the tree.

Am 25.10.2017 um 11:06 schrieb Brandon Allbery <[hidden email]>:

Discussion I've been seeing is Hadrian will not be ready for production use for 8.4. Pulling it in tree is scheduled for 8.4 mostly to make it easier to keep up to date with other changes and to make it easier to work on and test the various missing pieces: as I understand it, Hadrian can't yet deal with all of the various platform special cases, etc. that an actual release requires. 

On Tue, Oct 24, 2017 at 8:02 PM, Manuel M T Chakravarty <[hidden email]> wrote:
Why are you saying that? 

I think, Moritz is right. Hadrian is supposed to be the build system for 8.4. Adding new functionality for cross-compilation to the old build system is frustrating.

Manuel

Brandon Allbery <[hidden email]>:

On Tue, Oct 24, 2017 at 5:02 AM, Moritz Angermann <[hidden email]> wrote:
However, I am now again at the point where I start hacking on the build
system, while Hadrian is imminent.  And this is quite depressing

Realistically, while Hadrian going into the tree may be imminent, as I understand it Hadrian becoming the primary build system --- or even a viable alternative build system --- is not. It's more of a repo logistics speed bump. Just let it go for now.

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs




--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Moritz Angermann-2
Hi,

just to elaborate a bit.  We *can* build cross compiler, and cross compile ghc with
the current make based build system.  I'm not certain how well packaging binary
distributions of cross compiled ghc's work.  I've only run into several when trying
to build binary distributions for cross compilers (compiler host != compiler target).

This is also where the relocatable topic came up. If the build system would build ghc
in such a way that I could just take the stage1/stage2/stageN folder and relocate it
into it's final place.  Building binary distributions would be a simple as packaging
up that folder.  If we want to do some additional configuration on the install host,
we could add an automake script, but it wouldn't be strictly necessary.  For the case
of cross compilers, I'm pretty much sure, I do want almost none of the current
distrib configure script. My toolchain is locked down pretty hard, so I know the tools
I *need* to use.  Anything that the configure script at install time could mess up
is just going to result in more issues down the line.

Now as mentioned I was able to get most of the relocatable logic sorted by dropping
the wrapper script, and have ghc determine it's topdir on it's own (for mac and linux)
by reusing the codepaths already in place for windows.  The package db *does* understand
$topdir and ${pkgroot}.  Again reusing the $topdir logic, present for windows, allows
relocatable package databases.

Now the final issue that came up is dynamic libraries, which on macOS to some degree
encode their location.  However at build time the layout of libraries is sufficiently
different from the layout of libraries once installed.  Which would require to do
some additional path manipulation with the insall_name_tool at install time.

To make this easier, it would be preferable to have the same layout at build time as
it will be at install time.

Another issue I have with the current build system is, that everything is built *in
tree*.  This means that a) if cleaning doesn't work properly, I might have some
stale data lurking around and b) I am unable to build multiple configurations from
the same source tree (or modifications thereof) without resorting to some commit/push/pull
on local copies of the source tree, or wiping out the source tree for each build.

Therefore, after trying to relocate build artifacts in the current build system such
that build and install layouts are more similar.  I hereby declare defeat.  Not only
would this change the way the current build system works quite a bit, it's als pretty
hard to refactor the current aged build system in that way, and I believe it would
result in a number of additional bugs.  And finally, even if that change would make
it into GHC, Hadrian would have to be adapted to it as well.

I have now started trying to graft this different layout ontop of Hadrian.  If this
works out as I hope.  I believe it would drastically simplify the installation rules
as well as binary distribution rules in Hadrian.  It might also provide me with some
knowledge about Hadrian, and how much I like/dislike it over the current make system.

Maybe this is the direction we need to take to make Hadrian a viable option for the
build system.  Otherwise I fear Hadrian will never make it into ghc, if the current
build system keeps evolving and Hadrian fails to keep up.  Ideally we'd probably have
features in Hadrian that the current build system is lacking, which make Hadrian the
attractive alternative. E.g. moving Hadrian to "can do X better" instead of "can also
build GHC, but doesn't have all the features that the current build system has".

Cheers,
 Moritz
 

> On Oct 25, 2017, at 8:12 AM, Manuel M T Chakravarty <[hidden email]> wrote:
>
> That doesn’t mean it can’t be used for cross-compilation once it is in the tree.
>
>> Am 25.10.2017 um 11:06 schrieb Brandon Allbery <[hidden email]>:
>>
>> Discussion I've been seeing is Hadrian will not be ready for production use for 8.4. Pulling it in tree is scheduled for 8.4 mostly to make it easier to keep up to date with other changes and to make it easier to work on and test the various missing pieces: as I understand it, Hadrian can't yet deal with all of the various platform special cases, etc. that an actual release requires.
>>
>> On Tue, Oct 24, 2017 at 8:02 PM, Manuel M T Chakravarty <[hidden email]> wrote:
>> Why are you saying that?
>>
>> I think, Moritz is right. Hadrian is supposed to be the build system for 8.4. Adding new functionality for cross-compilation to the old build system is frustrating.
>>
>> Manuel
>>
>>> Brandon Allbery <[hidden email]>:
>>>
>>> On Tue, Oct 24, 2017 at 5:02 AM, Moritz Angermann <[hidden email]> wrote:
>>> However, I am now again at the point where I start hacking on the build
>>> system, while Hadrian is imminent.  And this is quite depressing
>>>
>>> Realistically, while Hadrian going into the tree may be imminent, as I understand it Hadrian becoming the primary build system --- or even a viable alternative build system --- is not. It's more of a repo logistics speed bump. Just let it go for now.
>>>
>>> --
>>> brandon s allbery kf8nh                               sine nomine associates
>>> [hidden email]                                  [hidden email]
>>> unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
>>> _______________________________________________
>>> ghc-devs mailing list
>>> [hidden email]
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>>
>>
>> --
>> brandon s allbery kf8nh                               sine nomine associates
>> [hidden email]                                  [hidden email]
>> unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
>> _______________________________________________
>> ghc-devs mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Relocatable dist

Joachim Breitner-2
Hi,

right now I’d like to compare the compiler at my feature branch with
and without the last commit on that branch. I have a working copy at
the end of my feature branch. If the worktree was relocatable, I could
now simply `cp -r` the whole worktree, pop the top commit, run
`make -C ghc 2` and would be good to go.

This would be much more pleasant that the current thing where I have to
 create a new work tree and rebuild everything from scratch…

(Again, nothing constructive in this mail, it's just motivational.
Or maybe someone can enlighten me with a better workflow.)

Joachim

--
Joachim Breitner
  [hidden email]
  http://www.joachim-breitner.de/

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment