Question about binary distributions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about binary distributions

Mathieu Boespflug-3
Hi all,

GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll focus the dicussion below on these three platforms. The binary distributions for Linux and macOS are designed to be unpacked, then the user types ./configure && make install. This is not the case for Windows.

On all platforms it's possible to create "relocatable" installations, such that GHC doesn't really care where it's installed, and commands will still work if the install directory changes location on the filesystem. So my question is, why do we have a ./configure step on Linux and macOS? Why could we not have bindists for all platforms that work like the Windows one? I.e. a binary distribution that you just unpack, in any directory of your choice, without any configuration or installation step.



_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Question about binary distributions

Moritz Angermann-2
Hi Mathieu,

you can! See http://hackage.mobilehaskell.org/; it's been one of the
design goals I had when I was hacking on hadrian. The whole configure
&& make install shenanigans were just too much.
Initially I wanted to drop that outright, but was convinced that
configure and make install is something distributions want (that
install into different locations) as well as some people who prefer to
install
into ghc into custom locations.

GHC has for a while now had relocatable support on our major
platforms, which means you don't even need that wrapper script anymore
as long as the bin and lib folder are next to each other and you
operating system can find the path of the executable. I'm told AIX
can't if the executable is a symlink.

I'm all for having "unpack and run" bindists with an optional
configure && make install phase for those who want it. As we are
moving over to hadrian as the primary build system I think this should
work,
but might have regressed?

Cheers,
 Moritz

On Fri, Aug 7, 2020 at 9:50 PM Mathieu Boespflug <[hidden email]> wrote:

>
> Hi all,
>
> GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll focus the dicussion below on these three platforms. The binary distributions for Linux and macOS are designed to be unpacked, then the user types ./configure && make install. This is not the case for Windows.
>
> On all platforms it's possible to create "relocatable" installations, such that GHC doesn't really care where it's installed, and commands will still work if the install directory changes location on the filesystem. So my question is, why do we have a ./configure step on Linux and macOS? Why could we not have bindists for all platforms that work like the Windows one? I.e. a binary distribution that you just unpack, in any directory of your choice, without any configuration or installation step.
>
>
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Question about binary distributions

Ben Gamari-3
In reply to this post by Mathieu Boespflug-3
"Mathieu Boespflug" <[hidden email]> writes:

> Hi all,
>
> GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll
> focus the dicussion below on these three platforms. The binary
> distributions for Linux and macOS are designed to be unpacked, then
> the user types ./configure && make install. This is not the case for
> Windows.
>
> On all platforms it's possible to create "relocatable" installations,
> such that GHC doesn't really care where it's installed, and commands
> will still work if the install directory changes location on the
> filesystem. So my question is, why do we have a ./configure step on
> Linux and macOS? Why could we not have bindists for all platforms that
> work like the Windows one? I.e. a binary distribution that you just
> unpack, in any directory of your choice, without any configuration or
> installation step.
There are a few reasons:

 * Relocatable GHC builds have only been supported for only a few
   releases now and only under the Hadrian build system, which is not
   currently used to produce our binary distributions (hopefully this
   will change for 9.2).

 * On Windows we have the luxury of having a very well-controlled
   environment as we rely on essentially nothing from the host
   system. We provide our own mingw toolchain, statically link
   against libc, and have no additional dynamic dependencies.

   By contrast, on Linux we have to deal with a much larger
   configuration space:

    * several linkers, each with their own bugs

    * several C compilers, supporting various subsets of functionality
      and quirks (e.g. some distributions enable -pie by default, others
      do not)

    * various LLVM packaging schemes

   Since it would be quite expensive to probe the toolchain
   characteristics on every compiler invocation, we rather do this once
   in the configure script during bindist installation and package the
   result in the installed `settings` file.

 * On Linux we may have additional dynamic dependencies (e.g. libdw,
   numactl) which we check for during configuration time, lest the user
   be faced with an unsightly linker error if they happen to be missing
   a library.

In principle we could perhaps avoid the need for many of these checks
by creating one binary distribution per operating system distribution.
However, we will first need to move to Hadrian to build our binary
distributions.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Question about binary distributions

John Ericson-2

Per https://gitlab.haskell.org/ghc/ghc/issues/17191 I do hope to break up our configure script soon.[1] Then the bindist will need not ship the "entire" configure script, but just what is necessary to fill in the settings file(s) which have that information Ben mentions.

I think that will improve the optics of the situation a bit; for example, I don't think the reduced bindist configure script should need to worry about directories at all since GHC is relocatable (when built by Hadrian).

John

[1]: I will be able to resume work on that once I get to the bottom of https://gitlab.haskell.org/ghc/ghc/merge_requests/1102. All help greatly appreciated!

On 8/7/20 11:15 AM, Ben Gamari wrote:
"Mathieu Boespflug" [hidden email] writes:

Hi all,

GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll
focus the dicussion below on these three platforms. The binary
distributions for Linux and macOS are designed to be unpacked, then
the user types ./configure && make install. This is not the case for
Windows.

On all platforms it's possible to create "relocatable" installations,
such that GHC doesn't really care where it's installed, and commands
will still work if the install directory changes location on the
filesystem. So my question is, why do we have a ./configure step on
Linux and macOS? Why could we not have bindists for all platforms that
work like the Windows one? I.e. a binary distribution that you just
unpack, in any directory of your choice, without any configuration or
installation step.
There are a few reasons:

 * Relocatable GHC builds have only been supported for only a few
   releases now and only under the Hadrian build system, which is not
   currently used to produce our binary distributions (hopefully this
   will change for 9.2).

 * On Windows we have the luxury of having a very well-controlled
   environment as we rely on essentially nothing from the host
   system. We provide our own mingw toolchain, statically link
   against libc, and have no additional dynamic dependencies.

   By contrast, on Linux we have to deal with a much larger
   configuration space:

    * several linkers, each with their own bugs

    * several C compilers, supporting various subsets of functionality
      and quirks (e.g. some distributions enable -pie by default, others
      do not)

    * various LLVM packaging schemes

   Since it would be quite expensive to probe the toolchain
   characteristics on every compiler invocation, we rather do this once
   in the configure script during bindist installation and package the
   result in the installed `settings` file.

 * On Linux we may have additional dynamic dependencies (e.g. libdw,
   numactl) which we check for during configuration time, lest the user
   be faced with an unsightly linker error if they happen to be missing
   a library.

In principle we could perhaps avoid the need for many of these checks
by creating one binary distribution per operating system distribution.
However, we will first need to move to Hadrian to build our binary
distributions.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Question about binary distributions

Brandon Allbery
It's only relocatable given some assumptions which are violated by
various distributions (AIX was already mentioned; and the bin and lib
directories may not be next to each other with some distributions'
preferred configurations). Basically the configure mechanism gives us
some flexibility not otherwise available, without requiring a full
build from source (which is fairly involved and requires an existing
ghc).

On 8/7/20, John Cotton Ericson <[hidden email]> wrote:

> Per https://gitlab.haskell.org/ghc/ghc/issues/17191 I do hope to break
> up our configure script soon.[1] Then the bindist will need not ship the
> "entire" configure script, but just what is necessary to fill in the
> settings file(s) which have that information Ben mentions.
>
> I think that will improve the optics of the situation a bit; for
> example, I don't think the reduced bindist configure script should need
> to worry about directories at all since GHC is relocatable (when built
> by Hadrian).
>
> John
>
> [1]: I will be able to resume work on that once I get to the bottom of
> https://gitlab.haskell.org/ghc/ghc/merge_requests/1102. All help greatly
> appreciated!
>
> On 8/7/20 11:15 AM, Ben Gamari wrote:
>> "Mathieu Boespflug" <[hidden email]> writes:
>>
>>> Hi all,
>>>
>>> GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll
>>> focus the dicussion below on these three platforms. The binary
>>> distributions for Linux and macOS are designed to be unpacked, then
>>> the user types ./configure && make install. This is not the case for
>>> Windows.
>>>
>>> On all platforms it's possible to create "relocatable" installations,
>>> such that GHC doesn't really care where it's installed, and commands
>>> will still work if the install directory changes location on the
>>> filesystem. So my question is, why do we have a ./configure step on
>>> Linux and macOS? Why could we not have bindists for all platforms that
>>> work like the Windows one? I.e. a binary distribution that you just
>>> unpack, in any directory of your choice, without any configuration or
>>> installation step.
>> There are a few reasons:
>>
>>   * Relocatable GHC builds have only been supported for only a few
>>     releases now and only under the Hadrian build system, which is not
>>     currently used to produce our binary distributions (hopefully this
>>     will change for 9.2).
>>
>>   * On Windows we have the luxury of having a very well-controlled
>>     environment as we rely on essentially nothing from the host
>>     system. We provide our own mingw toolchain, statically link
>>     against libc, and have no additional dynamic dependencies.
>>
>>     By contrast, on Linux we have to deal with a much larger
>>     configuration space:
>>
>>      * several linkers, each with their own bugs
>>
>>      * several C compilers, supporting various subsets of functionality
>>        and quirks (e.g. some distributions enable -pie by default, others
>>        do not)
>>
>>      * various LLVM packaging schemes
>>
>>     Since it would be quite expensive to probe the toolchain
>>     characteristics on every compiler invocation, we rather do this once
>>     in the configure script during bindist installation and package the
>>     result in the installed `settings` file.
>>
>>   * On Linux we may have additional dynamic dependencies (e.g. libdw,
>>     numactl) which we check for during configuration time, lest the user
>>     be faced with an unsightly linker error if they happen to be missing
>>     a library.
>>
>> In principle we could perhaps avoid the need for many of these checks
>> by creating one binary distribution per operating system distribution.
>> However, we will first need to move to Hadrian to build our binary
>> distributions.
>>
>> Cheers,
>>
>> - Ben
>>
>> _______________________________________________
>> ghc-devs mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>


--
brandon s allbery kf8nh
[hidden email]
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs