Reinstallable lib:ghc (was: [core libraries] Upgradeable base (smallest step forward))

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Reinstallable lib:ghc (was: [core libraries] Upgradeable base (smallest step forward))

Herbert Valerio Riedel-3
Hi everyone,

As part of the recent GHC 8.2.1 release and the associated boot-library
publishing to Hackage, I picked up again experimenting with a
reinstallable lib:ghc package primarily for the benefit of uploading
haddocks for lib:ghc (w/ `--hyperlinked-source`) to Hackage:

  http://hackage.haskell.org/package/ghc

However, this also provides somewhat of a tech-preview of a
reinstallable lib:ghc package, which would help mitigate a big issue we
had in the past with lib:ghc and its dependencies being frozen to
specific versions, and which in combination with cabal's nix-style
pkg store that finally provide us with the ability to have multiple
instances of the same library versions co-existing in the same package
db, is no longer a requirement. So we could have lib:ghc more easily pick
dependencies up (but still within reason!) like e.g. lib:text without
forcing everyone to the single lib:text version bundled with GHC. And
we'd also gain the ability to publish patch-level releases to lib:ghc
only inbetween proper minor GHC releases (e.g. to support new versions
of library dependencies).

Consider this artificial example demo package which requires a different
version of binary than the one bundled with GHC 8.2.1:

--8<---------------cut here---------------start------------->8---
-- ghc-demo.cabal
name:                ghc-demo
version:             0
build-type:          Simple
cabal-version:       >=2.0

executable ghc-demo
  main-is:             Main.hs
  build-depends:       base ^>= 4.10, ghc ^>= 8.2.1, binary == 0.8.5.0
  default-language:    Haskell2010
--8<---------------cut here---------------end--------------->8---

Trying to build this will have the cabal solver complain that the
installed instance of ghc requires a different (& installed) version of
binary, i.e. binary-0.8.5.1.

Moreover, the source version of lib:ghc on Hackage has the manual
`buildable` cabal flag which defaults to false, and currently prevents
lib:ghc to be reinstalled automatically (by enabling an unsatisfiable
constraint):

--8<---------------cut here---------------start------------->8---
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: base-4.10.0.0/installed-4.1... (dependency of ghc-demo-0)
next goal: ghc (dependency of ghc-demo-0)
rejecting: ghc-8.2.1/installed-8.2... (conflict: binary==0.8.5.0, ghc => binary==0.8.5.1/installed-0.8...)
trying: ghc-8.2.1
rejecting: ghc-8.2.1:-buildable (conflict: base==4.10.0.0/installed-4.1..., ghc -buildable => base<0)
rejecting: ghc-8.2.1:+buildable (manual flag can only be changed explicitly)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: ghc-demo, binary, base, ghc, ghc-8.2.1:buildable
--8<---------------cut here---------------end--------------->8---

However, if we unlock this by toggling the flag (NB: this only works on
linux/x86_64; it's explained later why), e.g.

--8<---------------cut here---------------start------------->8---
-- cabal.project file
packages: .

package ghc
  flags: +buildable
--8<---------------cut here---------------end--------------->8---

The cabal solver would come up with an install-plan:

--8<---------------cut here---------------start------------->8---
Resolving dependencies...
Build profile: -w ghc-8.2.1 -O1
In order, the following will be built (use -v for more details):
 - binary-0.8.5.0 {binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac} (lib) (requires download & build)
 - ghc-boot-8.2.1 {ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb} (lib) (requires build)
 - ghci-8.2.1 {ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1} (lib) (requires build)
 - ghc-8.2.1 {ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c} (lib) +buildable (requires build)
 - ghc-demo-0 {ghc-demo-0-inplace-ghc-demo} (exe:ghc-demo) (first run)

Configuring  binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib)
Building     binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib)
Installing   binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib)
Finished     binary-0.8.5.0-7f686cf5c40c3c685540306709c4a2a30d740679f5d31ad196283d242a4d70ac (lib)
Configuring  ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Building     ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Installing   ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Finished     ghc-boot-8.2.1-7930c3d78690c5fa476415d38abbb7717b05cf65e80cc79c12e782c9886507cb (lib)
Configuring  ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Building     ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Installing   ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Finished     ghci-8.2.1-6292924619a045fd18e9fde0cfa350632672ad7439cb84a5c56ff0e4781346b1 (lib)
Configuring  ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)
Building     ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)
Installing   ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)
Finished     ghc-8.2.1-bc10931b0f169622deb74d84cf10e315730dfd8f2edcf4cad9073d142cf5da0c (lib)

Configuring executable 'ghc-demo' for ghc-demo-0..
Preprocessing executable 'ghc-demo' for ghc-demo-0..
Building executable 'ghc-demo' for ghc-demo-0..
[1 of 1] Compiling Main             ( Main.hs, /tmp/lll/dist-newstyle/build/x86_64-linux/ghc-8.2.1/ghc-demo-0/c/ghc-demo/build/ghc-demo/ghc-demo-tmp/Main.o )
Linking /tmp/lll/dist-newstyle/build/x86_64-linux/ghc-8.2.1/ghc-demo-0/c/ghc-demo/build/ghc-demo/ghc-demo ...
--8<---------------cut here---------------end--------------->8---


So, cabal was able to succesfully produce an executable which uses a
reinstalled lib:ghc w/ a different lib:binary version!


But now for the limitations: this currently works at best with a GHC
8.2.1 installation whose configuration matches the generated files I
manually included in the modified lib:ghc package in

  http://hackage.haskell.org/package/ghc-8.2.1/src/autogen/

which as you can see from the generated files (see e.g. Config.hs) was a
Linux/x86_64/libgmp/internal-libffi/... configuration.

...and this finally brings me to the purpose of why I wrote this email:
In order to make lib:ghc properly reinstallable we'd need to either

 a) have a way to regenerate the files autogen/* via a custom Setup.hs
    (by the likes of genprimcode or similiar)

or the lazy option,

 b) simply have those files installed in a place we can easily locate
    (again, from a custom Setup.hs)

Does this make any sense; which option do you prefer?

Also, I'd like to know if you can think of reasons why or situations
when the reinstalled lib:ghc wouldn't work; or other reasons why this is
a bad idea.


Cheers,
  hvr

On 2016-06-05 at 19:02:29 +0200, Herbert Valerio Riedel wrote:

>> As an ultimate goal for a project like this, it would be nice if you could
>> do something like upgrade your base package but continue to use the ghc
>> package without incompatible types.
>
> That's one way to approach this. However, I've been experimenting with a
> more flexible way: Thanks to cabal-1.24's new nix-based tech-preview
> which gets rid of that `--force-reinstall` abomination for good, the
> prospect of simply recompiling the `ghc` cabal package becomes more
> attractive.
>
> This would get rid of the primary reason which currently constraints us
> to using the package versions bundled in GHC (like e.g. for
> `bytestring`) as soon as `ghc` enters an install-plan.
>
> I'm still investigating this, but early experiments were promising, but
> I need to reorganise GHC's source-tree a little bit to know for sure if
> this is feasable. But if this as I hope is possible, this could open up
> a lot of new possibilities!
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reinstallable lib:ghc (was: [core libraries] Upgradeable base (smallest step forward))

Joachim Breitner-2
Hi,

Am Montag, den 24.07.2017, 12:24 +0200 schrieb Herbert Valerio Riedel:
> Also, I'd like to know if you can think of reasons why or situations
> when the reinstalled lib:ghc wouldn't work; or other reasons why this
> is a bad idea.

I’d am mostly worried about ABI compatibility. Will the .hi files
written by the compiler be readable by some tool that was built with an
upgraded ghc? Which dependencies can affect the binary format (if any)?

Or will the rebuilt ghc get its own, randomly generated “GHC version”
(similar to a development build where the build date is part of the GHC
version) and hence never try to interact with build artifacts created
from the host ghc?


Also, if we can `cabal install ghc-the-library`, can we also `cabal
install ghc-the-program`, possibly at a different version? (It wouldn’t
be normally usable without bootstrapping a RTS and base library, but it
would be a step.)

Greetings,
Joachim

--
Joachim “nomeata” Breitner
  [hidden email]https://www.joachim-breitner.de/
  XMPP: [hidden email] • OpenPGP-Key: 0xF0FBF51F
  Debian Developer: [hidden email]
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reinstallable lib:ghc (was: [core libraries] Upgradeable base (smallest step forward))

Edward Z. Yang
Reinstalled GHC should be completely ABI compatible.  This should
actually not be difficult to achieve, since we already rely on this
in the GHC build system: we assume that ghc-stage1 generated hi files
can be compatibly read and used by ghc-stage2.

Excerpts from Joachim Breitner's message of 2017-07-24 15:04:21 -0400:

> Hi,
>
> Am Montag, den 24.07.2017, 12:24 +0200 schrieb Herbert Valerio Riedel:
> > Also, I'd like to know if you can think of reasons why or situations
> > when the reinstalled lib:ghc wouldn't work; or other reasons why this
> > is a bad idea.
>
> I’d am mostly worried about ABI compatibility. Will the .hi files
> written by the compiler be readable by some tool that was built with an
> upgraded ghc? Which dependencies can affect the binary format (if any)?
>
> Or will the rebuilt ghc get its own, randomly generated “GHC version”
> (similar to a development build where the build date is part of the GHC
> version) and hence never try to interact with build artifacts created
> from the host ghc?
>
>
> Also, if we can `cabal install ghc-the-library`, can we also `cabal
> install ghc-the-program`, possibly at a different version? (It wouldn’t
> be normally usable without bootstrapping a RTS and base library, but it
> would be a step.)
>
> Greetings,
> Joachim
>
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Loading...