Re: Hadrian

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Hadrian

Boespflug, Mathieu
Hi all,

As a user who tried to be an early adopter of Hadrian, I can attest to
Andrey's remarks about GHC progress sometimes (frequently?) breaking
Hadrian.

Ben, Andrey - how about this strawman proposal:

we merge Hadrian into GHC *now*, not because ./validate with Hadrian
works (it doesn't yet), but because the build of GHC succeeds with
Hadrian. The resulting compiler might well be garbage. But that's fine
- we can iterate in the official ghc repo, all the while knowing that
CI has our back if ever some change makes Hadrian no longer even build
a compiler. Once ./validate passes with Hadrian, the guarantee that CI
gives will become stronger still: we'll know if any change would make
the Hadrian-produced compiler garbage again.

Maybe that does sound totally bonkers to you? :) Maybe it's radical,
but sounds to me like the best way to get the benefit *now* of
ensuring Make and Hadrian move forward in lock step thanks to CI.

The above all assumes that we're committed to Hadrian being the future
of GHC's build system. But I take it that's a given by now.

Best,

Mathieu


On 19 October 2017 at 19:05, Andrey Mokhov
<[hidden email]> wrote:

> Hi Ben, Manuel and all,
>
> Ben has already covered most questions in his answer, but let me add a couple of comments.
>
>> So, Mathieu had the clever idea that having the two build system in
>> GHC side-by-side and then build in CI both ways might be a good way of
>> making sure that keep constant tabs on Hadian and get a clear (and
>> continuous picture) of where it is and what is missing.
>
> It would be great to build every GHC commit both by Make and Hadrian. Not only this would allow to monitor the readiness of Hadrian, it would also make it easier for us to catch up with changes in GHC and the Make build system. At the moment, Hadrian is frequently broken by changes in GHC/Make and we need to do detective work of figuring out which commit broke what and fix Hadrian accordingly. These fixes are trivial in many cases (e.g. adding a new flag to one of the builders) so GHC developers would probably be able to easily mirror these changes in Hadrian if only their commits were marked as Hadrian-breaking by the common CI.
>
>> In other words, why can’t we have Hadrian *today*?
>
> I'd say the biggest issue is #299 -- i.e. making sure that the GHC built by Hadrian passes validation. At the moment we still have 100-200 unexpected failures, most of which probably have to do with outdated command line flags (GHC's build system continuously evolves and it wasn't possible to keep track of all flag changes over time, so there are certainly some differences in Hadrian that lead to validation failures). This is where help is currently needed.
>
> Cheers,
> Andrey
>
> -----Original Message-----
> From: Ben Gamari [mailto:[hidden email]]
> Sent: 19 October 2017 15:08
> To: Manuel M T Chakravarty <[hidden email]>
> Cc: Mathieu Boespflug <[hidden email]>; Moritz Angermann <[hidden email]>; Jonas Pfenniger Chevalier <[hidden email]>; Andrey Mokhov <[hidden email]>
> Subject: Re: Hadrian
>
> CCing Andrey and Zhen, who are the principle authors of Hadrian.
>
> Manuel M T Chakravarty <[hidden email]> writes:
>
>> Hi Ben,
>>
>> I our exploration of cross-compilation for GHC and the CI integration,
>> we talked with Moritz and got to the topic of Hadrian. It seems that
>> there are few interdependent issues here, and I am really interested
>> in getting your take on them.
>>
>> * Hadrian is slated for inclusion for 8.4, but I couldn’t find any
>> tickets except #12599. Is this because Hadrian has its own issue
>> tracker on GitHub? How can I get an overview over what’s missing in
>> getting Hadrian into GHC?
>>
> Right, Hadrian tickets are currently tracked in its own issue tracker.
>
>> * For Moritz’ work on ARM and also for CI cross-compiling is
>> important, but Moritz pointed out rightly that it seems wasteful put
>> much time into improving this in the make-based build system if that
>> is going to go soon anyway. However, because Hadrian is not in yet,
>> doing the work on Hadrian now also doesn’t really make sense.
>>
> Indeed. Hadrian is one of the things I am currently working on for
> precisely this reason.
>
>> * Obviously, one issue with Hadrian is stability and feature
>> completeness. This has ramifications on our desire to really get 8.4
>> out of the door in February and not delay the release and also get the
>> CI done, because good CI is the best way of making sure the Hadrian
>> integration goes smoothly. (There is obviously a cyclic dependence
>> between this point and the previous.)
>>
>> So, Mathieu had the clever idea that having the two build system in
>> GHC side-by-side and then build in CI both ways might be a good way of
>> making sure that keep constant tabs on Hadian and get a clear (and
>> continuous picture) of where it is and what is missing. It would also
>> allow Moritz to do cross-compilation work right there on Hadrian.
>>
>> In other words, why can’t we have Hadrian *today*?
>>
> At this point the plan is for Hadrian and the Make-based build system to
> coexist in 8.4, with the make system serving as a backup.
>
> Well, we could (and potentially should) merge Hadrian
>
>> If the limiting factor is resources, maybe we can do something about
>> that, but it would require a good solid plan that we can execute
>> perfectly when given the opportunity.
>>
> The blocking issues in Hadrian are tagged with the "GHC Merge" project
> in the issue tracker. The overall themes currently are,
>
>  * Fixing validation support (finishing #187)
>
>  * Support the remaining test targets (#360)
>
>  * Documentation, so we don't overwhelm Andrey with questions when
>    things break. (#308)
>
>  * Finish and test install rules (#318, #373, #379)
>
>  * bindist install (the last part of #219). I started working on this,
>    although for the moment bugs and 8.2.2 have taken priority.
>
> Andrey, do correct me if I've missed something.
>
> Help is of course always welcome!
>
> Cheers,
>
> - Ben
>
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: Hadrian

Andrey Mokhov
Hi Mathieu,

Yes, in principle we can merge right now, as long as it's clear that Hadrian still requires more work before taking over.

My only concern is that merging will make it more difficult for us to quickly iterate on Hadrian: the GitHub workflow is more convenient (at least for me) than the Phabricator one. Perhaps, we can keep Hadrian on GitHub as a submodule? This also has the advantage that we can keep all existing references to GitHub issues/PRs without migrating everything to GHC Trac. It would be very unfortunate to lose all history during the merge.

Cheers,
Andrey

-----Original Message-----
From: Boespflug, Mathieu [mailto:[hidden email]]
Sent: 19 October 2017 19:21
To: Andrey Mokhov <[hidden email]>
Cc: Ben Gamari <[hidden email]>; Manuel M T Chakravarty <[hidden email]>; Moritz Angermann <[hidden email]>; Jonas Pfenniger Chevalier <[hidden email]>; ghc-devs <[hidden email]>
Subject: Re: Hadrian

Hi all,

As a user who tried to be an early adopter of Hadrian, I can attest to
Andrey's remarks about GHC progress sometimes (frequently?) breaking
Hadrian.

Ben, Andrey - how about this strawman proposal:

we merge Hadrian into GHC *now*, not because ./validate with Hadrian
works (it doesn't yet), but because the build of GHC succeeds with
Hadrian. The resulting compiler might well be garbage. But that's fine
- we can iterate in the official ghc repo, all the while knowing that
CI has our back if ever some change makes Hadrian no longer even build
a compiler. Once ./validate passes with Hadrian, the guarantee that CI
gives will become stronger still: we'll know if any change would make
the Hadrian-produced compiler garbage again.

Maybe that does sound totally bonkers to you? :) Maybe it's radical,
but sounds to me like the best way to get the benefit *now* of
ensuring Make and Hadrian move forward in lock step thanks to CI.

The above all assumes that we're committed to Hadrian being the future
of GHC's build system. But I take it that's a given by now.

Best,

Mathieu


On 19 October 2017 at 19:05, Andrey Mokhov
<[hidden email]> wrote:

> Hi Ben, Manuel and all,
>
> Ben has already covered most questions in his answer, but let me add a couple of comments.
>
>> So, Mathieu had the clever idea that having the two build system in
>> GHC side-by-side and then build in CI both ways might be a good way of
>> making sure that keep constant tabs on Hadian and get a clear (and
>> continuous picture) of where it is and what is missing.
>
> It would be great to build every GHC commit both by Make and Hadrian. Not only this would allow to monitor the readiness of Hadrian, it would also make it easier for us to catch up with changes in GHC and the Make build system. At the moment, Hadrian is frequently broken by changes in GHC/Make and we need to do detective work of figuring out which commit broke what and fix Hadrian accordingly. These fixes are trivial in many cases (e.g. adding a new flag to one of the builders) so GHC developers would probably be able to easily mirror these changes in Hadrian if only their commits were marked as Hadrian-breaking by the common CI.
>
>> In other words, why can’t we have Hadrian *today*?
>
> I'd say the biggest issue is #299 -- i.e. making sure that the GHC built by Hadrian passes validation. At the moment we still have 100-200 unexpected failures, most of which probably have to do with outdated command line flags (GHC's build system continuously evolves and it wasn't possible to keep track of all flag changes over time, so there are certainly some differences in Hadrian that lead to validation failures). This is where help is currently needed.
>
> Cheers,
> Andrey
>
> -----Original Message-----
> From: Ben Gamari [mailto:[hidden email]]
> Sent: 19 October 2017 15:08
> To: Manuel M T Chakravarty <[hidden email]>
> Cc: Mathieu Boespflug <[hidden email]>; Moritz Angermann <[hidden email]>; Jonas Pfenniger Chevalier <[hidden email]>; Andrey Mokhov <[hidden email]>
> Subject: Re: Hadrian
>
> CCing Andrey and Zhen, who are the principle authors of Hadrian.
>
> Manuel M T Chakravarty <[hidden email]> writes:
>
>> Hi Ben,
>>
>> I our exploration of cross-compilation for GHC and the CI integration,
>> we talked with Moritz and got to the topic of Hadrian. It seems that
>> there are few interdependent issues here, and I am really interested
>> in getting your take on them.
>>
>> * Hadrian is slated for inclusion for 8.4, but I couldn’t find any
>> tickets except #12599. Is this because Hadrian has its own issue
>> tracker on GitHub? How can I get an overview over what’s missing in
>> getting Hadrian into GHC?
>>
> Right, Hadrian tickets are currently tracked in its own issue tracker.
>
>> * For Moritz’ work on ARM and also for CI cross-compiling is
>> important, but Moritz pointed out rightly that it seems wasteful put
>> much time into improving this in the make-based build system if that
>> is going to go soon anyway. However, because Hadrian is not in yet,
>> doing the work on Hadrian now also doesn’t really make sense.
>>
> Indeed. Hadrian is one of the things I am currently working on for
> precisely this reason.
>
>> * Obviously, one issue with Hadrian is stability and feature
>> completeness. This has ramifications on our desire to really get 8.4
>> out of the door in February and not delay the release and also get the
>> CI done, because good CI is the best way of making sure the Hadrian
>> integration goes smoothly. (There is obviously a cyclic dependence
>> between this point and the previous.)
>>
>> So, Mathieu had the clever idea that having the two build system in
>> GHC side-by-side and then build in CI both ways might be a good way of
>> making sure that keep constant tabs on Hadian and get a clear (and
>> continuous picture) of where it is and what is missing. It would also
>> allow Moritz to do cross-compilation work right there on Hadrian.
>>
>> In other words, why can’t we have Hadrian *today*?
>>
> At this point the plan is for Hadrian and the Make-based build system to
> coexist in 8.4, with the make system serving as a backup.
>
> Well, we could (and potentially should) merge Hadrian
>
>> If the limiting factor is resources, maybe we can do something about
>> that, but it would require a good solid plan that we can execute
>> perfectly when given the opportunity.
>>
> The blocking issues in Hadrian are tagged with the "GHC Merge" project
> in the issue tracker. The overall themes currently are,
>
>  * Fixing validation support (finishing #187)
>
>  * Support the remaining test targets (#360)
>
>  * Documentation, so we don't overwhelm Andrey with questions when
>    things break. (#308)
>
>  * Finish and test install rules (#318, #373, #379)
>
>  * bindist install (the last part of #219). I started working on this,
>    although for the moment bugs and 8.2.2 have taken priority.
>
> Andrey, do correct me if I've missed something.
>
> Help is of course always welcome!
>
> Cheers,
>
> - Ben
>
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Hadrian

Ben Gamari-2
In reply to this post by Boespflug, Mathieu
"Boespflug, Mathieu" <[hidden email]> writes:

> Hi all,
>
> As a user who tried to be an early adopter of Hadrian, I can attest to
> Andrey's remarks about GHC progress sometimes (frequently?) breaking
> Hadrian.
>
> Ben, Andrey - how about this strawman proposal:
>
> we merge Hadrian into GHC *now*, not because ./validate with Hadrian
> works (it doesn't yet), but because the build of GHC succeeds with
> Hadrian. The resulting compiler might well be garbage. But that's fine
> - we can iterate in the official ghc repo, all the while knowing that
> CI has our back if ever some change makes Hadrian no longer even build
> a compiler. Once ./validate passes with Hadrian, the guarantee that CI
> gives will become stronger still: we'll know if any change would make
> the Hadrian-produced compiler garbage again.
>
> Maybe that does sound totally bonkers to you? :) Maybe it's radical,
> but sounds to me like the best way to get the benefit *now* of
> ensuring Make and Hadrian move forward in lock step thanks to CI.
>
> The above all assumes that we're committed to Hadrian being the future
> of GHC's build system. But I take it that's a given by now.
>
That's not so different from my existing plan, which is to import
Hadrian after 8.2.2.

That being said, if it's okay with Andrey to import now then it is also
okay with me. There are, of course some details to be worked out: do we
import hadrian as a git subtree, retaining history, or simply squash a
snapshot? Where should tickets now be filed from now on? Should we move
the scripts in the top-level of the hadrian repo to the top-level of the
GHC repo?

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Hadrian

Ben Gamari-3
In reply to this post by Boespflug, Mathieu
Andrey Mokhov <[hidden email]> writes:

> Hi Mathieu,
>
> Yes, in principle we can merge right now, as long as it's clear that Hadrian still requires more work before taking over.
>
> My only concern is that merging will make it more difficult for us to
> quickly iterate on Hadrian: the GitHub workflow is more convenient (at
> least for me) than the Phabricator one. Perhaps, we can keep Hadrian
> on GitHub as a submodule? This also has the advantage that we can keep
> all existing references to GitHub issues/PRs without migrating
> everything to GHC Trac. It would be very unfortunate to lose all
> history during the merge.
>
Okay, so if we want to preserve history then I would suggest that we go
the subtree route. That is pretty much precisely the use-case which
git subtree was designed to address. This will allow us to have Hadrian,
with history, in the GHC tree and you can continue to develop it on
GitHub until things have stabilized. The only question is how to ensure
that the subtree remains up-to-date.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Hadrian

Andrey Mokhov
Thanks Ben,

Just to clarify: By history I mean not just commits, but GitHub issues and PRs as well -- together they contain a lot of valuable interlinked information for GHC/Hadrian developers.

> That is pretty much precisely the use-case which
> git subtree was designed to address. This will allow us to have Hadrian,
> with history, in the GHC tree and you can continue to develop it on
> GitHub until things have stabilized.

Sounds great. I haven't used git subtrees before, so I'll need to do some reading, but if everyone is happy with merging Hadrian as is, I can prepare a patch over the weekend!

Cheers,
Andrey

-----Original Message-----
From: Ben Gamari [mailto:[hidden email]]
Sent: 19 October 2017 20:50
To: Andrey Mokhov <[hidden email]>; Boespflug, Mathieu <[hidden email]>
Cc: Manuel M T Chakravarty <[hidden email]>; Moritz Angermann <[hidden email]>; Jonas Pfenniger Chevalier <[hidden email]>; ghc-devs <[hidden email]>
Subject: RE: Hadrian

Andrey Mokhov <[hidden email]> writes:

> Hi Mathieu,
>
> Yes, in principle we can merge right now, as long as it's clear that Hadrian still requires more work before taking over.
>
> My only concern is that merging will make it more difficult for us to
> quickly iterate on Hadrian: the GitHub workflow is more convenient (at
> least for me) than the Phabricator one. Perhaps, we can keep Hadrian
> on GitHub as a submodule? This also has the advantage that we can keep
> all existing references to GitHub issues/PRs without migrating
> everything to GHC Trac. It would be very unfortunate to lose all
> history during the merge.
>
Okay, so if we want to preserve history then I would suggest that we go
the subtree route. That is pretty much precisely the use-case which
git subtree was designed to address. This will allow us to have Hadrian,
with history, in the GHC tree and you can continue to develop it on
GitHub until things have stabilized. The only question is how to ensure
that the subtree remains up-to-date.

Cheers,

- Ben
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: Hadrian

Ben Gamari-3
Andrey Mokhov <[hidden email]> writes:

> Thanks Ben,
>
> Just to clarify: By history I mean not just commits, but GitHub issues
> and PRs as well -- together they contain a lot of valuable interlinked
> information for GHC/Hadrian developers.
>
Well, the GitHub repo will still exist. Is that enough?

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Hadrian

Andrey Mokhov
Hi Ben,

> Well, the GitHub repo will still exist. Is that enough?

Yes, but I think I'll need to do some clean up in the code so that it's obvious where to look for answers. For example, here is a random comment from a Hadrian source file:

-- Objdump is only required on OpenBSD and AIX, as mentioned in #211.

The reader might confuse this with GHC ticket #211, so I guess this should be replaced with a full link https://github.com/snowleopard/hadrian/issues/211. There may be other potential pitfalls, but hopefully nothing difficult to handle.

I've created an issue to discuss and prepare for the merge: https://github.com/snowleopard/hadrian/issues/440.

Cheers,
Andrey

-----Original Message-----
From: Ben Gamari [mailto:[hidden email]]
Sent: 19 October 2017 21:50
To: Andrey Mokhov <[hidden email]>; Boespflug, Mathieu <[hidden email]>
Cc: Jonas Pfenniger Chevalier <[hidden email]>; Manuel M T Chakravarty <[hidden email]>; ghc-devs <[hidden email]>
Subject: RE: Hadrian

Andrey Mokhov <[hidden email]> writes:

> Thanks Ben,
>
> Just to clarify: By history I mean not just commits, but GitHub issues
> and PRs as well -- together they contain a lot of valuable interlinked
> information for GHC/Hadrian developers.
>
Well, the GitHub repo will still exist. Is that enough?

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Hadrian

Moritz Angermann
Hi,

two things.

a) why a git subtree if we keep working on github with hadrian, wouldn't that imply using a submodule? We use submodules for
   all the other ghc boot libs that are not part of the tree and are developed on github as well, no? As far as I understand
   a subtree, it's essentially integrating everything into the main tree.  So this looks more like the submodule to subtree
   conversion, when hadrian development is switched over to phabricator?
b) should we really hardcode the full url to hadrian tickets? wouldn't hadrian/#xxx suffice? Assuming for a moment hadrian
   would move into the github.com/haskell org or something, while the linkes would likely (due to githubs url redirection)
   keep on working, until someone recreates a snowleopard/hardian repo, but not necessarily correct anymore.

Anyway, let's do this now rather than later.

Cheers,
 moritz

> On Oct 20, 2017, at 7:06 AM, Andrey Mokhov <[hidden email]> wrote:
>
> Hi Ben,
>
>> Well, the GitHub repo will still exist. Is that enough?
>
> Yes, but I think I'll need to do some clean up in the code so that it's obvious where to look for answers. For example, here is a random comment from a Hadrian source file:
>
> -- Objdump is only required on OpenBSD and AIX, as mentioned in #211.
>
> The reader might confuse this with GHC ticket #211, so I guess this should be replaced with a full link https://github.com/snowleopard/hadrian/issues/211. There may be other potential pitfalls, but hopefully nothing difficult to handle.
>
> I've created an issue to discuss and prepare for the merge: https://github.com/snowleopard/hadrian/issues/440.
>
> Cheers,
> Andrey
>
> -----Original Message-----
> From: Ben Gamari [mailto:[hidden email]]
> Sent: 19 October 2017 21:50
> To: Andrey Mokhov <[hidden email]>; Boespflug, Mathieu <[hidden email]>
> Cc: Jonas Pfenniger Chevalier <[hidden email]>; Manuel M T Chakravarty <[hidden email]>; ghc-devs <[hidden email]>
> Subject: RE: Hadrian
>
> Andrey Mokhov <[hidden email]> writes:
>
>> Thanks Ben,
>>
>> Just to clarify: By history I mean not just commits, but GitHub issues
>> and PRs as well -- together they contain a lot of valuable interlinked
>> information for GHC/Hadrian developers.
>>
> Well, the GitHub repo will still exist. Is that enough?
>
> Cheers,
>
> - Ben
>
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Hadrian

Herbert Valerio Riedel-3
Hi Moritz (et al.),

On 2017-10-20 at 09:32:29 +0800, Moritz Angermann wrote:

> a) why a git subtree if we keep working on github with hadrian, wouldn't that imply using a submodule? We use submodules for
>    all the other ghc boot libs that are not part of the tree and are developed on github as well, no? As far as I understand
>    a subtree, it's essentially integrating everything into the main tree.  So this looks more like the submodule to subtree
>    conversion, when hadrian development is switched over to phabricator?

Fwiw, using a submodule makes sense imho if hadrian is supposed to
remain a submodule on the long-term; as transitioning from submodule <->
non-submodule is something Git doesn't handle to well automatically
(we've had fun with that when we restructured ghc.git into the current
structure...)

Also, a submodule has the benefit of ticket references clearly referring
to a different numbering namespace w/o having to rewrite all ticket &
commit-hash references (which is often overlooked; I for one tend to
refer quite often to other commit shas in my commits, and they also
result when you 'git cherry-pick -x')

So, what's the long-term plan for Hadrian repo-wise? Is it going to
remain externally maintained on GitHub, or will it be integrated into
GHC HQ's infrastucture which means using Phabricator (which may pose its
own challenge, as Phabricator, like Trac, uses globally unique ticket
numbers, rather than per-project numbering)?

If it is to be a submodule, we (and by that I mean myself; it doesn't
take long) also need to setup mirroring from snowleopard/hardian to
git://git.haskell.org, and from there to to github.com/ghc/hadrian for
technical reasons.

> b) should we really hardcode the full url to hadrian tickets? wouldn't hadrian/#xxx suffice? Assuming for a moment hadrian
>    would move into the github.com/haskell org or something, while the linkes would likely (due to githubs url redirection)
>    keep on working, until someone recreates a snowleopard/hardian
>    repo, but not necessarily correct anymore.

Note that GitHub doesn't support the abbreviated hadrian#xxx syntax; you
*have* to prefix it by the organization/user, in order for GitHub to
recognize a ticket reference; i.e. you *have* to use one of


- https://github.com/snowleopard/hadrian/issues/123  (or possibly .../pull/123)

- snowleopard/hadrian#123

- #123

- GH-123

for GitHub to recognize the issue reference.

anything else won't be recognized.

Moreover, github.com/haskell/ is not a good place for Hadrian since it's
a too GHC-specific tooling; if anywhere, it would rather end up together
with the other GHC repos in the /ghc/ org at
https://github.com/ghc/hadrian (or something slightly different; we need
reconcile this with our GHC mirroring scheme; that's may be a bit
tricky).

-- hvr
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Hadrian

Moritz Angermann

> On Oct 20, 2017, at 6:24 PM, Herbert Valerio Riedel <[hidden email]> wrote:
>
> Hi Moritz (et al.),
>
> On 2017-10-20 at 09:32:29 +0800, Moritz Angermann wrote:
>
>> a) why a git subtree if we keep working on github with hadrian, wouldn't that imply using a submodule? We use submodules for
>>   all the other ghc boot libs that are not part of the tree and are developed on github as well, no? As far as I understand
>>   a subtree, it's essentially integrating everything into the main tree.  So this looks more like the submodule to subtree
>>   conversion, when hadrian development is switched over to phabricator?
>
> Fwiw, using a submodule makes sense imho if hadrian is supposed to
> remain a submodule on the long-term; as transitioning from submodule <->
> non-submodule is something Git doesn't handle to well automatically
> (we've had fun with that when we restructured ghc.git into the current
> structure...)
>
> Also, a submodule has the benefit of ticket references clearly referring
> to a different numbering namespace w/o having to rewrite all ticket &
> commit-hash references (which is often overlooked; I for one tend to
> refer quite often to other commit shas in my commits, and they also
> result when you 'git cherry-pick -x')
>
> So, what's the long-term plan for Hadrian repo-wise? Is it going to
> remain externally maintained on GitHub, or will it be integrated into
> GHC HQ's infrastucture which means using Phabricator (which may pose its
> own challenge, as Phabricator, like Trac, uses globally unique ticket
> numbers, rather than per-project numbering)?
>
> If it is to be a submodule, we (and by that I mean myself; it doesn't
> take long) also need to setup mirroring from snowleopard/hardian to
> git://git.haskell.org, and from there to to github.com/ghc/hadrian for
> technical reasons.

I would assume that hadrian will be developed on github for a while, to not
disturb the current workflow. But I'm happy to be corrected, when wrong here.
After reading up on git-subtree[1][2], which sounds to me like subsuming one
repo in another one and requiring a set of different commands to handle it.
Maybe I'm just too used to git submodules. But as all the other externals in
the ghc tree are submodules, I'd prefer the consistency.  Knowing myself, I'd
probably screw up pushing changes back upstream into hadrian if its workflow
is different from e.g. the libraries, ... libffi-tarballs with the orphan
branches is already something that needs to be taken special care of...

>> b) should we really hardcode the full url to hadrian tickets? wouldn't hadrian/#xxx suffice? Assuming for a moment hadrian
>>   would move into the github.com/haskell org or something, while the linkes would likely (due to githubs url redirection)
>>   keep on working, until someone recreates a snowleopard/hardian
>>   repo, but not necessarily correct anymore.
>
> Note that GitHub doesn't support the abbreviated hadrian#xxx syntax; you
> *have* to prefix it by the organization/user, in order for GitHub to
> recognize a ticket reference; i.e. you *have* to use one of
>
>
> - https://github.com/snowleopard/hadrian/issues/123  (or possibly .../pull/123)
>
> - snowleopard/hadrian#123
>
> - #123
>
> - GH-123
>
> for GitHub to recognize the issue reference.
>
> anything else won't be recognized.
>
> Moreover, github.com/haskell/ is not a good place for Hadrian since it's
> a too GHC-specific tooling; if anywhere, it would rather end up together
> with the other GHC repos in the /ghc/ org at
> https://github.com/ghc/hadrian (or something slightly different; we need
> reconcile this with our GHC mirroring scheme; that's may be a bit
> tricky).

I didn't specifically care about github auto-linking features here. Primarily
about giving tickets a prefix. And not necessarily hardcoding some github url
into ticket numbers.

Cheers,
 Moritz

--
[1]: https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree
[2]: http://alistra.ghost.io/2014/11/30/git-subtree-a-better-alternative-to-git-submodule/

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: Hadrian

Andrey Mokhov
Hi Moritz and Herbert,

Thank you for detailed comments! We clearly need to carefully think through our options for merging Hadrian.

Can I invite you both and everyone else to continue the discussion in https://github.com/snowleopard/hadrian/issues/440? Long email threads tend to become hard to read/follow and get lost.

In my view, the two most important requirements in the long term are:

1) Preserving the commit/issue/pull-request history. A GHC developer fighting a strange build failure should be able to find a relevant discussion not only now but in 5 years from now. This may be solved via documentation, i.e. gradually moving all discussions from GitHub to docs/comments. That's a lot of hard work.

2) Making it convenient for GHC developers to work on Hadrian. To me, git submodules are not convenient at all, but maybe there is just no other option given the requirement (1). Is git subtree a solution?

Cheers,
Andrey

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: Hadrian

Ben Gamari-2
Andrey Mokhov <[hidden email]> writes:

> Hi Moritz and Herbert,
>
> Thank you for detailed comments! We clearly need to carefully think through our options for merging Hadrian.
>
> Can I invite you both and everyone else to continue the discussion in
> https://github.com/snowleopard/hadrian/issues/440? Long email threads
> tend to become hard to read/follow and get lost.
>
Yes, let's pick this up on the ticket.

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment