more submodule questions

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

more submodule questions

Richard Eisenberg-5
Hi devs,


---
The CI pipeline of ghc/ghc> includes a linting step to ensure that all submodules refer only to "persistent" commits of the upstream repositories (e.g. not wip/ branches, which may disappear in the future). Specifically, the linter checks that any submodules refer to commits that are reachable by at least one branch that doesn't begin with the prefix wip/. Consequently, you must ensure that any submodule changes introduced in a ghc/ghc> merge request are merged upstream before the merge request is added to the merge queue.
---

I don't understand what this means.

- By citing "ghc/ghc>", does this mean that the linter only checks for this on branches of the ghc/ghc repo? If I have a fork (e.g. rae/ghc), are these checks disabled?

- Does this linter stop CI from progressing to, say, running the testsuite? If so, then how can we run the testsuite via CI if we have any submodule changes? We want to run the testsuite while the work is still in progress.

- By "you must ensure ... before the merge request is added to the merge queue": this makes me wonder whether the linter is just a warning or an error. That is, if I must ensure it, then it suggests that CI is not ensuring it.

Sorry to be dense here!

Thanks,
Richard

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: more submodule questions

Ben Gamari-2
Richard Eisenberg <[hidden email]> writes:

> Hi devs,
>
> I recently found this text at the end of https://gitlab.haskell.org/ghc/ghc/wikis/working-conventions/git/submodules <https://gitlab.haskell.org/ghc/ghc/wikis/working-conventions/git/submodules> :
>
> ---
> The CI pipeline of ghc/ghc> <applewebdata://9E3625B8-2065-48C6-A8FA-16817EA6E642/ghc/ghc> includes a linting step to ensure that all submodules refer only to "persistent" commits of the upstream repositories (e.g. not wip/ branches, which may disappear in the future). Specifically, the linter checks that any submodules refer to commits that are reachable by at least one branch that doesn't begin with the prefix wip/. Consequently, you must ensure that any submodule changes introduced in a ghc/ghc> <applewebdata://9E3625B8-2065-48C6-A8FA-16817EA6E642/ghc/ghc> merge request are merged upstream before the merge request is added to the merge queue.
> ---
>
> I don't understand what this means.
>
I have amended the text, hopefully clearing things up.

To summarize:

> - By citing "ghc/ghc>", does this mean that the linter only checks for
> this on branches of the ghc/ghc repo? If I have a fork (e.g. rae/ghc),
> are these checks disabled?
>
Forks run the same CI configuration as ghc/ghc and are subject to the
same linter.

> - Does this linter stop CI from progressing to, say, running the
> testsuite? If so, then how can we run the testsuite via CI if we have
> any submodule changes? We want to run the testsuite while the work is
> still in progress.
>
> - By "you must ensure ... before the merge request is added to the
> merge queue": this makes me wonder whether the linter is just a
> warning or an error. That is, if I must ensure it, then it suggests
> that CI is not ensuring it.
>
The linter does not hold up builds for merge requests but will hold up a
"pre-merge" validation job (e.g. a validation of an MR created by
@marge-bot). This ensures that a patch containing a wip/ submodule
reference will not be merged to master.

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: more submodule questions

Richard Eisenberg-5


On Jan 27, 2020, at 8:19 PM, Ben Gamari <[hidden email]> wrote:

The linter does not hold up builds for merge requests but will hold up a
"pre-merge" validation job (e.g. a validation of an MR created by
@marge-bot). This ensures that a patch containing a wip/ submodule
reference will not be merged to master.

Very interesting! Are there other such checks? I always assume that if an MR passes CI, then it is suitable for merging. Of course, what you describe makes perfect sense here -- we don't require upstream to have our commits during CI, but we do during merging. I'm just wondering if there are other such scenarios that are checked for.

Thanks,
Richard

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: more submodule questions

Ben Gamari-2
Richard Eisenberg <[hidden email]> writes:

>> On Jan 27, 2020, at 8:19 PM, Ben Gamari <[hidden email]> wrote:
>>
>> The linter does not hold up builds for merge requests but will hold up a
>> "pre-merge" validation job (e.g. a validation of an MR created by
>> @marge-bot). This ensures that a patch containing a wip/ submodule
>> reference will not be merged to master.
>
> Very interesting! Are there other such checks? I always assume that if
> an MR passes CI, then it is suitable for merging. Of course, what you
> describe makes perfect sense here -- we don't require upstream to have
> our commits during CI, but we do during merging. I'm just wondering if
> there are other such scenarios that are checked for.
>
Indeed we run a whole suite of linters in the `lint` stage of the CI
pipeline. See, for instance, [1]. These linters generally fall into a
few categories:

 * Some (e.g. the submodule and changelog linters) enforce policy that
   would otherwise be mere social convention. The former we already
   discussed. The latter checks that the string TBD doesn't appear in
   core library changelogs, ensuring that we take the necessary steps to
   finalize the changelogs before cutting a release.

 * Some (e.g. the makefile, CPP, and shellcheck linters) check for portability
   issues that we may not otherwise stumble upon in the set of platforms
   that we routinely test

 * Some (e.g. the typecheck-testsuite and lint-testsuite linters) are
   simply cheaper checks for obvious mistakes that we would otherwise
   catch stumble upon late in the build.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment