How to merge your patch

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

How to merge your patch

Ben Gamari-3
tl;dr. Our beloved @marge-bot is behaving much more reliably now thanks
       to improvements in CI reliability and build batching. To merge
       an (accepted) merge request simply designate @marge-bot as its
       assignee. Marge will handle the rest. If things go awry let me
       know.


Hi everyone,

As you may have noticed, over the last weeks we have been feeling out
how best to leverage our new CI infrastructure, particularly when it
comes to merging patches.

As mentioned a few weeks ago, we have introduced a bot, Marge, to help
us work around some temporary limitations of GitLab's merge workflow.
Unfortunately, Marge had her own set of quirks which have taken a while
to sort out. In particular, fragile tests tended to result in repeated
merge attempts which tended to clog up CI, leading to an avalanche of
waiting, build failures, and general despair.

However, we have been working to improve this situation in three ways:

 * provisioning more builder capacity to reduce wait times

 * fixing or disabling fragile tests to reduce the need for retries

 * enable Marge's batched merge functionality, reducing the number of
   builds necessary per merged patch [1]

In light of this I just wanted to reiterate the previous guidance on
merging patches. If you have a merge request you would like to merge
simply do the following:

 1. make sure that it has at least one approval. This should
    happen in the course of code review but do ping if this was
    forgotten.

 2. assign the merge request to @marge-bot using the assignee field in
    the right-hand sidebar of the merge request page.

 3. next time Marge does a batch of merges she will fold in your MR
    (leaving a helpful comment to let you know) and, if the batch passes
    CI, merge it. If not she will leave a comment letting you know there
    was an issue. I will try to step in to sort out the mess when this
    happens.

Do let me know if you have any questions.

Cheers,

- Ben


[1] thanks to Matthew Pickering for picking this up

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to merge your patch

Matthew Pickering
I am continuing to monitor the situation. There still seems to be some
teething issues.

Matt

On Thu, Feb 7, 2019 at 3:58 AM Ben Gamari <[hidden email]> wrote:

>
> tl;dr. Our beloved @marge-bot is behaving much more reliably now thanks
>        to improvements in CI reliability and build batching. To merge
>        an (accepted) merge request simply designate @marge-bot as its
>        assignee. Marge will handle the rest. If things go awry let me
>        know.
>
>
> Hi everyone,
>
> As you may have noticed, over the last weeks we have been feeling out
> how best to leverage our new CI infrastructure, particularly when it
> comes to merging patches.
>
> As mentioned a few weeks ago, we have introduced a bot, Marge, to help
> us work around some temporary limitations of GitLab's merge workflow.
> Unfortunately, Marge had her own set of quirks which have taken a while
> to sort out. In particular, fragile tests tended to result in repeated
> merge attempts which tended to clog up CI, leading to an avalanche of
> waiting, build failures, and general despair.
>
> However, we have been working to improve this situation in three ways:
>
>  * provisioning more builder capacity to reduce wait times
>
>  * fixing or disabling fragile tests to reduce the need for retries
>
>  * enable Marge's batched merge functionality, reducing the number of
>    builds necessary per merged patch [1]
>
> In light of this I just wanted to reiterate the previous guidance on
> merging patches. If you have a merge request you would like to merge
> simply do the following:
>
>  1. make sure that it has at least one approval. This should
>     happen in the course of code review but do ping if this was
>     forgotten.
>
>  2. assign the merge request to @marge-bot using the assignee field in
>     the right-hand sidebar of the merge request page.
>
>  3. next time Marge does a batch of merges she will fold in your MR
>     (leaving a helpful comment to let you know) and, if the batch passes
>     CI, merge it. If not she will leave a comment letting you know there
>     was an issue. I will try to step in to sort out the mess when this
>     happens.
>
> Do let me know if you have any questions.
>
> Cheers,
>
> - Ben
>
>
> [1] thanks to Matthew Pickering for picking this up
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: How to merge your patch

Matthew Pickering
I have observed the problem in the logs and have stopped the Marge
service for now.

On Thu, Feb 7, 2019 at 6:57 AM Matthew Pickering
<[hidden email]> wrote:

>
> I am continuing to monitor the situation. There still seems to be some
> teething issues.
>
> Matt
>
> On Thu, Feb 7, 2019 at 3:58 AM Ben Gamari <[hidden email]> wrote:
> >
> > tl;dr. Our beloved @marge-bot is behaving much more reliably now thanks
> >        to improvements in CI reliability and build batching. To merge
> >        an (accepted) merge request simply designate @marge-bot as its
> >        assignee. Marge will handle the rest. If things go awry let me
> >        know.
> >
> >
> > Hi everyone,
> >
> > As you may have noticed, over the last weeks we have been feeling out
> > how best to leverage our new CI infrastructure, particularly when it
> > comes to merging patches.
> >
> > As mentioned a few weeks ago, we have introduced a bot, Marge, to help
> > us work around some temporary limitations of GitLab's merge workflow.
> > Unfortunately, Marge had her own set of quirks which have taken a while
> > to sort out. In particular, fragile tests tended to result in repeated
> > merge attempts which tended to clog up CI, leading to an avalanche of
> > waiting, build failures, and general despair.
> >
> > However, we have been working to improve this situation in three ways:
> >
> >  * provisioning more builder capacity to reduce wait times
> >
> >  * fixing or disabling fragile tests to reduce the need for retries
> >
> >  * enable Marge's batched merge functionality, reducing the number of
> >    builds necessary per merged patch [1]
> >
> > In light of this I just wanted to reiterate the previous guidance on
> > merging patches. If you have a merge request you would like to merge
> > simply do the following:
> >
> >  1. make sure that it has at least one approval. This should
> >     happen in the course of code review but do ping if this was
> >     forgotten.
> >
> >  2. assign the merge request to @marge-bot using the assignee field in
> >     the right-hand sidebar of the merge request page.
> >
> >  3. next time Marge does a batch of merges she will fold in your MR
> >     (leaving a helpful comment to let you know) and, if the batch passes
> >     CI, merge it. If not she will leave a comment letting you know there
> >     was an issue. I will try to step in to sort out the mess when this
> >     happens.
> >
> > Do let me know if you have any questions.
> >
> > Cheers,
> >
> > - Ben
> >
> >
> > [1] thanks to Matthew Pickering for picking this up
> > _______________________________________________
> > ghc-devs mailing list
> > [hidden email]
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: How to merge your patch

GHC - devs mailing list
In reply to this post by Ben Gamari-3
|   2. assign the merge request to @marge-bot using the assignee field in
|      the right-hand sidebar of the merge request page.

This sounds great. What email will we receive in these three cases?

a) the merge is successful
b) the merge fails because I made a mistake and it just doesn't validate
c) the merge failed because the CI infrastructure failed somehow
   (should no longer happen, I know).

Examples would be helpful.

Thanks

Simon

|  -----Original Message-----
|  From: ghc-devs <[hidden email]> On Behalf Of Ben Gamari
|  Sent: 07 February 2019 03:58
|  To: GHC developers <[hidden email]>
|  Subject: How to merge your patch
|  
|  tl;dr. Our beloved @marge-bot is behaving much more reliably now thanks
|         to improvements in CI reliability and build batching. To merge
|         an (accepted) merge request simply designate @marge-bot as its
|         assignee. Marge will handle the rest. If things go awry let me
|         know.
|  
|  
|  Hi everyone,
|  
|  As you may have noticed, over the last weeks we have been feeling out how
|  best to leverage our new CI infrastructure, particularly when it comes to
|  merging patches.
|  
|  As mentioned a few weeks ago, we have introduced a bot, Marge, to help us
|  work around some temporary limitations of GitLab's merge workflow.
|  Unfortunately, Marge had her own set of quirks which have taken a while
|  to sort out. In particular, fragile tests tended to result in repeated
|  merge attempts which tended to clog up CI, leading to an avalanche of
|  waiting, build failures, and general despair.
|  
|  However, we have been working to improve this situation in three ways:
|  
|   * provisioning more builder capacity to reduce wait times
|  
|   * fixing or disabling fragile tests to reduce the need for retries
|  
|   * enable Marge's batched merge functionality, reducing the number of
|     builds necessary per merged patch [1]
|  
|  In light of this I just wanted to reiterate the previous guidance on
|  merging patches. If you have a merge request you would like to merge
|  simply do the following:
|  
|   1. make sure that it has at least one approval. This should
|      happen in the course of code review but do ping if this was
|      forgotten.
|  
|   2. assign the merge request to @marge-bot using the assignee field in
|      the right-hand sidebar of the merge request page.
|  
|   3. next time Marge does a batch of merges she will fold in your MR
|      (leaving a helpful comment to let you know) and, if the batch passes
|      CI, merge it. If not she will leave a comment letting you know there
|      was an issue. I will try to step in to sort out the mess when this
|      happens.
|  
|  Do let me know if you have any questions.
|  
|  Cheers,
|  
|  - Ben
|  
|  
|  [1] thanks to Matthew Pickering for picking this up
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: How to merge your patch

Matthew Pickering
In reply to this post by Ben Gamari-3
I have attempted to fix the problem (which happened right at the end
of her work) and have now redeployed her.

I will continue to monitor the situation. Her progress can be tracked
https://gitlab.haskell.org/ghc/ghc/merge_requests/307

Matt

On Thu, Feb 7, 2019 at 3:58 AM Ben Gamari <[hidden email]> wrote:

>
> tl;dr. Our beloved @marge-bot is behaving much more reliably now thanks
>        to improvements in CI reliability and build batching. To merge
>        an (accepted) merge request simply designate @marge-bot as its
>        assignee. Marge will handle the rest. If things go awry let me
>        know.
>
>
> Hi everyone,
>
> As you may have noticed, over the last weeks we have been feeling out
> how best to leverage our new CI infrastructure, particularly when it
> comes to merging patches.
>
> As mentioned a few weeks ago, we have introduced a bot, Marge, to help
> us work around some temporary limitations of GitLab's merge workflow.
> Unfortunately, Marge had her own set of quirks which have taken a while
> to sort out. In particular, fragile tests tended to result in repeated
> merge attempts which tended to clog up CI, leading to an avalanche of
> waiting, build failures, and general despair.
>
> However, we have been working to improve this situation in three ways:
>
>  * provisioning more builder capacity to reduce wait times
>
>  * fixing or disabling fragile tests to reduce the need for retries
>
>  * enable Marge's batched merge functionality, reducing the number of
>    builds necessary per merged patch [1]
>
> In light of this I just wanted to reiterate the previous guidance on
> merging patches. If you have a merge request you would like to merge
> simply do the following:
>
>  1. make sure that it has at least one approval. This should
>     happen in the course of code review but do ping if this was
>     forgotten.
>
>  2. assign the merge request to @marge-bot using the assignee field in
>     the right-hand sidebar of the merge request page.
>
>  3. next time Marge does a batch of merges she will fold in your MR
>     (leaving a helpful comment to let you know) and, if the batch passes
>     CI, merge it. If not she will leave a comment letting you know there
>     was an issue. I will try to step in to sort out the mess when this
>     happens.
>
> Do let me know if you have any questions.
>
> Cheers,
>
> - Ben
>
>
> [1] thanks to Matthew Pickering for picking this up
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

RE: How to merge your patch

Ben Gamari-3
In reply to this post by GHC - devs mailing list
Simon Peyton Jones via ghc-devs <[hidden email]> writes:

> |   2. assign the merge request to @marge-bot using the assignee field in
> |      the right-hand sidebar of the merge request page.
>
> This sounds great. What email will we receive in these three cases?
>
When Marge picks up an MR and incorporates it into a batch merge she
leaves a message [1] of the form

>>> I will attempt to batch this MR (!303 (closed))...

[1] https://gitlab.haskell.org/ghc/ghc/merge_requests/58#note_5354



> a) the merge is successful

At this point Marge will close the MR and leave a note [2] of the form

>>> @marge-bot merged 9 hours ago

[2] https://gitlab.haskell.org/ghc/ghc/merge_requests/58#note_5385



> b) the merge fails because I made a mistake and it just doesn't
> validate

Unfortunately I've not seen a case where this has happened since we
enabled batching so I can't comment yet.

However, from previous experience I suspect the message will look
something like this [3]

>>> I couldn't merge this branch: CI failed!

[3] https://gitlab.haskell.org/ghc/ghc/merge_requests/257#note_4788



> c) the merge failed because the CI infrastructure failed somehow
>    (should no longer happen, I know).

From Marge's perspective this typically looks no different from (b). In
the past the infrastructure issues fell into a few buckets:

 * the Windows or Darwin builders ran out of disk space due to [4],
   resulting in spurious failures

 * some fragile test failed


Cheers,

- Ben


[4] https://gitlab.com/gitlab-org/gitlab-runner/issues/3856#note_127887227

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to merge your patch

Matthew Pickering
Latest update. I had to deploy her again because the CI timeout was
set too low (10h). The timeout is now 24 hours.

Sorry for the noisy emails this is generating. Fingers crossed for this time.

Cheers,

Matt

On Thu, Feb 7, 2019 at 4:10 PM Ben Gamari <[hidden email]> wrote:

>
> Simon Peyton Jones via ghc-devs <[hidden email]> writes:
>
> > |   2. assign the merge request to @marge-bot using the assignee field in
> > |      the right-hand sidebar of the merge request page.
> >
> > This sounds great. What email will we receive in these three cases?
> >
> When Marge picks up an MR and incorporates it into a batch merge she
> leaves a message [1] of the form
>
> >>> I will attempt to batch this MR (!303 (closed))...
>
> [1] https://gitlab.haskell.org/ghc/ghc/merge_requests/58#note_5354
>
>
>
> > a) the merge is successful
>
> At this point Marge will close the MR and leave a note [2] of the form
>
> >>> @marge-bot merged 9 hours ago
>
> [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/58#note_5385
>
>
>
> > b) the merge fails because I made a mistake and it just doesn't
> > validate
>
> Unfortunately I've not seen a case where this has happened since we
> enabled batching so I can't comment yet.
>
> However, from previous experience I suspect the message will look
> something like this [3]
>
> >>> I couldn't merge this branch: CI failed!
>
> [3] https://gitlab.haskell.org/ghc/ghc/merge_requests/257#note_4788
>
>
>
> > c) the merge failed because the CI infrastructure failed somehow
> >    (should no longer happen, I know).
>
> From Marge's perspective this typically looks no different from (b). In
> the past the infrastructure issues fell into a few buckets:
>
>  * the Windows or Darwin builders ran out of disk space due to [4],
>    resulting in spurious failures
>
>  * some fragile test failed
>
>
> Cheers,
>
> - Ben
>
>
> [4] https://gitlab.com/gitlab-org/gitlab-runner/issues/3856#note_127887227
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs