GitLab migration status

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

GitLab migration status

Ben Gamari-3
TL;DR. Given somewhat slower-than-expected progress on the Trac import I
       suggest that we implement a pared-down migration on Tuesday.
       See "The Plan" below.


Hello everyone,

Over the last few weeks we have been hard at work preparing the
migration to GitLab. Currently the following things are ready:

 * Hosting of GHC's repositories and those of its mirrors have been
   prepared.

 * Continuous integration has been configured for GHC.

   All-in-all the GitLab migration has been quite timely since we were
   recently notified by CircleCI of billing changes which will soon make
   it quite difficult for us to continue using their services (see the
   thread on ghc-devops for details).

   Thankfully, moving CI to GitLab has been mostly painless and has
   even enabled us to introduce testing of platforms which were
   previously inaccessible to us under CircleCI.

 * The various linters which previously ran via `arc lint` and gitolite
   post-receive hooks have been ported to CI jobs.

 * The Trac ticket migration is looking good although there are still a
   significant number of details which need to be sorted out.

 * The Wiki migration is in a similar state.

Over the past weeks we have been in constant contact with GitLab's FOSS
outreach group, who have been quite helpful in getting the eyes of
GitLab employees on the issues affecting our transition. Thanks to
especially to David Planella for his help so far.

Unfortunately, there is one issue in particular [1] which is currently
blocking the Trac migration. From my discussions with GitLab's upstream
it sounds like it may be possible for them to prioritize a fix in the
short-term. However, our aggressive migration timeline is a fair bit
faster than GitLab's development cycle and consequently this certainly
won't happen before our planned migration on Tuesday.


# The Plan

Given what remains to be done in the Trac migration I believe it would
be a mistake to move ahead with the full migration as planned. However,
in the interest of re-gaining functional continuous integration of
patches as soon as possible I propose that we move ahead with moving
code review on Tuesday.

The plan would be as follows:

 1. We setup the final gitlab.haskell.org instance tomorrow; since the
    Trac migration will not be run will need to create new accounts on
    instance.

 2. We begin officially accepting merge requests on this fresh GitLab
    instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will
    become GHC's official upstream repository.

 3. We allow a week of transition time where new Differentials will
    continue to be accepted via Phabricator.

 4. After this transition period we place Phabricator in read-only mode.

 5. When we are confident in the Trac migration (likely after the new
    year) we move ahead with importing tickets and the wiki

Previously I was skeptical of any plan that involved running the Trac
migration against a live GitLab instance. However, further reflection
I believe such a migration is safe and feasible. Moreover, given the
constraints set upon us by the impending CircleCI changes, I think this
is our best option to ensure continuity of CI.

Thoughts?

Cheers,

- Ben


[1] https://gitlab.com/gitlab-org/gitlab-ce/issues/46980

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [GHC DevOps Group] GitLab migration status

Simon Marlow-7
Hi Ben - this sounds good, a couple of questions:

- What about the performance issue we noticed last week?
- What will happen to Phabricator diffs that are still mid-review? It would be a shame to have to move them to gitlab and interrupt the review trail. Can't we just shut Phabricator to new diffs but keep the possibility of working on existing ones?

Cheers
Simon

On Mon, 17 Dec 2018 at 05:29, Ben Gamari <[hidden email]> wrote:
TL;DR. Given somewhat slower-than-expected progress on the Trac import I
       suggest that we implement a pared-down migration on Tuesday.
       See "The Plan" below.


Hello everyone,

Over the last few weeks we have been hard at work preparing the
migration to GitLab. Currently the following things are ready:

 * Hosting of GHC's repositories and those of its mirrors have been
   prepared.

 * Continuous integration has been configured for GHC.

   All-in-all the GitLab migration has been quite timely since we were
   recently notified by CircleCI of billing changes which will soon make
   it quite difficult for us to continue using their services (see the
   thread on ghc-devops for details).

   Thankfully, moving CI to GitLab has been mostly painless and has
   even enabled us to introduce testing of platforms which were
   previously inaccessible to us under CircleCI.

 * The various linters which previously ran via `arc lint` and gitolite
   post-receive hooks have been ported to CI jobs.

 * The Trac ticket migration is looking good although there are still a
   significant number of details which need to be sorted out.

 * The Wiki migration is in a similar state.

Over the past weeks we have been in constant contact with GitLab's FOSS
outreach group, who have been quite helpful in getting the eyes of
GitLab employees on the issues affecting our transition. Thanks to
especially to David Planella for his help so far.

Unfortunately, there is one issue in particular [1] which is currently
blocking the Trac migration. From my discussions with GitLab's upstream
it sounds like it may be possible for them to prioritize a fix in the
short-term. However, our aggressive migration timeline is a fair bit
faster than GitLab's development cycle and consequently this certainly
won't happen before our planned migration on Tuesday.


# The Plan

Given what remains to be done in the Trac migration I believe it would
be a mistake to move ahead with the full migration as planned. However,
in the interest of re-gaining functional continuous integration of
patches as soon as possible I propose that we move ahead with moving
code review on Tuesday.

The plan would be as follows:

 1. We setup the final gitlab.haskell.org instance tomorrow; since the
    Trac migration will not be run will need to create new accounts on
    instance.

 2. We begin officially accepting merge requests on this fresh GitLab
    instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will
    become GHC's official upstream repository.

 3. We allow a week of transition time where new Differentials will
    continue to be accepted via Phabricator.

 4. After this transition period we place Phabricator in read-only mode.

 5. When we are confident in the Trac migration (likely after the new
    year) we move ahead with importing tickets and the wiki

Previously I was skeptical of any plan that involved running the Trac
migration against a live GitLab instance. However, further reflection
I believe such a migration is safe and feasible. Moreover, given the
constraints set upon us by the impending CircleCI changes, I think this
is our best option to ensure continuity of CI.

Thoughts?

Cheers,

- Ben


[1] https://gitlab.com/gitlab-org/gitlab-ce/issues/46980
_______________________________________________
Ghc-devops-group mailing list
[hidden email]
https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: [GHC DevOps Group] GitLab migration status

GHC - devs mailing list
In reply to this post by Ben Gamari-3
Hi Ben,

in my experience Gitlab has been extremely slow at showing commit diffs to the point that it gives up and returns a 502: https://gitlab.staging.haskell.org/ghc/ghc/issues/15944

Is this possibly related to any resource constraints on our instance?

Cheers,
Simon

Am Mo., 17. Dez. 2018 um 06:29 Uhr schrieb Ben Gamari <[hidden email]>:
TL;DR. Given somewhat slower-than-expected progress on the Trac import I
       suggest that we implement a pared-down migration on Tuesday.
       See "The Plan" below.


Hello everyone,

Over the last few weeks we have been hard at work preparing the
migration to GitLab. Currently the following things are ready:

 * Hosting of GHC's repositories and those of its mirrors have been
   prepared.

 * Continuous integration has been configured for GHC.

   All-in-all the GitLab migration has been quite timely since we were
   recently notified by CircleCI of billing changes which will soon make
   it quite difficult for us to continue using their services (see the
   thread on ghc-devops for details).

   Thankfully, moving CI to GitLab has been mostly painless and has
   even enabled us to introduce testing of platforms which were
   previously inaccessible to us under CircleCI.

 * The various linters which previously ran via `arc lint` and gitolite
   post-receive hooks have been ported to CI jobs.

 * The Trac ticket migration is looking good although there are still a
   significant number of details which need to be sorted out.

 * The Wiki migration is in a similar state.

Over the past weeks we have been in constant contact with GitLab's FOSS
outreach group, who have been quite helpful in getting the eyes of
GitLab employees on the issues affecting our transition. Thanks to
especially to David Planella for his help so far.

Unfortunately, there is one issue in particular [1] which is currently
blocking the Trac migration. From my discussions with GitLab's upstream
it sounds like it may be possible for them to prioritize a fix in the
short-term. However, our aggressive migration timeline is a fair bit
faster than GitLab's development cycle and consequently this certainly
won't happen before our planned migration on Tuesday.


# The Plan

Given what remains to be done in the Trac migration I believe it would
be a mistake to move ahead with the full migration as planned. However,
in the interest of re-gaining functional continuous integration of
patches as soon as possible I propose that we move ahead with moving
code review on Tuesday.

The plan would be as follows:

 1. We setup the final gitlab.haskell.org instance tomorrow; since the
    Trac migration will not be run will need to create new accounts on
    instance.

 2. We begin officially accepting merge requests on this fresh GitLab
    instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will
    become GHC's official upstream repository.

 3. We allow a week of transition time where new Differentials will
    continue to be accepted via Phabricator.

 4. After this transition period we place Phabricator in read-only mode.

 5. When we are confident in the Trac migration (likely after the new
    year) we move ahead with importing tickets and the wiki

Previously I was skeptical of any plan that involved running the Trac
migration against a live GitLab instance. However, further reflection
I believe such a migration is safe and feasible. Moreover, given the
constraints set upon us by the impending CircleCI changes, I think this
is our best option to ensure continuity of CI.

Thoughts?

Cheers,

- Ben


[1] https://gitlab.com/gitlab-org/gitlab-ce/issues/46980
_______________________________________________
Ghc-devops-group mailing list
[hidden email]
https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: [GHC DevOps Group] GitLab migration status

Ben Gamari-3
Simon Jakobi via ghc-devs <[hidden email]> writes:

> Hi Ben,
>
> in my experience Gitlab has been extremely slow at showing commit diffs to
> the point that it gives up and returns a 502:
> https://gitlab.staging.haskell.org/ghc/ghc/issues/15944
>
> Is this possibly related to any resource constraints on our instance?
>
Yes, I have also noticed the general slowness of our instance. Last week
I identified the cause as being the fact that it is being hosted on an
AArch64 box, which apparently isn't a configuration that is very well
supported by the Ruby interpreter's optimised implementation.

The same instance running on x86_64 is much more responsive [1].

Cheers,

- Ben


[1] https://gitlab.staging2.haskell.org/ghc/ghc/merge_requests/11

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [GHC DevOps Group] GitLab migration status

Ben Gamari-3
In reply to this post by Simon Marlow-7
Simon Marlow <[hidden email]> writes:

> Hi Ben - this sounds good, a couple of questions:
>
> - What about the performance issue we noticed last week?

I identified the problem as being the ARM environment which the staging
instance is running on. The final instance will run on x86_64.

> - What will happen to Phabricator diffs that are still mid-review? It would
> be a shame to have to move them to gitlab and interrupt the review trail.
> Can't we just shut Phabricator to new diffs but keep the possibility of
> working on existing ones?
>
Of course. That is perhaps a better path forward.

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [GHC DevOps Group] GitLab migration status

Richard Eisenberg-4


On Dec 17, 2018, at 10:52 AM, Ben Gamari <[hidden email]> wrote:


- What will happen to Phabricator diffs that are still mid-review? It would
be a shame to have to move them to gitlab and interrupt the review trail.
Can't we just shut Phabricator to new diffs but keep the possibility of
working on existing ones?

Of course. That is perhaps a better path forward.

I second this notion, that we stop accepting new diffs on Phab but continue to work with existing diffs until they are merged or abandoned (depending on how resource-intensive it is to do this).

Thanks,
Richard

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: [GHC DevOps Group] GitLab migration status

Carter Schonwald
In reply to this post by GHC - devs mailing list
Hey simon!
I actually did some digging. And it looks like currently gitlab basically does a git diff of the two branches and does that computation on every page reload.  

1). There currently is no caching —- some of the folks on the Haskell infra team could help perhaps but they need to be given access to the setup for that to happen. 

2) the complexity of the git diff is linear in the commit history length between the two branches being compared 


Some of these things could be mitigated easily, but I spoke with one of the folks on the Haskell infra team and it sounds like ultimately either patching the gitlab gitaly server or providing a more efficient api compatible replacement are what would be needed. 

On Mon, Dec 17, 2018 at 3:18 AM Simon Jakobi via ghc-devs <[hidden email]> wrote:
Hi Ben,

in my experience Gitlab has been extremely slow at showing commit diffs to the point that it gives up and returns a 502: https://gitlab.staging.haskell.org/ghc/ghc/issues/15944

Is this possibly related to any resource constraints on our instance?

Cheers,
Simon

Am Mo., 17. Dez. 2018 um 06:29 Uhr schrieb Ben Gamari <[hidden email]>:
TL;DR. Given somewhat slower-than-expected progress on the Trac import I
       suggest that we implement a pared-down migration on Tuesday.
       See "The Plan" below.


Hello everyone,

Over the last few weeks we have been hard at work preparing the
migration to GitLab. Currently the following things are ready:

 * Hosting of GHC's repositories and those of its mirrors have been
   prepared.

 * Continuous integration has been configured for GHC.

   All-in-all the GitLab migration has been quite timely since we were
   recently notified by CircleCI of billing changes which will soon make
   it quite difficult for us to continue using their services (see the
   thread on ghc-devops for details).

   Thankfully, moving CI to GitLab has been mostly painless and has
   even enabled us to introduce testing of platforms which were
   previously inaccessible to us under CircleCI.

 * The various linters which previously ran via `arc lint` and gitolite
   post-receive hooks have been ported to CI jobs.

 * The Trac ticket migration is looking good although there are still a
   significant number of details which need to be sorted out.

 * The Wiki migration is in a similar state.

Over the past weeks we have been in constant contact with GitLab's FOSS
outreach group, who have been quite helpful in getting the eyes of
GitLab employees on the issues affecting our transition. Thanks to
especially to David Planella for his help so far.

Unfortunately, there is one issue in particular [1] which is currently
blocking the Trac migration. From my discussions with GitLab's upstream
it sounds like it may be possible for them to prioritize a fix in the
short-term. However, our aggressive migration timeline is a fair bit
faster than GitLab's development cycle and consequently this certainly
won't happen before our planned migration on Tuesday.


# The Plan

Given what remains to be done in the Trac migration I believe it would
be a mistake to move ahead with the full migration as planned. However,
in the interest of re-gaining functional continuous integration of
patches as soon as possible I propose that we move ahead with moving
code review on Tuesday.

The plan would be as follows:

 1. We setup the final gitlab.haskell.org instance tomorrow; since the
    Trac migration will not be run will need to create new accounts on
    instance.

 2. We begin officially accepting merge requests on this fresh GitLab
    instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will
    become GHC's official upstream repository.

 3. We allow a week of transition time where new Differentials will
    continue to be accepted via Phabricator.

 4. After this transition period we place Phabricator in read-only mode.

 5. When we are confident in the Trac migration (likely after the new
    year) we move ahead with importing tickets and the wiki

Previously I was skeptical of any plan that involved running the Trac
migration against a live GitLab instance. However, further reflection
I believe such a migration is safe and feasible. Moreover, given the
constraints set upon us by the impending CircleCI changes, I think this
is our best option to ensure continuity of CI.

Thoughts?

Cheers,

- Ben


[1] https://gitlab.com/gitlab-org/gitlab-ce/issues/46980
_______________________________________________
Ghc-devops-group mailing list
[hidden email]
https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: GitLab migration status

Joachim Breitner-2
In reply to this post by Ben Gamari-3
Hi,

Am Montag, den 17.12.2018, 00:29 -0500 schrieb Ben Gamari:
>  2. We begin officially accepting merge requests on this fresh GitLab
>     instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will
>     become GHC's official upstream repository.

I guess this works only because GitLab has a different name (number)
space for merge requests and issues, so creating MRs does not create
collisions with the issues merged later … so yay :-)

Cheers,
Joachim
--
Joachim Breitner
  [hidden email]
  http://www.joachim-breitner.de/


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment