Proposal: Gitolite for repository management

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Proposal: Gitolite for repository management

Austin Seipp-4
Hello all,

Recently with the new haskell.org server move, a few of us have taken
roles of administrating the new server infrastructure including
ghc.haskell.org, containing the GHC repositories. (Previously, the GHC
repos were on abbot.haskell.org, which was maintained by Galois. The
new servers are community managed.) I'm one of these people, so first
off if you have any problems, let me know!

I should also say Herbert Valerio Riedel has also stepped up to help
administrate the GHC Trac instance. He's quite experienced in these
sorts of matters, and his help is greatly appreciated. If there's
anything wrong in this area, he can also be of help. :)

Anyway, the real topic: Recently, we have been discussing the way
GHC's repositories are managed, and it's slightly suboptimal for
several reasons. We would instead like to deploy Gitolite, a smart
git-access wrapper. This will not only solve some annoying issues
(like Simon's recent permissions error when pushing to testsuite,) but
also make ghc.haskell.org more secure and easier to maintain.

We have a proposal with preliminary details up here:

http://ghc.haskell.org/trac/ghc/wiki/GitolitePlan

Please refer to it for the exact details. But the visible overview for
all the active developers will be:

 * Shell accounts will go away. You'll only have access to the repositories.

 * Your SSH push URL will change very slightly.

 * sync-all will probably need to change a bit for the new remotes.

This will all need to happen within a small window of downtime. As
outlined above, we believe we can pull off a switch with minimal
interruption. So on that end, we need to know a few things too. What
we'd like to know is:

 * Does any developer who has shell access to ghc.haskell.org actually
*need it*? Outside of administrative tasks, I'm not sure who should
and should not have access. At the least, your privileges will be
slightly reduced after we're done (since the darcs group won't be
needed.)

 * Who is an active committer? I'm not really sure what to do here,
but we can easily transplant all the current users in the 'darcs'
group. Alternatively we can establish it for most of the core
committers, and add people who commit less frequently on a rolling
basis (they can just contact me.)

 * When should this be done? The downtime window will be small
hopefully, and I don't think this would really inconvenience anyone
too much if we did it soon, but I feel we should ask.

--
Regards,
Austin - PGP: 4096R/0x91384671



Reply | Threaded
Open this post in threaded view
|

Proposal: Gitolite for repository management

Geoffrey Mainland
On 07/30/2013 10:41 AM, Austin Seipp wrote:

> Hello all,
>
> Recently with the new haskell.org server move, a few of us have taken
> roles of administrating the new server infrastructure including
> ghc.haskell.org, containing the GHC repositories. (Previously, the GHC
> repos were on abbot.haskell.org, which was maintained by Galois. The
> new servers are community managed.) I'm one of these people, so first
> off if you have any problems, let me know!
>
> I should also say Herbert Valerio Riedel has also stepped up to help
> administrate the GHC Trac instance. He's quite experienced in these
> sorts of matters, and his help is greatly appreciated. If there's
> anything wrong in this area, he can also be of help. :)
>
> Anyway, the real topic: Recently, we have been discussing the way
> GHC's repositories are managed, and it's slightly suboptimal for
> several reasons. We would instead like to deploy Gitolite, a smart
> git-access wrapper. This will not only solve some annoying issues
> (like Simon's recent permissions error when pushing to testsuite,) but
> also make ghc.haskell.org more secure and easier to maintain.
>
> We have a proposal with preliminary details up here:
>
> http://ghc.haskell.org/trac/ghc/wiki/GitolitePlan
>
> Please refer to it for the exact details. But the visible overview for
> all the active developers will be:
>
>  * Shell accounts will go away. You'll only have access to the repositories.
>
>  * Your SSH push URL will change very slightly.
>
>  * sync-all will probably need to change a bit for the new remotes.
>
> This will all need to happen within a small window of downtime. As
> outlined above, we believe we can pull off a switch with minimal
> interruption. So on that end, we need to know a few things too. What
> we'd like to know is:
>
>  * Does any developer who has shell access to ghc.haskell.org actually
> *need it*? Outside of administrative tasks, I'm not sure who should
> and should not have access. At the least, your privileges will be
> slightly reduced after we're done (since the darcs group won't be
> needed.)
>
>  * Who is an active committer? I'm not really sure what to do here,
> but we can easily transplant all the current users in the 'darcs'
> group. Alternatively we can establish it for most of the core
> committers, and add people who commit less frequently on a rolling
> basis (they can just contact me.)
>
>  * When should this be done? The downtime window will be small
> hopefully, and I don't think this would really inconvenience anyone
> too much if we did it soon, but I feel we should ask.

Sounds like a great plan!

Is it possible to have the old server spit back a message when people
try to use it? If so, your suggestion to re-establish less frequent
committers on a rolling basis sounds reasonable---when they use the old
server, it would tell them who to contact to get access to the repository.

Both Simons are on holiday at the moment, so perhaps now is a good time
for the switch? Amazingly, the most recent commit to ghc is my own from
over a week ago!

Geoff




Reply | Threaded
Open this post in threaded view
|

Proposal: Gitolite for repository management

Ian Lynagh-2
In reply to this post by Austin Seipp-4

I know nothing about gitolite, but one detail:

On Tue, Jul 30, 2013 at 04:41:37AM -0500, Austin Seipp wrote:
>
> http://ghc.haskell.org/trac/ghc/wiki/GitolitePlan

In the "Developer changes" you talk about changing the remote URL. It
should be possible to do this for all repos with a sync-all command or
two, but I will leave working out exactly what command(s) to a git
expert.

I don't think any changes to sync-all should be needed, but I could be
wrong.


Thanks
Ian
--
Ian Lynagh, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/



Reply | Threaded
Open this post in threaded view
|

Proposal: Gitolite for repository management

Simon Peyton Jones
In reply to this post by Austin Seipp-4
Austin, Herbert,

All good with me.  Thanks for working on this.

Some thoughts:

* Which repos are covered here?  Just the ones that used to be on abbott?  That is, the ones maintained by GHC HQ?  Better make that clear.

* Does commit permission cover all repos?  If they are just the old GHC HQ ones, fine.  If more, it may need to be more granular?

* Can anyone have a repo on git.haskell.org?  I assume not.

* I have lots of check-out repos.  Each contains lots of .git/config files
   because of the multi-repo nature of the GHC build system.  It would be
   a pain to have to edit each individually.  Maybe you can tell us a sync-all
   command to update the URL and pushurl configs, once per tree.

* This might be a good moment to clean out our committer list.   I suggest the following
   - Start a wiki page on the GHC Trac for committers.  Maybe a section of
        http://ghc.haskell.org/trac/ghc/wiki/Contributors
     or maybe a separate page

   - Send email to all existing committers inviting them to
        * create an entry on the GHC Committers page, saying who they are, where
            they work, and what they work on in GHC
        * reply to you asking for continued commit permission
     Lacking such a reply, you can omit them.  In this way we'll get an up to date
     list.

  - Perhaps we should have a convention that no commits in a year means  you lose
     commit permission.  You can be reinstated by asking, but it means we don't have
     a list filled with ex-committers.

Simon


|  -----Original Message-----
|  From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of Austin
|  Seipp
|  Sent: 30 July 2013 10:42
|  To: ghc-devs at haskell.org; Herbert Valerio Riedel
|  Subject: Proposal: Gitolite for repository management
|  
|  Hello all,
|  
|  Recently with the new haskell.org server move, a few of us have taken
|  roles of administrating the new server infrastructure including
|  ghc.haskell.org, containing the GHC repositories. (Previously, the GHC
|  repos were on abbot.haskell.org, which was maintained by Galois. The
|  new servers are community managed.) I'm one of these people, so first
|  off if you have any problems, let me know!
|  
|  I should also say Herbert Valerio Riedel has also stepped up to help
|  administrate the GHC Trac instance. He's quite experienced in these
|  sorts of matters, and his help is greatly appreciated. If there's
|  anything wrong in this area, he can also be of help. :)
|  
|  Anyway, the real topic: Recently, we have been discussing the way
|  GHC's repositories are managed, and it's slightly suboptimal for
|  several reasons. We would instead like to deploy Gitolite, a smart
|  git-access wrapper. This will not only solve some annoying issues
|  (like Simon's recent permissions error when pushing to testsuite,) but
|  also make ghc.haskell.org more secure and easier to maintain.
|  
|  We have a proposal with preliminary details up here:
|  
|  http://ghc.haskell.org/trac/ghc/wiki/GitolitePlan
|  
|  Please refer to it for the exact details. But the visible overview for
|  all the active developers will be:
|  
|   * Shell accounts will go away. You'll only have access to the repositories.
|  
|   * Your SSH push URL will change very slightly.
|  
|   * sync-all will probably need to change a bit for the new remotes.
|  
|  This will all need to happen within a small window of downtime. As
|  outlined above, we believe we can pull off a switch with minimal
|  interruption. So on that end, we need to know a few things too. What
|  we'd like to know is:
|  
|   * Does any developer who has shell access to ghc.haskell.org actually
|  *need it*? Outside of administrative tasks, I'm not sure who should
|  and should not have access. At the least, your privileges will be
|  slightly reduced after we're done (since the darcs group won't be
|  needed.)
|  
|   * Who is an active committer? I'm not really sure what to do here,
|  but we can easily transplant all the current users in the 'darcs'
|  group. Alternatively we can establish it for most of the core
|  committers, and add people who commit less frequently on a rolling
|  basis (they can just contact me.)
|  
|   * When should this be done? The downtime window will be small
|  hopefully, and I don't think this would really inconvenience anyone
|  too much if we did it soon, but I feel we should ask.
|  
|  --
|  Regards,
|  Austin - PGP: 4096R/0x91384671
|  
|  _______________________________________________
|  ghc-devs mailing list
|  ghc-devs at haskell.org
|  http://www.haskell.org/mailman/listinfo/ghc-devs





Reply | Threaded
Open this post in threaded view
|

Proposal: Gitolite for repository management

Nicolas Frisby
On Tue, Jul 30, 2013 at 8:21 PM, Simon Peyton-Jones
<simonpj at microsoft.com>wrote:

> Austin, Herbert,
>
> All good with me.  Thanks for working on this.
>
>    - Send email to all existing committers inviting them to
>         * create an entry on the GHC Committers page, saying who they are,
> where
>             they work, and what they work on in GHC
>         * reply to you asking for continued commit permission
>      Lacking such a reply, you can omit them.  In this way we'll get an up
> to date
>      list.
>
>   - Perhaps we should have a convention that no commits in a year means
>  you lose
>      commit permission.  You can be reinstated by asking, but it means we
> don't have
>      a list filled with ex-committers.
>
>
Please consider asking recipients of this email to also update
http://ghc.haskell.org/trac/ghc/wiki/ActiveBranches

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130730/619cf1bc/attachment.htm>

Reply | Threaded
Open this post in threaded view
|

Proposal: Gitolite for repository management

Herbert Valerio Riedel
In reply to this post by Simon Peyton Jones
On 2013-07-31 at 03:21:48 +0200, Simon Peyton-Jones wrote:

[...]

> * I have lots of check-out repos.  Each contains lots of .git/config files
>    because of the multi-repo nature of the GHC build system.  It would be
>    a pain to have to edit each individually.  Maybe you can tell us a sync-all
>    command to update the URL and pushurl configs, once per tree.

The following should do the trick with the current `./sync-all` for developers:

  ./sync-all -r ssh://git at git.haskell.org remote set-url origin
  ./sync-all -r ssh://git at git.haskell.org remote set-url --push origin

Non-authenticated users who have been using only the http:// urls don't
have to switch right-away (as the old http://darcs.haskell.org uris will
continue to work), but in the long-term they'll want to update their
Git urls via

  ./sync-all -r http://git.haskell.org remote set-url origin

not the least to benefit from the more efficient Git's smart http
transport[1] (should we decide to enable that on
http://git.haskell.org/).

PS: The reason I modification to `sync-all` are mentioned is because it
    contains references to the old Git urls in its emitted help texts.

[1]: http://git-scm.com/blog/2010/03/04/smart-http.html

Cheers,
  hvr



Reply | Threaded
Open this post in threaded view
|

Proposal: Gitolite for repository management

Simon Peyton Jones
Excellent -- but don't just tell me! Could you add that guidance to the wiki?

Thanks

SImon

|  -----Original Message-----
|  From: Herbert Valerio Riedel [mailto:hvr at gnu.org]
|  Sent: 31 July 2013 10:06
|  To: Simon Peyton-Jones
|  Cc: Austin Seipp; ghc-devs at haskell.org; Herbert Valerio Riedel
|  Subject: Re: Proposal: Gitolite for repository management
|  
|  On 2013-07-31 at 03:21:48 +0200, Simon Peyton-Jones wrote:
|  
|  [...]
|  
|  > * I have lots of check-out repos.  Each contains lots of .git/config files
|  >    because of the multi-repo nature of the GHC build system.  It would be
|  >    a pain to have to edit each individually.  Maybe you can tell us a sync-all
|  >    command to update the URL and pushurl configs, once per tree.
|  
|  The following should do the trick with the current `./sync-all` for developers:
|  
|    ./sync-all -r ssh://git at git.haskell.org remote set-url origin
|    ./sync-all -r ssh://git at git.haskell.org remote set-url --push origin
|  
|  Non-authenticated users who have been using only the http:// urls don't
|  have to switch right-away (as the old http://darcs.haskell.org uris will
|  continue to work), but in the long-term they'll want to update their
|  Git urls via
|  
|    ./sync-all -r http://git.haskell.org remote set-url origin
|  
|  not the least to benefit from the more efficient Git's smart http
|  transport[1] (should we decide to enable that on
|  http://git.haskell.org/).
|  
|  PS: The reason I modification to `sync-all` are mentioned is because it
|      contains references to the old Git urls in its emitted help texts.
|  
|  [1]: http://git-scm.com/blog/2010/03/04/smart-http.html
|  
|  Cheers,
|    hvr