Repository Reorganization Question

classic Classic list List threaded Threaded
44 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Austin Seipp-5
Hi all,

While discussing something with Herbert this week in preparation of
making a new stable branch, he brought a good point to my attention,
which is that if we go ahead and reorganize the repository situation
post 7.8, merging things to the stable branch from HEAD will become a
bit harder.

Notably, we had planned to fold testsuite (and perhaps some other
repositories) into the GHC tree. Once we do this, the two branches
will have diverged quite a bit, so merging from HEAD to STABLE will
become harder* (because HEAD would have rolled in testsuite changes
for example, but the STABLE branch would not have this history.)

Thinking about it, the best time to do such a move is, basically, when
there is no active stable branch. Unfortunately this time is right
now, but I'm not sure how everyone feels about this.

So, the question is: should we go ahead and pull the trigger on some
of these perhaps? Herbert collected some numbers on the git
repositories and outlined all the basic details here:

https://ghc.haskell.org/trac/ghc/wiki/GitRepoReorganization

The only thing I'd honestly propose right now is folding 'testsuite'
into the main repository, but of course we should see what people
think - perhaps we should keep base/etc off the table for now, since
they seem more controversial.

* I'll point out they will only become *slightly* harder in most
cases, because I can always instead apply unified diffs, rather than
cherry pick or something. But it does lose the original metadata from
commits too. But I won't cry if people vote against this.

--
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Simon Peyton Jones
I have no objection to doing it now, though my opinion should count for little, since I know so little. The main person who is affected is Austin, since he has to do the merging. And I'm not sure that even 'base' is really controversial, is it?  
       
Simon

| -----Original Message-----
| From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of
| Austin Seipp
| Sent: 04 December 2013 21:25
| To: ghc-devs at haskell.org
| Subject: Repository Reorganization Question
|
| Hi all,
|
| While discussing something with Herbert this week in preparation of
| making a new stable branch, he brought a good point to my attention,
| which is that if we go ahead and reorganize the repository situation
| post 7.8, merging things to the stable branch from HEAD will become a
| bit harder.
|
| Notably, we had planned to fold testsuite (and perhaps some other
| repositories) into the GHC tree. Once we do this, the two branches
| will have diverged quite a bit, so merging from HEAD to STABLE will
| become harder* (because HEAD would have rolled in testsuite changes
| for example, but the STABLE branch would not have this history.)
|
| Thinking about it, the best time to do such a move is, basically, when
| there is no active stable branch. Unfortunately this time is right
| now, but I'm not sure how everyone feels about this.
|
| So, the question is: should we go ahead and pull the trigger on some
| of these perhaps? Herbert collected some numbers on the git
| repositories and outlined all the basic details here:
|
| https://ghc.haskell.org/trac/ghc/wiki/GitRepoReorganization
|
| The only thing I'd honestly propose right now is folding 'testsuite'
| into the main repository, but of course we should see what people
| think - perhaps we should keep base/etc off the table for now, since
| they seem more controversial.
|
| * I'll point out they will only become *slightly* harder in most
| cases, because I can always instead apply unified diffs, rather than
| cherry pick or something. But it does lose the original metadata from
| commits too. But I won't cry if people vote against this.
|
| --
| Regards,
|
| Austin Seipp, Haskell Consultant
| Well-Typed LLP, http://www.well-typed.com/
| _______________________________________________
| ghc-devs mailing list
| ghc-devs at haskell.org
| http://www.haskell.org/mailman/listinfo/ghc-devs

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Carter Schonwald
i'm fine with them being merged, otoh I don't have any large in progress
patches afoot, and having testsuite and ghc repos merged will make it much
easier for some of the patches i have planned for end of december.

-Carter


On Wed, Dec 4, 2013 at 5:13 PM, Simon Peyton-Jones <simonpj at microsoft.com>wrote:

> I have no objection to doing it now, though my opinion should count for
> little, since I know so little. The main person who is affected is Austin,
> since he has to do the merging. And I'm not sure that even 'base' is really
> controversial, is it?
>
> Simon
>
> | -----Original Message-----
> | From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of
> | Austin Seipp
> | Sent: 04 December 2013 21:25
> | To: ghc-devs at haskell.org
> | Subject: Repository Reorganization Question
> |
> | Hi all,
> |
> | While discussing something with Herbert this week in preparation of
> | making a new stable branch, he brought a good point to my attention,
> | which is that if we go ahead and reorganize the repository situation
> | post 7.8, merging things to the stable branch from HEAD will become a
> | bit harder.
> |
> | Notably, we had planned to fold testsuite (and perhaps some other
> | repositories) into the GHC tree. Once we do this, the two branches
> | will have diverged quite a bit, so merging from HEAD to STABLE will
> | become harder* (because HEAD would have rolled in testsuite changes
> | for example, but the STABLE branch would not have this history.)
> |
> | Thinking about it, the best time to do such a move is, basically, when
> | there is no active stable branch. Unfortunately this time is right
> | now, but I'm not sure how everyone feels about this.
> |
> | So, the question is: should we go ahead and pull the trigger on some
> | of these perhaps? Herbert collected some numbers on the git
> | repositories and outlined all the basic details here:
> |
> | https://ghc.haskell.org/trac/ghc/wiki/GitRepoReorganization
> |
> | The only thing I'd honestly propose right now is folding 'testsuite'
> | into the main repository, but of course we should see what people
> | think - perhaps we should keep base/etc off the table for now, since
> | they seem more controversial.
> |
> | * I'll point out they will only become *slightly* harder in most
> | cases, because I can always instead apply unified diffs, rather than
> | cherry pick or something. But it does lose the original metadata from
> | commits too. But I won't cry if people vote against this.
> |
> | --
> | Regards,
> |
> | Austin Seipp, Haskell Consultant
> | Well-Typed LLP, http://www.well-typed.com/
> | _______________________________________________
> | ghc-devs mailing list
> | ghc-devs at haskell.org
> | http://www.haskell.org/mailman/listinfo/ghc-devs
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131204/f76f5914/attachment.html>

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Joachim Breitner-2
In reply to this post by Austin Seipp-5
Hi,

Am Mittwoch, den 04.12.2013, 15:24 -0600 schrieb Austin Seipp:
> The only thing I'd honestly propose right now is folding 'testsuite'
> into the main repository, but of course we should see what people
> think - perhaps we should keep base/etc off the table for now, since
> they seem more controversial.

yes please! It will make live much easier for me. And it would allow to
remove "expect_broken" flags with the same commit that unbreaks them.

Greetings,
Joachim
--
Joachim ?nomeata? Breitner
  mail at joachim-breitner.de ? http://www.joachim-breitner.de/
  Jabber: nomeata at joachim-breitner.de  ? GPG-Key: 0x4743206C
  Debian Developer: nomeata at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131204/733a3ea7/attachment.sig>

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Herbert Valerio Riedel
In reply to this post by Austin Seipp-5
On 2013-12-04 at 22:24:40 +0100, Austin Seipp wrote:
> So, the question is: should we go ahead and pull the trigger on some
> of these perhaps? Herbert collected some numbers on the git
> repositories and outlined all the basic details here:
>
> https://ghc.haskell.org/trac/ghc/wiki/GitRepoReorganization
>
> The only thing I'd honestly propose right now is folding 'testsuite'
> into the main repository, but of course we should see what people
> think

Fyi, I've drafted how the change would look like in the new ghc.git
branch 'wip/T8545' so we can test/evaluate the effects/fallout before
peforming this operation on 'master'.

So running

 git clone -b wip/T8545 git://git.haskell.org/ghc.git
 cd ghc/
 ./sync-all get

should result in a new checkout including the folded-in testsuite/
folder.

PS: If we decide to fold in `base` too (which imho would make life
    easier for GHC devs as it reduces exposure to confusing
    git-submodule effects) I think we should also fold in
    ghc-prim/integer-{gmp,simple} in, as `base` depends on those and
    they're even more tightly coupled to GHC than `base` is and so imho
    don't benefit much from being kept in a separate Git repo.

Cheers,
  hvr

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Herbert Valerio Riedel-3
On 2013-12-05 at 11:17:01 +0100, Herbert Valerio Riedel wrote:

[...]

> Fyi, I've drafted how the change would look like in the new ghc.git
> branch 'wip/T8545' so we can test/evaluate the effects/fallout before
> peforming this operation on 'master'.
>
> So running
>
>  git clone -b wip/T8545 git://git.haskell.org/ghc.git
>  cd ghc/
>  ./sync-all get
>
> should result in a new checkout including the folded-in testsuite/
> folder.

PS: I didn't merge in testsuite's Git history as that would bloat
    ghc.git quite a bit; however, 'git blame' functionality can be
    recovered in a local checkout by using something like Git's grafting
    feature:

    # make available old testsuite Git objects in local ghc.git
    git remote add -f old-testsuite git://git.haskell.org/testsuite.git

    # add 2nd parent commit to e45b9f57a90 pointing to testsuite.git
    echo e45b9f57a9044e8a20e3cc13bcff86b12b3da405 \
         1860dae3a7e377f085f3a4134f532a7f577fccbe \
         3e66489ebcef0f4cd86968c6781a1d4ad1981f94 > .git/info/grafts

    This way when peforming 'git blame' on files in the the testsuite/
    folder results in sensible information dating back before the
    history-cut-off point.


Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Simon Peyton Jones
What (if anything) do we need to do when updating existing local repos.  Will everything be ok if I just do
   sync-all pull

Simon

| -----Original Message-----
| From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of
| Herbert Valerio Riedel
| Sent: 05 December 2013 11:15
| To: ghc-devs at haskell.org
| Subject: Re: Repository Reorganization Question
|
| On 2013-12-05 at 11:17:01 +0100, Herbert Valerio Riedel wrote:
|
| [...]
|
| > Fyi, I've drafted how the change would look like in the new ghc.git
| > branch 'wip/T8545' so we can test/evaluate the effects/fallout before
| > peforming this operation on 'master'.
| >
| > So running
| >
| >  git clone -b wip/T8545 git://git.haskell.org/ghc.git
| >  cd ghc/
| >  ./sync-all get
| >
| > should result in a new checkout including the folded-in testsuite/
| > folder.
|
| PS: I didn't merge in testsuite's Git history as that would bloat
|     ghc.git quite a bit; however, 'git blame' functionality can be
|     recovered in a local checkout by using something like Git's
| grafting
|     feature:
|
|     # make available old testsuite Git objects in local ghc.git
|     git remote add -f old-testsuite git://git.haskell.org/testsuite.git
|
|     # add 2nd parent commit to e45b9f57a90 pointing to testsuite.git
|     echo e45b9f57a9044e8a20e3cc13bcff86b12b3da405 \
|          1860dae3a7e377f085f3a4134f532a7f577fccbe \
|          3e66489ebcef0f4cd86968c6781a1d4ad1981f94 > .git/info/grafts
|
|     This way when peforming 'git blame' on files in the the testsuite/
|     folder results in sensible information dating back before the
|     history-cut-off point.
|
| _______________________________________________
| ghc-devs mailing list
| ghc-devs at haskell.org
| http://www.haskell.org/mailman/listinfo/ghc-devs

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Joachim Breitner-2
In reply to this post by Herbert Valerio Riedel-3
Hi,

Am Donnerstag, den 05.12.2013, 12:15 +0100 schrieb Herbert Valerio
Riedel:
> PS: I didn't merge in testsuite's Git history as that would bloat
>     ghc.git quite a bit;

would that really be a problem? How different are the numbers?

I?m a fan of keeping history readily available, so unless it really
hurts I suggest to do a proper merge.


Greetings,
Joachim

--
Joachim ?nomeata? Breitner
  mail at joachim-breitner.de ? http://www.joachim-breitner.de/
  Jabber: nomeata at joachim-breitner.de  ? GPG-Key: 0x4743206C
  Debian Developer: nomeata at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131205/ffd1cc99/attachment.sig>

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Herbert Valerio Riedel
Hello Joachim,

On 2013-12-05 at 12:56:55 +0100, Joachim Breitner wrote:
> Am Donnerstag, den 05.12.2013, 12:15 +0100 schrieb Herbert Valerio
> Riedel:
>> PS: I didn't merge in testsuite's Git history as that would bloat
>>     ghc.git quite a bit;
>
> would that really be a problem? How different are the numbers?

Here's a rough estimate:

testsuite.git current single packfile:

 1.8M Dec  5 14:18 .git/objects/pack/pack-5d85ce17a3003e44e0e36d757564ce7df09275d4.idx
  27M Dec  5 14:18 .git/objects/pack/pack-5d85ce17a3003e44e0e36d757564ce7df09275d4.pack

whereas, when I create a new git repo containing only the HEAD commit
from testsuite.git, the resulting single packfile:

 204K Dec  5 14:19 .git/objects/pack/pack-27355d714321978fd34c21ce341a7b55f416719a.idx
 2.5M Dec  5 14:19 .git/objects/pack/pack-27355d714321978fd34c21ce341a7b55f416719a.pack

this seemed to be a significant increase to me;

> I?m a fan of keeping history readily available, so unless it really
> hurts I suggest to do a proper merge.

btw, it'd be easy to provide a simple script which would re-attach the
testsuite history (and any other repositories with truncated history)

but there's another subtle issue; there's multiple ways to merge in the
old testsuite repo, one is without any path-translation, as accomplished
by the grafting example I gave; the other is to first rewrite the
'testsuite.git' to have its root-folder being located in a 'testsuite/'
folder, so that Git doesn't have to follow renames and thus maybe also
simplify navigating/querying the Git history.

Cheers,
  hvr

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Johan Tibell-2
Lets not lose our history or make it annoying to access. Disk is cheap.


On Thu, Dec 5, 2013 at 2:32 PM, Herbert Valerio Riedel <hvr at gnu.org> wrote:

> Hello Joachim,
>
> On 2013-12-05 at 12:56:55 +0100, Joachim Breitner wrote:
> > Am Donnerstag, den 05.12.2013, 12:15 +0100 schrieb Herbert Valerio
> > Riedel:
> >> PS: I didn't merge in testsuite's Git history as that would bloat
> >>     ghc.git quite a bit;
> >
> > would that really be a problem? How different are the numbers?
>
> Here's a rough estimate:
>
> testsuite.git current single packfile:
>
>  1.8M Dec  5 14:18
> .git/objects/pack/pack-5d85ce17a3003e44e0e36d757564ce7df09275d4.idx
>   27M Dec  5 14:18
> .git/objects/pack/pack-5d85ce17a3003e44e0e36d757564ce7df09275d4.pack
>
> whereas, when I create a new git repo containing only the HEAD commit
> from testsuite.git, the resulting single packfile:
>
>  204K Dec  5 14:19
> .git/objects/pack/pack-27355d714321978fd34c21ce341a7b55f416719a.idx
>  2.5M Dec  5 14:19
> .git/objects/pack/pack-27355d714321978fd34c21ce341a7b55f416719a.pack
>
> this seemed to be a significant increase to me;
>
> > I?m a fan of keeping history readily available, so unless it really
> > hurts I suggest to do a proper merge.
>
> btw, it'd be easy to provide a simple script which would re-attach the
> testsuite history (and any other repositories with truncated history)
>
> but there's another subtle issue; there's multiple ways to merge in the
> old testsuite repo, one is without any path-translation, as accomplished
> by the grafting example I gave; the other is to first rewrite the
> 'testsuite.git' to have its root-folder being located in a 'testsuite/'
> folder, so that Git doesn't have to follow renames and thus maybe also
> simplify navigating/querying the Git history.
>
> Cheers,
>   hvr
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131205/a8bad917/attachment.html>

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Herbert Valerio Riedel
In reply to this post by Simon Peyton Jones
On 2013-12-05 at 12:31:40 +0100, Simon Peyton-Jones wrote:
> What (if anything) do we need to do when updating existing local repos.  Will everything be ok if I just do
>    sync-all pull

...assuming there's no important uncommitted data left in testsuite/
(and ideally nowhere else in the source-tree), it *should* suffice to
just './sync-all pull' as this operation *should* overwrite any
testsuite/* files laying around (at least that's what I've observed so
far in my tests)

However, if the testsuite/ was already checked out before the 'sync-all
pull', the 'testsuite/.git' folder won't be removed automatically (and
it shouldn't hurt either, as 'sync-all' won't traverse it anymore after
ghc.git was updated)

Cheers,
  hvr

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Ian Lynagh
On Thu, Dec 05, 2013 at 03:03:42PM +0100, Herbert Valerio Riedel wrote:
>
> However, if the testsuite/ was already checked out before the 'sync-all
> pull', the 'testsuite/.git' folder won't be removed automatically (and
> it shouldn't hurt either, as 'sync-all' won't traverse it anymore after
> ghc.git was updated)

But git commands under testsuite/ will use the wrong .git, won't they?


Thanks
Ian


Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Herbert Valerio Riedel
On 2013-12-05 at 15:17:53 +0100, Ian Lynagh wrote:
> On Thu, Dec 05, 2013 at 03:03:42PM +0100, Herbert Valerio Riedel wrote:
>>
>> However, if the testsuite/ was already checked out before the 'sync-all
>> pull', the 'testsuite/.git' folder won't be removed automatically (and
>> it shouldn't hurt either, as 'sync-all' won't traverse it anymore after
>> ghc.git was updated)
>
> But git commands under testsuite/ will use the wrong .git, won't they?

good point, didn't think of that :-/

So I guess 'sync-all' should be tweaked to either delete the .git/
folder or warn the user to remove it;

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Geoffrey Mainland
In reply to this post by Austin Seipp-5
I'm all for converting to submodules. Since we will have submodules
anyway, we could also convert testsuite et al to submodules and see how
painful that is before deciding to fold them in to the main repo. I'm
not clear on the pros/cons of having, e.g., testsuite, as a submodule vs
folded in. The submodule approach will certainly maintain history!

Geoff

On 12/04/2013 04:24 PM, Austin Seipp wrote:

> Hi all,
>
> While discussing something with Herbert this week in preparation of
> making a new stable branch, he brought a good point to my attention,
> which is that if we go ahead and reorganize the repository situation
> post 7.8, merging things to the stable branch from HEAD will become a
> bit harder.
>
> Notably, we had planned to fold testsuite (and perhaps some other
> repositories) into the GHC tree. Once we do this, the two branches
> will have diverged quite a bit, so merging from HEAD to STABLE will
> become harder* (because HEAD would have rolled in testsuite changes
> for example, but the STABLE branch would not have this history.)
>
> Thinking about it, the best time to do such a move is, basically, when
> there is no active stable branch. Unfortunately this time is right
> now, but I'm not sure how everyone feels about this.
>
> So, the question is: should we go ahead and pull the trigger on some
> of these perhaps? Herbert collected some numbers on the git
> repositories and outlined all the basic details here:
>
> https://ghc.haskell.org/trac/ghc/wiki/GitRepoReorganization
>
> The only thing I'd honestly propose right now is folding 'testsuite'
> into the main repository, but of course we should see what people
> think - perhaps we should keep base/etc off the table for now, since
> they seem more controversial.
>
> * I'll point out they will only become *slightly* harder in most
> cases, because I can always instead apply unified diffs, rather than
> cherry pick or something. But it does lose the original metadata from
> commits too. But I won't cry if people vote against this.
>


Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Herbert Valerio Riedel
In reply to this post by Herbert Valerio Riedel
On 2013-12-05 at 14:32:10 +0100, Herbert Valerio Riedel wrote:

[...]

> whereas, when I create a new git repo containing only the HEAD commit
> from testsuite.git, the resulting single packfile:
>
>  204K Dec  5 14:19 .git/objects/pack/pack-27355d714321978fd34c21ce341a7b55f416719a.idx
>  2.5M Dec  5 14:19 .git/objects/pack/pack-27355d714321978fd34c21ce341a7b55f416719a.pack
>
> this seemed to be a significant increase to me;

PS: if anyone wonders why the testsuite.git history is so large: there
were a few *huge* binary files with bad compressibility checked in by
accident, such as the one removed via

 http://git.haskell.org/testsuite.git/commitdiff/68acef7dd144452db12689db3299ae7106d87f16

 tests/haddock/should_compile_flag_nohaddock/a.out   | Bin 2745963 -> 0 bytes
 tests/haddock/should_compile_noflag_nohaddock/a.out | Bin 2745963 -> 0 bytes

or

 http://git.haskell.org/testsuite.git/commitdiff/cb540135b26504cffe557fd57fa3bb936e413769

 tests/ghc-regress/dph/diophantine/dph-diophantine-fast | Bin 16854700 -> 0 bytes
 tests/ghc-regress/dph/diophantine/dph-diophantine-opt  | Bin 17017376 -> 0 bytes
 tests/ghc-regress/dph/primespj/dph-primespj-fast       | Bin 16783780 -> 0 bytes
 tests/ghc-regress/dph/quickhull/dph-quickhull-fast     | Bin 17092732 -> 0 bytes
 tests/ghc-regress/dph/smvm/dph-smvm                    | Bin 16581028 -> 0 bytes
 tests/ghc-regress/dph/sumnats/dph-sumnats              | Bin 16101268 -> 0 bytes
 tests/ghc-regress/dph/words/dph-words-fast             | Bin 17580076 -> 0 bytes

which account for most of the ~20MiB history....

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Joachim Breitner-2
Hi,

Am Freitag, den 06.12.2013, 11:05 +0100 schrieb Herbert Valerio Riedel:
> PS: if anyone wonders why the testsuite.git history is so large: there
> were a few *huge* binary files with bad compressibility checked in by
> accident, such as the one removed via
>
>[..]
>
> s/dph/words/dph-words-fast             | Bin 17580076 -> 0 bytes
>
> which account for most of the ~20MiB history....

so if we were to do the marge including a pathname rewrite (which I tend
to think of as fine), we could remove these files from the history as
well, and get a reasonably sized repository?

Greetings,
Joachim

--
Joachim ?nomeata? Breitner
  mail at joachim-breitner.de ? http://www.joachim-breitner.de/
  Jabber: nomeata at joachim-breitner.de  ? GPG-Key: 0x4743206C
  Debian Developer: nomeata at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131206/a6cb3478/attachment.sig>

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Johan Tibell-2
Hi,

When we merge in the testsuite repo, can we still keep the old commit IDs?
They're referenced from all over the place.


On Fri, Dec 6, 2013 at 12:07 PM, Joachim Breitner
<mail at joachim-breitner.de>wrote:

> Hi,
>
> Am Freitag, den 06.12.2013, 11:05 +0100 schrieb Herbert Valerio Riedel:
> > PS: if anyone wonders why the testsuite.git history is so large: there
> > were a few *huge* binary files with bad compressibility checked in by
> > accident, such as the one removed via
> >
> >[..]
> >
> > s/dph/words/dph-words-fast             | Bin 17580076 -> 0 bytes
> >
> > which account for most of the ~20MiB history....
>
> so if we were to do the marge including a pathname rewrite (which I tend
> to think of as fine), we could remove these files from the history as
> well, and get a reasonably sized repository?
>
> Greetings,
> Joachim
>
> --
> Joachim ?nomeata? Breitner
>   mail at joachim-breitner.de ? http://www.joachim-breitner.de/
>   Jabber: nomeata at joachim-breitner.de  ? GPG-Key: 0x4743206C
>   Debian Developer: nomeata at debian.org
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131206/33fb69e5/attachment.html>

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Joachim Breitner-2
Hi,

Am Freitag, den 06.12.2013, 13:01 +0100 schrieb Johan Tibell:

> When we merge in the testsuite repo, can we still keep the old commit
> IDs? They're referenced from all over the place.

that depends on the style of merge:

 * With pathname rewriting:
   + git can easily trace the history of a file
   + we can remove the large binary blobs while we are at it
 * Without pathname rewriting:
   + commit IDs stay the same

Greetings,
Joachim


--
Joachim ?nomeata? Breitner
  mail at joachim-breitner.de ? http://www.joachim-breitner.de/
  Jabber: nomeata at joachim-breitner.de  ? GPG-Key: 0x4743206C
  Debian Developer: nomeata at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131206/298b81bd/attachment.sig>

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Herbert Valerio Riedel
In reply to this post by Johan Tibell-2
On 2013-12-06 at 13:01:41 +0100, Johan Tibell wrote:
> When we merge in the testsuite repo, can we still keep the old commit IDs?
> They're referenced from all over the place.

...if we want to preserve the old testsuite's commit-ids, then we'll
have to live with carrying around those superflous large blobs in
testsuite's history nobody really needs, as afaik[1] we can't remove the
blobs w/o causing the commit-ids to change.

I'm not so much concerned about disk-space, but rather the wasted
network bandwidth involved (and yes, we're already suffering from that
now with the current testsuite.git). But I don't feel very strongly on
this one, if there's agreement that we want to keep around those dead
weights in the Git history in order to retain testsuite.git's original
commit ids *inside* ghc.git (nb: the old testsuite.git repo won't be
deleted and thus remain available, it just won't be pushed into
anymore).

Otoh, which use-cases exactly do you have in mind wrt the testsuite.git
commit-ids?


 [1]: maybe someone with more Git-foo knows a trick here?

Reply | Threaded
Open this post in threaded view
|

Repository Reorganization Question

Johan Tibell-2
Whichever way to go, we should write down the options and consequences and
communicating them widely enough so no core devs get surprised.

Commit IDs for the test suite are referenced in e.g. various Trac issues,
on mailing lists (although rarely), and perhaps even in code.


On Fri, Dec 6, 2013 at 1:15 PM, Herbert Valerio Riedel <hvr at gnu.org> wrote:

> On 2013-12-06 at 13:01:41 +0100, Johan Tibell wrote:
> > When we merge in the testsuite repo, can we still keep the old commit
> IDs?
> > They're referenced from all over the place.
>
> ...if we want to preserve the old testsuite's commit-ids, then we'll
> have to live with carrying around those superflous large blobs in
> testsuite's history nobody really needs, as afaik[1] we can't remove the
> blobs w/o causing the commit-ids to change.
>
> I'm not so much concerned about disk-space, but rather the wasted
> network bandwidth involved (and yes, we're already suffering from that
> now with the current testsuite.git). But I don't feel very strongly on
> this one, if there's agreement that we want to keep around those dead
> weights in the Git history in order to retain testsuite.git's original
> commit ids *inside* ghc.git (nb: the old testsuite.git repo won't be
> deleted and thus remain available, it just won't be pushed into
> anymore).
>
> Otoh, which use-cases exactly do you have in mind wrt the testsuite.git
> commit-ids?
>
>
>  [1]: maybe someone with more Git-foo knows a trick here?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20131206/f833641f/attachment.html>

123