Getting latest GHC-head/libraries takes forever

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Getting latest GHC-head/libraries takes forever

Austin Seipp
For the purpose of experimenting with NDP I went through the
process of getting the GHC head from darcs.haskell.org. As
specified in the developer wiki[1], using darcs get is basically
not possible because there're so many patches. So I downloaded

http://darcs.haskell.org/ghc-HEAD-2008-06-06-ghc-corelibs-testsuite.tar.bz2

This alone took over *7 hours* using wget because it could never get
a connection faster than 5kb/s.

After downloading I changed to the directory and ran 'darcs pull -a'.
Nothing bad happened here, darcs pulled most of the changes pretty quick
so I had an up to date GHC.

However, when trying to use the ./darcs-all script to pull in the
latest library/testsuite changes, things go horribly, horribly wrong.

I let darcs sit for over another hour and a half trying to pull patches
from darcs.haskell.org for the libraries and the testsuite.

It never got past packages/base. In fact, it never even got to *getting* the
patches from the base repository, it sat there, stuck at the 'identifying repository'
stage (darcs 2.0.2) for over an hour, making my processor go round and round at
99% CPU usage the entire way, never accomplishing anything. What makes it even stranger
is that it got the latest patches for the testsuite and the array package pretty quick.
It just got permanently stuck on base for some reason and never went further.

Also, I tried pulling package-ndp a few days ago while using an older development snapshot
of GHC (not the darcs repository, a nightly HEAD tarball of ghc.)
Just pulling 500 patches from http://darcs.haskell.org/packages/ndp took over
an hour, but at least it got done. It didn't build properly so I thought I'd
get the latest darcs version of GHC instead, and here I am.

Has anybody experienced anything like this recently? Off the top of my head the only
things that possibly come to mind are:

1) darcs.haskell.org is just really slow or under -constant- heavy load.
2) the darcs repositories are in darcs-1 format, so darcs-2 is having issues when
 trying to pull the patches.
3) Diabolical connection-killing ninjas.

None of these 3 possibilities are really 'great,' and none of them help me get the
latest version of GHC, the libs, and NDP any faster then a narcoleptic snail.

If instead this is a techincal issue with, say, darcs that someone is aware of, I'm
using OS X Leopard, 10.5.4 with a statically linked darcs-2.0.2 binary from [2].

[1] http://hackage.haskell.org/trac/ghc/wiki/Building/GettingTheSources
[2] http://wiki.darcs.net/DarcsWiki/CategoryBinaries#head-25c8108e9d719be30a8cc6bcc86ce243a78b8c25
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Getting latest GHC-head/libraries takes forever

Brandon S Allbery KF8NH

On 2008 Jul 15, at 20:45, Austin Seipp wrote:

> For the purpose of experimenting with NDP I went through the
> process of getting the GHC head from darcs.haskell.org. As
> specified in the developer wiki[1], using darcs get is basically
> not possible because there're so many patches. So I downloaded
>
> http://darcs.haskell.org/ghc-HEAD-2008-06-06-ghc-corelibs-testsuite.tar.bz2
>
> This alone took over *7 hours* using wget because it could never get
> a connection faster than 5kb/s.

FWIW I did manage to get it using darcs (!) having not checked the  
wiki to see that it was supposedly impossible these days.  (Well, I  
did get --partial.  --all is a definite mistake.)

> I let darcs sit for over another hour and a half trying to pull  
> patches
> from darcs.haskell.org for the libraries and the testsuite.
>
> It never got past packages/base. In fact, it never even got to  
> *getting* the
> patches from the base repository, it sat there, stuck at the  
> 'identifying repository'
> stage (darcs 2.0.2) for over an hour, making my processor go round  
> and round at
> 99% CPU usage the entire way, never accomplishing anything. What  
> makes it even stranger
> is that it got the latest patches for the testsuite and the array  
> package pretty quick.
> It just got permanently stuck on base for some reason and never went  
> further.

Mine stopped at array.  That said, it does seem really slow; given  
your comment about downloading, this sounds like a problem with  
darcs.haskell.org in general.  But it might be worth grabbing darcs1  
for Leopard and trying again.

--
brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [hidden email]
system administrator [openafs,heimdal,too many hats] [hidden email]
electrical and computer engineering, carnegie mellon university    KF8NH


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Getting latest GHC-head/libraries takes forever

Duncan Coutts
In reply to this post by Austin Seipp

On Tue, 2008-07-15 at 19:45 -0500, Austin Seipp wrote:
> For the purpose of experimenting with NDP I went through the
> process of getting the GHC head from darcs.haskell.org. As
> specified in the developer wiki[1], using darcs get is basically
> not possible because there're so many patches. So I downloaded
>
> http://darcs.haskell.org/ghc-HEAD-2008-06-06-ghc-corelibs-testsuite.tar.bz2
>
> This alone took over *7 hours* using wget because it could never get
> a connection faster than 5kb/s.

darcs.haskell.org is still being throttled. The server move means the
machine itself has more bandwidth but the apache http server is still
configured to limit connection speed. I believe this is supposed to
change at some point but I don't know when.

I've no idea about what's going on with darcs though. I know there are
some issues with case-insensitive file systems or as you speculate it
might be a darcs 1 vs 2 issue. Ask a darcs expert.

Duncan

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe