ByteStrings and the Terabyte barrier

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

ByteStrings and the Terabyte barrier

Donald Bruce Stewart
Just for fun, I was able to filter a 1 Terabyte stream through a
Haskell program, using Data.ByteString.Lazy, with a constant 2M
heapsize, in

    $ time ( repeat 100 ; cat /home/dons/data/10G | ./lazy > /dev/null )
    12752.37s user 4069.35s system 58% cpu 8:03:11.42 total

8 hours.

Just so we can say it is possible :)

-- Don
Libraries mailing list
[hidden email]