Re: Loading a csv file with ~200 columns into Haskell Record

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: Loading a csv file with ~200 columns into Haskell Record

Dominic Steinitz-2
I don’t know if this helps but here’s something I created in 5 minutes yesterday. Once you have the data you can convert to e.g. frames with all the help that types can give. I once used frames on a file with 10,000 columns and it worked (probably I shouldn’t have done this).

Also an option is in Chris Done’s blog: https://www.fpcomplete.com/blog/2016/09/data-haskell

PS My script below will fail if I have missing data but then I can use something better than read.

> {-# OPTIONS_GHC -Wall #-}
>
> import qualified Data.Vector as V
> import qualified Data.ByteString.Lazy as BL
> import qualified Data.ByteString.Lazy.Char8 as BLC
> import Data.Csv
> import Data.List
> import Plots
> import Diagrams.Prelude
> import Diagrams.Backend.Rasterific
>
> main :: IO ()
> main = do
>   evalsCsv <- BLC.readFile "/Users/dom/.local/share/simwork/data/Models/Lorenz.csv"
>   let records :: Either String (V.Vector (V.Vector BL.ByteString))
>       records = decode NoHeader $ BLC.unlines $ BLC.lines evalsCsv
>   case records of
>     Left err -> putStrLn err
>     Right evals -> do
>       let hdrs :: V.Vector String
>           hdrs = V.map BLC.unpack $ V.head evals
>       let zss :: [[Double]]
>           zss = map (map read) $ map (map BLC.unpack) $
>                 transpose $ V.toList $ V.map V.toList $ V.drop 1 evals
>       renderRasterific "diagrams/lorenz.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!0) (zss!!1))
>       renderRasterific "diagrams/lorenz1.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!0) (zss!!2))
>       renderRasterific "diagrams/lorenz2.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!0) (zss!!3))
>       renderRasterific "diagrams/lorenz3.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!1) (zss!!2))
>       renderRasterific "diagrams/lorenz4.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!1) (zss!!3))
>       renderRasterific "diagrams/lorenz5.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!2) (zss!!3))
>
> jSaxis :: [Double] -> [Double] -> Axis B V2 Double
> jSaxis ts xs = r2Axis &~ do
>   linePlot' $ zip ts xs


Dominic Steinitz
[hidden email]
http://idontgetoutmuch.wordpress.com

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.