Billion-triple RDF store

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Billion-triple RDF store

Joel Reymont
Folks,

How would you go about implementing a RDF triple store in Haskell  
capable of handling a billion triples? I'm looking for ideas.

For example, you could use tries to store all strings but you could  
easily get to the point where you can't load them all into, say, 1Gb  
of memory which is all I have on my machine. I'm thinking that the  
required data could be pulled from disk lazily (as needed) and  
laziness is a Haskell forte.

        Thanks, Joel

P.S. I know about Swish but would go for a far more compact and  
efficient representation.

--
http://wagerlabs.com/





_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Billion-triple RDF store

Bulat Ziganshin-2
Hello Joel,

Monday, April 24, 2006, 5:50:57 AM, you wrote:

> For example, you could use tries to store all strings but you could
> easily get to the point where you can't load them all into, say, 1Gb  
> of memory which is all I have on my machine. I'm thinking that the  
> required data could be pulled from disk lazily (as needed) and  
> laziness is a Haskell forte.

this should be rather easy to implement:

type Key = String
type Value = String

-- | Opens a file with given name and returns function that maps keys to values
open :: String -> IO (Key -> Value)
open filename = do h <- openBinaryFile filename ReadMode
                   let lookup key = unsafePerformIO $ do
                         -- your genuine algorithm in IO monad that
                         -- finds value for `key` and returns it
                         return value
                   return lookup

main = do lookup <- open "database"
          print (lookup "key1", lookup "key2")

--
Best regards,
 Bulat                            mailto:[hidden email]

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe