Datomic-like db for Haskell?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Datomic-like db for Haskell?

Ovidiu Deac
Is there any package in Haskell that provides something similar to Datomic http://www.datomic.com/?

Basically what I am looking for is a database which could keep all the versions of my data in an efficient way, the same way the persistent data structures do.

Ideally it should be implemented over some scalable storage.

I looked at Datomic itself but I couldn't find a Haskell client library. The only way to connect to it from a Haskell application is through the REST server which they have just deprecated (http://docs.datomic.com/rest.html)

Thanks!


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Datomic-like db for Haskell?

Ertugrul Söylemez-3
> Basically what I am looking for is a database which could keep all the
> versions of my data in an efficient way, the same way the persistent
> data structures do.
>
> Ideally it should be implemented over some scalable storage.

There are a few options, if you consider that you don't necessarily need
special support for versioning:

  * Would using a snapshot-based revision control system like Git be an
    option?  If yes, you could use the [filestore] library.

  * Another option is to run an insertion-only relational model on
    really any database system you want like PostgreSQL.  There is
    nothing about preserving history that needs special handling to be
    efficient.

  * Yet another option is to model your application state with history
    and use [acid-state].  However, acid-state only makes an in-memory
    Haskell value (the "database") persistent with ACID guarantees.  If
    you mostly work with the current state, keeping the whole history in
    memory all the time would be a waste.  Therefore this option may be
    less useful.

[acid-state]: https://hackage.haskell.org/package/acid-state
[filestore]:  https://hackage.haskell.org/package/filestore


Greets
ertes

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

signature.asc (463 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Fwd: Datomic-like db for Haskell?

Ovidiu Deac
  * Would using a snapshot-based revision control system like Git be an
    option?  If yes, you could use the [filestore] library.
It's not exactly what I had in mind but I'll think about it.
 

  * Another option is to run an insertion-only relational model on
    really any database system you want like PostgreSQL.  There is
    nothing about preserving history that needs special handling to be
    efficient.
I assume that some sort  of structural sharing is needed once the collections grow above a certain size. Also some support for caching would be nice.


  * Yet another option is to model your application state with history
    and use [acid-state].  However, acid-state only makes an in-memory
    Haskell value (the "database") persistent with ACID guarantees.  If
    you mostly work with the current state, keeping the whole history in
    memory all the time would be a waste.  Therefore this option may be
    less useful.
The database will probably grow bigger than the memory so this is not an option.

...and thanks for the pointers

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Datomic-like db for Haskell?

Bardur Arantsson-2
On 2016-12-23 18:41, Ovidiu Deac wrote:
>       * Another option is to run an insertion-only relational model on
>         really any database system you want like PostgreSQL.  There is
>         nothing about preserving history that needs special handling to be
>         efficient.
>
> I assume that some sort  of structural sharing is needed once the
> collections grow above a certain size. Also some support for caching
> would be nice.

A similar option might be something like

   https://github.com/BardurArantsson/cqrs

*However*, I'm not sure I'd actually recommend using my particular
library at this time. I'm probably going to be reworking it (slowly)
quite a bit and currently there's no support for persistent[1] queries
(see the CQRS/ES description linked from the README).

Still, the ideas are sound and pretty simple. (See the PostgreSQL
package for exactly what queries CQRS/ES maps to.)

[1] As in: stored on disk.

Regards,


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Datomic-like db for Haskell?

Ovidiu Deac
In reply to this post by Ertugrul Söylemez-3
I'll do some research to evaluate my options. So far I have:

Homegrown implementation on top of some DB (Riak/Postgres/Mongo...)
https://github.com/agentm/project-m36
https://hackage.haskell.org/package/filestore

Thanks for the answers!

On Fri, Dec 23, 2016 at 8:07 PM, Ertugrul Söylemez <[hidden email]> wrote:
>>   * Another option is to run an insertion-only relational model on
>>     really any database system you want like PostgreSQL.  There is
>>     nothing about preserving history that needs special handling to be
>>     efficient.
>
> I assume that some sort of structural sharing is needed once the
> collections grow above a certain size. Also some support for caching
> would be nice.

Yeah, sharing could improve disk usage and cache utilisation.  I'm not
aware of any DBMS-side sharing features in systems I have used
(PostgreSQL and SQLite mainly).  The usual way to get sharing in
relational systems is to normalise the schema.  That also allows you to
use domain-specific knowledge to get even better sharing.

Caching is of course supported, at least by PostgreSQL.  It aims to keep
as much of the database in memory as possible (or as you tell it to).


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Datomic-like db for Haskell?

Paul Brauner-3
Would you mind sharing the result of your research with the mailing list once you're done?

On Sat, Dec 24, 2016 at 4:50 PM Ovidiu Deac <[hidden email]> wrote:
I'll do some research to evaluate my options. So far I have:

Homegrown implementation on top of some DB (Riak/Postgres/Mongo...)
https://github.com/agentm/project-m36
https://hackage.haskell.org/package/filestore

Thanks for the answers!


On Fri, Dec 23, 2016 at 8:07 PM, Ertugrul Söylemez <[hidden email]> wrote:
>>   * Another option is to run an insertion-only relational model on
>>     really any database system you want like PostgreSQL.  There is
>>     nothing about preserving history that needs special handling to be
>>     efficient.
>
> I assume that some sort of structural sharing is needed once the
> collections grow above a certain size. Also some support for caching
> would be nice.

Yeah, sharing could improve disk usage and cache utilisation.  I'm not
aware of any DBMS-side sharing features in systems I have used
(PostgreSQL and SQLite mainly).  The usual way to get sharing in
relational systems is to normalise the schema.  That also allows you to
use domain-specific knowledge to get even better sharing.

Caching is of course supported, at least by PostgreSQL.  It aims to keep
as much of the database in memory as possible (or as you tell it to).

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.