Creating a instance for (Storable Text)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Creating a instance for (Storable Text)

Cody Goodman
Any ideas on how I'd write this instance? I know for it to be storable it needs a size, couldn't I just get the length of the text? Has anyone did anything similar?

I ran into a surprise when trying to use storable-record and derive-storable and tried using them with a record that contained a Text field.

My end goal is having a vector of records that could contain Text, UTCTime, or other similarly complex types which I can write as an mmap to disk and then read back to the original format.

Thanks!

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Creating a instance for (Storable Text)

Michael Snoyman

It can't be done. Instances of Storable need to have a fixed serialized size, and Text values don't.


On Thu, Nov 24, 2016, 8:21 PM Cody Goodman <[hidden email]> wrote:
Any ideas on how I'd write this instance? I know for it to be storable it needs a size, couldn't I just get the length of the text? Has anyone did anything similar?

I ran into a surprise when trying to use storable-record and derive-storable and tried using them with a record that contained a Text field.

My end goal is having a vector of records that could contain Text, UTCTime, or other similarly complex types which I can write as an mmap to disk and then read back to the original format.

Thanks!
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Creating a instance for (Storable Text)

Ken Bateman

Maybe you want to use Data.Serialize?  Encode to and from a bytestring.


On Nov 24, 2016 1:24 PM, "Michael Snoyman" <[hidden email]> wrote:

It can't be done. Instances of Storable need to have a fixed serialized size, and Text values don't.


On Thu, Nov 24, 2016, 8:21 PM Cody Goodman <[hidden email]> wrote:
Any ideas on how I'd write this instance? I know for it to be storable it needs a size, couldn't I just get the length of the text? Has anyone did anything similar?

I ran into a surprise when trying to use storable-record and derive-storable and tried using them with a record that contained a Text field.

My end goal is having a vector of records that could contain Text, UTCTime, or other similarly complex types which I can write as an mmap to disk and then read back to the original format.

Thanks!
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Creating a instance for (Storable Text)

Chris Wong-2
In reply to this post by Cody Goodman
Since Text is represented as UTF-16 underneath, you can retrieve the
length in terms of 16-bit code units.

I don't think you'll be able to mmap them though, since the backing
array may be moved by the garbage collector.

See https://hackage.haskell.org/package/text-1.2.2.1/docs/Data-Text-Foreign.html

On Fri, Nov 25, 2016 at 7:20 AM, Cody Goodman
<[hidden email]> wrote:

> Any ideas on how I'd write this instance? I know for it to be storable it
> needs a size, couldn't I just get the length of the text? Has anyone did
> anything similar?
>
> I ran into a surprise when trying to use storable-record and derive-storable
> and tried using them with a record that contained a Text field.
>
> My end goal is having a vector of records that could contain Text, UTCTime,
> or other similarly complex types which I can write as an mmap to disk and
> then read back to the original format.
>
> Thanks!
>
> _______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.



--
Chris Wong (https://lambda.xyz)

"I had not the vaguest idea what this meant and when I could not
remember the words, my tutor threw the book at my head, which did not
stimulate my intellect in any way." -- Bertrand Russell
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Creating a instance for (Storable Text)

David Feuer
Chris Wong, the Storable class documents that sizeOf and alignmentOf
are not supposed to depend on the values they're passed. If the class
were being designed today, we'd likely have something more like

sizeOf :: proxy a -> Int

or possibly

sizeOf# :: Proxy# a -> Int#

But when Storable was designed, the proxy-passing "pattern" had not
yet been established.

On Thu, Nov 24, 2016 at 8:59 PM, Chris Wong <[hidden email]> wrote:

> Since Text is represented as UTF-16 underneath, you can retrieve the
> length in terms of 16-bit code units.
>
> I don't think you'll be able to mmap them though, since the backing
> array may be moved by the garbage collector.
>
> See https://hackage.haskell.org/package/text-1.2.2.1/docs/Data-Text-Foreign.html
>
> On Fri, Nov 25, 2016 at 7:20 AM, Cody Goodman
> <[hidden email]> wrote:
>> Any ideas on how I'd write this instance? I know for it to be storable it
>> needs a size, couldn't I just get the length of the text? Has anyone did
>> anything similar?
>>
>> I ran into a surprise when trying to use storable-record and derive-storable
>> and tried using them with a record that contained a Text field.
>>
>> My end goal is having a vector of records that could contain Text, UTCTime,
>> or other similarly complex types which I can write as an mmap to disk and
>> then read back to the original format.
>>
>> Thanks!
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> To (un)subscribe, modify options or view archives go to:
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> Only members subscribed via the mailman list are allowed to post.
>
>
>
> --
> Chris Wong (https://lambda.xyz)
>
> "I had not the vaguest idea what this meant and when I could not
> remember the words, my tutor threw the book at my head, which did not
> stimulate my intellect in any way." -- Bertrand Russell
> _______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.