runGet exits prematurely

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

runGet exits prematurely

Quentin Liu

Hi,


I am using Data.Binary to decode binary files and found that the `runGet` function would throw an exception “not enough bytes” even though it has not consumed all the input.


Specifically, in the loop I am repeatedly trying to parse the binary file until it has consumed all the input


 parsePPackets xs = do

  empty <- isEmpty

  if empty

   then return xs

   else do p <- parseB6034

        parsePPackets (p:xs)


When I try to run this function with `runGet`, the exception “Data.Binary.Get.runGet at position 3293603: not enough bytes” would be thrown, while the total length of input is 5864230, a number significantly larger. The function `parseB6034` consumes no more than 250 bytes in each round. In addition, the `parsePPackets` works well when fed with a small amount of data.


Is it the problem of my code or an error with the package binary? The ByteString fed into `runGet` is lazy ByteString, as required by `runGet` function.


Best Regards,

Qingbo Liu


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: runGet exits prematurely

Quentin Liu
Thank you for the suggestion! I am still not sure about the correct way to communicate on mailing-list. 

But I have figured out the problem which is caused by arithmetic overflow. I was using Word8 to parse the packet length and presumed the packet length would be larger than 46. The subtraction of 47 from the Word8 packet length gave the largest number Word8 could represent, which definitely exceeds the length of the whole file. 

Regards,
Qingbo Liu

On Feb 26, 2018, 02:41 -0500, Robin Palotai <[hidden email]>, wrote:
[-list]

FYI Given the lack of responses, I think more context would help. For example github / gist link to code, minimal case to reproduce.

2018-02-23 16:59 GMT+01:00 Quentin Liu <[hidden email]>:

Hi,


I am using Data.Binary to decode binary files and found that the `runGet` function would throw an exception “not enough bytes” even though it has not consumed all the input.


Specifically, in the loop I am repeatedly trying to parse the binary file until it has consumed all the input


 parsePPackets xs = do

  empty <- isEmpty

  if empty

   then return xs

   else do p <- parseB6034

        parsePPackets (p:xs)


When I try to run this function with `runGet`, the exception “Data.Binary.Get.runGet at position 3293603: not enough bytes” would be thrown, while the total length of input is 5864230, a number significantly larger. The function `parseB6034` consumes no more than 250 bytes in each round. In addition, the `parsePPackets` works well when fed with a small amount of data.


Is it the problem of my code or an error with the package binary? The ByteString fed into `runGet` is lazy ByteString, as required by `runGet` function.


Best Regards,

Qingbo Liu


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.