R/W from/to partitions (on linux)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

R/W from/to partitions (on linux)

Silent Leaf
Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!

_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: R/W from/to partitions (on linux)

Sylvain Henry-2

Hi,

It is not Haskell specific. You just have to read from the partition device special file (e.g., something like /dev/sdb2) as you would do with a normal file. You must have the permission to do so (e.g., be root). Be careful as you can destroy your system if you write something incorrect in your partitions.

Repositioning handles: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#g:13

Read/write: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf

Sylvain


On 26/06/2017 07:35, Silent Leaf wrote:
Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: R/W from/to partitions (on linux)

Silent Leaf
Darn quick answer! Thanks Sylvain, that may be all i need to start!

2017-06-26 7:51 GMT+02:00 Sylvain Henry <[hidden email]>:

Hi,

It is not Haskell specific. You just have to read from the partition device special file (e.g., something like /dev/sdb2) as you would do with a normal file. You must have the permission to do so (e.g., be root). Be careful as you can destroy your system if you write something incorrect in your partitions.

Repositioning handles: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#g:13

Read/write: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf

Sylvain


On 26/06/2017 07:35, Silent Leaf wrote:
Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: R/W from/to partitions (on linux)

Silent Leaf
i'm reading on the doc of BS.Lazy.hGetContents:
"Once EOF is encountered, the Handle is closed."

what does that imply if i'm using it inside of withFile? no risk of getting prematurely out of the function right? that doesn seem possible in a pure function but i'm asking either way.
if say i do something that reads the whole file, say calculating its length, does it mean since EOF will be reached i'll have to open the file again? i think i'm a bit lost...

i'm trying to find how to read big chunks of two files, do stuff with each pair of chunk, and so on till the EOF, which may or may not happen at the same time for both... i don't really know how lazy bytestrings handle, for example, taking too much from a file. one way would be to calculate the length, ofc, but for files (partition) of several dozens of gigabytes, it's a bit delicate... the ideal would be to get the length from the system itself rather than calculate the whole string ...

2017-06-26 7:57 GMT+02:00 Silent Leaf <[hidden email]>:
Darn quick answer! Thanks Sylvain, that may be all i need to start!

2017-06-26 7:51 GMT+02:00 Sylvain Henry <[hidden email]>:

Hi,

It is not Haskell specific. You just have to read from the partition device special file (e.g., something like /dev/sdb2) as you would do with a normal file. You must have the permission to do so (e.g., be root). Be careful as you can destroy your system if you write something incorrect in your partitions.

Repositioning handles: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#g:13

Read/write: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf

Sylvain


On 26/06/2017 07:35, Silent Leaf wrote:
Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners




_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: R/W from/to partitions (on linux)

Silent Leaf
also what happens to the result of Lazy.hGetContents after i close the handle i used for accessing the file content? can i still browse the part of th file cached? will there be an exception? it does not seem to be the case from tests on GHCI but then...

2017-06-26 10:35 GMT+02:00 Silent Leaf <[hidden email]>:
i'm reading on the doc of BS.Lazy.hGetContents:
"Once EOF is encountered, the Handle is closed."

what does that imply if i'm using it inside of withFile? no risk of getting prematurely out of the function right? that doesn seem possible in a pure function but i'm asking either way.
if say i do something that reads the whole file, say calculating its length, does it mean since EOF will be reached i'll have to open the file again? i think i'm a bit lost...

i'm trying to find how to read big chunks of two files, do stuff with each pair of chunk, and so on till the EOF, which may or may not happen at the same time for both... i don't really know how lazy bytestrings handle, for example, taking too much from a file. one way would be to calculate the length, ofc, but for files (partition) of several dozens of gigabytes, it's a bit delicate... the ideal would be to get the length from the system itself rather than calculate the whole string ...

2017-06-26 7:57 GMT+02:00 Silent Leaf <[hidden email]>:
Darn quick answer! Thanks Sylvain, that may be all i need to start!

2017-06-26 7:51 GMT+02:00 Sylvain Henry <[hidden email]>:

Hi,

It is not Haskell specific. You just have to read from the partition device special file (e.g., something like /dev/sdb2) as you would do with a normal file. You must have the permission to do so (e.g., be root). Be careful as you can destroy your system if you write something incorrect in your partitions.

Repositioning handles: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#g:13

Read/write: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf

Sylvain


On 26/06/2017 07:35, Silent Leaf wrote:
Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners





_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: R/W from/to partitions (on linux)

Stefan Risberg
In reply to this post by Silent Leaf
I would use some streaming library instead of lazy bytestring to keep memory at reasonably low levels. It will also help with reading chunks, and then composing actions on it. 

For library need you got: conduits, iostreams and pipes




On 26 Jun. 2017 10:37, "Silent Leaf" <[hidden email]> wrote:
i'm reading on the doc of BS.Lazy.hGetContents:
"Once EOF is encountered, the Handle is closed."

what does that imply if i'm using it inside of withFile? no risk of getting prematurely out of the function right? that doesn seem possible in a pure function but i'm asking either way.
if say i do something that reads the whole file, say calculating its length, does it mean since EOF will be reached i'll have to open the file again? i think i'm a bit lost...

i'm trying to find how to read big chunks of two files, do stuff with each pair of chunk, and so on till the EOF, which may or may not happen at the same time for both... i don't really know how lazy bytestrings handle, for example, taking too much from a file. one way would be to calculate the length, ofc, but for files (partition) of several dozens of gigabytes, it's a bit delicate... the ideal would be to get the length from the system itself rather than calculate the whole string ...

2017-06-26 7:57 GMT+02:00 Silent Leaf <[hidden email]>:
Darn quick answer! Thanks Sylvain, that may be all i need to start!

2017-06-26 7:51 GMT+02:00 Sylvain Henry <[hidden email]>:

Hi,

It is not Haskell specific. You just have to read from the partition device special file (e.g., something like /dev/sdb2) as you would do with a normal file. You must have the permission to do so (e.g., be root). Be careful as you can destroy your system if you write something incorrect in your partitions.

Repositioning handles: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#g:13

Read/write: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf

Sylvain


On 26/06/2017 07:35, Silent Leaf wrote:
Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners




_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: R/W from/to partitions (on linux)

Silent Leaf
yes, conduits really seem optimal! i'll probably use that in my final version, thanks!
however i'll first try without using pre-made tools, to get the handle (ha) of manually using (binary) files with haskell, as after all my program isn that complicated.

2017-06-26 10:42 GMT+02:00 Stefan Risberg <[hidden email]>:
I would use some streaming library instead of lazy bytestring to keep memory at reasonably low levels. It will also help with reading chunks, and then composing actions on it. 

For library need you got: conduits, iostreams and pipes




On 26 Jun. 2017 10:37, "Silent Leaf" <[hidden email]> wrote:
i'm reading on the doc of BS.Lazy.hGetContents:
"Once EOF is encountered, the Handle is closed."

what does that imply if i'm using it inside of withFile? no risk of getting prematurely out of the function right? that doesn seem possible in a pure function but i'm asking either way.
if say i do something that reads the whole file, say calculating its length, does it mean since EOF will be reached i'll have to open the file again? i think i'm a bit lost...

i'm trying to find how to read big chunks of two files, do stuff with each pair of chunk, and so on till the EOF, which may or may not happen at the same time for both... i don't really know how lazy bytestrings handle, for example, taking too much from a file. one way would be to calculate the length, ofc, but for files (partition) of several dozens of gigabytes, it's a bit delicate... the ideal would be to get the length from the system itself rather than calculate the whole string ...

2017-06-26 7:57 GMT+02:00 Silent Leaf <[hidden email]>:
Darn quick answer! Thanks Sylvain, that may be all i need to start!

2017-06-26 7:51 GMT+02:00 Sylvain Henry <[hidden email]>:

Hi,

It is not Haskell specific. You just have to read from the partition device special file (e.g., something like /dev/sdb2) as you would do with a normal file. You must have the permission to do so (e.g., be root). Be careful as you can destroy your system if you write something incorrect in your partitions.

Repositioning handles: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#g:13

Read/write: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf

Sylvain


On 26/06/2017 07:35, Silent Leaf wrote:
Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners




_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: R/W from/to partitions (on linux)

Silent Leaf
i'm trying to get for which purpose BS.strict and BS.lazy are adapted. am i right to think that:

lazy bytestrings are not optimal when needing to read huge files of several gigabytes, bc they are only lazy to the extent that the very long bytestring is not read to the end. aka it's like haskell's natural laziness, if i compute [0..2^20] == [0..2^20] it won't just take a lot of time but will also (try to) keep in memory both huge lists, aka the operation is not intrinsically on minimal resources, comparing each item of each list and discarding the previous ones at the same time; more generally the lists or bytestrings, be they lazy, are still only deallocated when the whole variable (the whole list/array) is dereferenced.

thus if i need say to read very huge files and compare them together block by block, it's better for me to use strict bytestrings for each pair of chunks and manually move the handle from one block to the other, looping over the whole file. correct?

2017-06-26 12:53 GMT+02:00 Silent Leaf <[hidden email]>:
yes, conduits really seem optimal! i'll probably use that in my final version, thanks!
however i'll first try without using pre-made tools, to get the handle (ha) of manually using (binary) files with haskell, as after all my program isn that complicated.

2017-06-26 10:42 GMT+02:00 Stefan Risberg <[hidden email]>:
I would use some streaming library instead of lazy bytestring to keep memory at reasonably low levels. It will also help with reading chunks, and then composing actions on it. 

For library need you got: conduits, iostreams and pipes




On 26 Jun. 2017 10:37, "Silent Leaf" <[hidden email]> wrote:
i'm reading on the doc of BS.Lazy.hGetContents:
"Once EOF is encountered, the Handle is closed."

what does that imply if i'm using it inside of withFile? no risk of getting prematurely out of the function right? that doesn seem possible in a pure function but i'm asking either way.
if say i do something that reads the whole file, say calculating its length, does it mean since EOF will be reached i'll have to open the file again? i think i'm a bit lost...

i'm trying to find how to read big chunks of two files, do stuff with each pair of chunk, and so on till the EOF, which may or may not happen at the same time for both... i don't really know how lazy bytestrings handle, for example, taking too much from a file. one way would be to calculate the length, ofc, but for files (partition) of several dozens of gigabytes, it's a bit delicate... the ideal would be to get the length from the system itself rather than calculate the whole string ...

2017-06-26 7:57 GMT+02:00 Silent Leaf <[hidden email]>:
Darn quick answer! Thanks Sylvain, that may be all i need to start!

2017-06-26 7:51 GMT+02:00 Sylvain Henry <[hidden email]>:

Hi,

It is not Haskell specific. You just have to read from the partition device special file (e.g., something like /dev/sdb2) as you would do with a normal file. You must have the permission to do so (e.g., be root). Be careful as you can destroy your system if you write something incorrect in your partitions.

Repositioning handles: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#g:13

Read/write: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf

Sylvain


On 26/06/2017 07:35, Silent Leaf wrote:
Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners




_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners




_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: R/W from/to partitions (on linux)

Silent Leaf
btw i just tried my comparison of lists, and it seems there's an intermediate type holding the concept of boundaries and so on, aka it does not need to compute them beyond comparing their boundaries. clever haskell! a bit too much, you never know how smart it exactly is...

2017-06-26 13:33 GMT+02:00 Silent Leaf <[hidden email]>:
i'm trying to get for which purpose BS.strict and BS.lazy are adapted. am i right to think that:

lazy bytestrings are not optimal when needing to read huge files of several gigabytes, bc they are only lazy to the extent that the very long bytestring is not read to the end. aka it's like haskell's natural laziness, if i compute [0..2^20] == [0..2^20] it won't just take a lot of time but will also (try to) keep in memory both huge lists, aka the operation is not intrinsically on minimal resources, comparing each item of each list and discarding the previous ones at the same time; more generally the lists or bytestrings, be they lazy, are still only deallocated when the whole variable (the whole list/array) is dereferenced.

thus if i need say to read very huge files and compare them together block by block, it's better for me to use strict bytestrings for each pair of chunks and manually move the handle from one block to the other, looping over the whole file. correct?

2017-06-26 12:53 GMT+02:00 Silent Leaf <[hidden email]>:
yes, conduits really seem optimal! i'll probably use that in my final version, thanks!
however i'll first try without using pre-made tools, to get the handle (ha) of manually using (binary) files with haskell, as after all my program isn that complicated.

2017-06-26 10:42 GMT+02:00 Stefan Risberg <[hidden email]>:
I would use some streaming library instead of lazy bytestring to keep memory at reasonably low levels. It will also help with reading chunks, and then composing actions on it. 

For library need you got: conduits, iostreams and pipes




On 26 Jun. 2017 10:37, "Silent Leaf" <[hidden email]> wrote:
i'm reading on the doc of BS.Lazy.hGetContents:
"Once EOF is encountered, the Handle is closed."

what does that imply if i'm using it inside of withFile? no risk of getting prematurely out of the function right? that doesn seem possible in a pure function but i'm asking either way.
if say i do something that reads the whole file, say calculating its length, does it mean since EOF will be reached i'll have to open the file again? i think i'm a bit lost...

i'm trying to find how to read big chunks of two files, do stuff with each pair of chunk, and so on till the EOF, which may or may not happen at the same time for both... i don't really know how lazy bytestrings handle, for example, taking too much from a file. one way would be to calculate the length, ofc, but for files (partition) of several dozens of gigabytes, it's a bit delicate... the ideal would be to get the length from the system itself rather than calculate the whole string ...

2017-06-26 7:57 GMT+02:00 Silent Leaf <[hidden email]>:
Darn quick answer! Thanks Sylvain, that may be all i need to start!

2017-06-26 7:51 GMT+02:00 Sylvain Henry <[hidden email]>:

Hi,

It is not Haskell specific. You just have to read from the partition device special file (e.g., something like /dev/sdb2) as you would do with a normal file. You must have the permission to do so (e.g., be root). Be careful as you can destroy your system if you write something incorrect in your partitions.

Repositioning handles: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#g:13

Read/write: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf

Sylvain


On 26/06/2017 07:35, Silent Leaf wrote:
Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners




_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners





_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: R/W from/to partitions (on linux)

Carl Eyeinsky
AFAIK, in the list comparison case you mention, it will just build and destroy the lists until it reaches the end or finds a non-equal pair and returns False at that point. So no keeping the list in memory, since computing equality for lists doesn't require it.

On Jun 26, 2017 2:39 PM, "Silent Leaf" <[hidden email]> wrote:
btw i just tried my comparison of lists, and it seems there's an intermediate type holding the concept of boundaries and so on, aka it does not need to compute them beyond comparing their boundaries. clever haskell! a bit too much, you never know how smart it exactly is...

2017-06-26 13:33 GMT+02:00 Silent Leaf <[hidden email]>:
i'm trying to get for which purpose BS.strict and BS.lazy are adapted. am i right to think that:

lazy bytestrings are not optimal when needing to read huge files of several gigabytes, bc they are only lazy to the extent that the very long bytestring is not read to the end. aka it's like haskell's natural laziness, if i compute [0..2^20] == [0..2^20] it won't just take a lot of time but will also (try to) keep in memory both huge lists, aka the operation is not intrinsically on minimal resources, comparing each item of each list and discarding the previous ones at the same time; more generally the lists or bytestrings, be they lazy, are still only deallocated when the whole variable (the whole list/array) is dereferenced.

thus if i need say to read very huge files and compare them together block by block, it's better for me to use strict bytestrings for each pair of chunks and manually move the handle from one block to the other, looping over the whole file. correct?

2017-06-26 12:53 GMT+02:00 Silent Leaf <[hidden email]>:
yes, conduits really seem optimal! i'll probably use that in my final version, thanks!
however i'll first try without using pre-made tools, to get the handle (ha) of manually using (binary) files with haskell, as after all my program isn that complicated.

2017-06-26 10:42 GMT+02:00 Stefan Risberg <[hidden email]>:
I would use some streaming library instead of lazy bytestring to keep memory at reasonably low levels. It will also help with reading chunks, and then composing actions on it. 

For library need you got: conduits, iostreams and pipes




On 26 Jun. 2017 10:37, "Silent Leaf" <[hidden email]> wrote:
i'm reading on the doc of BS.Lazy.hGetContents:
"Once EOF is encountered, the Handle is closed."

what does that imply if i'm using it inside of withFile? no risk of getting prematurely out of the function right? that doesn seem possible in a pure function but i'm asking either way.
if say i do something that reads the whole file, say calculating its length, does it mean since EOF will be reached i'll have to open the file again? i think i'm a bit lost...

i'm trying to find how to read big chunks of two files, do stuff with each pair of chunk, and so on till the EOF, which may or may not happen at the same time for both... i don't really know how lazy bytestrings handle, for example, taking too much from a file. one way would be to calculate the length, ofc, but for files (partition) of several dozens of gigabytes, it's a bit delicate... the ideal would be to get the length from the system itself rather than calculate the whole string ...

2017-06-26 7:57 GMT+02:00 Silent Leaf <[hidden email]>:
Darn quick answer! Thanks Sylvain, that may be all i need to start!

2017-06-26 7:51 GMT+02:00 Sylvain Henry <[hidden email]>:

Hi,

It is not Haskell specific. You just have to read from the partition device special file (e.g., something like /dev/sdb2) as you would do with a normal file. You must have the permission to do so (e.g., be root). Be careful as you can destroy your system if you write something incorrect in your partitions.

Repositioning handles: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#g:13

Read/write: https://www.stackage.org/haddock/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf

Sylvain


On 26/06/2017 07:35, Silent Leaf wrote:
Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners




_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners





_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Loading...