addFinalizer to which object? (fwd from cafe)

Previous Topic Next Topic
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view

addFinalizer to which object? (fwd from cafe)

Henning Thielemann

The documentation of System.Mem.Weak.addFinalizer suggests to me, that this
function is very fragile, because the object we track, might be optimized away.
In my case this applies. I have a resource that is needed for generating chunks
of a lazy byte string. If the byte string is created completely or the byte
string cannot be accessed anymore, this resource must be freed. It must also be
freed within Haskell code, thus addForeignPtrFinalizer does not help.
To illustrate that I have attached a small example, where I use a StablePtr as
resource, that points to a list. This Haskell list is read from C code and
written to a chunk of a lazy ByteString. A Haskell function creates the
ByteString lazily and calls the C function for every new chunk. I have added a
finalizer to the StablePtr itself. That works without optimization, but using
optimization the finalizer is run immediately and the program aborts with
segmentation fault.
Thus my question: How to get reliable finalization?

#include "ProcessList_stub.h"
The function copies elements from a Haskell list to a piece of memory
and returns the number of bytes that could be fetched.
int copy_list_to_chunk (unsigned long int n, HsStablePtr list, char *p) {
   unsigned long int k = n;
   while (k>0) {
      if (!get_next_element(list, p)) {break; }
   return (n-k);

# generate C stubs
ghc -c ProcessList.hs
# generate executable
ghc -O --make ProcessChunk.c ProcessList.hs

Use a C function for filling chunks of a lazy ByteString
with data from a Haskell list.
A StablePtr is needed to pass the list to C and back to Haskell.
Somehow we must assert that the StablePtr is deleted
and the list can be garbage collected,
once the ByteString is constructed.
We concatenate infinitely many such lists,
thus many StablePtrs and referenced lists must be freed.
Use addFinalizer? To which object shall I add the finalizer?
Without optimization adding the finalizer to the StablePtr works,
with optimization, its finalizer is run too early.
I also tried to add the finalizer to the end of the list,
which is not quite correct,
but then the finalizer is run never.
Whatever I tried: Finalizer is run immediately or never.
What is the right way?

{-# LANGUAGE ForeignFunctionInterface #-}
module Main where

import qualified Data.ByteString as B
import qualified Data.ByteString.Internal as BI
import qualified Data.ByteString.Lazy as BL
import qualified Data.ByteString.Lazy.Internal as BLI

import System.IO.Unsafe (unsafePerformIO, unsafeInterleaveIO, )
import System.Mem.Weak (addFinalizer, )
import Foreign.StablePtr (StablePtr, newStablePtr, freeStablePtr, deRefStablePtr, )
import Foreign.Storable (poke, )
import Foreign.C.Types (CULong, )
import Foreign.Ptr (FunPtr, Ptr, nullPtr, castPtr, )
import Data.IORef (IORef, newIORef, readIORef, writeIORef, )
import Data.Word (Word8, Word32, )

import Control.Monad (liftM2, )

getNextElement ::
   StablePtr (IORef [Word8]) -> Ptr Word8 -> IO Bool
getNextElement stable elemPtr = do
   listRef <- deRefStablePtr stable
   xt <- readIORef listRef
   case xt of
      [] -> return False
      x:xs -> do
         poke elemPtr x
         writeIORef listRef xs
         return True

foreign export ccall "get_next_element"
   getNextElement :: StablePtr (IORef [Word8]) -> Ptr Word8 -> IO Bool

foreign import ccall "copy_list_to_chunk"
   copyListToChunk :: CULong -> StablePtr (IORef [Word8]) -> Ptr Word8 -> IO CULong

chunkSize :: Int
chunkSize = 13

byteStringFromList :: [Word8] -> BL.ByteString
byteStringFromList list = unsafePerformIO $ do
   stable <- newStablePtr =<< newIORef list
   addFinalizer stable
      (putStrLn "free stable pointer" >> freeStablePtr stable)
   let go =
         unsafeInterleaveIO $ do
            v <- BI.createAndTrim chunkSize $
               fmap fromIntegral .
               copyListToChunk (fromIntegral chunkSize) stable
            fmap (BLI.chunk v) $
               if B.length v < chunkSize
                 then return BL.empty
                 else go

main :: IO ()
main =
   print $ BL.unpack $ -- BL.take 1000 $
   BL.concat $
     (BL.take 42 . byteStringFromList . iterate (1+))
     (iterate (1+) 0)

FFI mailing list
[hidden email]
Reply | Threaded
Open this post in threaded view

Re: addFinalizer to which object? (fwd from cafe)

Henning Thielemann

On Tue, 19 Jan 2010, Henning Thielemann wrote:

> The documentation of System.Mem.Weak.addFinalizer suggests to me, that this
> function is very fragile, because the object we track, might be optimized
> away. ...

After even more web search I found the bytestring-mmap package that
brought my attention to the module Foreign.Concurrent. This actually
solves my problem! I would like to have pointers in System.Mem.Weak and
Foreign.ForeignPtr to this module, otherwise it is very hard to find! The
function names in this module are so similar to Foreign.ForeignPtr that it
is likely that you do not even recognize easily that it is actually
different functionality, not only the same functionality for concurrent
  I would also like to read about examples, where
System.Mem.Weak.addFinalizer can be actually used and does not interfer
unexpectedly with unboxing. If there is no such example there might be a
warning like "addFinalizer is certainly not, what you are looking for" or
FFI mailing list
[hidden email]