External system connections

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

External system connections

Richard Wallace
Hello all,

I'm trying to understand how to properly structure a connection to an
external system.  I'm writing an application that processes requests
(it's an IRC bot - ya, I've looked at lambabot but it's a bit beyond
my current understanding and I'm really trying to learn this stuff and
find the best way to do that is to write it myself) and sends requests
to an external system, via SOAP.  The SOAP requests should be sent on
separate threads.  The SOAP server requires an initial request for
authentication be made to obtain a token used on subsequent requests.

There are three ways I can structure this.  One is to do the
authentication with the SOAP server when the application starts up and
just use that token for all subsequent requests.  This will fall over
if the token times out and I'd have to restart the application.
Another way is to do authenticate with the server and get a new token
for each request.  This is obviously really inefficient.

What I'd like to do is something a bit smarter.  When a request to the
SOAP server is to be made, if we have a token then we try and use it.
If it fails, we reauthenticate and get a new token.  When establishing
a new token, other threads trying to send SOAP requests should block
until a new token is available.  There are other conditions that
should be handled - such as an exponential back-off when the SOAP
server can't be reached - but I feel like reestablishing the
authentication token is really the key concern here.  Unfortunately, I
have no idea right now how to write this in Haskell.

Any pointers would be awesome.  Thanks!

Rich

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Felipe Lessa
You don't need to do it, it is already done =).  See the pool package
by Michael Snoyman on Hackage [1].  More specifically, see
createPoolCheckAlive [2].

Cheers,

[1] http://hackage.haskell.org/package/pool
[2] http://hackage.haskell.org/packages/archive/pool/0.1.0.2/doc/html/Data-Pool.html#v:createPoolCheckAlive

--
Felipe.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Brandon Allbery
In reply to this post by Richard Wallace
On Sun, Jul 10, 2011 at 15:04, Richard Wallace
<[hidden email]> wrote:
> What I'd like to do is something a bit smarter.  When a request to the
> SOAP server is to be made, if we have a token then we try and use it.
> If it fails, we reauthenticate and get a new token.  When establishing
> a new token, other threads trying to send SOAP requests should block
> until a new token is available.  There are other conditions that

Typically you'd use a Chan or TChan for communication with the SOAP
thread.  Other threads write to the Chan and block if the reading
thread is busy.  There's also a non-blocking version.  (TChan uses
STM, which provides atomic transactions; in this case it may be
overkill, especially if you're just getting started.)  I wouldn't be
surprised if there's already a (or several) package on Hackage for
it....

--
brandon s allbery                                      [hidden email]
wandering unix systems administrator (available)     (412) 475-9364 vm/sms

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Richard Wallace
In reply to this post by Richard Wallace
On Sun, Jul 10, 2011 at 2:54 PM, Brandon Allbery <[hidden email]> wrote:

> On Sun, Jul 10, 2011 at 17:34, Richard Wallace
> <[hidden email]> wrote:
>> Rather than a single separate thread that makes requests I was hoping to
>> make several soap requests concurrently, rather than have them be made
>> serially.  I would only want to block other threads making soap request if
>> one of them returns a response indicating the token they were using is no
>> longer valid.
>
> Is there one shared token, or one per requesting thread, or some other
> arrangement?

There should be one shared token.

> In either case, separation of concerns makes me think
> the token handling belongs on the other end:  a thread writes a
> request down a Chan, a queue dispatcher thread reads it if there are
> available workers and the token is available (with callers blocking
> otherwise), dispatcher sends request to available worker.  If the
> worker finds the token is invalid, it calls back in to the dispatcher
> with a token renewal request, which causes anything else that needs
> that token to block as above, and itself blocks until a valid token is
> returned.

Ahhh... I see.  That sounds like a much better plan.  It also keeps
the fact that the token is "mutable" confined to a single thread, the
dispatcher thread.

One thing I'd have to be careful to handle is iIf multiple worker
threads find that the token is now invalid.  In that case they would
all send the token renewal request to the dispatcher and they would
each try and do the re-auth.  To avoid that, part of the token renewal
request could be the used token.  When the dispatcher does the token
renewal, it checks if the used token is the same as the current token.
 If so, it does the token renewal.  If not, it dispatches to the SOAP
worker thread with the new token.

> Another advantage of this is that adding more workers is
> done in an obvious place (the queuer thread).
>
> That said, I still don't know enough details to say if that token
> management implementation actually makes sense.  I'd still go with the
> worker/queuer thread setup, which is I think what you were pointed to
> (the pool hackage).
>

Ok, I still don't see how the Pool package helps.  I had assumed the
pooled resource would be the token, so the size of the pool would be
1.   Now that I rethink it, it seems like you are suggesting the
resource is the worker threads.  Is that right?  When I read what you
were talking about above with the dispatcher thread I had thought it
would use forkIO to start the worker threads.  The worker threads
would read from a Chan that the dispatcher wrote to.    In that way,
the dispatcher doesn't have to worry about worker threads being
available.  Am I misunderstanding something?

Thanks,
Rich

> --
> brandon s allbery                                      [hidden email]
> wandering unix systems administrator (available)     (412) 475-9364 vm/sms
>

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Brandon Allbery
On Sun, Jul 10, 2011 at 19:16, Richard Wallace
<[hidden email]> wrote:
> On Sun, Jul 10, 2011 at 2:54 PM, Brandon Allbery <[hidden email]> wrote:
> One thing I'd have to be careful to handle is iIf multiple worker
> threads find that the token is now invalid.  In that case they would
> all send the token renewal request to the dispatcher and they would
> each try and do the re-auth.  To avoid that, part of the token renewal

One of the reasons to send the request back to the dispatcher instead
of doing it inline is so that the dispatcher can note that a renewal
request is already in flight (which it needs to know anyway, so it can
block other requests) and wake all threads waiting on it when it's
done, instead of having multiple renewals in flight.

> 1.   Now that I rethink it, it seems like you are suggesting the
> resource is the worker threads.  Is that right?  When I read what you
> were talking about above with the dispatcher thread I had thought it
> would use forkIO to start the worker threads.  The worker threads
> would read from a Chan that the dispatcher wrote to.    In that way,
> the dispatcher doesn't have to worry about worker threads being
> available.  Am I misunderstanding something?

The point of a pool is so (a) you can throttle it in special cases,
such as when you need to renew the token, and (b) so you don't find
yourself juggling a couple thousand threads if things get unexpectedly
busy (or buggy).  You can limit the pool to something sensible
(probably something like 4 during development and debugging so things
can't get too out of hand) and increased later; plus, the pool manager
will provide the primitives to deal with managing shared resources
(such as your token) within the pool.

--
brandon s allbery                                      [hidden email]
wandering unix systems administrator (available)     (412) 475-9364 vm/sms

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Richard Wallace
On Sun, Jul 10, 2011 at 4:38 PM, Brandon Allbery <[hidden email]> wrote:
> One of the reasons to send the request back to the dispatcher instead
> of doing it inline is so that the dispatcher can note that a renewal
> request is already in flight (which it needs to know anyway, so it can
> block other requests) and wake all threads waiting on it when it's
> done, instead of having multiple renewals in flight.
>

Ok, I can see that.  Though I was thinking that the worker threads
would send the request back to the dispatcher by way of the same Chan
the dispatcher reads requests from.  Obviously I was thinking the
dispatcher would use the original token to filter requests in the
Chan.  If I understand what you are talking about, the dispatcher
would do the token renewal in a separate thread, continuing to process
it's incoming Chan while that is going on, accumulating incoming
requests somewhere, perhaps in another Chan.  Then, when the token
renewal is complete, the dispatcher stops forwarding incoming requests
to the secondary Chan, processes the accumulated requests using the
new token, and, when done with those, the switches back to processing
the incoming Chan.

Is that something like what you had in mind, or did I make it more
complicated than necessary?

> The point of a pool is so (a) you can throttle it in special cases,
> such as when you need to renew the token, and (b) so you don't find
> yourself juggling a couple thousand threads if things get unexpectedly
> busy (or buggy).  You can limit the pool to something sensible
> (probably something like 4 during development and debugging so things
> can't get too out of hand) and increased later; plus, the pool manager
> will provide the primitives to deal with managing shared resources
> (such as your token) within the pool.
>

Hmm.  I'm still not entirely seeing it and I think the problem is just
my lack of knowledge of Haskell concurrency.

If the pool is of threads, how do you start the threads?  How do you
submit work to the threads?  The only way I know of in Haskell of
creating threads to do work is forkIO.  That takes a function and runs
to completion.  Would a worker thread just be one that loops forever
and checks a MVar for something to do?  So the pool would really
consist of (MVar, ThreadId) pairs.  When we speak of getting a thread
out of a pool and giving it work to do, are we really talking about
sticking a value in an MVar that the thread is blocking on, waiting
for data to be available to do something with, so that it can go and
do some work?  And when we are done with the thread we return it to
the pool?

Rich

P.S. Thanks for talking me through this!  I'm learning a ton about
concurrency in Haskell.

> --
> brandon s allbery                                      [hidden email]
> wandering unix systems administrator (available)     (412) 475-9364 vm/sms
>

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Brandon Allbery
On Sun, Jul 10, 2011 at 20:19, Richard Wallace
<[hidden email]> wrote:
> On Sun, Jul 10, 2011 at 4:38 PM, Brandon Allbery <[hidden email]> wrote:
> Ok, I can see that.  Though I was thinking that the worker threads
> would send the request back to the dispatcher by way of the same Chan
> the dispatcher reads requests from.  Obviously I was thinking the
> dispatcher would use the original token to filter requests in the
> Chan.  If I understand what you are talking about, the dispatcher
> would do the token renewal in a separate thread, continuing to process

The same Chan is used to send a request; the thread processing token
renewal might or might not be otherwise a normal worker thread, but
it's separated out as a Maybe ThreadId instead of being in a pool,
because (a) there can only be zero or one of them, and (b) if it's not
Nothing then the dispatcher thread accepts only token renewals.  (This
actually requires either multiple Chans or something more complex than
a normal Chan, since you can't filter a Chan based on the type of
message.)  You also need some way to block the sender, which suggests
that a message written down a Chan must include an MVar which will be
signaled when the operation is complete.  This suggests to me
something along the lines of

> data WorkRequest = SOAPData ... (MVar Bool)
>                  | TokenRequest Token (MVar Token)
> --               | ReadyForWork (MVar WorkRequest)

where the requestor allocates an MVar, writes it as part of the
WorkRequest, and then does a takeMVar to wait for the response.  The
dispatcher reads WorkRequests, dispatches any it can to available
workers, and queues the rest internally; if it's a TokenRequest then
it's queued separately and all SOAPData requests get queued regardless
of whether there are free workers.  When the single token processor
returns, all entries in the TokenRequest queue get awakened (putMVar
threadMVar newToken) and normal processing of the standard request
queue resumes.

Or you can see if the pool hackage handles the ugly details here
automatically; I haven't looked.

> If the pool is of threads, how do you start the threads?  How do you
> submit work to the threads?  The only way I know of in Haskell of
> creating threads to do work is forkIO.  That takes a function and runs
> to completion.  Would a worker thread just be one that loops forever

Yes; the dispatcher keeps a list of workers, which are forkIO-d
threads that are waiting on an MVar or Chan for work to do.  When they
receive something, they go off and do it, write the result into
another MVar or Chan which was specified in the request, and go back
to waiting on the initial MVar/Chan for something to do.  If the list
is shorter than the maximum, more workers are forkIO-d to fill it as
needed; if longer, idle workers are sent "shut down" requests.  (The
latter is "polite" handling of program shutdown, and also allows for
the pool size to be modified dynamically if needed.)  I think doing
this right also requires that a worker that's ready for more work
explicitly check in, so the dispatcher knows it's available; that
could be handled by an additional WorkRequest type (see commented-out
line above, where a worker that's ready to handle another request
passes its input MVar to the dispatcher)... but there may be better
ways; I have some grasp of concurrency, but my Haskell library fu is
still somewhat weak.  Hopefully someone else will jump in if
appropriate.

(You can see how quickly this becomes complex, though; if the canned
solution does what you need, you might want to avoid reinventing this
particular wheel unless you're doing it for educational purposes.)

--
brandon s allbery                                      [hidden email]
wandering unix systems administrator (available)     (412) 475-9364 vm/sms

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Richard Wallace
Alright, I'll have to think on this some more but I think we're
speaking the same language now - and what's more I even understand it!

Thanks again for all your help,
Rich

On Sun, Jul 10, 2011 at 6:46 PM, Brandon Allbery <[hidden email]> wrote:

> On Sun, Jul 10, 2011 at 20:19, Richard Wallace
> <[hidden email]> wrote:
>> On Sun, Jul 10, 2011 at 4:38 PM, Brandon Allbery <[hidden email]> wrote:
>> Ok, I can see that.  Though I was thinking that the worker threads
>> would send the request back to the dispatcher by way of the same Chan
>> the dispatcher reads requests from.  Obviously I was thinking the
>> dispatcher would use the original token to filter requests in the
>> Chan.  If I understand what you are talking about, the dispatcher
>> would do the token renewal in a separate thread, continuing to process
>
> The same Chan is used to send a request; the thread processing token
> renewal might or might not be otherwise a normal worker thread, but
> it's separated out as a Maybe ThreadId instead of being in a pool,
> because (a) there can only be zero or one of them, and (b) if it's not
> Nothing then the dispatcher thread accepts only token renewals.  (This
> actually requires either multiple Chans or something more complex than
> a normal Chan, since you can't filter a Chan based on the type of
> message.)  You also need some way to block the sender, which suggests
> that a message written down a Chan must include an MVar which will be
> signaled when the operation is complete.  This suggests to me
> something along the lines of
>
>> data WorkRequest = SOAPData ... (MVar Bool)
>>                  | TokenRequest Token (MVar Token)
>> --               | ReadyForWork (MVar WorkRequest)
>
> where the requestor allocates an MVar, writes it as part of the
> WorkRequest, and then does a takeMVar to wait for the response.  The
> dispatcher reads WorkRequests, dispatches any it can to available
> workers, and queues the rest internally; if it's a TokenRequest then
> it's queued separately and all SOAPData requests get queued regardless
> of whether there are free workers.  When the single token processor
> returns, all entries in the TokenRequest queue get awakened (putMVar
> threadMVar newToken) and normal processing of the standard request
> queue resumes.
>
> Or you can see if the pool hackage handles the ugly details here
> automatically; I haven't looked.
>
>> If the pool is of threads, how do you start the threads?  How do you
>> submit work to the threads?  The only way I know of in Haskell of
>> creating threads to do work is forkIO.  That takes a function and runs
>> to completion.  Would a worker thread just be one that loops forever
>
> Yes; the dispatcher keeps a list of workers, which are forkIO-d
> threads that are waiting on an MVar or Chan for work to do.  When they
> receive something, they go off and do it, write the result into
> another MVar or Chan which was specified in the request, and go back
> to waiting on the initial MVar/Chan for something to do.  If the list
> is shorter than the maximum, more workers are forkIO-d to fill it as
> needed; if longer, idle workers are sent "shut down" requests.  (The
> latter is "polite" handling of program shutdown, and also allows for
> the pool size to be modified dynamically if needed.)  I think doing
> this right also requires that a worker that's ready for more work
> explicitly check in, so the dispatcher knows it's available; that
> could be handled by an additional WorkRequest type (see commented-out
> line above, where a worker that's ready to handle another request
> passes its input MVar to the dispatcher)... but there may be better
> ways; I have some grasp of concurrency, but my Haskell library fu is
> still somewhat weak.  Hopefully someone else will jump in if
> appropriate.
>
> (You can see how quickly this becomes complex, though; if the canned
> solution does what you need, you might want to avoid reinventing this
> particular wheel unless you're doing it for educational purposes.)
>
> --
> brandon s allbery                                      [hidden email]
> wandering unix systems administrator (available)     (412) 475-9364 vm/sms
>

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Michael Orlitzky
In reply to this post by Felipe Lessa
On 07/10/11 15:37, Felipe Almeida Lessa wrote:

> You don't need to do it, it is already done =).  See the pool
> package by Michael Snoyman on Hackage [1].  More specifically, see
> createPoolCheckAlive [2].
>
> Cheers,
>
> [1] http://hackage.haskell.org/package/pool [2]
> http://hackage.haskell.org/packages/archive/pool/0.1.0.2/doc/html/Data-Pool.html#v:createPoolCheckAlive
>
>

How do people use this stuff? The README is empty, there's no
documentation, and none of the functions are commented. I'm forced to
conclude that there are people capable of looking at this,

  createPoolCheckAlive :: MonadControlIO m
                          => IO a
                          -> (a -> IO ())
                          -> Int
                          -> (Pool a -> m b)
                          -> (a -> IO Bool)
                          -> m b

and knowing immediately what it does.

I'm still a beginner, and I don't think I have a use for this package
(what does it do?), but I've run into similar situations with other
packages and would genuinely like to know (as opposed to just complain
about it).

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Brandon Allbery
On Mon, Jul 11, 2011 at 11:27, Michael Orlitzky <[hidden email]> wrote:
> How do people use this stuff? The README is empty, there's no
> documentation, and none of the functions are commented. I'm forced to
> conclude that there are people capable of looking at this,

I can *almost* work it out just from the type and knowing what it
should be doing.  But in this case, the main problem is that it was
split off of yesod (http://www.yesodweb.com/), and it was only
documented in the context of that.

--
brandon s allbery                                      [hidden email]
wandering unix systems administrator (available)     (412) 475-9364 vm/sms

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Felipe Lessa
In reply to this post by Michael Orlitzky
On Mon, Jul 11, 2011 at 12:27 PM, Michael Orlitzky <[hidden email]> wrote:

> How do people use this stuff? The README is empty, there's no
> documentation, and none of the functions are commented. I'm forced to
> conclude that there are people capable of looking at this,
>
>  createPoolCheckAlive :: MonadControlIO m
>                          => IO a
>                          -> (a -> IO ())
>                          -> Int
>                          -> (Pool a -> m b)
>                          -> (a -> IO Bool)
>                          -> m b
>
> and knowing immediately what it does.

Yes, there's a documentation problem.  But you can guess by the types =).

IO a: opens a new resource a.
a -> IO (): disposes the given resource.
Int: maximum number of open resources.
Pool a -> m b: the action you want to execute; after it executes, the
pool is destroyed.
a -> IO Bool: check if the resource is still alive.
m b: the result of running your given action with a new pool.

The key to understanding all of this is looking backwards from the result.

1) The result is m b.  Where did this come from?
2) Well, there's a 'Pool a -> m b'.  So it runs the action for me.  So
this is like a 'withFile :: FilePath -> (Handle -> IO a) -> IO a'
function.
3) So if it is a 'Pool a', then the resource is of type 'a'.
4) The only function returning 'a' is 'IO a', so this creates the resource.
5) By the same reason, 'a -> IO ()' disposes it.
6) 'a -> IO Bool' is confusing, but there's a doc for that.
7) 'Int' is also confusing, but since this is a pool, this must be the
maximum number of open resources.

HTH,

--
Felipe.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

ozataman
In reply to this post by Richard Wallace
Have you looked at Bryan O'Sullivan's resource-pool[1] library?

I've used it with MySQL, Redis and Riak connection pooling. It's been robust and easy to use. It seems like it would be a good fit for your application.

Cheers,
Ozgun

[1] http://hackage.haskell.org/package/resource-pool

On Sun, Jul 10, 2011 at 3:04 PM, Richard Wallace <[hidden email]> wrote:
Hello all,

I'm trying to understand how to properly structure a connection to an
external system.  I'm writing an application that processes requests
(it's an IRC bot - ya, I've looked at lambabot but it's a bit beyond
my current understanding and I'm really trying to learn this stuff and
find the best way to do that is to write it myself) and sends requests
to an external system, via SOAP.  The SOAP requests should be sent on
separate threads.  The SOAP server requires an initial request for
authentication be made to obtain a token used on subsequent requests.

There are three ways I can structure this.  One is to do the
authentication with the SOAP server when the application starts up and
just use that token for all subsequent requests.  This will fall over
if the token times out and I'd have to restart the application.
Another way is to do authenticate with the server and get a new token
for each request.  This is obviously really inefficient.

What I'd like to do is something a bit smarter.  When a request to the
SOAP server is to be made, if we have a token then we try and use it.
If it fails, we reauthenticate and get a new token.  When establishing
a new token, other threads trying to send SOAP requests should block
until a new token is available.  There are other conditions that
should be handled - such as an exponential back-off when the SOAP
server can't be reached - but I feel like reestablishing the
authentication token is really the key concern here.  Unfortunately, I
have no idea right now how to write this in Haskell.

Any pointers would be awesome.  Thanks!

Rich

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Michael Orlitzky
In reply to this post by Felipe Lessa
On 07/11/11 11:41, Felipe Almeida Lessa wrote:

>
> Yes, there's a documentation problem.  But you can guess by the types =).
>
> IO a: opens a new resource a.
> a -> IO (): disposes the given resource.
> Int: maximum number of open resources.
> Pool a -> m b: the action you want to execute; after it executes, the
> pool is destroyed.
> a -> IO Bool: check if the resource is still alive.
> m b: the result of running your given action with a new pool.
>
> The key to understanding all of this is looking backwards from the result.
>
> 1) The result is m b.  Where did this come from?
> 2) Well, there's a 'Pool a -> m b'.  So it runs the action for me.  So
> this is like a 'withFile :: FilePath -> (Handle -> IO a) -> IO a'
> function.
> 3) So if it is a 'Pool a', then the resource is of type 'a'.
> 4) The only function returning 'a' is 'IO a', so this creates the resource.
> 5) By the same reason, 'a -> IO ()' disposes it.
> 6) 'a -> IO Bool' is confusing, but there's a doc for that.
> 7) 'Int' is also confusing, but since this is a pool, this must be the
> maximum number of open resources.

Magic, got it =)

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Michael Snoyman
In reply to this post by Michael Orlitzky
On Mon, Jul 11, 2011 at 6:27 PM, Michael Orlitzky <[hidden email]> wrote:

> On 07/10/11 15:37, Felipe Almeida Lessa wrote:
>> You don't need to do it, it is already done =).  See the pool
>> package by Michael Snoyman on Hackage [1].  More specifically, see
>> createPoolCheckAlive [2].
>>
>> Cheers,
>>
>> [1] http://hackage.haskell.org/package/pool [2]
>> http://hackage.haskell.org/packages/archive/pool/0.1.0.2/doc/html/Data-Pool.html#v:createPoolCheckAlive
>>
>>
>
> How do people use this stuff? The README is empty, there's no
> documentation, and none of the functions are commented. I'm forced to
> conclude that there are people capable of looking at this,
>
>  createPoolCheckAlive :: MonadControlIO m
>                          => IO a
>                          -> (a -> IO ())
>                          -> Int
>                          -> (Pool a -> m b)
>                          -> (a -> IO Bool)
>                          -> m b
>
> and knowing immediately what it does.
>
> I'm still a beginner, and I don't think I have a use for this package
> (what does it do?), but I've run into similar situations with other
> packages and would genuinely like to know (as opposed to just complain
> about it).
>

Author of the package speaking. I agree, that's a problem :). I've
just uploaded a new version[1] that is properly documented, though it
will take a bit for Hackage to generates the Haddocks. As Brandon
said, the reason this wasn't documented in the first place is that it
was an internally used module for Persistent that I decided after the
fact to release separately.

As for Bryan's resource-pool: currently I would strongly recommend
*against* using it for any purpose. It is based on
MonadCatchIO-transformers[2], which is a subtly broken package. In
particular, when I tried using it for pool/persistent in the first
place, I ended up with double-free bugs from SQLite. As an abridged
history, I ended up writing a replacement called MonadInvertIO, which
later was superceded by MonadPeelIO[3] and MonadControlIO[4]. The
latter two packages are both much more correct that MonadCatchIO, and
either one should be used in its place.

I did email Bryan about this a bit ago, but he didn't get back (he
*is* a very busy guy). Ideally, I think that we don't need to have
both pool and resource-pool, but such is the situation right now.

Michael

[1] http://hackage.haskell.org/package/pool-0.1.0.3
[2] http://hackage.haskell.org/package/MonadCatchIO-transformers-0.2.2.2
[3] http://hackage.haskell.org/package/monad-peel
[4] http://hackage.haskell.org/package/monad-control-0.2.0.1

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Michael Orlitzky
On 07/11/11 13:49, Michael Snoyman wrote:
>
> Author of the package speaking. I agree, that's a problem :). I've
> just uploaded a new version[1] that is properly documented, though it
> will take a bit for Hackage to generates the Haddocks. As Brandon
> said, the reason this wasn't documented in the first place is that it
> was an internally used module for Persistent that I decided after the
> fact to release separately.

Thanks for humoring me. I guess I'll begin tracking down other authors
and making them feel guilty!

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Bryan O'Sullivan
In reply to this post by Michael Snoyman
On Mon, Jul 11, 2011 at 10:49 AM, Michael Snoyman <[hidden email]> wrote:
I did email Bryan about this a bit ago, but he didn't get back [...]

Thanks for jogging my memory. I've released an updated version of resource-pool that drops the dependency on that package.


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Alistair Bayley
In reply to this post by Michael Snoyman
 12 July 2011 05:49, Michael Snoyman <[hidden email]> wrote:
>
> As for Bryan's resource-pool: currently I would strongly recommend
> *against* using it for any purpose. It is based on
> MonadCatchIO-transformers[2], which is a subtly broken package. In
> particular, when I tried using it for pool/persistent in the first
> place, I ended up with double-free bugs from SQLite.

Do you have a reference explaining this brokenness? e.g. a mailing
list message? I wasn't aware of this. Are the other MonadCatchIO-*
packages also broken?

Alistair

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: External system connections

Daniel Gorín

On Jul 11, 2011, at 10:48 PM, Alistair Bayley wrote:

> 12 July 2011 05:49, Michael Snoyman <[hidden email]> wrote:
>>
>> As for Bryan's resource-pool: currently I would strongly recommend
>> *against* using it for any purpose. It is based on
>> MonadCatchIO-transformers[2], which is a subtly broken package. In
>> particular, when I tried using it for pool/persistent in the first
>> place, I ended up with double-free bugs from SQLite.
>
> Do you have a reference explaining this brokenness? e.g. a mailing
> list message? I wasn't aware of this. Are the other MonadCatchIO-*
> packages also broken?
>

http://www.haskell.org/pipermail/haskell-cafe/2010-October/084890.html


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe