forkProcess behaviour

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

forkProcess behaviour

PICCA Frederic-Emmanuel
Hello,

I try to write a service which execute an IO using forkProcess
This service contain a web server whcih avwait for job published by users.
for each job, I create a forkProcess of this job. I need to fork the process in order to change the uid and gid of the process for each of the job dependencing on who request the job.

I have a least two questions

1) is this forkProcess a fork of all the current process, or is it just an executin of the IO in another process id ?

I ask this because sometime one of my job hang and a process keep running. Since the original process is binded to a port, it is not possible to restart the server, saying that the port is already in use.

2 ) How can I catch the exceptions thrown from the child process in order to process them in the parent process.

thanks for your help answering these questions.


Frederic
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: forkProcess behaviour

David McBride
I'm not familiar with forkProcess.

As for exception handling, with asynchronous exceptions, it's a  pretty in depth topic and I can't say I have any mastery of it.  fpcomplete did a talk recently that I suspect could answer most of your questions.  https://www.youtube.com/watch?v=T5y8sFmCFnA

On Tue, Jul 17, 2018 at 4:48 AM, PICCA Frederic-Emmanuel <[hidden email]> wrote:
Hello,

I try to write a service which execute an IO using forkProcess
This service contain a web server whcih avwait for job published by users.
for each job, I create a forkProcess of this job. I need to fork the process in order to change the uid and gid of the process for each of the job dependencing on who request the job.

I have a least two questions

1) is this forkProcess a fork of all the current process, or is it just an executin of the IO in another process id ?

I ask this because sometime one of my job hang and a process keep running. Since the original process is binded to a port, it is not possible to restart the server, saying that the port is already in use.

2 ) How can I catch the exceptions thrown from the child process in order to process them in the parent process.

thanks for your help answering these questions.


Frederic
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: forkProcess behaviour

PICCA Frederic-Emmanuel
Hello thanks for the link, I already saw theses slides :)

but it seems to me that forkProcess is a different beast :))
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: forkProcess behaviour

Michael Snoyman
In reply to this post by PICCA Frederic-Emmanuel
Hi Frederic,

Let me give a high level recommendation: don't use forkProcess. It's a very dangerous function, which can result in confusing behavior and impossible-to-debug problems. There are cases where it can be made to work, but (1) it's complicated, (2) I'm not sure anyone has ever figured out all of those caveats, and (3) it's certainly not documented properly.

forkProcess is little more than a call to the fork() system call, creating a brand new child process which will run the IO action provided. The runtimes of the two processes will not be connected to each other at all. It would be impossible to, say, throw an exception from a thread in the parent process to a thread in the child process.

I could say a lot more about this, but I think I'll just reiterate my original recommendation: don't use forkProcess :)

Instead, for this kind of use case of changing user/group IDs, I'd recommend using a normal external process call via the process package[1]. I'm not sure of your use case exactly, but I see three ways of making this work:

* Generate two Haskell executables
* Put the code for both the parent and child into a single executable, and use command line arguments or env vars to control which behavior is run
* If you don't have any real logic in the Haskell code, and instead are just using some other program: you can call that directly

HTH,
Michael

[1] Or the typed-process package instead, which I typically recommend these days. Note that I'm the maintainer of process, but typed-process has a brand new API I think is easier to use correctly. More info at: https://haskell-lang.org/library/typed-process

On Tue, Jul 17, 2018 at 3:36 PM PICCA Frederic-Emmanuel <[hidden email]> wrote:
Hello,

I try to write a service which execute an IO using forkProcess
This service contain a web server whcih avwait for job published by users.
for each job, I create a forkProcess of this job. I need to fork the process in order to change the uid and gid of the process for each of the job dependencing on who request the job.

I have a least two questions

1) is this forkProcess a fork of all the current process, or is it just an executin of the IO in another process id ?

I ask this because sometime one of my job hang and a process keep running. Since the original process is binded to a port, it is not possible to restart the server, saying that the port is already in use.

2 ) How can I catch the exceptions thrown from the child process in order to process them in the parent process.

thanks for your help answering these questions.


Frederic
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners

_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: forkProcess behaviour

PICCA Frederic-Emmanuel
> Hi Frederic,

Hello Michael

Thanks a lot for your informative answer.

> Let me give a high level recommendation: don't use forkProcess. It's a very dangerous function, which can result in confusing behavior and impossible-to-debug problems. There are cases where it can be made to work, but (1) it's complicated, (2) I'm not sure anyone has ever figured out all of those caveats, and (3) it's certainly not documented properly.

> forkProcess is little more than a call to the fork() system call, creating a brand new child process which will run the IO action provided. The runtimes of the two processes will not be > connected to each other at all. It would be impossible to, say, throw an exception from a thread in the parent process to a thread in the child process.

> I could say a lot more about this, but I think I'll just reiterate my original recommendation: don't use forkProcess :)

> Instead, for this kind of use case of changing user/group IDs, I'd recommend using a normal external process call via the process package[1]. I'm not sure of your use case exactly, > but I see three ways of making this work:

> * Generate two Haskell executables
> * Put the code for both the parent and child into a single executable, and use command line arguments or env vars to control which behavior is run
> * If you don't have any real logic in the Haskell code, and instead are just using some other program: you can call that directly

I am quite close to this model, since I have only once executable and the command line parameters allows to execute the different job like this

autoprocessing-exe [exec|submit|server] <jobname> parameters

I start my service via the server command without parameters.
Then from another computer, I can submit jobs to that server.
And locally I can execute jobs with the exec command.

What you proposed is just to execute the autoprocessing-command line with s/submit/exec in order to execute a child process.

I think that I can do this.

Now sometimes this process hang and I want to add a timeout, can I just use timeout for this purpose.

2) I do not want to have communication between my server process and the child one, so in that case is it worth changing the code in order to use process instead of forkPRocess.

With process I will need to had lot's of code in oder to convert my job type into command line.

I have for now the right Parser command line -> jobtype, but not the other way around.
This is why I used forkProcess et first.

Once again thanks for your valuable comments.

Frédéric
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: forkProcess behaviour

Michael Snoyman


On Fri, Jul 20, 2018 at 10:19 AM PICCA Frederic-Emmanuel <[hidden email]> wrote:
> Hi Frederic,

Hello Michael

Thanks a lot for your informative answer.

> Let me give a high level recommendation: don't use forkProcess. It's a very dangerous function, which can result in confusing behavior and impossible-to-debug problems. There are cases where it can be made to work, but (1) it's complicated, (2) I'm not sure anyone has ever figured out all of those caveats, and (3) it's certainly not documented properly.

> forkProcess is little more than a call to the fork() system call, creating a brand new child process which will run the IO action provided. The runtimes of the two processes will not be > connected to each other at all. It would be impossible to, say, throw an exception from a thread in the parent process to a thread in the child process.

> I could say a lot more about this, but I think I'll just reiterate my original recommendation: don't use forkProcess :)

> Instead, for this kind of use case of changing user/group IDs, I'd recommend using a normal external process call via the process package[1]. I'm not sure of your use case exactly, > but I see three ways of making this work:

> * Generate two Haskell executables
> * Put the code for both the parent and child into a single executable, and use command line arguments or env vars to control which behavior is run
> * If you don't have any real logic in the Haskell code, and instead are just using some other program: you can call that directly

I am quite close to this model, since I have only once executable and the command line parameters allows to execute the different job like this

autoprocessing-exe [exec|submit|server] <jobname> parameters

I start my service via the server command without parameters.
Then from another computer, I can submit jobs to that server.
And locally I can execute jobs with the exec command.

What you proposed is just to execute the autoprocessing-command line with s/submit/exec in order to execute a child process.

I think that I can do this.

Now sometimes this process hang and I want to add a timeout, can I just use timeout for this purpose.


For the most part, yes. You may need to reach deeper and use SIGKILL occasionally, depending on how stubborn the child process is. The `timeout` will only kill off the parent thread's call to block on the child's exit code.
 
2) I do not want to have communication between my server process and the child one, so in that case is it worth changing the code in order to use process instead of forkPRocess.

With process I will need to had lot's of code in oder to convert my job type into command line.

I have for now the right Parser command line -> jobtype, but not the other way around.
This is why I used forkProcess et first.

Once again thanks for your valuable comments.


Yes, you should avoid forkProcess in this case, it will have unpredictable and confusing results.
 
Frédéric
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners

_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: forkProcess behaviour

PICCA Frederic-Emmanuel
Hello

I have done the first solution and call the program changing the command line in oder to execute each child task.
It works well and I can change the uid gid of these process.

> For the most part, yes. You may need to reach deeper and use SIGKILL occasionally, depending on how stubborn the child process is. The `timeout` will only kill off the parent thread's call to block on the child's exit code.

Since all these tasks are executing also child process via other system cals.
So I have a hyerarchy of process.

Do you know how to manage that sort of things if something goes wrong and I need to kill the first child and all its children's ?
Is it possible for CreateProcess to track all these process and kill them all  on request ?

> Yes, you should avoid forkProcess in this case, it will have unpredictable and confusing results.

I take you advices seriously and now I start to build something (I hope) more robust.

Thansk

Frédéric
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: forkProcess behaviour

Michael Snoyman


On Tue, Jul 24, 2018 at 10:05 AM PICCA Frederic-Emmanuel <[hidden email]> wrote:
Hello

I have done the first solution and call the program changing the command line in oder to execute each child task.
It works well and I can change the uid gid of these process.

> For the most part, yes. You may need to reach deeper and use SIGKILL occasionally, depending on how stubborn the child process is. The `timeout` will only kill off the parent thread's call to block on the child's exit code.

Since all these tasks are executing also child process via other system cals.
So I have a hyerarchy of process.

Do you know how to manage that sort of things if something goes wrong and I need to kill the first child and all its children's ?
Is it possible for CreateProcess to track all these process and kill them all  on request ?


Funnily enough, there's another discussion going on about this right now:


The basic answer is:

* You can create a process group and then kill the whole process group
* But a misbehaving program would still be able to escape it by creating its own process group, only something like cgroups can be fully reliable here
 
> Yes, you should avoid forkProcess in this case, it will have unpredictable and confusing results.

I take you advices seriously and now I start to build something (I hope) more robust.

Thansk

Frédéric
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners

_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: forkProcess behaviour

PICCA Frederic-Emmanuel
> Funnily enough, there's another discussion going on about this right now:

> https://mail.haskell.org/pipermail/haskell-cafe/2018-July/129646.html


:))

> The basic answer is:

> * You can create a process group and then kill the whole process group
> * But a misbehaving program would still be able to escape it by creating its own process group, only something like cgroups can be fully reliable here

My server will be managed via systemd, so I hope that it will be able to  managed this  when restarting.
Is there an haskell API in order to work with cgoups ?

Cheers

Fred
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: forkProcess behaviour

Michael Snoyman


On Wed, Jul 25, 2018 at 11:27 AM PICCA Frederic-Emmanuel <[hidden email]> wrote:
> Funnily enough, there's another discussion going on about this right now:

> https://mail.haskell.org/pipermail/haskell-cafe/2018-July/129646.html


:))

> The basic answer is:

> * You can create a process group and then kill the whole process group
> * But a misbehaving program would still be able to escape it by creating its own process group, only something like cgroups can be fully reliable here

My server will be managed via systemd, so I hope that it will be able to  managed this  when restarting.
Is there an haskell API in order to work with cgoups ?


None that I'm aware of.
 
Cheers

Fred
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners

_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners