Dynamically choosing the main function

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Dynamically choosing the main function

Harendra Kumar
Hi,

GHC allows choosing a main function at link time using the "-main-is" option. I was wondering if there is a possibility to choose the main function at runtime. Or even better, if something equivalent to "ghc -e"  is somehow possible in a linked binary executable. If not, are there any plans to achieve something like that in future? Are there any theoretical, practical obstacles to that?

Thanks,
Harendra

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Dynamically choosing the main function

Shea Levy
What would you be able to achieve with this that you couldn't achieve
with branching in a fixed custom main function?

Thanks,
Shea

Harendra Kumar <[hidden email]> writes:

> Hi,
>
> GHC allows choosing a main function at link time using the "-main-is"
> option. I was wondering if there is a possibility to choose the main
> function at runtime. Or even better, if something equivalent to "ghc -e"
> is somehow possible in a linked binary executable. If not, are there any
> plans to achieve something like that in future? Are there any theoretical,
> practical obstacles to that?
>
> Thanks,
> Harendra
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Dynamically choosing the main function

Harendra Kumar
For example, I can easily define entry points for arbitrary functions. At run time I can choose which entry point to call and benchmark that entry point using existing performance benchmarking tools like the very powerful and comprehensive "perf" tool on linux. This provides a very powerful way to analyze the performance for free and in a convenient manner with little instrumentation. I do not want the main entry point to be clobbered for such things. Ideally I would just use a simple annotation for any arbitrary function to convert it into an entry point that can be chosen at runtime. I am just giving a general idea here without going into specifics.

-harendra

On 12 November 2017 at 21:58, Shea Levy <[hidden email]> wrote:
What would you be able to achieve with this that you couldn't achieve
with branching in a fixed custom main function?

Thanks,
Shea

Harendra Kumar <[hidden email]> writes:

> Hi,
>
> GHC allows choosing a main function at link time using the "-main-is"
> option. I was wondering if there is a possibility to choose the main
> function at runtime. Or even better, if something equivalent to "ghc -e"
> is somehow possible in a linked binary executable. If not, are there any
> plans to achieve something like that in future? Are there any theoretical,
> practical obstacles to that?
>
> Thanks,
> Harendra
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Dynamically choosing the main function

Ben Gamari-2
In reply to this post by Harendra Kumar
Harendra Kumar <[hidden email]> writes:

> Hi,
>
> GHC allows choosing a main function at link time using the "-main-is"
> option. I was wondering if there is a possibility to choose the main
> function at runtime. Or even better, if something equivalent to "ghc -e"
> is somehow possible in a linked binary executable. If not, are there any
> plans to achieve something like that in future? Are there any theoretical,
> practical obstacles to that?
>
In general it's not easy to get a reference to a an arbitrary function
in an object file by it's source name. Unless the symbol is exported the
compiler is free to optimize the function or even drop it altogether.

If you can guarantee that the functions you are about are exported then
in principle you could probably do what you ask from a library. Just
`dlopen` the executable (which would need to be compiled as a position
independent exectuable), and dlsym the appropriately mangled name.

Either way, I don't think think this request is in scope for the
compiler.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Dynamically choosing the main function

Brandon Allbery
In reply to this post by Harendra Kumar
On Sun, Nov 12, 2017 at 11:18 AM, Harendra Kumar <[hidden email]> wrote:
GHC allows choosing a main function at link time using the "-main-is" option. I was wondering if there is a possibility to choose the main function at runtime. Or even better, if something equivalent to "ghc -e"  is somehow possible in a linked binary executable. If not, are there any plans to achieve something like that in future? Are there any theoretical, practical obstacles to that?

-e is just running the compiler's bytecode backend on an expression from the command line instead of a declaration from a file. It's not related to this.

I might point out that this is not at all common from any language. A limited (and Unix-specific) mechanism is to have main pick from a list of fixed operating modes based on the program's basename, which can be set in the filesystem by making a hard link to the executable with a different basename.

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Dynamically choosing the main function

Harendra Kumar
In reply to this post by Ben Gamari-2


On 12 November 2017 at 23:18, Ben Gamari <[hidden email]> wrote:
Harendra Kumar <[hidden email]> writes:

> Hi,
>
> GHC allows choosing a main function at link time using the "-main-is"
> option. I was wondering if there is a possibility to choose the main
> function at runtime. Or even better, if something equivalent to "ghc -e"
> is somehow possible in a linked binary executable. If not, are there any
> plans to achieve something like that in future? Are there any theoretical,
> practical obstacles to that?
>
In general it's not easy to get a reference to a an arbitrary function
in an object file by it's source name. Unless the symbol is exported the
compiler is free to optimize the function or even drop it altogether.

I understand that. I was also wondering if there is way (some sort of annotation, pragma or any other workaround) to keep the symbols around without actually exporting them.  Exporting either clobbers the module interface from user point of view or the modules need a wrapper to avoid exporting such symbols in user exposed modules. I would also expect an option to remove the effect of any such annotation so that the production build is not under optimized because of this. This is in fact one of the problems that I have been facing in general. I do not know of a good way to hide symbols from users but use them in tests and benchmarks during dev. What is the GHC recommended way to achieve this?


If you can guarantee that the functions you are about are exported then
in principle you could probably do what you ask from a library. Just
`dlopen` the executable (which would need to be compiled as a position
independent exectuable), and dlsym the appropriately mangled name.

Yeah that is a good option for dynamically linked executables. In fact I have been using dlopen to implement Haskell plugins but did not think of using it for this use case. Thanks for reminding me. In a static executable, does it make sense to make the RTS choose an exported symbol as entry point using an RTS option?

-harendra


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Dynamically choosing the main function

Saurabh Nanda
> In fact I have been using dlopen to implement Haskell plugins but did not think of using it for this use case. 

Absolutely OT. Is this technique different from what facebook/simonmar have recently open-sourced? Please blog about it. 

-- Saurabh. 




-harendra


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs




_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Dynamically choosing the main function

Harendra Kumar


On 13 November 2017 at 00:29, Saurabh Nanda <[hidden email]> wrote:
> In fact I have been using dlopen to implement Haskell plugins but did not think of using it for this use case. 

Absolutely OT. Is this technique different from what facebook/simonmar have recently open-sourced? Please blog about it. 


-harendra 

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Dynamically choosing the main function

Ben Gamari-2
In reply to this post by Harendra Kumar
Harendra Kumar <[hidden email]> writes:

> On 12 November 2017 at 23:18, Ben Gamari <[hidden email]> wrote:
>
>> In general it's not easy to get a reference to a an arbitrary function
>> in an object file by it's source name. Unless the symbol is exported the
>> compiler is free to optimize the function or even drop it altogether.
>>
>
> I understand that. I was also wondering if there is way (some sort of
> annotation, pragma or any other workaround) to keep the symbols around
> without actually exporting them.  Exporting either clobbers the module
> interface from user point of view or the modules need a wrapper to avoid
> exporting such symbols in user exposed modules. I would also expect an
> option to remove the effect of any such annotation so that the production
> build is not under optimized because of this. This is in fact one of the
> problems that I have been facing in general. I do not know of a good way to
> hide symbols from users but use them in tests and benchmarks during dev.
> What is the GHC recommended way to achieve this?
>
The usual way of dealing with this is to expose the symbols from a
`.Internal` module.

>> If you can guarantee that the functions you are about are exported then
>> in principle you could probably do what you ask from a library. Just
>> `dlopen` the executable (which would need to be compiled as a position
>> independent exectuable), and dlsym the appropriately mangled name.
>>
>
> Yeah that is a good option for dynamically linked executables. In fact I
> have been using dlopen to implement Haskell plugins but did not think of
> using it for this use case. Thanks for reminding me. In a static
> executable, does it make sense to make the RTS choose an exported symbol as
> entry point using an RTS option?
>
I don't see much precedent for this in other compilers. Moreover, I am
generally not convinced that it should be the compiler's job to provide
such a feature when the same end could be reached with far less magic by
the user.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Dynamically choosing the main function

Harendra Kumar


On 13 November 2017 at 03:47, Ben Gamari <[hidden email]> wrote:
Harendra Kumar <[hidden email]> writes:

> On 12 November 2017 at 23:18, Ben Gamari <[hidden email]> wrote:
>
>> In general it's not easy to get a reference to a an arbitrary function
>> in an object file by it's source name. Unless the symbol is exported the
>> compiler is free to optimize the function or even drop it altogether.
>>
>
> I understand that. I was also wondering if there is way (some sort of
> annotation, pragma or any other workaround) to keep the symbols around
> without actually exporting them.  Exporting either clobbers the module
> interface from user point of view or the modules need a wrapper to avoid
> exporting such symbols in user exposed modules. I would also expect an
> option to remove the effect of any such annotation so that the production
> build is not under optimized because of this. This is in fact one of the
> problems that I have been facing in general. I do not know of a good way to
> hide symbols from users but use them in tests and benchmarks during dev.
> What is the GHC recommended way to achieve this?
>
The usual way of dealing with this is to expose the symbols from a
`.Internal` module.

That's what I meant when I referred to a wrapper module above. However, it requires you to create two modules even if one extra symbol is needed for dev purposes, a lot of boilerplate for a simple thing. Also, the symbols are anyway exposed to the users, we just ask the users to not look at those. I am wondering if using a preprocessor (CPP) conditional around the optional exports is a better way then, you will have to build differently but you won't have to create another module and expose extra symbols?

-harendra

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Dynamically choosing the main function

Brandon Allbery
On Mon, Nov 13, 2017 at 2:46 AM, Harendra Kumar <[hidden email]> wrote:
Also, the symbols are anyway exposed to the users, we just ask the users to not look at those.

Only if you built a dynamic executable, or built for debugging. Default static executables are stripped.

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs