Can I get the internal name of a package at runtime?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Can I get the internal name of a package at runtime?

MarLinn
Hi.

I'm experimenting with plug-ins right now. I did manage to dynamically
load functions at runtime. The caveat: Something (cabal? ghc?) mangles
the package names. For example, to load a function called "theFunction"
from a module called "Callee" in a package "Plugin", I had to address it
via the name
"Pluginzm0zi0zi0zi0zm2QaFQQzzYhnKJSPRXA7VtPe_Callee_theFunction_closure".
O…K. Most parts of that are clear, and thanks for making my package
cooler by appending a "z", but who is this Ozi guy and why is he rapping
about modems? Without knowing Ozi, the only way I found to get at this
magic string is to manually look at the actual ELF-header of the
compiled module. While that might be a robust way, it seems neither
portable nor elegant.

The "plugins" library failed too, probably for the same reason. (Or it's
under-documented. Probably both.) The "dynamic-loader" library does
something via c, therefore no.

Which brings me to the question: Is there any way for a module to get at
its own internal package name? Or even at the internal name of an
external package? If not, can I somehow recreate the magic mangling at
runtime? At first I thought the functions in the "Module", "Name" etc
modules of GHC might help – but it seems I either need an existing Name
(that I have no idea how to get) or I have to create one (with no idea
what magic mangler to call).

I'm asking this question here rather than on café as I feel that if
there is a solution, it's probably buried in the details of GHC.

Thanks for any suggestions,
MarLinn

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Can I get the internal name of a package at runtime?

Edward Z. Yang
Hi MarLinn,

The mangling name is "z-encoded".  It is documented here:
https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/SymbolNames

Edward

Excerpts from MarLinn's message of 2017-10-14 17:35:28 +0200:

> Hi.
>
> I'm experimenting with plug-ins right now. I did manage to dynamically
> load functions at runtime. The caveat: Something (cabal? ghc?) mangles
> the package names. For example, to load a function called "theFunction"
> from a module called "Callee" in a package "Plugin", I had to address it
> via the name
> "Pluginzm0zi0zi0zi0zm2QaFQQzzYhnKJSPRXA7VtPe_Callee_theFunction_closure".
> O…K. Most parts of that are clear, and thanks for making my package
> cooler by appending a "z", but who is this Ozi guy and why is he rapping
> about modems? Without knowing Ozi, the only way I found to get at this
> magic string is to manually look at the actual ELF-header of the
> compiled module. While that might be a robust way, it seems neither
> portable nor elegant.
>
> The "plugins" library failed too, probably for the same reason. (Or it's
> under-documented. Probably both.) The "dynamic-loader" library does
> something via c, therefore no.
>
> Which brings me to the question: Is there any way for a module to get at
> its own internal package name? Or even at the internal name of an
> external package? If not, can I somehow recreate the magic mangling at
> runtime? At first I thought the functions in the "Module", "Name" etc
> modules of GHC might help – but it seems I either need an existing Name
> (that I have no idea how to get) or I have to create one (with no idea
> what magic mangler to call).
>
> I'm asking this question here rather than on café as I feel that if
> there is a solution, it's probably buried in the details of GHC.
>
> Thanks for any suggestions,
> MarLinn
>
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Can I get the internal name of a package at runtime?

MarLinn
Hi Edward,

thank you.
That knowledge revealed that the "Ozi" part was actually the version number.

So the "actual" package name seems to be
"Plugin-0.0.0.0-2QaFQQzYhnKJSPRXA7VtPe".
That leaves the random(?) characters behind the version number to be
explained.
But at least now I can exploit the fact that a
"libHSPlugin-0.0.0.0-2QaFQQzYhnKJSPRXA7VtPe.a" file is generated. So if
I don't find the complete answer I still have a more portable way for
discovery than inspecting headers.

That's quite useful.

Cheers,
MarLinn


On 2017-10-14 18:01, Edward Z. Yang wrote:

> Hi MarLinn,
>
> The mangling name is "z-encoded".  It is documented here:
> https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/SymbolNames
>
> Edward
>
> Excerpts from MarLinn's message of 2017-10-14 17:35:28 +0200:
>> Hi.
>>
>> I'm experimenting with plug-ins right now. I did manage to dynamically
>> load functions at runtime. The caveat: Something (cabal? ghc?) mangles
>> the package names. For example, to load a function called "theFunction"
>> from a module called "Callee" in a package "Plugin", I had to address it
>> via the name
>> "Pluginzm0zi0zi0zi0zm2QaFQQzzYhnKJSPRXA7VtPe_Callee_theFunction_closure".
>> O…K. Most parts of that are clear, and thanks for making my package
>> cooler by appending a "z", but who is this Ozi guy and why is he rapping
>> about modems? Without knowing Ozi, the only way I found to get at this
>> magic string is to manually look at the actual ELF-header of the
>> compiled module. While that might be a robust way, it seems neither
>> portable nor elegant.
>>
>> The "plugins" library failed too, probably for the same reason. (Or it's
>> under-documented. Probably both.) The "dynamic-loader" library does
>> something via c, therefore no.
>>
>> Which brings me to the question: Is there any way for a module to get at
>> its own internal package name? Or even at the internal name of an
>> external package? If not, can I somehow recreate the magic mangling at
>> runtime? At first I thought the functions in the "Module", "Name" etc
>> modules of GHC might help – but it seems I either need an existing Name
>> (that I have no idea how to get) or I have to create one (with no idea
>> what magic mangler to call).
>>
>> I'm asking this question here rather than on café as I feel that if
>> there is a solution, it's probably buried in the details of GHC.
>>
>> Thanks for any suggestions,
>> MarLinn
>>

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Can I get the internal name of a package at runtime?

Brandon Allbery
On Sat, Oct 14, 2017 at 12:48 PM, MarLinn <[hidden email]> wrote:
So the "actual" package name seems to be "Plugin-0.0.0.0-2QaFQQzYhnKJSPRXA7VtPe".
That leaves the random(?) characters behind the version number to be explained.

ABI hash of that specific package build, which is needed because compiling with different optimization levels etc. will change what part of the internals gets exposed in the .hi file for inlining into other modules; mismatches there lead to *really* weird behavior. (If you're lucky, it'll "just" be a type mismatch in code you didn't write, because it came from the .hi file. If unlucky, it compiles but dumps core at runtime.)

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Can I get the internal name of a package at runtime?

MarLinn
That sounds reasonable, but also like there *can not be* a way to obtain
that hash at runtime. And therefore, no way to discover the true package
name.

Which in turn makes discovery and loading of plug-ins a bit harder.
Well, I guess it's for a good reason so I'll have to work around it.
Good to know.

Thanks for helping out!

Cheers,
MarLinn


On 2017-10-14 20:11, Brandon Allbery wrote:

> On Sat, Oct 14, 2017 at 12:48 PM, MarLinn <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     So the "actual" package name seems to be
>     "Plugin-0.0.0.0-2QaFQQzYhnKJSPRXA7VtPe".
>     That leaves the random(?) characters behind the version number to
>     be explained.
>
>
> ABI hash of that specific package build, which is needed because
> compiling with different optimization levels etc. will change what
> part of the internals gets exposed in the .hi file for inlining into
> other modules; mismatches there lead to *really* weird behavior. (If
> you're lucky, it'll "just" be a type mismatch in code you didn't
> write, because it came from the .hi file. If unlucky, it compiles but
> dumps core at runtime.)
>
> --
> brandon s allbery kf8nh sine nomine associates
> [hidden email] <mailto:[hidden email]>
> [hidden email] <mailto:[hidden email]>
> unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Can I get the internal name of a package at runtime?

Brandon Allbery
On Sat, Oct 14, 2017 at 3:59 PM, MarLinn <[hidden email]> wrote:
That sounds reasonable, but also like there *can not be* a way to obtain that hash at runtime. And therefore, no way to discover the true package name.

I can think of a hacky way: make sure the symbol table is still present in the executable and that you can find said executable, z-encode the part of the package+version you know and a symbol you know, and look for symbols that start with one and end with the other; the hash will be in between.

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Can I get the internal name of a package at runtime?

Brandon Allbery
Oh, wait, that won;t work because you're trying to find the name of something to load at runtime. But you can still z-encode the known parts of the name and look for files starting and ending with those components; the ABI hash won't matter because a dynamically loaded plugin can't use the inlinings from the .hi file anyway. If you get multiple matches, you could warn about it but because you can't use the .hi inlinings in that context it won't matter which one you load.

On Sat, Oct 14, 2017 at 4:19 PM, Brandon Allbery <[hidden email]> wrote:
On Sat, Oct 14, 2017 at 3:59 PM, MarLinn <[hidden email]> wrote:
That sounds reasonable, but also like there *can not be* a way to obtain that hash at runtime. And therefore, no way to discover the true package name.

I can think of a hacky way: make sure the symbol table is still present in the executable and that you can find said executable, z-encode the part of the package+version you know and a symbol you know, and look for symbols that start with one and end with the other; the hash will be in between.

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net



--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Can I get the internal name of a package at runtime?

Daniel Gröber
In reply to this post by MarLinn
Hi,

I think you might be interrested in my rts-loader package,
particularly the [mkSymbol function](https://github.com/DanielG/rts-loader/blob/master/System/Loader/RTS.hs#L275).

It should demonstrate how you can construct the symbol names for
dynamic loading and how to coax the information needed out of Cabal,
see the README and the rest of the source.

--Daniel

On Sat, Oct 14, 2017 at 09:59:03PM +0200, MarLinn wrote:

> That sounds reasonable, but also like there *can not be* a way to obtain
> that hash at runtime. And therefore, no way to discover the true package
> name.
>
> Which in turn makes discovery and loading of plug-ins a bit harder. Well, I
> guess it's for a good reason so I'll have to work around it. Good to know.
>
> Thanks for helping out!
>
> Cheers,
> MarLinn
>
>
> On 2017-10-14 20:11, Brandon Allbery wrote:
> > On Sat, Oct 14, 2017 at 12:48 PM, MarLinn <[hidden email]
> > <mailto:[hidden email]>> wrote:
> >
> >     So the "actual" package name seems to be
> >     "Plugin-0.0.0.0-2QaFQQzYhnKJSPRXA7VtPe".
> >     That leaves the random(?) characters behind the version number to
> >     be explained.
> >
> >
> > ABI hash of that specific package build, which is needed because
> > compiling with different optimization levels etc. will change what part
> > of the internals gets exposed in the .hi file for inlining into other
> > modules; mismatches there lead to *really* weird behavior. (If you're
> > lucky, it'll "just" be a type mismatch in code you didn't write, because
> > it came from the .hi file. If unlucky, it compiles but dumps core at
> > runtime.)
> >
> > --
> > brandon s allbery kf8nh sine nomine associates
> > [hidden email] <mailto:[hidden email]> [hidden email]
> > <mailto:[hidden email]>
> > unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net
>
> _______________________________________________
> ghc-devs mailing list
> [hidden email]
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Can I get the internal name of a package at runtime?

MarLinn
Hi Daniel,

that looks very interesting. I think it'll take some time to understand
what's going on, but I already got some good parts. And even if I won't
end up using it, this seems like a good way to learn some stuff. So
thanks a lot!

For now I don't need the full power, so I think I'll take what I learned
here and stick to a simple, hacky solution along the lines of Brandon's
suggestions. Like enumerating object files in some specified directory,
then mapping

     readCreateProcessWithExitCode (shell $ "readelf --symbols --wide " ++ path ++ " | grep closure | tr --squeeze-repeat ' ' | cut --delimiter=' ' --fields=9") ""

over them and z-encoding back and forth to discover what the heck I'm
actually loading.
Elegant? No. Secure? No. Portable? …sufficiently. Works? …hopefully.

I got the feeling there is no good non-hacky way. Somewhere there's
always some extra c code or something. I'm just glad my current goal is
to just load object files, not compile user-supplied "scripts" into a
running project or something.

So thanks again to you all!

Cheers,
MarLinn

On 2017-10-15 02:30, Daniel Gröber wrote:

> Hi,
>
> I think you might be interrested in my rts-loader package,
> particularly the [mkSymbol function](https://github.com/DanielG/rts-loader/blob/master/System/Loader/RTS.hs#L275).
>
> It should demonstrate how you can construct the symbol names for
> dynamic loading and how to coax the information needed out of Cabal,
> see the README and the rest of the source.
>
> --Daniel
>
> On Sat, Oct 14, 2017 at 09:59:03PM +0200, MarLinn wrote:
>> That sounds reasonable, but also like there *can not be* a way to obtain
>> that hash at runtime. And therefore, no way to discover the true package
>> name.
>>
>> Which in turn makes discovery and loading of plug-ins a bit harder. Well, I
>> guess it's for a good reason so I'll have to work around it. Good to know.
>>
>> Thanks for helping out!
>>
>> Cheers,
>> MarLinn
>>
>>
>> On 2017-10-14 20:11, Brandon Allbery wrote:
>>> On Sat, Oct 14, 2017 at 12:48 PM, MarLinn <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>      So the "actual" package name seems to be
>>>      "Plugin-0.0.0.0-2QaFQQzYhnKJSPRXA7VtPe".
>>>      That leaves the random(?) characters behind the version number to
>>>      be explained.
>>>
>>>
>>> ABI hash of that specific package build, which is needed because
>>> compiling with different optimization levels etc. will change what part
>>> of the internals gets exposed in the .hi file for inlining into other
>>> modules; mismatches there lead to *really* weird behavior. (If you're
>>> lucky, it'll "just" be a type mismatch in code you didn't write, because
>>> it came from the .hi file. If unlucky, it compiles but dumps core at
>>> runtime.)
>>>
>>> --
>>> brandon s allbery kf8nh sine nomine associates
>>> [hidden email] <mailto:[hidden email]> [hidden email]
>>> <mailto:[hidden email]>
>>> unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net
>> _______________________________________________
>> ghc-devs mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs