Subject: Re: [boost] Is there interest in a library for object (especially STL object) marshalling?
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2013-10-11 13:00:50
On 11 Oct 2013 at 11:30, Gavin Lambert wrote:
> On 10/10/2013 3:07 PM, Quoth Roger Sanders:
> > The only compiler requirement here is that each assembly uses a compatible
> > C++ ABI, or at least, as much as is required to allow calling a virtual
> > member function on an object that was constructed within another assembly.
>
> That could be a steep requirement, since AFAIK vtable location and
> layout isn't standardised, especially once you start getting into
> multiple and virtual inheritance. And name mangling can get in the way
> of finding the things to be called in the first place.
We had an internal prototype at a former employer of mine which
interoped between the Itanium ABI (GCC) and MSVC as a thought
exercise of just how bad things could get. Single inheritance vtables
are actually identical between the two, but virtual inheritance is
not so we didn't allow that, and multiple inheritance can be worked
around with some difficulty (amongst the issues MSVC doesn't do empty
base class optimisation at times, and persuading GCC to respect that
is non-obvious). An interesting caveat is that x86 MSVC uses stdcall
calling convention for member function calls, but x64 MSVC uses
cdecl. Therefore on x86 the interop thunk has to flip the parameter
stack and do callee cleanup. You also have to avoid all returns
completely, because MSVC and GCC don't do the same thing, so
returning via a pointer write is the only portable way.
Mangling was very straightforward: MSVC's is far more rich up to the
point of needing a non-trivial tokenising parser, and everything the
Itanium ABI specifies is easily representable in MSVC's, albeit
sometimes not obviously (MSVC inverts what an array type means in
mangling at times depending on context). For the other way round
generating a link error where it can't interop is straightforward.
> And memory management is always entertaining given that all sorts of
> weird and different allocators can be in use even before you introduce a
> different compiler into the mix. (Though this is where shared_ptr and
> unique_ptr's pointer+deleter concept can get you out of a jam, although
> it's more common to use opaque handles and explicit destroy functions.)
C libraries are actually somewhat binary interchangeable, at least
more so than STLs. Local C library malloc was always used in our
prototype.
> Having said that: if you can make it work, it would be awesome.
Getting the above system to work reliably needs a graph database to
mark out what needs to be done with what, and what does work versus
the majority which does not work. Such a graph database needs to work
at the binary load layer, and therefore needs to be the filing system
and little extra. For such a graph database to have any chance of
performance, it needs async batch file i/o.
And hence me writing Boost.AFIO, with hopefully that thin graph
database coming next. It'll take at least a year :)
Niall
-- Currently unemployed and looking for work. Work Portfolio: http://careers.stackoverflow.com/nialldouglas/