Re: A vision for Parrot

Sorry for the repost at perl, I didn't release the post was crossposted (I
entered via Ruby).


"Neil Madden" <[e-mail]> wrote in message
news:LRaA9.1325$[e-mail]...

> Hmm.. the more I think about this, the more problems it seems to present.
I'd
> love to be able to write an extension, and have it instantly work with x
> different langauges. Also, I'd love to be able to use Python and Perl
> extensions from Tcl, without loading seperate interpreters and all that.
Am I
> dreaming of an impossible Utopia?

I wrote some DLL helper logic that makes it easy to query a vtable by name
in another DLL.
The vtables can be created in whatever way, but the vtable is not assocated
with allocated memory, as is the case with COM. This makes it quite flexible
without really being limited because one could decide that the first
function should return a 'this' pointer and the second function should be
the destructor, or whatever.

In any DLL that can link statically with a piece of C, you can use this
framework to pass functions around. (Even if you can only link dynamically,
you could have a helper dll to expose vtables). I use some #define macros to
create the vtables statically (see snippet below). It's then very easy to
take arbitrary C functions and wrap them up in one or more vtables. However,
vtables need not be statically allocated (and this is significantly
different from public dll functions).
Because the vtables are looked up by name initially, you could handle
versioning like in COM: "mycomponent.vtablename.1", where
"mycomponent.vtablename" would refer to the most recent version - but it is
preferable to always specificy an exact version.
Contrary to COM this is completely cross platform as there is no OS magic
involved.
When a vtable is created, it is helpful to write a C prototype struct that
matches, because this makes things easier clientside - but it is not
required (see snippet below).
On the clientside I wrote some small wrappers to ease loading dll's
dynamically and in C++ to wrap the optional prototype. I only did this for
Windows, but the principles are essentially the same on Linux, just like
Ruby loads .so files.

The framework is generic - here is what I did to access functionality in a
OCaml parser application I wrapped up as selfcontained dll - without using
public dll functions except for init, query vtable and uninit functions: I
picked all the most relevant OCaml API logic for allocating memory on the
OCaml runtime stack and created a set of vtables approximately one for each
..h file in the API. The Ocaml OCaml functions I wrote (parse_file) I had to
dynamically call a function to locate the OCaml function address. In the dll
init logic I performed this operation and added the result to a vtable
already prepared for the purpse. The client of the dll loads the dll, calls
init and then queries relevant vtables, but in principle has no clue that it
really is OCaml the executes the logic (in principle because in praxis I
wanted to know in order to efficiently allocate memory).

The same thing could be done in Ruby, wrapping rb_... functions into vtables
and having separate vtables for calling Ruby code - this vtables would be
created dynamically or at least filled dynamically.

Any language that supports C integration and dynamic link libraries can use
this framework.

Incidentally Parrot works a lot with vtables, so there could be some overlap
here (I didn't know that at the time I wrote the framework though).

I have not put the code online, but if anyone is interested, let my know.

Below is a readme snippet, a bit technical and not the only way to use the
framework (it would be possible to map Ruby functions into dynamically
created vtables for instance).

Mikkel

<snip>
    Interfaces are organized in vtable maps which are statically allocated
arrays of VtblMapEntries.

    VTBL_MAP_BEGIN(<vtbl_map_name>)
    VTBL_MAP_ENTRY(name, vtbl)
    ...  more entries here ...
    VTBL_MAP_END

    The name <vtbl_map_name> is later used be the linker, such that the map
can be hooked into a master
    map of all vtable interfaces compiled together.
    The master map is located in a central compilation unit such that new
vtable maps can easily be
    added to the master map without modifying the source of any of the
existing map providers.

    Before entering the map into the mastermap, it must be declared - unless
the map is earlier in the same
    compilation unit as the master map:

    VTBL_DECLARE_MAP(<vtbl_map_name>)
    ... more maps declared here ...

    Following the map declareations, there is a master map which is scanned
by the default lookup function:

    VTBL_MASTER_MAP_BEGIN(<vtbl_master_map_name>)
    VTBL_MASTER_MAP_ENTRY(<vtbl_map_name>)
    ... more master map entries here ...
    VTBL_MASTER_MAP_END

    Typically, a dynamically loaded library (dll) will have a purpose
specific mastermap using a selection
    of available vtbl interface maps.

    Example:
    we have the following functions in a .c file. Moreover, we have the
malloc and free functions
    from the std library. We want all four functions wrapped in two
interfaces: FooBar and Mem.

    First we create a header file for the interfaces:
    <file "examples.h">
    struct
    {
        int (*get)(int x);
        void (*set)(int x, int val);
        void *(*create)(int x);
    } ExamplesVectorVtbl;
    struct
    {
        void *(*allocate)(size_t size);
        void (*deallocate)(void *p);
    } ExamplesMemoryVtbl;
    </file>
    <file "examples.c">
    #include <memory.h>
    #include "vtbl.h"
    #include "examples.h"
    void set(void *p, int x, int val) { return ((int*)p)[x] = val; };
    int get(void *p, int x) { return ((int*)p)[x]; }
    void *create(int x) { return calloc(x * sizeof(int)); };
    ExamplesVectorVtbl { get, set, create } vector_vtbl;
    /* shows that existing library functions can be packaged as well */
    ExamplesMemoryVtbl { calloc, free } mem_vtbl;

    VTBL_MAP_BEGIN(examples_vtbl_map)
    VTBL_MAP_ENTRY("Examples.Vector", vector_vtbl)
    VTBL_MAP_ENTRY("Examples.Memory", mem_vtbl)
    VTBL_MAP_END
    </file>
    <file "master.c">

    /* to get vtbl.h and the lookup function in vtbl.c */
    #include "vtbl.c"

    VTBL_DECLARE_MAP(examples_vtbl_map)

    VTBL_MASTER_MAP_BEGIN(vtbl_master_map)
    VTBL_MASTER_MAP_ENTRY(examples_vtbl_map)
    VTBL_MASTER_MAP_END

    void *GetNamedInterface(char *name)
    {
        return vtbl_master_map_lookup(vtbl_master_map, name);
    }

    </file>
    <file "client.c">
    #include "examples.h"
    void *GetNamedInterface(char *name);
    void test()
    {
        /* since interfaces are static, pMem and pVec need not be
deallocated */
        Examples_Memory *pMem =
(Examples_Memory*)GetNamedInterface("Examples.Memory");
        Examples_Vector *pVec =
(Examples_Vector*)GetNamedInterface("Examples.Vector");
        void *v1 = pMem->allocate(sizeof(int[4]));
        void *v2 = pVec->create(4);
        pVec->set(v1, 2, 42);
        pVec->set(v2, 0, pVec->get(v1, 2));
        pMem->deallocate(v1);
        pMem->deallocate(v2);
    }
    </file>

    Typically the client would have loaded a dynamically linked library and
found the address of the published
    GetNamedInterface function. Once that function is avaible, it is easy to
access all the remaining functions
    via the named interfaces.

</snip>