Smoke for KDE4

I've seen mumblings from the KDE folk about increasing their binding support in KDE4. Specifically, aseigo has mentioned making KJS and Python bindings standard.

My recommendation is to base the bindings on Qt4's metaobject system. The idea would be to adopt the Qt4 meta-calling convention for the next version of Smoke.

How would the Qt4 Smoke bindings work?

The goal would be for every function in every class to be available through QMetaObject::invokeMethod() Like Smoke does now, it would allow AUTOLOAD/missing_method based language bindings.
However, the metaobject system only gets us 50% of the way there. There are some obstacles to overcome:

1. There must be a way to call static functions -- like constructors.
2. Not every Qt class inherits from QObject. We need to interface those as well
3. Virtual functions!
   

I haven't worked out how to solve these issues entirely, yet. Here's where my thoughts have lead me so far.

Static functions

The answer here is probably to generate a factory class which is aware of the static functions.
Lets try writing Hello World..

// Lets say the smoke4 bindings export factory() functions for each class
extern "C" void* qapplication_factory();
extern "C" void* qpushbutton_factory();
extern "C" void* qstring_factory();

// public call interface of smoke
extern "C" void metacall_wrapper(void* object, int index, void** args) {
   // call qt_metacall()
   QObject *o = static_cast<QObject*>(object);
   o->qt_metacall(QMetaObject::InvokeMetaMember, index, args);
}

extern "C" int indexOfMember_wrapper(void* object, const char* member) {
   // indexOfMember() looks up the method name in the metaobject
   QObject *o = static_cast<QObject*>(object);
   return o->metaObject()->indexOfMember(member);
}

extern "C" void* qt_metacast_wrapper(void* object, const char* className) {
   QObject *o = static_cast<QObject*>(object);
   return o->qt_metacast(className);
}

extern "C" void smoke_call(void* object, const char* member, void** args) {
   // helper function to do everything at once
   int _i = indexOfMember_wrapper(object, member);
   qt_metacall_wrapper(object, _i, args);
}

int main(int argc, char **argv) {
  // lets assume these *factory() functions are coming from smoke
  void *appfactory = qapplication_factory();
  void *pbfactory = qpushbutton_factory();
  void *strfactory = qstring_factory();

  void *app;
  void *_a0[] = { &app, &argc, &argv };
  smoke_call(appfactory, "new(int,char**)", _a0);

At this point, the app variable contains the return-value from new(). In order for us to be able to call qt_metacall() on that variable, we need to mandate that the return-value from new() can be safely cast with static_cast<QObject*>. If we want the derived pointer, we can use qt_metacast().

   // allocate a new QString
   void *string;
   char *_d1 = "Hello World!";
   void *_a1[] = { &string, &_d1 };
   smoke_call(strfactory, "new(const char*)", _a1);

In order to automate the QString object, the constructor returns a proxy QObject instead of a bare QString, so we can invoke methods on it with qt_metacall(). The question here is how do we get at the bare QString object if it's being wrapped in a QObject? The obvious solution would be to hijack qt_metacast() in the proxy object to break the encapsulation. This would be the only legal method to see a naked QString pointer. It would also allow us to cast up/down the inheritance chain for non-QObject classes by implementing it in qt_metacast().

   void *qstring = qt_metacast_wrapper(string, "QString");
   void *button;
   void *_a2[] = { &button, qstring };
   // and construct the new QPushButton
   smoke_call(pbfactory, "new(const QString&)", _a2);

Finally, we finish everything off;

   int w = 100, h = 30;
   void *_a3[] = { 0, &w, &h };
   smoke_call(hello, "resize(int,int)", _a3);
   smoke_call(hello, "show()", 0);
   int ret;
   void *_a4[] = { &ret };
   smoke_call(app, "exec()", _a4);
   return ret;
}

That covers Hello World. As a perk, every function in every class is now a valid slot. From a language binding, you can even connect() to a function in a non-QObject class.

my $str = new QString;
# there's no QString::set(const QString&) function?
$str->connect($listview, currentTextChanged => 'operator=');

Virtual Functions

If we've figured out the calling convention, how do we implement virtual functions? As with Smoke v3, it'll be necessary to override every virtual function in every class. I suspect if you fiddle with gcc4's symbol visibility and compile the virtual-function subclass into the same library as the original function, the linking overhead for doing this should be ~0. That overhead was a nasty performance hit for the smoke library, especially since it was built as a monolithic .so for all of Qt.

Each virtual function should emit itself as a signal. By default, each virtual function would be connected to its own slot. When a binding language wants to override a virtual function, it'll be free to connect() the signal to any valid slot. Even a slot in a different object! The binding should take care to disconnect() the default slot connection if the programmer reimplements the virtual function.

If the virtual function hasn't been connect()ed up, we're free to optimize away the signal emission if we want and call the original function directly.

Just to put some code to this idea...

#include <QWidget>
#include <QSize>
#include <QCloseEvent>
#include <QMetaObject>

namespace SMOKE {

class QWidget : public ::QWidget {
public:
    virtual ::QSize sizeHint() const;
    virtual void closeEvent(::QCloseEvent*);
};

// SMOKE::QWidget implementation

::QSize QWidget::sizeHint() const {
   ::QSize _ret;
   void _a[] = { &_ret };
   qt_metacall(::QMetaObject::InvokeMetaMember, metaObject().memberOffset() + 12, _a);
   return _ret;
}

void QWidget::closeEvent(::QCloseEvent *e) {
   void _a[] = { 0, &e };
   qt_metacall(::QMetaObject::InvokeMetaMember, metaObject().memberOffser() + 13, _a);
}

}

Method discovery

Now that we can call all these methods, we need to know they're there. Somehow we need to provide a listing of available methods to the language bindings. QMetaObject doesn't provide what Smoke needs in this regard.

The current Smoke method discovery could be adapted to use the Qt4 moc data-structure, which is just byte offsets into a string which contains every method and type used separated by \0. The Smoke::Index values would be converted into metaObject() offsets. A few other tweaks would be necessary, but moc seems to have similar requirements to Smoke as far as method descriptions go. We'll probably want to keep Smoke3's method$$ munging syntax since it's so handy for overload resolution.

Signals and Slots

If you notice from my connect() example above, I think we can do away with specifying signal and slot arguments. For signals declared in the binding language, we're going to cheat! Every signal will be declared as having zero arguments. When you call connect(), with a language signal being emitted, the call doesn't get passed on to QObject::connect. Instead, the binding needs to keep track of its own connections. Picture this:

sub foo : signal;

# this actually declares a function which looks like:

sub foo {
    my($self, @args);
    our %slots;
    my @candidates;
    for my $slot (@{ $slots{'foo'} }) {
        push @candidates, $slot
            if compatibleArguments($slot, \@args);
    }

    # of the slots with the given name which can accept these arguments
    # pick the one with the longest string length
    @candidates = sort {
        length($a->signature) <=> length($b->signature) || $a cmp $b
    } @candidates;

    magically_call($candidates[0]);
}

For slots, we can dynamically create slots which match all the matching signals:

$foo->connect($bar, signalName => slotName);

# signalName(const QString&)
# signalName(int)
sub slotName : slot {
    # this function gets called for either/both of those signals,
    # unless it specifies a signature in the slot declaration
}

That exhausts all I've thought through so far.