Embeddable languages, an implementation

I read Havocs post on embeddable languages with great joy. I very much agree that this is a good model for writing a larger application. I.E. you write the core of your application in C/C++ and then do the higher levels in an embedded dynamic scripting language. All the reasons Havoc lists are important, especially for the Gnome platform (a swarm-of-processes model where we want to avoid duplicating the platform in each language used).

There are a lot of dynamic scripting languages around, but only a few of them really qualify as embeddable languages in the form Havoc proposes. I would say the list is basically just: lua, JavaScript and some of the lisp dialects.

I’m far from a JavaScript lover, but it seems pretty obvious to me that from these alternatives it is the best choice. For a variety of reasons:

  • Its a modern dynamic language
    Its dynamically typed, it has precise gargabe collection, lambda functions, generators, array comprehension, etc
  • Its object oriented (sorta)
    We want to wrap the GObject type system which is object oriented, so this is important. The JavaScript prototype-based model is a bit weird, but its simple and it works.
  • It has no “platform” of its own
    If we inject the Gnome platform (Gtk+, Gio, etc) into JavaScript it doesn’t have to fight with any “native” versions of the same functionallity, and we won’t be duplicating anything causing bloat and conversions. (JS does have a Date object, but that is basically all.)
  • Lots of people know it, and if not there are great docs
    Chances of finding someone who knows JS is far higher than finding someone who knows e.g. lua or scheme, and having a large set of potential developers is important for a free software project.
  • Lots of activity around the language and implementations
    The language continually evolves, and there is a standards body trying to shephard this. There is also a multitude of competetive implementations, each trying to be best. This leads to things like the huge performance increases with TraceMoney and Google v8. Such performance work and focus is unlikely to happen in smaller and less used languages.

To experiment with this I started working on GScript (gscript module in gnome svn). It is two things. First of all its an API for easily embedding JavaScript (using SpiderMonkey atm) in a Gtk+ application. Secondly its a binding of GObject and GObject-based libraries to JavaScript.

Here is a quick example:

GScriptEngine *engine;
GScriptValue *res;
engine = g_script_engine_new ();
res = g_script_engine_evaluate_script (engine, "5+5");
g_print ("script result: %s", g_script_value_to_string (res));
g_object_unref (res);

If you want to expose a gobject (such as a GtkLabel) as a global property in JS you can do:

GtkLabel *label = gtk_label_new ("Test");
GScriptValue *val = g_script_value_new_from_gobject (G_OBJECT (label));
g_script_value_set_property (g_script_engine_get_global (engine), "the_label", val);

Then you can access the label from  javascript by the name “the_label”. You can read and set the object properties, connect and emit the signals and call all the methods that are availible via introspection.

You can also easily call from javascript into native code.

static GScriptValue *native_function (int n_args, GScriptValue **args);
GScriptValue *f = g_script_value_new_from_function (native_function, num_args);
g_script_value_set_property (g_script_engine_get_global (engine), "function", f);

The JavaScript wrappers are fully automatic, and lazily binds objects/classes as  they are used in JS. The object properties and signal information are extracted from the GType machinery. Method calls are done using the new GObject-Introspection system.

More work is clearly needed on the details of the JS bindings, but this is already a usable piece of code. I’m very interested in feedback about interest in something like this, and discussions about how the JS bindings should look.

30 thoughts on “Embeddable languages, an implementation”

  1. Hmm, I think we need some sort of standard query language and DOM to hide all of the complexities. As I have said before the beauty of JS is not the language but the environment it works in. If for instance we used GtkBuilder as the DOM the API could be such that you just register the glade/gtkbuilder tree and can query by name (even using libraries such as jQuery which makes the job even easier).

  2. J5 – should be possible to look up GtkWidgets by name; the GtkBuildable interface exposes an _get_name call for all buildable widgets. Also, gobject-introspection gives you bindings to Hippo Canvas, GooCanvas as well as WebKit.

  3. John (J5) Palmieri: What a nice idea.

    window.getObjectsByType(“GtkLabel”);
    window.getObjectByName(“location-entry”); # s/Name/Id/ ?

    notebook = window.getObjectByName(“document-notebook”);
    notebook.set_tab_pos(notebook.POS_LEFT);

  4. John: Introducing a dom and query language is interesting. Using gobject introspection we could easily expose getters and other properties to t the query language, eg so you can easily do “child.children[0].child.label”.
    properties, getters, fields (to be deprecated in 3.0), widget hierarchies are possible.

  5. I really love the idea of having a scriptable “platform” – Python is the current gnomish of doing that. As a longtime Python user (1.5.x 🙂 I loved it.

    (Random scripting language thoughts ahead 🙂

    But I would love a language that I can use for both scripting and to build applications out of without the overhead of an interpreter. Javascript is popular, but is a horrible language in its current form. So while I applaud gscript and Miguel’s new csharp scripting, I still yearn for a decent language that can be run as a script, and compiled efficiently when needed.

    Maybe a form of Vala scripting? Or maybe seed7 – http://seed7.sourceforge.net? Or a compiler for the Blue language? http://www.lechak.info/wiki/index.php?title=Blue

    For the moment I’m learning Seed7 and using both its interpreter and compiler.

  6. I don’t think a query language is quite the right thing. You really want to get the precise thing you want, not something that “matches the query”. However, it should be very easy to use the wrapped version of:

    GObject *gtk_builder_get_object (GtkBuilder *builder, const gchar *name);

    To look up an object from the UI you built with GtkBuilder and then connect up the signals and whatnot.

    Even more interesting would be if you could define some script snippets in the gtkbuilder files themselves. Is the format extensible enough to support that?

  7. Conrad:
    You should look up the work done in TraceMonkey and Google V8. There is some serious performance work happening around JavaScript.

    I don’t mind python, but its not really a great embeddable language, because it comes with too much baggage. It has tons and tons of libraries, most that “conflict” with the stuff in the Gtk/Gnome development platform. Its also more heavyweight than javascript I believe.

  8. alexl: IMO the library that comes with Python is a big advantage. Even for simple plugin-like scripts, it’s useful to have all those utility functions for strings, regexps, path names, date/time… For Javascript, the Gnome platform would have to offer all that stuff on its own.

  9. Andy – ‘giavascript’ doesn’t look quite right to me. How about ‘guavascript’? Begins with G, rhymes with javascript, and ‘guava’ immediately offers a visual theme for the project…

  10. Personally I would like to see a general framework for working with scripts. I.e. We have a single frontend (GScript) and plugins (lua-plugin, js-plugin, schema-plugin, python-plugin etc.). In such way every languaguage is/can be included.

  11. oliver:

    When writing an application in python the python standard libs are great. In fact, they are often the reason you’re using python in the first place. However, if you’re writing your application in C using the gtk+/gnome platform and just using scripting to tie together the highlevel behaviour of your app its not so good.

    For instance, none of the python objects can be passed into the lower levels of your app. The python libs don’t follow the behavior of the platform (gconf settings, desktop specified http proxy, etc, etc). And you get double implementations of all the things as most stuff exists in both set of libaries.

    Maciej:

    Thats seems like a very bad idea to me. Everything using different languages is a total pain in the ass for maintainability. It will also lead to multiple languages used in the same desktop/app leading to much bloat. Its also technically very hard as these languages behave very differently wrt how they look for the embedder and how they manage memory and other resources.

  12. Would be great to, say, have this go into glib proper eventually, and the API be fully scripting-language-independent and you install modules to use individual languages for scripting. Then all apps (gedit, totem, …) can use scripts in any language you have a scripting module installed for…

  13. behdad:

    I can’t really see that. The API isn’t really language agnostic and cannot be, because it depends on the language you’re embedding. For instance, python does not have a global object like JS does, and the native values spec:ed by GScriptValue are very much different between different languages. So are details like how calls to native functions work, how objects/derivation work, etc, etc.

  14. I fail to see how MONO is never mentioned in these discussions?

    1. A large part of the GNOME community is also involved with MONO
    2. MONO can run scripting languages such as IronPython and IronRuby
    3. Once MONO can fully host the dynamic language runtime (DLR) which from what I can understand shouldn’t be too soon, even more languages will be added to that list (including JavaScript).
    4. A lot of developers (outside of the GNOME/Linux community) is working with .NET programming, the threshold for them to begin GNOME development will be lowered.
    5. The philosophy of .NET/MONO is very similar to that of glib, taken one step further. glib is written in C, in part to make it easier for people to create bindings for other languages (as we all can see, this has succeded), CIL is a bytecode standard in .NET and MONO which all languages compile to, it is designed to support a variety of higher level languages. glib is also written with multiplatform ability in mind, so also for CIL.

  15. Markus:

    It is exactly your nr 5 that makes mono unsuitable for this kind of embedded scripting. Its not a compliment to the Gnome stack but largely a replacement. So, you get double implementations of most things that don’t interchange well. Mono is also rather heavy-weight.

    I’m not saying mono is bad in general. Its an excellent way to write an app, but not in the C core plus highlevel behaviour scripting model. If anything I would rather use C# as the base of the app and then script it in a DLR based .net scripting language like IronPython.

  16. Yeah, I see your point, but should one make (power-)users suffer because one love ones own code too much? Maybe the way to go is the mono way, it is IMHO a superior technology.

  17. Markus:

    To each his own. Some people like .net, some not. I have no interest in arguing about that.

    Anyway, i don’t think the exact language picked is really that important for the users. One can write good or bad programs in all languages, and arguing that you must use a specific language to make the user happy is just lame.

  18. alexl

    you have my full support on this. Zoo of languages is very uneasy environment to live in. And JS must-know today, because of web. Also it is impossable to incorporate useful scripts done by users(can’t imagine firefox exentions written in 15+ scripting langs) if there are no standard lang.

    >vaScript prototype-based model is a bit weird,

    if i’m not mistaken, next version of standard has separate classes and objects, not prototype based…

    sorry for bad english

  19. rainwoodman:
    You can already create object with e.g.:
    new GType.GtkLabel(“label”, “some text”)

    There is no code atm to import a full namespace, so the functions of a namespace are not visible. However, the actual call is very similar to the one for method calls (they are both just c functions really).

    Some work is happening on getting this running atm. More info later.

  20. Hi,

    This is an area I’ve been thinking about quite a lot recently and I’ve been considering development of an all-GLib implementation of ECMA-262. A GLib implementation would avoid duplication of code and hopefully be easier for Gnome developers to maintain. Perhaps a collaboration is in order? What do you think?

    Matt.

  21. Matt:

    Eh, i’m not sure what you mean, but a (re-)implementation of javascript just to use glib is both duplication of code (you’ll be running two js implementations if you also run firefox) and harder to maintain (spidermonkey is already maintained, the new implementation is not).

    Not to mention that we wont’ get any of the advantages of all the important performance work done on the mainline js engines.

  22. Hi Alex,

    By duplication of code, I mean Spidermonkey implementing its own data structures, data manipulation functions, unicode handling etc when Glib already provides such facilities. This might not be a problem on the server or desktop, but on memory constrained embedded devices (where Gnome is really starting to gain some momentum), this is an issue.

    If one is to use an existing Javascript engine, perhaps JavaScriptCore would be a better choice seeing as WebKit seems to be winning the embedded browser wars, otherwise systems are going to end up with all of GLib’s facilities, some duplicated by SpiderMonkey and some duplicated WebKit simultaneously. That is a really messy overall solution. Of course, bringing in a 3rd javascript engine and running that alongside webkit is not ideal either, but unless someone comes up with a grade A 100% Gtk+ and Glib-based web browser (which lets face it, isn’t likely to happen anytime soon), there is not much choice about that.

    I don’t yet have a convincing argument for implementing yet another Javascript engine, as I said in my original post, I’m just considering things at the moment. Such a task is hardly trivial, so there are lots of reasons not to do it. I feel uneasy about using SpiderMonkey and JavaScriptCore because they’ve both been hacked around and are pretty messy inside (e.g. one minute the engine is designed to be standalone, the next minute is integrated into a browser, then it is hacked out again). My comment about a Glib-based Javascript engine being easier for Gnome developers to maintain meant that the code would be more in fitting with the GTK+ way of doing things from the inside out. Of course, another project is going to involve more work than using what is already available. There are pros and cons either way.

    Matt.

Leave a Reply to Matt Bucknall Cancel reply

Your email address will not be published. Required fields are marked *