Archive for the ‘General’ Category

A small preview of subwindowless Gtk+

Wednesday, November 5th, 2008

Before:

After:

This code is still early in progress work, but it shows a lot of promise.

Note that the effect in the videos above is somewhat exagurated by doing two ssh forwards over the network so that i could screengrab the flicker, its not as obvious in plain local use.

Gtk can haz aliens too!

Towards a Ridley-based platform

Tuesday, October 7th, 2008

Today I finally made the last changes to remove libgnome and its dependencies from nautilus. Its not in svn yet because some patches to gnome-desktop has to go in first.

Before:

ldd `which nautilus` | wc -l
91

After:

ldd `which nautilus` | wc -l
60

So, we are now linking to  30 libraries less! Libraries which we hardly used but still take time to load, initialize and resolve symbols from.

As a comparison, gtk-demo links to 37 libraries on my system. The additional libraries come from session management, thread use, dbus, gconf, libgnomedesktop and some other nautilus-specific features.

So, Project Ridley is alive and kicking (even if the wiki page is a bit outdated). I’m just waiting for dconf/GtkSettings to be finished and then we’ll really have a competitive next generation platform without all the old deprecated libgnome era stuff.

UPDATE: I’ve now commited this to trunk. You need new gnome-desktop, eel and nautilus

I’m back! (sorta)

Monday, September 22nd, 2008

After a long parental leave I’m now partially back to work. I’ll be working half-time until the end of the year.

I’ll try to get a sense of the status of the modules I work on, and I’m glad to help answer the list of questions people have built up over the last 6 months. I’m on email, and will hang out on irc around 9:00 to 15:00 CET.

Async I/O made “easy” using JavaScript

Tuesday, September 16th, 2008

Writing code that does async I/O is a total pain in the ass. And I’ve written a whole lot of async I/O code, so I should know.

However, if you’re working in a language with full support for continuations it is a lot easier. JavaScript doesn’t support continuations fully, but it has generators, which is like a lite-version of continuations. That is part of the reason why I’m interested in JavaScript scripting.

So, I’ve been experimenting with  how to bind GIO-style async operations in JavaScript so that its easier to write async I/O code. I’ve got something going now that looks pretty ok.

Here is an example that calls an async function “$open” to read three files. It also demonstrates how to do calls from one async generator to another.

do_stuff = new AsyncRunner (function  (filenames) {
    for (let i in filenames) {
        var data = yield $open (filenames[i], 0);
        print ("data for " + filenames[i] + "  = " + data);
    }
    var ret = yield do_stuff2.call("other.txt");
    print ("AsyncRunner call returned: " + ret);
});
do_stuff2 = new AsyncRunner (function (filename) {
    var data = yield $open (filename, 0);
    print ("data for " + filename + " = " + data);
    yield "the end (of do_stuff2)"
});
do_stuff.start(null, ["test.txt", "test2.txt"]);

Its IMHO pretty easy to understand. At every async op we yield which suspends execution and restarts it at that point when we have the operation results. The “$” in “$open” is just to make it different from the sync variant and still short.

This code is far easier than it would look in C, where we would have to split out a new function at each async function call. The loop in do_stuff makes this quite tricky to do.

The code that implements this is here. Its somewhat complicated, using things like currying, closures and generators, so read it carefully.

I’d like some feedback on this. Are there any tricks I’ve missed that could make this nicer? I really wish javascript had a macro facility so that the yield keyword could be put inside the async calls and we could avoid the boilderplate around the AsyncRunner creation. I would also like to make the AsyncRunner objects callable like “do_stuff2()”, but that doesn’t seem possible with standard javascript (although it is the AsyncRunner class is implemented natively via the spidermonkey APIs…)

Embeddable languages, an implementation

Tuesday, September 9th, 2008

I read Havocs post on embeddable languages with great joy. I very much agree that this is a good model for writing a larger application. I.E. you write the core of your application in C/C++ and then do the higher levels in an embedded dynamic scripting language. All the reasons Havoc lists are important, especially for the Gnome platform (a swarm-of-processes model where we want to avoid duplicating the platform in each language used).

There are a lot of dynamic scripting languages around, but only a few of them really qualify as embeddable languages in the form Havoc proposes. I would say the list is basically just: lua, JavaScript and some of the lisp dialects.

I’m far from a JavaScript lover, but it seems pretty obvious to me that from these alternatives it is the best choice. For a variety of reasons:

  • Its a modern dynamic language
    Its dynamically typed, it has precise gargabe collection, lambda functions, generators, array comprehension, etc
  • Its object oriented (sorta)
    We want to wrap the GObject type system which is object oriented, so this is important. The JavaScript prototype-based model is a bit weird, but its simple and it works.
  • It has no “platform” of its own
    If we inject the Gnome platform (Gtk+, Gio, etc) into JavaScript it doesn’t have to fight with any “native” versions of the same functionallity, and we won’t be duplicating anything causing bloat and conversions. (JS does have a Date object, but that is basically all.)
  • Lots of people know it, and if not there are great docs
    Chances of finding someone who knows JS is far higher than finding someone who knows e.g. lua or scheme, and having a large set of potential developers is important for a free software project.
  • Lots of activity around the language and implementations
    The language continually evolves, and there is a standards body trying to shephard this. There is also a multitude of competetive implementations, each trying to be best. This leads to things like the huge performance increases with TraceMoney and Google v8. Such performance work and focus is unlikely to happen in smaller and less used languages.

To experiment with this I started working on GScript (gscript module in gnome svn). It is two things. First of all its an API for easily embedding JavaScript (using SpiderMonkey atm) in a Gtk+ application. Secondly its a binding of GObject and GObject-based libraries to JavaScript.

Here is a quick example:

GScriptEngine *engine;
GScriptValue *res;
engine = g_script_engine_new ();
res = g_script_engine_evaluate_script (engine, "5+5");
g_print ("script result: %s", g_script_value_to_string (res));
g_object_unref (res);

If you want to expose a gobject (such as a GtkLabel) as a global property in JS you can do:

GtkLabel *label = gtk_label_new ("Test");
GScriptValue *val = g_script_value_new_from_gobject (G_OBJECT (label));
g_script_value_set_property (g_script_engine_get_global (engine), "the_label", val);

Then you can access the label from  javascript by the name “the_label”. You can read and set the object properties, connect and emit the signals and call all the methods that are availible via introspection.

You can also easily call from javascript into native code.

static GScriptValue *native_function (int n_args, GScriptValue **args);
GScriptValue *f = g_script_value_new_from_function (native_function, num_args);
g_script_value_set_property (g_script_engine_get_global (engine), "function", f);

The JavaScript wrappers are fully automatic, and lazily binds objects/classes as  they are used in JS. The object properties and signal information are extracted from the GType machinery. Method calls are done using the new GObject-Introspection system.

More work is clearly needed on the details of the JS bindings, but this is already a usable piece of code. I’m very interested in feedback about interest in something like this, and discussions about how the JS bindings should look.

Making backtraces readable

Thursday, February 7th, 2008

Reading gdb backtraces from Gtk+ applications can be a pain, as the signal emissions tend make them very long and hard to read. Today i wrote a small application that compresses signal emissions and some other common gnome stuff in gdb backtraces.

For example, take this 147 frames long backtrace i’m currently working on. Its pretty much a pain to read.

With my app it turns into this instead, which is much nicer. Here are some examples:

#7 ... segfault caught by bug-buddy
#8 <signal handler called> ()
#9 __kernel_vsyscall ()

#36 gtk_scrolled_window_destroy (object=0x82bc538) at gtkscrolledwindow.c:799
     ...
#43 gtk_object_dispose (gobject=0x82bc538) at gtkobject.c:418
#44 gtk_widget_dispose (object=0x82bc538) at gtkwidget.c:7851
#45 fm_tree_view_dispose (object=0x82bc538) at fm-tree-view.c:1457

#115 gtk_object_destroy (object=0x82d0200) at gtkobject.c:403
#120 ... emitting signal 219 on instance 0x5954
#121 _gtk_action_emit_activate (action=0x82c8e60) at gtkaction.c:872

#130 gtk_window_key_press_event (widget=0x82d0200, event=0x83716e8) at gtkwindow.c:4961
#140 ... dispatching GdkEvent 0x83716e8 to widget 0x82d0200
#141 g_main_context_dispatch (context=0x819fca0) at gmain.c:2064

Any chance something like this could be integrated with bugzilla?

Nautilus gio-branch merged – be careful

Friday, November 30th, 2007

Today I got most of file moving working in nautilus-gio, so that means that most functionality is there, even if its quite raw and not very tested. Its time to get more people involved in testing and working on the new codebase.

So, I just merged the nautilus gio-branch to HEAD.

WARNING: Its still far from done, and it might eat your files. Don’t use in anger! If it breaks you get to keep both pieces.

There is a list of things left todo on the wiki. But even after that is finished there is bound to be a bunch of stuf to be fixed.

So, use with caution, and give feedback.

UPDATE:  Please file bugs against the GIO component of nautilus in bugzilla, so we can track this

gio-standalone merged into glib

Monday, November 26th, 2007

I just commited the gio code into glib:

185 files changed, 52768 insertions(+), 8 deletions(-)

This is a pretty large piece of work. Its still got some areas that need work, and the docs are far from ready, however the API is imho pretty powerful and easy to understand. If your app uses file I/O in any way, please take a look at the GIO APIs and give feedback.

The revolution is coming

Friday, September 21st, 2007

I’ve started working on converting Nautilus to use the gio API instead of gnome-vfs. Its nowhere near done, but today I finished converting the basic file information reading and handling to gio.

Nautilus screenshot

Here it is, in all its glory!

Of course, it looks exactly the same as before, but its new and shiny on the inside. :-)

The code is available in the “nautilus-gio” branch in subversion. At the moment it only works with the standard gio local files backend. If you install gvfs it crashes due to some dbus mainloop integration conflict issues.

Experiments with runtime-less app-bundles

Tuesday, August 7th, 2007

I’ve always admired the klik
project. At least the idea behind it, because the implementation was
(by necessity) a bit ugly. There are two main problems, first it uses
loopback mounts (requires root/setuid, has weird limitations, etc) and
secondly it does relocation of packages (so that they can find their
files) by patching the binaries.

Thats why I was so happy when i saw this blog
entry
. The klik developers are working on klik2, which is working
to clean up these issues. I took a quick look at it, and they seem to
be using fuse instead of loopback, which is very good. To solve the
relocation problem they seem to be using fuse to implement a full
union filesystem.

I wasn’t completely sold on the union filesystem thing, as I thought
that was a bit overkill. All that really needs to be virtualized is one
directory, accessible via a fixed name for the running
application. Then you can install things in that given prefix, make a bundle of it and it
will run fine.

A few days later when I was lying on the beach I came up with various ways this
could be done. After a while I realized that it would be
possible to use fuse to do a runtime-less bundle system. In other
words, all you needed to install was a standard fuse, and then you can download the
app as one file and just run it. I’m on vacation and shouldn’t even be
touching my computer, but I was so exited about this idea I just had
to see if I could do it. After a day or so of hacking, i was able to make it work.

Introducing glick

glick bundles are standard ELF files that you can run like any other
binary, but they contain all the files, libs and binaries needed for
the application in that one file. glick files embedd a filesystem in the
rodata section of the elf file, and the startup code contains a fuse
filesystem to mount this filesystem, and some magic glue code to make
it all work together.

Here are two example bundles: fancy
greeter
, katach.

I’ve tested these on F7, and it seems to work. (fuse-lib and fuse must be
installed, and due to a broken Fedora packaging of fuse the user must be a
member of the fuse group or fusermount needs to be chmod:ed).

Implementation details

On application launch the glick code is started. It then creates a
temporary directory where the filesytem will be mounted. Then it creates a pipe to
talk to a child process and forks. The child process starts up the
fuse filesystem (reading from the automatically mmap:ed rodata section) and mounts that on the temp directory. Then it talks
to the parent by writing to the pipe.

The parent code opens the directory, and uses dup2() to store the file
descriptor to it as fd number 1023. Then it creates (or ensures that
there already exists) a symlink called “/tmp/glick_root” that points
to “/proc/self/fd/1023″. This is the magic part. Since we have the
target directory opened at fd 1023 the /tmp/glick_root symlink will
point to the mounted fuse filesystem for this process and all its children
(that don’t close fd 1023). This means the code in the filesystem can
hardcode /tmp/glick_root to access their files. (Having this in /tmp
is slightly non-ideal. A better place would be in say /var/glick_root,
but then it wouldn’t be a runtime-less system
).

Then it reads from the pipe, waiting for the mount to finish. When the
mount is up it it execs /tmp/glick_root/start which is a script or a
symlink that starts up the real application inside the fuse mount.
When the application dies the pipe to the fuse child is automatically
closed. The child notices this, unmounts the mount, removes the
temporary mountpoint and exits.

Licensing issues?

The implementation uses e2fsprogs-libs to implement the fuse
filesystem (based on fuse-ext2 by Jeff Garzik). This library is
unfortunately GPL, which is not ideal for maximal reuse (my code in glick is X-style
BSD). Does embedding an ext2 filesystem inside an elf
file that contains GPL code make the files in the filesystem a derived
work of the GPL code, or is this a “mere aggregation” in the terms of
the GPL? License questions abound… Anyway, perhaps using fuse-cramfs
that the klik people are working on is a better idea anyway.

The code

The code is availible via git here. Play
around with it, experiment, have fun. I did.

Note: The name “glick” is a pun on the kde:ish name of the the klik
project. This is not ment to be a slight or anything like that. I
greatly appreciate the work of the klik project (although I do believe
that their choice of name unfortunately is costing them mind-share
amongs gnome users). Hopefully the klik people will look at my work and steal the appropriate ideas.