Archive for the ‘General’ Category

How to remove flicker from Gtk+

Tuesday, February 10th, 2009

In between spending time taking care of a sick kid, a sick wife and being sick myself I have slowly been working on the remaining issues in the client-side-windows branch of Gtk+. The initial and main interest in having client side windows is that it lets us emulate all thats needed for widgets to work without any server side windows, which lets us do things like put Gtk+ widgets inside clutter, etc. However another interesting, and not entierly obvious advantage of client side windows is that it allows us to remove flicker. This post will describe how this works and show the effects.

Gtk+ already does quite a lot of things to avoid flicker. For instance, all drawing in expose events is automatically double buffered so that you never see partially drawn results. The remaining flickering is related to the effect of moving or resizing server side subwindows. Although even these are minimized by Gtk+, since many widgets don’t use such windows or only use input-only windows which don’t cause any visual effects. However, there are still some areas where subwindows are used, mostly in cases where scrolling is involved.

Lets start with an example on how scrolling works:

Evince

This is a regular Evince window showing a pdf, and we want to scroll down. This happens in several steps. First we copy the bottom area of the window to the top of the window:

Evince 2

Then we mark the newly scrolled in area at the bottom as invalid:

Evince scrolling 3

As a result of this Gtk+ will call the application to redraw the invalid region as soon as it has finished handling the incomming events:

scrolling-4

Voila! We have scrolled. (In reality more happened above, the scrollbar area was marked invalid and repainted also, but lets ignore that for now.)

This example also makes it easy to see where flicker comes from. The drawing of the newly exposed area is double buffered, so the newly drawn area is replaced atomically, however the initial copy is not done with the Gtk+ drawing system, instead its done with a XCopyArea directly on the window (not a subwindow move, but with similar effect). So, the xserver will display that immediately, while there might be some delay before the expose of the scrolled in area is drawn causing visual tearing.

Another common problem is widget resizing/move that can be seen in my previous blog entry.  In this case what happens is that a widget with a subwindow is moved and/or resized and it ends up over another widget. The window move operation is done immediately in the server and results in a copy similar to the above, and then there is some delay before the widgets are redrawn to match that.

Now, client side windows don’t by itself fix this, but the copies above and all rendering is now under control of the client (i.e. the app) so we have the tools to do something about it. The solution is to delay the copying until we’re ready to draw everything that will be drawn, so we never show any partial results. Whenever some region of a window is copied we just record the area to be copied and by how much. When we’re handling the expose events for the invalid area we handle the expose up to the point of drawing everything in the double buffer. At this point we replay all the copies we recorded, except we don’t care about copying anything that will draw into the area which will be drawn by the expose. Then we blit out the final result of the expose event.

Furthermore, in practice it often happens that we do several moves/scrolls of the same region before it its drawn. This works with the above approach, but is a bit wasteful as some regions are copied twice. So, instead of just simply keeping track of all copies being made we try to combine such double copies into a single copy, thus minimizing the actual copies we make in the end.

So, how does this look in the end? Its kind of hard to capture this kind of flicker with a screengrabber, so here is a video I took with my phone:

http://www.vimeo.com/3148023

Can you tell which one uses the standard Gtk+?

Flicker free Gtk+ continued

Tuesday, December 9th, 2008

Remember my preview of subwindowless Gtk+? It got rid of some flicker, but it was still pretty raw.

I’ve been working on a new version of the subwindowless patch, and today I implemented a cool trick that gives fully flicker-free subwindow move/resize:

This video is done with the same kind of increased X latency as the previous ones, but no flickering is detectable.

Gdb is dead. Long live Gdb!

Tuesday, November 18th, 2008

All hail Archer and Tom Tromeys python gdb integration:

Before:


(gdb) bt
#0  nautilus_file_unref (file=0xdbbc60) at nautilus-file.c:724
#1  0x004a9e74 in nautilus_directory_async_state_changed (directory=0x7bf090) at nautilus-directory-async.c:4891
#2  0x004ae363 in nautilus_directory_call_when_ready_internal (directory=0x7bf090, file=0xdbbc60, file_attributes=3, wait_for_file_list=0,
directory_callback=<value optimized out>, file_callback=0x4c4b80 <file_list_file_ready_callback>, callback_data=0x9729c0) at nautilus-directory-async.c:1344
#3  0x004fcad1 in vfs_file_call_when_ready (file=0xc19768, file_attributes=0, callback=0, callback_data=<value optimized out>) at nautilus-vfs-file.c:68
#4  0x004c69e1 in nautilus_file_list_call_when_ready (file_list=<value optimized out>, attributes=3, handle=0x972880, callback=0x4eaa80 <activate_activation_uris_ready_callback>, callback_data=0x972830) at nautilus-file.c:6900
#5  0x0046f52a in fm_directory_view_activate_files (view=0xa07430, files=0x1109480, mode=NAUTILUS_WINDOW_OPEN_ACCORDING_TO_MODE, flags=0, confirm_multiple=1) at fm-directory-view.c:703
#6  0xf4fbf6fd in IA__g_closure_invoke (closure=0xa43740, returnvalue=0x0, n_param_values=2, param_values=0x110d950, invocation_hint=0x7fffffffcc10) at gclosure.c:767
#7  0xf4fd6760 in signal_emit_unlocked_R (node=0xa360b0, detail=0, instance=0x92d340, emission_return=0x0, instance_and_params=0x110d950) at gsignal.c:3244
#8  0xf4fd7e49 in IA__g_signal_emit_valist (instance=0x92d340, signal_id=<value optimized out>, detail=0, var_args=0x7fffffffcdf0) at gsignal.c:2977
#9  0xf4fd8383 in IA__g_signal_emit (instance=0xdbbc60, signal_id=12687208, detail=0) at gsignal.c:3034
#10 0x004d5405 in activate_selected_items (container=0x92d340) at nautilus-icon-container.c:6669
#11 0x0004df20d in item_event_callback (item=<value optimized out>, event=0x7fffffffd3b0, data=<value optimized out>) at nautilus-icon-container.c:6237
#12 0xf7dcde38 in eel_marshal_BOOLEAN__BOXED (closure=0xf0b7f0, returnvalue=0x7fffffffd130, n_param_values=<value optimized out>, param_values=0x1113d80,
invocation_hint=<value optimized out>, marshal_data=0x4decc0) at eel-marshal.c:121
#13 0xf4fbf6fd in IA__g_closure_invoke (closure=0xf0b7f0, returnvalue=0x7fffffffd130, n_param_values=2, param_values=0x1113d80, invocation_hint=0x7fffffffd0f0) at gclosure.c:767
#14 0xf4fd6760 in signal_emit_unlocked_R (node=0xa3ade0, detail=0, instance=0xd6ed00, emission_return=0x7fffffffd270, instance_and_params=0x1113d80) at gsignal.c:3244
#15 0xf4fd7cf8 in IA__g_signal_emit_valist (instance=0xd6ed00, signal_id=<value optimized out>, detail=0, var_args=0x7fffffffd2d0) at gsignal.c:2987
#16 0xf4fd8383 in IA__g_signal_emit (instance=0xdbbc60, signal_id=12687208, detail=0) at gsignal.c:3034
#17 0xf7dacca8 in emit_event (canvas=<value optimized out>, event=<value optimized out>) at eel-canvas.c:2518
#18 0x004dd749 in button_press_event (widget=0x92d340, event=0x100a100) at nautilus-icon-container.c:4183
#19 0xf67ac148 in _gtk_marshal_BOOLEAN__BOXED (closure=0x7ee870, returnvalue=0x7fffffffd6c0, n_param_values=<value optimized out>, param_values=0x112c0f0,
invocation_hint=<value optimized out>, marshal_data=0x4dd6b0) at gtkmarshalers.c:84
#20 0xf4fbf6fd in IA__g_closure_invoke (closure=0x7ee870, returnvalue=0x7fffffffd6c0, n_param_values=2, param_values=0x112c0f0, invocation_hint=0x7fffffffd680) at gclosure.c:767
#21 0x4fd6432 in signal_emit_unlocked_R (node=0x7eea00, detail=0, instance=0x92d340, emission_return=0x7fffffffd800, instance_and_params=0x112c0f0) at gsignal.c:3282
#22 0xf4fd7cf8 in IA__g_signal_emit_valist (instance=0x92d340, signal_id=<value optimized out>, detail=0, var_args=0x7fffffffd860) at gsignal.c:2987
#23 0xf4fd8383 in IA__g_signal_emit (instance=0xdbbc60, signal_id=12687208, detail=0) at gsignal.c:3034
#24 0xf68aec3e in gtk_widget_event_internal (widget=0x92d340, event=0x100a100) at gtkwidget.c:4745
...

After: (interesting areas marked out)

(gdb) gbt
#0  nautilus_file_unref (file=<NautilusVFSFile:0xdbbc60>) at nautilus-file.c:724
#1  0x004a9e74 in nautilus_directory_async_state_changed (directory=<NautilusVFSDirectory:0x7bf090>) at nautilus-directory-async.c:4891
#2  0x004ae363 in nautilus_directory_call_when_ready_internal (directory=<NautilusVFSDirectory:0x7bf090>, file=<NautilusVFSFile:0xdbbc60>, file_attributes=3, wait_for_file_list=0, directory_callback=<value optimized out>, file_callback=0x4c4b80 <file_list_file_ready_callback>, callback_data=0x9729c0) at nautilus-directory-async.c:1344
#3  0x004fcad1 in vfs_file_call_when_ready (file=0xc19768, file_attributes=0, callback=0, callback_data=<value optimized out>) at nautilus-vfs-file.c:68
#4  0x004c69e1 in nautilus_file_list_call_when_ready (file_list=<value optimized out>, attributes=3, handle=0x972880, callback=0x4eaa80 <activate_activation_uris_ready_callback>, callback_data=0x972830) at nautilus-file.c:6900
# Emit signal activate on instance <FMIconContainer:0x92d340>
#9  0xf4fd8383 in g_signal_emit (instance=0xdbbc60, signal_id=12687208, detail=0) at gsignal.c:3034
#10 0x004d5405 in activate_selected_items (container=<FMIconContainer:0x92d340>) at nautilus-icon-container.c:6669
#11 0x004df20d in item_event_callback (item=<value optimized out>, event=0x7fffffffd3b0, data=<value optimized out>) at nautilus-icon-container.c:6237
# Emit signal event on instance <NautilusIconCanvasItem:0xd6ed00>
#16 0xf4fd8383 in g_signal_emit (instance=0xdbbc60, signal_id=12687208, detail=0) at gsignal.c:3034
#17 0xf7dacca8 in emit_event (canvas=<value optimized out>, event=<value optimized out>) at eel-canvas.c:2518
#18 0x004dd749 in button_press_event (widget=<FMIconContainer:0x92d340>, event=0x100a100) at nautilus-icon-container.c:4183
# Emit signal button-press-event on instance <FMIconContainer:0x92d340>
#23 0xf4fd8383 in g_signal_emit (instance=0xdbbc60, signal_id=12687208, detail=0) at gsignal.c:3034
#24 0xf68aec3e in gtk_widget_event_internal (widget=<FMIconContainer:0x92d340>, event=0x100a100) at gtkwidget.c:4745
...

As a bonus, I throw in the break_finalize command:


(gdb) p file
$1 = (NautilusFile *) 0xdbbc60
(gdb) break_finalize file
Breakpoint 2 at 0x7ffff4fc3b00: file gobject.c, line 742.
(gdb) c
Continuing.
Breakpoint 2, g_object_finalize (object=0xdbbc60) at gobject.c:742
742     {

All code availible via git from http://www.gnome.org/~alexl/git/gnome-gdb.git. Needs latest version of the archer-tromey-python branch from Archer git.

If you have other cool gdb hacks please send them so we can collect them all in the same place.

A small preview of subwindowless Gtk+

Wednesday, November 5th, 2008

Before:

After:

This code is still early in progress work, but it shows a lot of promise.

Note that the effect in the videos above is somewhat exagurated by doing two ssh forwards over the network so that i could screengrab the flicker, its not as obvious in plain local use.

Gtk can haz aliens too!

Towards a Ridley-based platform

Tuesday, October 7th, 2008

Today I finally made the last changes to remove libgnome and its dependencies from nautilus. Its not in svn yet because some patches to gnome-desktop has to go in first.

Before:

ldd `which nautilus` | wc -l
91

After:

ldd `which nautilus` | wc -l
60

So, we are now linking to  30 libraries less! Libraries which we hardly used but still take time to load, initialize and resolve symbols from.

As a comparison, gtk-demo links to 37 libraries on my system. The additional libraries come from session management, thread use, dbus, gconf, libgnomedesktop and some other nautilus-specific features.

So, Project Ridley is alive and kicking (even if the wiki page is a bit outdated). I’m just waiting for dconf/GtkSettings to be finished and then we’ll really have a competitive next generation platform without all the old deprecated libgnome era stuff.

UPDATE: I’ve now commited this to trunk. You need new gnome-desktop, eel and nautilus

I’m back! (sorta)

Monday, September 22nd, 2008

After a long parental leave I’m now partially back to work. I’ll be working half-time until the end of the year.

I’ll try to get a sense of the status of the modules I work on, and I’m glad to help answer the list of questions people have built up over the last 6 months. I’m on email, and will hang out on irc around 9:00 to 15:00 CET.

Async I/O made “easy” using JavaScript

Tuesday, September 16th, 2008

Writing code that does async I/O is a total pain in the ass. And I’ve written a whole lot of async I/O code, so I should know.

However, if you’re working in a language with full support for continuations it is a lot easier. JavaScript doesn’t support continuations fully, but it has generators, which is like a lite-version of continuations. That is part of the reason why I’m interested in JavaScript scripting.

So, I’ve been experimenting with  how to bind GIO-style async operations in JavaScript so that its easier to write async I/O code. I’ve got something going now that looks pretty ok.

Here is an example that calls an async function “$open” to read three files. It also demonstrates how to do calls from one async generator to another.

do_stuff = new AsyncRunner (function  (filenames) {
    for (let i in filenames) {
        var data = yield $open (filenames[i], 0);
        print ("data for " + filenames[i] + "  = " + data);
    }
    var ret = yield do_stuff2.call("other.txt");
    print ("AsyncRunner call returned: " + ret);
});
do_stuff2 = new AsyncRunner (function (filename) {
    var data = yield $open (filename, 0);
    print ("data for " + filename + " = " + data);
    yield "the end (of do_stuff2)"
});
do_stuff.start(null, ["test.txt", "test2.txt"]);

Its IMHO pretty easy to understand. At every async op we yield which suspends execution and restarts it at that point when we have the operation results. The “$” in “$open” is just to make it different from the sync variant and still short.

This code is far easier than it would look in C, where we would have to split out a new function at each async function call. The loop in do_stuff makes this quite tricky to do.

The code that implements this is here. Its somewhat complicated, using things like currying, closures and generators, so read it carefully.

I’d like some feedback on this. Are there any tricks I’ve missed that could make this nicer? I really wish javascript had a macro facility so that the yield keyword could be put inside the async calls and we could avoid the boilderplate around the AsyncRunner creation. I would also like to make the AsyncRunner objects callable like “do_stuff2()”, but that doesn’t seem possible with standard javascript (although it is the AsyncRunner class is implemented natively via the spidermonkey APIs…)

Embeddable languages, an implementation

Tuesday, September 9th, 2008

I read Havocs post on embeddable languages with great joy. I very much agree that this is a good model for writing a larger application. I.E. you write the core of your application in C/C++ and then do the higher levels in an embedded dynamic scripting language. All the reasons Havoc lists are important, especially for the Gnome platform (a swarm-of-processes model where we want to avoid duplicating the platform in each language used).

There are a lot of dynamic scripting languages around, but only a few of them really qualify as embeddable languages in the form Havoc proposes. I would say the list is basically just: lua, JavaScript and some of the lisp dialects.

I’m far from a JavaScript lover, but it seems pretty obvious to me that from these alternatives it is the best choice. For a variety of reasons:

  • Its a modern dynamic language
    Its dynamically typed, it has precise gargabe collection, lambda functions, generators, array comprehension, etc
  • Its object oriented (sorta)
    We want to wrap the GObject type system which is object oriented, so this is important. The JavaScript prototype-based model is a bit weird, but its simple and it works.
  • It has no “platform” of its own
    If we inject the Gnome platform (Gtk+, Gio, etc) into JavaScript it doesn’t have to fight with any “native” versions of the same functionallity, and we won’t be duplicating anything causing bloat and conversions. (JS does have a Date object, but that is basically all.)
  • Lots of people know it, and if not there are great docs
    Chances of finding someone who knows JS is far higher than finding someone who knows e.g. lua or scheme, and having a large set of potential developers is important for a free software project.
  • Lots of activity around the language and implementations
    The language continually evolves, and there is a standards body trying to shephard this. There is also a multitude of competetive implementations, each trying to be best. This leads to things like the huge performance increases with TraceMoney and Google v8. Such performance work and focus is unlikely to happen in smaller and less used languages.

To experiment with this I started working on GScript (gscript module in gnome svn). It is two things. First of all its an API for easily embedding JavaScript (using SpiderMonkey atm) in a Gtk+ application. Secondly its a binding of GObject and GObject-based libraries to JavaScript.

Here is a quick example:

GScriptEngine *engine;
GScriptValue *res;
engine = g_script_engine_new ();
res = g_script_engine_evaluate_script (engine, "5+5");
g_print ("script result: %s", g_script_value_to_string (res));
g_object_unref (res);

If you want to expose a gobject (such as a GtkLabel) as a global property in JS you can do:

GtkLabel *label = gtk_label_new ("Test");
GScriptValue *val = g_script_value_new_from_gobject (G_OBJECT (label));
g_script_value_set_property (g_script_engine_get_global (engine), "the_label", val);

Then you can access the label from  javascript by the name “the_label”. You can read and set the object properties, connect and emit the signals and call all the methods that are availible via introspection.

You can also easily call from javascript into native code.

static GScriptValue *native_function (int n_args, GScriptValue **args);
GScriptValue *f = g_script_value_new_from_function (native_function, num_args);
g_script_value_set_property (g_script_engine_get_global (engine), "function", f);

The JavaScript wrappers are fully automatic, and lazily binds objects/classes as  they are used in JS. The object properties and signal information are extracted from the GType machinery. Method calls are done using the new GObject-Introspection system.

More work is clearly needed on the details of the JS bindings, but this is already a usable piece of code. I’m very interested in feedback about interest in something like this, and discussions about how the JS bindings should look.

Making backtraces readable

Thursday, February 7th, 2008

Reading gdb backtraces from Gtk+ applications can be a pain, as the signal emissions tend make them very long and hard to read. Today i wrote a small application that compresses signal emissions and some other common gnome stuff in gdb backtraces.

For example, take this 147 frames long backtrace i’m currently working on. Its pretty much a pain to read.

With my app it turns into this instead, which is much nicer. Here are some examples:

#7 ... segfault caught by bug-buddy
#8 <signal handler called> ()
#9 __kernel_vsyscall ()

#36 gtk_scrolled_window_destroy (object=0x82bc538) at gtkscrolledwindow.c:799
     ...
#43 gtk_object_dispose (gobject=0x82bc538) at gtkobject.c:418
#44 gtk_widget_dispose (object=0x82bc538) at gtkwidget.c:7851
#45 fm_tree_view_dispose (object=0x82bc538) at fm-tree-view.c:1457

#115 gtk_object_destroy (object=0x82d0200) at gtkobject.c:403
#120 ... emitting signal 219 on instance 0x5954
#121 _gtk_action_emit_activate (action=0x82c8e60) at gtkaction.c:872

#130 gtk_window_key_press_event (widget=0x82d0200, event=0x83716e8) at gtkwindow.c:4961
#140 ... dispatching GdkEvent 0x83716e8 to widget 0x82d0200
#141 g_main_context_dispatch (context=0x819fca0) at gmain.c:2064

Any chance something like this could be integrated with bugzilla?

Nautilus gio-branch merged – be careful

Friday, November 30th, 2007

Today I got most of file moving working in nautilus-gio, so that means that most functionality is there, even if its quite raw and not very tested. Its time to get more people involved in testing and working on the new codebase.

So, I just merged the nautilus gio-branch to HEAD.

WARNING: Its still far from done, and it might eat your files. Don’t use in anger! If it breaks you get to keep both pieces.

There is a list of things left todo on the wiki. But even after that is finished there is bound to be a bunch of stuf to be fixed.

So, use with caution, and give feedback.

UPDATE:  Please file bugs against the GIO component of nautilus in bugzilla, so we can track this