Neat hacks

I’m doing optimization tasks recently. The first one is covered on the gtk mailing list and bugzilla. It involves a filechooser, many thousands of files and deleting a lot of code. But I’m not gonna talk about it now.

What I’m gonna talk about started with a 37MB Firefox history from my windows days and Epiphany. Epiphany deletes all history after 10 days, because according to its developers the location bar autocomplete becomes so slow it’s unusable. As on WIndows I had configured the history to never delete anything, I had a large corpus to test with. So I converted the 100.000 entries into epiphany’s format and started the application. That took 2 minutes. Then I started entering text. When the autocomplete should have popped up, the application started taking 100% CPU. After 5 minutes I killed it. Clearly, deleting all old history entries is a good thing in its current state.
So I set out to fix it. I wrote a multithreaded incremental sqlite history backend that populates the history on-demand. I had response times of 2-20ms in my command-line tests, which is clearly acceptable. However, whenever I tried to use them in a GtkEntryCompletion, the app started “stuttering”: keys I pressed didn’t immediately appear on screen, the entry completion lagged behind. And all that without taking a lot of CPU or loading stuff from disk or anything noticable in sysprof. It looked exactly like on this 6MB gif – when you let it finish loading at least. And that’s where the neatest of all hacks came in.

That hack is documented in bug 588139 and is what you see scrolling past in the terminal in the background. It tells you which callbacks in the gtk main loop take too long. So if you have apps that seem to stutter, you can download the patch, apply it to glib, LD_PRELOAD the lib you just built and you’ll get prints of the callbacks that take too long. I was amazed at how useful that small hack is. Because it told me immediately what the problem was. And it was something I had never thought would be an issue: redraws of the tree view.

And that gets me back to the beginning and my filechooser work, because stuff there was sometimes still a bit laggy, too. So I ran it with this tool, and the same functions were showing up. It looks like a good idea to look into tree view redraws and why they can take half a second.


#1 Tom on 07.09.09 at 20:02

Sounds like a lot of fun =)

Any plans to clean up the “hack” and properly integrate it into Ephy? That would rock…

#2 AK on 07.09.09 at 22:43

Actually i think the hack is for glib to do “speed testing”.

this hack rocks for speed optimization… good job

#3 Rafael on 07.09.09 at 22:58

As a GNOME user, I find this very exciting.
Keep up with the good work, and don’t let it die!

#4 fazil on 07.10.09 at 03:59

will this solve the lag caused while resizing windows in the gnome environment

#5 Richard on 07.10.09 at 04:37

Thanks a lot. You make another human’s life better and more joyful with such optimisations. It’s really a delight to see obstacles and resistance removed from my daily work experience (whether it’s opening a file or trying to type a URL) :D


#6 Andy Wingo on 07.10.09 at 10:11


#7 June on 10.21.09 at 18:55

A very (v-e-r-y) nice hack.

What about a GTK+ application for managing BOOKMARKS/HISTORY (for & from ALL web browsers formats, i.e. Epiphany/RDF, Mozilla/JSON/SQLITE, Midori/XBEL HTML etc.)

It should be very easy with SQLITE…
QT3 –

People are taking about it, from time to time~

What do you think?

If I’LL (I really wish I will) HAVE the time, I’ll gladly work very hard on the GUI ;)