Neat hacks

I’m doing optimization tasks recently. The first one is covered on the gtk mailing list and bugzilla. It involves a filechooser, many thousands of files and deleting a lot of code. But I’m not gonna talk about it now.

What I’m gonna talk about started with a 37MB Firefox history from my windows days and Epiphany. Epiphany deletes all history after 10 days, because according to its developers the location bar autocomplete becomes so slow it’s unusable. As on WIndows I had configured the history to never delete anything, I had a large corpus to test with. So I converted the 100.000 entries into epiphany’s format and started the application. That took 2 minutes. Then I started entering text. When the autocomplete should have popped up, the application started taking 100% CPU. After 5 minutes I killed it. Clearly, deleting all old history entries is a good thing in its current state.
So I set out to fix it. I wrote a multithreaded incremental sqlite history backend that populates the history on-demand. I had response times of 2-20ms in my command-line tests, which is clearly acceptable. However, whenever I tried to use them in a GtkEntryCompletion, the app started “stuttering”: keys I pressed didn’t immediately appear on screen, the entry completion lagged behind. And all that without taking a lot of CPU or loading stuff from disk or anything noticable in sysprof. It looked exactly like on this 6MB gif – when you let it finish loading at least. And that’s where the neatest of all hacks came in.

That hack is documented in bug 588139 and is what you see scrolling past in the terminal in the background. It tells you which callbacks in the gtk main loop take too long. So if you have apps that seem to stutter, you can download the patch, apply it to glib, LD_PRELOAD the lib you just built and you’ll get prints of the callbacks that take too long. I was amazed at how useful that small hack is. Because it told me immediately what the problem was. And it was something I had never thought would be an issue: redraws of the tree view.

And that gets me back to the beginning and my filechooser work, because stuff there was sometimes still a bit laggy, too. So I ran it with this tool, and the same functions were showing up. It looks like a good idea to look into tree view redraws and why they can take half a second.