Context Switch

Using advogato (and my log), since I’m not on the Planet (yet).

I’ve just read Daniel’s post, and since shame is a powerful drive, here’s my non-requested answer.

I’m one guy – quite possibly, the only one – that switched from a full-fledged XML parser like libxml2 to GMarkup. Believe you me: I would have rather stayed within the blissfulness of DOM, within the ease of development of a complex and powerful parser, within the safety of one of the best XML parser around the F/OSS world.

I would have used libxml2 (and in fact, I did begin using it) – because of the work that DV (and every other developer involved) put into libxml2; I state that again: it’s a wonderful library, and it’s great to have it, and for it I’ll have to buy Daniel enough beer to knock him unconscious until Gnome 2.14. :-)

That said, I switched to GMarkup because libxml2 is also a heavy dependency for Gtk+. It’s a 1M+ library, and a dependency some devices can’t afford to have on the chain – I think specifically of embedded devices.

Supporting a platform standard like the storage for recent files and bookmarks only on desktop boxes, because they can afford to have libxml2 pre-installed is not an option.

I remember discussing on the XDG list with Daniel and others about the desktop-bookmark spec; the spec started as a GMarkup format, and then I was convinced to use XBEL. It was a good idea – and once properly standardised, it could lead to data sharing between various environments; a goal that the previous recent-files spec missed badly.

Having GMarkup to parse a valid XBEL stream, even with every limitation it has (UTF-8 only, a bit shaky XML:NS support, etc.), has been possible (even thinkable) just because I had beside my open Gedit window another window with a XBEL parser written using libxml2, reminding me how a full-fledged XML parser should work.

So: thanks Daniel for libxml2 – your great work has been and it’s still really appreciated and useful. Sadly, there are requirements, in this world, and many times they collide with what we would like.

hacking: That is, the GMarkup-based XBEL parser that should be used to parse “desktop bookmarks” (recent files, filechooser’s bookmarks, default locations, etc.)

Today I’ve worked hard on the namespace parsing mechanism, and even though I feel like it’s a little too fragile, and it doesn’t cover every conceivable XML namespace declaration, it’s a start. At the moment, it parser the XBEL streams that I pass to it, resulting from libxml2.

I’ll tighten up the namespace marking routines, in order for it to be XML:NS compliant, and hope that nobody messes up with his own bookmarks. :-)

life: Tonight, I made bread with Marta. On Sunday there will be a party at her parents’, and she’s going to cater for 25+ people. We had to go shopping for two days – and believe me: it has been very tiresome.

exams: tomorrow, a C exam.

Update 2005-09-16@09:39: the EggBookmarkFile code hit CVS tonight, and so the code in EggRecentManager using it. Profiling is still on, so it’ll require auto-foo patching, but everything works nicely at the moment. I’ll do more work in the API and more profiling on the widgets, as soon as I can compile sysprof.

Since I’ve been re-writing the XBEL parser in order to use GMarkup in place of libxml2, I’ve also felt the need to get some profile going.

I’ve always felt that one of the biggest bottlenecks in the recent-files was its use of GMarkup for the recent files storage; it appears that I was wrong – at least, as far as GMarkup is concerned.

I’ve added some profile code in my local development tree (I’ve yet to upload it to libegg, since it’s not finished or polished enough), and here’s some results:

P: [ load_from_file:000.000000] - loading from file 'test-file.xbel'
P:  [ load_from_data:000.000000] - begin loading from data
P:   [     parse_data:000.000000] - beginning parse_data
P:  [     parse_data:000.163825] - finished parsing
P: [ load_from_data:000.164151] - finished loading data
P:[ load_from_file:000.167050] - finished loading from file 'test-file.xbel'
782 bookmarks loaded successfully.

(numbers are seconds and microseconds timestamps, as returned by gettimeofday(3), launching from a cold cache state hot cache (the cache wasn’t too cold, as later profiling showed) – with the file and libraries in cache the times are pretty much halved). As you can see, test-file.xbel is an XBEL compliant stream containing 782 bookmarks. I know this is a Poor Man’s Profiling(tm), but better than nothing anyway.

As you cans ee, the parsing of the file with GMarkup takes up ~0.16 seconds, mostly due to the parsing (obviously); I’ve implemented the loading from a file using a buffered read (4KB a time, which means that the read cycle does ~400 iterations in order to store the test-file.xbel file); then I feed the buffer to a function that parses everything from a string in memory.

Next run, I’ll break down the parse_data section into the per-element parsing for the bookmark, mime-type and application elements, which should be the most time-consuming, parsing-wise:

P: [ load_from_file:000.000000] - loading from file 'test-file.xbel'
P:  [ load_from_data:000.000000] - begin loading from data
P:   [     parse_data:000.000000] - beginning parse_data
P:    [ parse_bookmark:000.000000] - begin bookmark element
P:   [ parse_bookmark:000.000486] - end bookmark element
P:    [     parse_mime:000.000000] - begin mime element
P:   [     parse_mime:000.000094] - end mime element
P:    [parse_application:000.000000] - begin application element
P:   [parse_application:000.000124] - end application element
...

As you can see, the bookmark element is the biggest offender, here; this because it has four attributes (href, added, modified and visited), three of which hold a timestamp stored is the ISO8601 format (“YYYYMMDDTHH:MM:SSZ”, where T and Z are fixed characters) – which must be parsed too. The application element holds another four attributes (name, exec, count and timestamp), so it’s the second place where the parser spends most of its time. GMarkup holds the attributes names and values passed to the callbacks in two string arrays, so scanning them is a O(n) operation; better would be accessing them using a attribute_value = g_markup_parse_context_get_attribute (context, element, "attribute_name") which internally uses an hash table: this would make the attribute parsing O(1) – but it would also break API; we could provide a flag for activating a new callback signature:

void (*start_element) (GMarkupParseContext  *context,
		       const gchar          *element_name,
		       GError              **error);
void (*end _element)  (GMarkupParseContext  *context,
		       const gchar          *element_name,
		       GError              **error);

...

G_CONST_RETURN gchar *
g_markup_parse_context_get_attribute (GMarkupParseContext  *context,
				      const gchar          *attribute_name,
				      GError              **error);

This would be neat, even though, given the numbers, this could be seen as micro-optimization.

Consideration:: most time, as shown by cold-cache runs, is spent in I/O; thus, GMarkup shows virtually no bottleneck, even with parsing a bit more intensive than plain dumb 0 == strcmp (element_name, element)) checking.