Documentation

In the last thirty minutes I’ve ported gnome-utils to the new, spiffy gnome-doc-utils system.

I hope that this could hit HEAD in the next few days (pending Vincent‘s approval)- the sheer amount of stuff deleted and/or changed is really high. This would allow better translations of the documentation of the tools inside gnome-utils.

Leaking/2

After some six hours, a chinese dinner, two large glasses of earl grey tea, and a coffee, I’m finally able to declare that, according to Valgrind, both the libegg/recent-files and libegg/recentchooser code are leak-free.

Okay, take this with a grain of salt, as I’m not that good with Valgrind, and some of the logic of my widgets might lead to leaks; as far as my mallocs are concerned, my code is leak-free

[]
[]
[]

Dictionary Applet/8

I’ve finally been able to begin working on the new GNOME Dictionary widgets. While GdictEntry still has issues – related to the speed of word look up and the entry completion – and has already hit CVS, in the last two days I’ve been hacking on the definition display widget, or GdictDefbox (I’ve retained the name from the old code).

The new Defbox code is a little bit more simple – things like links are not implemented (yet, even though I found them questionable), and the output is not quite as good looking as the old one; here’s the obligatory screenshot:

gdict-defbox

the new GdictDefbox

Aside from code cleanliness and the new multiple back-end system, what differences there are with the old Defbox? First of all, this is a composite widget and not just a TextView-derived widget; this allows neat things like the next feature, that is the embedded search pane:

gdict-defbox

the embedded search pane

It should Just Work®: Ctrl+F to view it, Ctrl+G for the next match, Esc to close it. Say goodbye to silly find dialogs.

gdict-defbox

the embedded search pane at work

The new GdictDefbox also allows jumping from one definition to another, by tracking them internally.

With GdictDefbox nailed down, a simple Dictionary application and applet might already be created; I plan to improve the definition box’s output and put it on par with the current one (plus enabling theming of the colors/sizes using GtkStyle) before actually coding the new Dictionary – but keep checking out the new-dictionary branch of gnome-utils in the next week or so.

Update 20051129@13:49: the spiffy new GdictDefbox is CVS as of some minutes. Go and try it using the gnome-dictionary/libgdict/test-defbox.c test case.

[]
[]
[]

Long-standing

I’ve just committed some code that should hopefully fix bug #172587 of the old recent-files code.

The bug was hard to track down – and for it I do own a beer to Sebastien Bacher: it appears that older versions of OpenOffice creeped unescaped & into the dot-recently-used file we use to store the recent documents. The parser code inside EggRecentModel has always been a little too fragile (a mistake I tried to avoid since the beginning of my BookmarkFile parser code, by feeding it invalid XML/XBEL streams and trying to get errors instead of crashers), and thus gnome-panel had the tendency to crash until the dot-recently-used file was deleted or corrected.

I hope to close #172587 for good before marking the whole code as deprecated: it’s personal, now.

Friends

I’ve just donated to the GNOME Foundation. Not much (but since I got back to studying for this year, my <marketspeak>revenue stream</marketspeak> has really slowed down), but I plan to donate more money, aside from my time as a developer, to the F/OSS projects that I use the most.

There are many F/OSS projects, out there, which asks for some support: from patches to documentation to translations; sometimes, even a little contribute if you find that project useful. Since many of these projects are free (as in beer, other than as in speach), a little donation might show some appreciacion for the wonderful jobs done by the people working on those projects.

Dictionary Applet/7

New week, new adventures

Actually, the number of new adventures has been strictly limited by my birthday first, and by a strange feeling of drowsiness that lasted all week (and still lasts as I’m writing this). Probably, it’s due to the fact that I’ve been sleeping like four hours per night since the last week-end. Yep, that’s probably it.

Dictionary Sources

The stuff (which should land in CVS as soon as I can fix up a test unit for it) coded in two days is the spiffy new dictionary source configuration file. Since we could have multiple dictionary sources, and thus multiple contextes, we also need a way to tell Dictionary what to use. Steal^H^H^H^HTaking inspiration from the new Gedit plugin systems which uses .desktop files for defining meta-data bound to a plug-in (author, name, description, etc.), I’ve designed a .desktop file for dictionary sources:

[Dictionary]
_Name=Default
_Description=Default Dictionary server
Transport=dictd
Hostname=dict.org
Port=2628

The Transport does the trick: it specifies which GdictContext implementation should be used. Each dictionary source file is interpreted by the GdictSource class, which loads the .desktop file from an absolute path and creates the right GdictContext for you.

These sources should go inside default directories – right now, the only hard-coded directory is $DATADIR/gdict-1.0/sources, but GNOME Dictionary will also check into $HOME/.gnome2/gdict-1.0/sources (and, in the end, I’d like to use $XDG_DATA_DIRS/gdict-1.0/sources). All the dictionary source files found in those directory will be loaded by the GdictSourceLoader object; using this object, you’ll be able to access the whole sources list or directly get the GdictSource for a particular name. Oh, and if you look at the format of the dictionary source file, you’ll see that both the Name and Description keys are localizable.

As soon as I begin adding transports (in form of GdictContext implementations) to GNOME Dictionary, more dictionary sources will be made available; if I add run-time plug-ins to libgdict in the future (maybe the next development cycle), those plug-ins will have to provide a dictionary source file in order to be used.

How does this change the code for a dictionary client? Now, you’ll have to load the dictionary sources and get the source you want, say the default one:

  GMainLoop *main_loop;
  GdictSourceLoader *loader;
  GdictSource *default;
  GdictContext *context;

  main_loop = g_main_loop_new (NULL, FALSE);

  loader = gdict_source_loader_new ();
  source = gdict_source_loader_get_source (loader, "Default");

  context = gdict_source_get_context (source);
  g_signal_connect (context, "definition-found",
                    G_CALLBACK (definition_found_cb), NULL);

  gdict_context_define_word (context, "vera", "GNOME", NULL);

  g_main_loop_run (main_loop);

  g_object_unref (context);
  g_object_unref (source);
  g_object_unref (loader);

As you see – nothing more complex that adding those lines.

Update 20051124@1435: I’ve committed my development trunk to the new-dictionary branch of gnome-utils. The src directory doesn’t build yet, since there’s not much to build, but everything works in libgdict, where all the fun is ATM. Now that we have most of the infrastructure in place, the UI should soon follow. I’ll make smaller check-ins, from now on.

Update 20051124@2126: I’ve coded in 10 minutes (while commuting from my university to home) and committed simple test suite for GdictSource and GdictSourceLoader (it works only if you install the dictionary source file – but it should give you an idea on how it works); I’ve also fixed a couple of dumb bugs (did I’ve already said that I’m a sloppy coder?) discovered when coding the test suite. While I’m waiting for Marta, I’ll begin porting GdictDefbox.

[]
[]

Dictionary Applet/6

The new GNOME Dictionary code has landed in CVS this morning! woot!

I hope I didn’t fuck everything up – as usual when I’m dealing with CVS. viewcvs seems to be fine – and the tree seems to build.

ATM, you won’t find much – just the context stuff and the dictionary protocol client implementation (with a test program); see the TODO for what comes next.

[]
[]
[]

Dictionary Applet/5

Don’t you hate it when you arrive near deployment stage and you have to get back to the design table because someone makes you notice that the architecture you’ve very cleverly layed out has one fatal flaw?

Well, to be honest, I hate it and yet I love it. Because it clearly puts your ego down and moves everything under a new perspective.

Let’s take GNOME Dictionary. Last week-end I was getting the UI down with the new GdictContext object; then, Reinout van Schouwen showed me bug #167366, and this week – aside from doing the (hopefully) last profiling session of BookmarkFile – I had to design a new architecture, flexible enough to be expanded with new back-ends.

Luckily, when I re-designed the whole implementation of the dictionary protocol client, I had it clearly separated from the whole code-base; this led to a session of “find-and-rename” and some minor code tweaking.

Now, the code in libgdict is a little more complex – but it’s expandible and should allow the creation of multiple back-ends more easily.

At top level, we still have GdictContext, but instead of being an GObject it’s now a GTypeInterface; every dictionary context must inherit it’s methods (and a locality property):

  gboolean (*get_databases)  (GdictContext  *context,
  			      GError       **error);
  gboolean (*get_strategies) (GdictContext  *context,
  			      GError       **error);
  gboolean (*match_word)     (GdictContext  *context,
  			      const gchar   *database,
  			      const gchar   *strategy,
  			      const gchar   *word,
  			      GError       **error);
  gboolean (*define_word)    (GdictContext  *context,
  			      const gchar   *database,
  			      const gchar   *word,
  			      GError       **error);

Also, we have the same signals as we had before the re-design:

  void (*database_found)   (GdictContext    *context,
  			    GdictDatabase   *database);
  void (*strategy_found)   (GdictContext    *context,
  			    GdictStrategy   *strategy);
  void (*match_found)      (GdictContext    *context,
  			    GdictMatch      *match);
  void (*definition_found) (GdictContext    *context,
  			    GdictDefinition *definition);

The first implementation of this interface is the GdictClientContext object, which is the dictionary protocol client object; in order to write a dictionary client, all you have to do is instantiate a new GdictContext implementation and use one of the GdictContext methods:

  GdictContext *context;
  GError *definition_error = NULL;

...

  /* we use a GdictClientContext */
  context = gdict_client_context_new ("dict.org", 2628);
  g_signal_connect (context, "definition-found",
     G_CALLBACK (on_definition_found), NULL);

  gdict_context_define_word (context, "penguin", &definition_error);

...

Adding contextes is a matter of writing a GdictContext implementation.

Note: Even adding run-time modules would be quite simple; right now, I don’t intend to – but nothing prevets you from doing it (hint! hint!)

Now that the architecture is a bit more solid, I’m going to land it into CVS. I’ll create a branch of gnome-utils for it; my code will live in a new directory, and will slowly move gnome-utils/gdictsrc out of build. Now the timeframe for my work has shifted a bit, but I’m still very confident on having a feature-equivalent version of GNOME Dictionary out before the end of December.

[]
[]
[]

Profiling GMarkup – Take 3

Yesterday, I was attending the GTK developers meeting and I was asked by Federico what performances the BookmarkFile parser yields. I replied:

  ebassi:  f_lunch, I/O plays a big role. a file with 3000 bookmarks gets
           parsed in 2 seconds, more or less (cold cache).
  ebassi:  f_lunch, which, I think, is as good as it gets
  ebassi:  (3000 bookmarks == ~1.5 MB on disk)
  f_lunch: ebassi: any estimates for a 120 KB file?  my ~/.recently-used
           is 120 KB right now
  f_lunch: that would be 0.2 sec
  f_lunch: too slow :)

I did some more tests, with my profiling code turned off, just in order to measure raw timings (using /usr/bin/time), and it seems that I was wrong (and with this being my code and my brain-child I’m somewhat concerned on the state of my memory). The numbers returned by /usr/bin/time are 270% better (with cold cache):

  $ /usr/bin/time ./testbookmarkfile
  3360 bookmarks loaded successfully.
  0.74user 0.01system 0:00.83elapsed 91%CPU (0avgtext+0avgdata 0maxresident)k
  0inputs+0outputs (0major+1297minor)pagefaults 0swaps

And, in a warm cache case it yields timings that are 500% better than what I presumed:

  $ /usr/bin/time ./testbookmarkfile
  3360 bookmarks loaded successfully.
  0.35user 0.01system 0:00.43elapsed 86%CPU (0avgtext+0avgdata 0maxresident)k
  0inputs+0outputs (0major+1298minor)pagefaults 0swaps

The file-size is:

  $ ls -lk test-file.xbel | awk '{ print $5; }'
  1565

Which is ten times the size of Federico’s ~/.recently-used file. And this, I think, is as good as it gets. Further optimizations would be moving the namespace resolution inside GMarkup, and also the constant time accessing to an element’s attributes. These optimizations would remove the O(n) (where n := number of attributes) scans of the attributes list that the GBookmarkFile parser does when checking the attributes payload and when building the namespaces resolution hashtable.

Further optimization could target the I/O which, as I said on -devel, plays a big role for reading files bigger than the size of a page of VM; I’d like to try using mmap() when loading files, and check what timings this yields. Also, using mmap() and g_file_set_contents() would remove the need of file locking when reading/writing; right now, I’m using lockf(), which means that I must use O_RD | O_WR when I want to lock a file when reading it – this supposedly generates spurious write events in file monitoring systems like FAM.

[]
[]
[]
[]

Dictionary Applet/4

In order to let others know how’s the status of GNOME Dictionary Breaking is progressing (and in order to force myself working on it regularly ;-)), I’m sending the

Not So Weekly Status Report on GNOME Dictionary

Containing the status of the review-slash-breaking-slash-rewrite process of the GNOME Dictionary application and applet. Okay, it won’t be weekly, and I could simply use my blog instead of email; nevertheless, having someone to report to makes room for public discussion, and since I’m breaking (mostly) working stuff better be sure that some discussion actually happens at all.

So, what have you been doing so far?

First of all, I tried to salvage the ugly mess that was implementing the dictionary protocol (gnome-utils/gdictsrc/dict.[ch]). For the lack of better (or non 12-years-old-sister safe) words, I won’t use the ones I came up while auditing the said code. Suffice is to say that the code couldn’t be salvaged without a massive rewrite process anyway, so I scrapped out the implementation, and created a new, GObject-based one. You can see a preliminary version of the API usage on my weblog. How’s the rewrite going? I’ve pretty much implemented all the needed client commands for a RFC2229-compliant client, testing it both on dict.org and on my local dictionary server. The MATCH command is still missing, but I evaluate that will be a matter of 15 minutes worth of coding, in the new infrastructure. About the new infrastructure: I’m pretty proud of GdictContext; for example, now the client will advertise itself (using a predefined string or a custom one), and all the commands get queued; thus, you can set up a batch of commands (e.g. connect-lookup databases-lookup strategies-disconnect) without having to actually *wait* inside the callback for a command to complete its run. This will allow lazy data loading, and close one of my pet peeves about the current GNOME Dictionary, that is the absence of a CLI output; I’d like to write into a terminal:

$ gnome-dictionary --lookup "GNOME" --no-window

and have my definition of “GNOME” with no fuss (yes, I know that there already is “dict”, but what’s the point of having a dictionary client in GNOME if I also have to install dict?).

Another point of the new infrastructure is that lives all inside a shared library; which means that we can now have the much needed separation between the back-end code and the UI and the application/applet. Also, this means we can provide language bindings for it, allowing other applications to interface to dictionary servers (and I specifically thinking about the Deskbar Applet, here, even though I think there already is a dictionary protocol implementation in Python, using one based only on platform stuff would shorten the dependency chain).

What are you planning to do next?

First of all, finishing the client implementation (also, authentication could be added, if I can come up with a UI for it); adding a way to get some data like server’s capabilities, caching of the databases and strategies lists, etc. Also, I’ll do some code polishing and consolidation. This shouldn’t take long – probably no more than a week.

The UI is what comes next. The GdictEntry will be the first to be checked; I’d like to use the SexyIconEntry by Christian Hammond as a base for this widget, in order to have something more appealing (for the applet, mostly)- but first I’ve got to check how that behaves with GtkEntryCompletion.

The next widget to be targeted will be GdictSpeller; I’m still dubious about the form of this widget, or if it’s useful at all.

Finally, GdictDefbox. The main widget is pretty much right as it is, so I’ll just update it in order to use the new GdictContext; I’ll move the “find” dialog into a bottom pane a la Firefox, and I think that will be the most relevant change, UI-wise.

This would complete the road-map for the GNOME Dictionary Library (libgdict): the application and the applet would simply fall into place, once this has been sorted out.

When do we see the code?

This is the worst part of all. Since I’m basically rewriting everything, up until I can actually provide a working version of GNOME Dictionary I’d like not to taint the CVS repository; but I also understand the need for looking at the code, and comment on it – so, as soon as I’m able to create a full text client using libgdict, I’ll upload a tarball of it on my web space. If it passes a review from the maintainer, I’ll begin by creating a branch of gnome-utils HEAD with the new code, and keep working on it.

[]
[]
[]
[]