introducing libinotify

due to my recent work with inotify i’m becoming increasingly convinced that a common library that provides a nice userspace interface to inotify is required. up until this point, everyone who wants to use inotify has written their own interface to it based on the raw syscalls. this means a lot of duplicated code and (probably) a lot of duplicated bugs.

a library with lots of users would carry all of the normal advantages of a single piece of code with many users. bugs are found more rapidly and these bug fixes help everyone. right now, if (for example) one of the beagle guys finds a bug in their implementation, then it gets fixed and nobody else benefits.

i also believe that a userspace library that provides an easy-to-use interface to inotify would make inotify more attractive to potential users.

as such, tonight i decided to release ‘libinotify’ based on the work i’ve been doing over the past week or so. it’s in gnome cvs with the module name ‘libinotify’.

it’s callback-based (with the customary user_data pointer) and it features automatic glib mainloop integration and a cracktastically easy-to-use public interface of two functions — inotify_monitor_add and inotify_monitor_remove. (there are some other functions but most users won’t need them.)

it tries to remain as true as is reasonable to the inotify api, allowing the user (for example) to deal natively with inotify event masks when requesting watches and receiving event notifications. it also provides the ‘name’ field of the event as reported by inotify (or null) directly to the user without any path fiddling.

perhaps most importantly, it handles the multiplexing that is required to support the user being able to register multiple watches on a single inode. it even allows multiple watches to be registered on the same inode with different event masks and calls callbacks only when they should be.

a very simple program might look like this:

#include <linux/inotify.h>
#include <libinotify.h>
#include <gtk/gtk.h>

static void
my_callback( INotifyHandle *inh, const char *monitor_name,
             const char *name, guint32 event_type,
             gpointer user_data )
{
  g_print( "Got event %x on %s (file %s)\n",
           event_type, monitor_name, name );
}

int main( int argc, char **argv )
{
  INotifyHandle *inh;

  gtk_init( &argc, &argv );

  inh = inotify_monitor_add( "/home/desrt", IN_ALL_EVENTS,
                              my_callback, NULL );

  if( inh == NULL )
    g_warning( "monitor_add failed" );

  /* maybe you want to do this later... */
  /* inotify_monitor_remove( inh ); */

  gtk_main();

  return 0;
}

as of now, there are still a few things left to implement. i need to figure out what (if anything) i want to do about reporting joint events (ie: moved_from/moved_to). also, the support for having multiple watches with different event masks doesn’t actually exist yet. the code isn’t yet very resilient against failure. the api is also very likely to change. as such, libinotify isn’t quite ready to be used for anything more than experimenting with.

i’m mostly making this post in hopes of receiving useful feedback. if you’re a current user of inotify (or plan to become one) i appreciate any thoughts or suggestions you might have regarding the direction of the libinotify api.

my summer at school

this past summer i was employed by the department of computing and software at mcmaster university. there were a bunch of us in one cubefarm together. william sat in the desk across from me. we introduced him to linux.

william's creation
(the best thing any of us did all summer)

a lowercase manifesto

this past weekend i went up to the cottage to get some time there before school started and kept me stuck in the city forever. i took my powerbook.

while i was there, i started looking into writing an inotify backend for gnome-vfs (so that it doesn’t have to use gam_server anymore). along the way of writing code to interface with inotify i noticed an evil problem but was able to write code to work around it. i then noticed another evil problem. with these two problems combined, i was able to convince myself that it was completely impossible to write a general-purpose monitoring system with inotify that would be free of both race conditions and the possibility of missed events.

at this point, i switched from writing code to writing a rant, in hopes that maybe i’d be able to see some of the problems fixed. i uploaded the rant on monday, while still at the cottage, and gave the url to a friend in #gnome-hackers.

http://manic.desrt.ca/inotify

a few things happened.

by tuesday, robert love had seen the rant and he told me on irc that, in fact, linux 2.6.13 had addressed one of the major issues in the rant by not reissuing ‘wd’ (watch descriptor) numbers. this removed the majority of the race condition problems. when at the cottage, i had been working with ubuntu breezy (which ships 2.6.12 and didn’t yet have the fix).

upon my hearing this, robert helped me track down the appropriate change in the kernel source and i filed a bug with the ubuntu people. it was fixed in breezy that night. i was reasonably impressed by the response of everyone involved here.

on wednesday, i was at work. john mccutchan (the guy who works on inotify with robert) contacted me privately on irc to ask if i’d make a note in my rant that most of the problems in inotify were in fact fixed. i made this adjustment. john and i got talking about the problems with inotify. he said he was glad that i was taking the time to learn inotify’s interface to the level that i was and that i was the first person other than him and robert to notice the race condition problems. we talked about how to fix the remaining problems as discussed in the rant and i proposed a IN_MASK_ADD field be added to allow watches to be added to an inode using a bitwise OR (as opposed to the current behaviour which replaces the watchmask outright). this addition would basically wrap up the rest of my problems with inotify.

i told john that i had to get back to work and that it was nice hearing from him.

what happened next astounded me. i checked my email a few hours later to notice that i had received emails from john, robert and andrew morton. i wasn’t a primary recipient of any of the emails, but rather just on the cc: list.

the first was a patch, written by john, addressing the last remaining issues in inotify.

the second email was robert, signing off on the changes.

the fifth was andrew morton acknowledging the patch for inclusion in 2.6.14.

in the span of a couple of hours, every fixable problem that i had with inotify went out the window.

this not only leaves me with renewed faith in inotify as a suitable api, but also gives me a really great feeling about the guys behind it. you can’t buy upstream support that’s this responsive.

later in the day, i proposed john’s patch on ubuntu’s bugzilla so that it might be folded into breezy.

in all fairness, i wasn’t around for the release of gnome 2.0. at that time i was still bouncing between being a gnome and kde user. i don’t know what collaboration between different people on parts of gnome was like at this time.

in 2005, though, it’s become very clear that it’s about more than just gnome. it’s about “the linux desktop”. this includes the kernel guys. this includes the distribution guys. the number of people involved has grown. the willingness of people (from both sides) to help is amazing. we can’t succeed without them. i’ve given one simple example above but i’ve seen cases time and time again that this is true.

i’m not particularly involved in gnome development, so i’m just observing “from the cheap seats”. even still, it is clear to me that this project (the “linux on the desktop” project — not just gnome) is a project that is not going downhill at all, but rather is at its peak and still gaining positive momentum.