a lowercase manifesto

this past weekend i went up to the cottage to get some time there before school started and kept me stuck in the city forever. i took my powerbook.

while i was there, i started looking into writing an inotify backend for gnome-vfs (so that it doesn’t have to use gam_server anymore). along the way of writing code to interface with inotify i noticed an evil problem but was able to write code to work around it. i then noticed another evil problem. with these two problems combined, i was able to convince myself that it was completely impossible to write a general-purpose monitoring system with inotify that would be free of both race conditions and the possibility of missed events.

at this point, i switched from writing code to writing a rant, in hopes that maybe i’d be able to see some of the problems fixed. i uploaded the rant on monday, while still at the cottage, and gave the url to a friend in #gnome-hackers.

http://manic.desrt.ca/inotify

a few things happened.

by tuesday, robert love had seen the rant and he told me on irc that, in fact, linux 2.6.13 had addressed one of the major issues in the rant by not reissuing ‘wd’ (watch descriptor) numbers. this removed the majority of the race condition problems. when at the cottage, i had been working with ubuntu breezy (which ships 2.6.12 and didn’t yet have the fix).

upon my hearing this, robert helped me track down the appropriate change in the kernel source and i filed a bug with the ubuntu people. it was fixed in breezy that night. i was reasonably impressed by the response of everyone involved here.

on wednesday, i was at work. john mccutchan (the guy who works on inotify with robert) contacted me privately on irc to ask if i’d make a note in my rant that most of the problems in inotify were in fact fixed. i made this adjustment. john and i got talking about the problems with inotify. he said he was glad that i was taking the time to learn inotify’s interface to the level that i was and that i was the first person other than him and robert to notice the race condition problems. we talked about how to fix the remaining problems as discussed in the rant and i proposed a IN_MASK_ADD field be added to allow watches to be added to an inode using a bitwise OR (as opposed to the current behaviour which replaces the watchmask outright). this addition would basically wrap up the rest of my problems with inotify.

i told john that i had to get back to work and that it was nice hearing from him.

what happened next astounded me. i checked my email a few hours later to notice that i had received emails from john, robert and andrew morton. i wasn’t a primary recipient of any of the emails, but rather just on the cc: list.

the first was a patch, written by john, addressing the last remaining issues in inotify.

the second email was robert, signing off on the changes.

the fifth was andrew morton acknowledging the patch for inclusion in 2.6.14.

in the span of a couple of hours, every fixable problem that i had with inotify went out the window.

this not only leaves me with renewed faith in inotify as a suitable api, but also gives me a really great feeling about the guys behind it. you can’t buy upstream support that’s this responsive.

later in the day, i proposed john’s patch on ubuntu’s bugzilla so that it might be folded into breezy.

in all fairness, i wasn’t around for the release of gnome 2.0. at that time i was still bouncing between being a gnome and kde user. i don’t know what collaboration between different people on parts of gnome was like at this time.

in 2005, though, it’s become very clear that it’s about more than just gnome. it’s about “the linux desktop”. this includes the kernel guys. this includes the distribution guys. the number of people involved has grown. the willingness of people (from both sides) to help is amazing. we can’t succeed without them. i’ve given one simple example above but i’ve seen cases time and time again that this is true.

i’m not particularly involved in gnome development, so i’m just observing “from the cheap seats”. even still, it is clear to me that this project (the “linux on the desktop” project — not just gnome) is a project that is not going downhill at all, but rather is at its peak and still gaining positive momentum.

6 Comments

  1. Posted September 2, 2005 at 5:18 am | Permalink

    That’s just awesome, it always makes me happy to see such reports of FOSS succeses.

    However, while moving gnome-vfs to just using inotify is a cool thing for Linux I can see a few problems.

    One you pointed out yourself, this will only work properly if it’s actually running a 2.6.14 or later kernel

    The other would be that it doesn’t actually do anything if you are using an older kernel or not using Linux at all.

    If I understand this right, this would introduce for FreeBSD, etc. a second layer compared to Linux, namely gamin (admittingly this would be more like removing a layer on Linux than adding one on the other platforms). Wouldn’t this be a possible cause of problems, instead of spotting all monitoring bugs in gamin, we now spot them in the inotify backend and in gamin.

    Don’t get me wrong, I still think it’s really cool.

  2. ryan lortie
    Posted September 2, 2005 at 10:39 am | Permalink

    as with all things, users of older versions of linux and other operating systems will continue to be supported through the old interfaces (ie: gamin).

    as for the reporting of bugs, it really just changes the demarcation point for what is considered a platform bug and what is a bug in gnome and who is responsible for fixing it.

    even now, with gamin in use on all platforms, each platform has its own in-kernel monitoring code (that feeds events to gamin). there could be different bugs in all of this different code.

    what would be -really- cool is if other platforms implemented native inotify or inotify-like interfaces for their kernels.

  3. alexl
    Posted September 2, 2005 at 5:25 pm | Permalink

    Uhm, john already has a patch for gnome-vfs inotify support at http://bugzilla.gnome.org/show_bug.cgi?id=314281

  4. ryan lortie
    Posted September 2, 2005 at 5:32 pm | Permalink

    i know about this patch. christian has reservations about it and encouraged me to write another patch.

  5. alexl
    Posted September 2, 2005 at 6:44 pm | Permalink

    Please add a comment about this to the bug.

  6. ryan lortie
    Posted September 3, 2005 at 2:18 am | Permalink

    i discussed it with john, personally (well, on irc) when we were talking about how to improve inotify. today i should be finished my implementation so i’ll attach it to the bug then.