There be divils here

I reckon a lot of hackers don’t realise how wary they need to
be when using enums. Problems occur frequently because people
(usually unwittingly) make assumptions about either the size,
alignment or signedness of the integer representation for the
enum which the compiler uses.

C99 says Each enumerated type shall be compatible with
an integer type. The choice of type is implementation-defined.
An implementation may delay the choice of which integer type
until all enumeration constants have been seen
. What
this means is that once the compiler has seen all the constants
you have defined for that enum, it will then decide what integer
representation to use and that could be a signed or unsigned
integer of any size.

So, GNOME hackers beware. gconf_enum_to_string() is
nasty. Have a look at the prototype:

gboolean  gconf_string_to_enum (GConfEnumStringPair  lookup_table[],
                                const gchar         *str,
                                gint                *enum_value_retloc);

Bugs I’ve seen include:

  enum { A, B } test_enum;

  gconf_enum_to_string (enum_to_str_table, str, (int *) &test_enum);

This is guaranteed to cause crashes on some architectures since
alignment of the enum is different from the expected
alignment of an int. So it should be:

  enum { A, B } test_enum;
  int test_int;

  if (gconf_enum_to_string (enum_to_str_table, str, &test_int))
      test_enum = test_int;

Also, this breaks:

enum { A, B } test_enum;
int test_int;

test_enum = -1;
gconf_enum_to_string (enum_to_str_table, str, &test_int);
test_enum = test_int;

if (test_enum
The problem here is that you're assuming the enum has a signed
representation when in fact the compiler may choose an unsigned
representation. If the conversion fails, if (test_enum
will not detect it.

Alex also points out that when adding values to an enum in an API
you need to be careful that you're not unwittingly changing the
ABI because the compiler chooses to use a large integer representation
for the enum than was previously used.


I hate now knowing how things work. Especially things that I
rely on heavily as a hacker. But I love it when I get a chance
to finally figure out how the damn thing works and it turns out
to be beautifully simple.

This week I found that I had to debug a small part of libtool’s
behaviour. It had always worked for me before and I had assumed
it worked a certain way, but in this case it just wasn’t
working properly. I could feel myself breaking out in a cold
sweat. Don’t make me debug libtool! Please God, noooo…

But the clouds parted. And it was simple.

When you build an executable, and it links against a library
in the same module, you need some way to be able to run that
uninstalled executable with the uninstalled library. So,
libtool puts a script where you think your uninstalled
executable should be and here’s what it does

  • When you run the script for the first time, it re-links
    your executable with --rpath $path-to-lib/.libs and
    places that executable in .libs/lt-myexec.
  • That causes the linker to to put a DT_RPATH ELF
    attribute in the executable which tells the dynamic loader
    where to looks for libraries. (See the manpage)
  • And finally, the script just runs the modified executable.

For example:

  $> ls -l .libs/*gdk-pixbuf-csource
  -rwxrwxr-x    1 markmc   markmc      51262 Feb 26 11:28 .libs/gdk-pixbuf-csource
  $> ./gdk-pixbuf-csource
  Usage: gdk-pixbuf-csource [options] [image]
  $> ls -l .libs/*gdk-pixbuf-csource
  -rwxrwxr-x    1 markmc   markmc      51262 Feb 26 11:28 .libs/gdk-pixbuf-csource
  -rwxrwxr-x    1 markmc   markmc      51294 Feb 28 15:48 .libs/lt-gdk-pixbuf-csource
  $> objdump -p .libs/gdk-pixbuf-csource | grep RPATH
  $> objdump -p .libs/lt-gdk-pixbuf-csource | grep RPATH
    RPATH       /gnome/head/cvs/gtk+/gdk-pixbuf/.libs:/gnome/head/INSTALL/lib
  $> ldd .libs/gdk-pixbuf-csource | grep gdk_pixbuf
 => /gnome/head/INSTALL/lib/ (0x0087d000)
  $> ldd .libs/lt-gdk-pixbuf-csource | grep gdk_pixbuf
 => /gnome/head/cvs/gtk+/gdk-pixbuf/.libs/ (0x004a0000)

Not exactly, earth shattering. But nice to know.

February 28, 2004

Its been a fun week completely focused on Fedora packaging and
bugzilla work. Had a nice wake up call on how huge a time sink
fixing non x86 builds can be. A day and a half debugging an ABI
mismatch on pcc breaking the gtk2 build! Sheesh.

Its nice to finish up a week with the feeling that you’ve got
a good handle on things, though.

February 25, 2004

I’ve been really taken by surprise at how my motivation has jumped
hugely with the move to Red Hat. I’m even enjoying things like bugzilla
work, packaging and release-team stuff these days. Clearly, this can’t
last. And I’ve got to get back hacking on things like Vino soon …

GNOME 2.6 looks like its pretty much on track. Things like the gtk+
schedule and the constant string freeze breakage has given the release
team the heebie jeebies somewhat, but after our irc meeting last night
I’m feeling pretty confident the release will come out without any huge

Alex and I have been updating the GNOME packages in Fedora Core to the
latest GNOME Beta ones. Hopefully, by the time FC2 test2 goes into devel
freeze on Friday we’ll have most of the packages updated. A big priority
is to get the new file selector out to the Fedora testing masses, so
I’m going to update gtk+/atk/pango/glib to the latest today. Also going
to shove the little gnome-netstatus applet in too 🙂

RFB Protocol

So, I had a discussion with the RFB protocol maintainers at
RealVNC about getting the
new RFB security types – which I added in Vino for doing the
encryption thing – registered in the protocol. As far as I can
make out at this point:

  • There are more security types registered than appear in
    the protocol specification.
  • Security types 1-16 are reserved for “standard” security
  • A security type becomes “standard” when it is added to the
    “reference implementation” … which just happens to be
    the same product RealVNC is trying to make money from.

I can’t make up my mind whether I care about this or whether
I should just leave sleeping dogs lie. My main worry is what
happens if we decide there are other additions we want to make
to the protocol. Oh well, cross that bridge when we come to it.
The RealVNC guys are friendly, so I don’t expect there would
actually be problems.

In the end, we agreed that I had to use different numbers to
identify the new security types and I had to modify the way
the new security type actually worked. End result – I’ve just
pushed a new version of Vino with incompatible changes in the


I’ve just imported Vino into CVS. Its “Remote Desktop” functionality
for GNOME using VNC’s RFB
protocol which I wrote while at Sun.

Currently, the most notable bits over other VNC servers are:

  • Support for encrypting the RFB protocol stream using SSL
  • A dialog which prompts the user before allowing a remote user
    to connect
  • Exports the local display, so you don’t have to start up a
    seperate VNC display
  • Properly integrated into GNOME

More details



list includes:

  • Use the new
    extension instead of polling the screen
  • Write a GTK+ based client – the current client is a Java applet
    but you can use any other vncviewer without encryption
  • Service discovery using DNS-SD and mDNS
  • Better authentication schemes

Modems are teh suck

A rather “interesting” day doing bugzilla work, disting and
pushing 2.5.4 tarballs and jhbuilding GNOME HEAD in parallel
over a 56k dialup line. I think I’ll be counting down the
minutes until the Eircom
fools manage to flick the magic DSL switch …

How on earth did I put up with this for an entire year in
New Zealand?

Leaving Sun

So, today is my last day working for Sun which feels very, very
strange. Its been a great few years and I’m genuinely going to
miss this place and the great people here. I still can’t believe
that my dream of working full time on free software has come
true. Sun made that possible – for that I’ll be ever grateful.

On the upside, though, it marks the start of a new and exciting
adventure – working for Red Hat. Its a little daunting, actually.
Having seen the awesome work the guys do, I’ve no idea if I’ll
be able to live up to their standards. I can’t wait to get
started though 🙂

I hope people don’t read into this something that isn’t there –
even though I’m leaving, I still think Sun is a great place to
work and that its a company with a seriously bright future.
My moving to Red Hat has nothing to do with which is a “better”
company, and everything to do with me wanting to take up the
opportunity of a new challenge.

I can’t get Robert Frost’s
The Road
Not Taken
out of my head right now. It pretty well sums
up how I’m feeling … but that could just be the hangover 🙂