today’s hack: pstimeouts

today i wrote a kernel patch to find out just how often processes are waking up as a result of setting timers for themselves. the intention is to provide a tool to make it extremely easy to spot poorly behaved applications.

the kernel patch recognises 5 types of “timers” that might cause a process to wake up:

  • poll returning due to timeout after sleeping
  • select, same as above
  • epoll, same as above
  • the ‘real time’ interval timer (SIGALRM)
  • anything else inside the kernel that uses schedule_timeout

the kernel patch makes this information available in /proc/pid/timeouts as 5 numbers (corresponding to the list above).

i wrote ‘pstimeouts’ as a small utility to read this data and present it to the user. it’s written in straight-up C with no library dependencies (not even glib) as to be usable by the biggest number of people.

here is a ‘screenshot':

desrt@acquiesce:~$ pstimeouts
  pid timeouts process
 4568       44 x-session-manager
 4607        7 /usr/bin/dbus-daemon
 4609      704 /usr/lib/libgconf2-4/gconfd-2
 4612        0 /usr/bin/gnome-keyring-daemon
 4615     6615 /usr/lib/control-center/gnome-settings-daemon
 4629     5428 /usr/bin/metacity
 4634    39574 gnome-panel
 4636    17635 nautilus
 4639     1391 gnome-volume-manager
 4643        0 /usr/lib/bonobo-activation/bonobo-activation-server
 4648    10346 update-notifier
 4655    37128 nm-applet
 4658      128 /usr/lib/gnome-vfs-2.0/gnome-vfs-daemon
 4661     4058 gnome-cups-icon
 4680    28102 gnome-power-manager
 4718     3388 /usr/lib/nautilus-cd-burner/mapping-daemon
 4742       59 /usr/lib/gnome-applets/gweather-applet-2
 4758   290997 gnome-terminal
 4766    21571 gnome-screensaver
 4768        0 -bash
10005    12195 /usr/lib/notification-daemon/notification-daemon
11068        0 -bash
11099        0 -bash
14420        0 -bash
14503        0 -bash
14529        0 -bash
14595        0 -bash
  539        0 pstimeouts

pstimeouts has -a and -u options that do the same as their ‘ps’ counterparts. -u will show each timeout type separately.

the kernel patch (which applies against ubuntu linux-source-2.6.17-7.20) and the source for pstimeouts are located here: http://desrt.mcmaster.ca/code/pstimeouts/. enjoy :)

future work: make a ‘top’ sort of utility.

9 thoughts on “today’s hack: pstimeouts”

  1. i looked at timertop and ran it. it only tracked when high precision timers caused wakeups.

    this patch tracks things like poll/select as well which is a lot more useful for the specific use case of finding gtk applications that schedule timers to run more often.

  2. You, sir, are a god.

    A minor god, to be sure — somewhere between Pellor and Mnemosyne, I’d say — but still quite bodacious.

  3. I appreciate this kind of work.

    I have a question though. I wrote a peace of code for gnuplot where I have to care about stdin from the command-line interface and at the same time to look for the GUI events (clicks, moves, keys) coming from the ‘plot’ window. The main thread is a loop on a function called waitforinput() which is supposed to return the next character from stdin while processing the GUI events when they arrive. There is a second thread which is a usual GUI loop, and when it processes an event, it forwards it to the main one using a mutex’ed list. I ended up using select() on stdin with a timeout to periodically check for the contents of this list. So gnuplot falls in your category of “bad programs”, but it’s because I don’t know how else to do ! Do you have an idea ?

  4. Timothée: the only real solution here is to use a single mainloop for X events and your stdin.

    Toolkits like GTK will let you register a fd watch (like for stdin).

    Otherwise, you can get the X socket fd from the display using the ConnectionNumber() call and add it to your stdin mainloop that way (this is what GDK does internally).

Comments are closed.