today i wrote a kernel patch to find out just how often processes are waking up as a result of setting timers for themselves. the intention is to provide a tool to make it extremely easy to spot poorly behaved applications.
the kernel patch recognises 5 types of “timers” that might cause a process to wake up:
- poll returning due to timeout after sleeping
- select, same as above
- epoll, same as above
- the ‘real time’ interval timer (SIGALRM)
- anything else inside the kernel that uses schedule_timeout
the kernel patch makes this information available in /proc/pid/timeouts as 5 numbers (corresponding to the list above).
i wrote ‘pstimeouts’ as a small utility to read this data and present it to the user. it’s written in straight-up C with no library dependencies (not even glib) as to be usable by the biggest number of people.
here is a ‘screenshot’:
desrt@acquiesce:~$ pstimeouts pid timeouts process 4568 44 x-session-manager 4607 7 /usr/bin/dbus-daemon 4609 704 /usr/lib/libgconf2-4/gconfd-2 4612 0 /usr/bin/gnome-keyring-daemon 4615 6615 /usr/lib/control-center/gnome-settings-daemon 4629 5428 /usr/bin/metacity 4634 39574 gnome-panel 4636 17635 nautilus 4639 1391 gnome-volume-manager 4643 0 /usr/lib/bonobo-activation/bonobo-activation-server 4648 10346 update-notifier 4655 37128 nm-applet 4658 128 /usr/lib/gnome-vfs-2.0/gnome-vfs-daemon 4661 4058 gnome-cups-icon 4680 28102 gnome-power-manager 4718 3388 /usr/lib/nautilus-cd-burner/mapping-daemon 4742 59 /usr/lib/gnome-applets/gweather-applet-2 4758 290997 gnome-terminal 4766 21571 gnome-screensaver 4768 0 -bash 10005 12195 /usr/lib/notification-daemon/notification-daemon 11068 0 -bash 11099 0 -bash 14420 0 -bash 14503 0 -bash 14529 0 -bash 14595 0 -bash 539 0 pstimeouts
pstimeouts has -a and -u options that do the same as their ‘ps’ counterparts. -u will show each timeout type separately.
the kernel patch (which applies against ubuntu linux-source-2.6.17-7.20) and the source for pstimeouts are located here: http://desrt.mcmaster.ca/code/pstimeouts/. enjoy :)
future work: make a ‘top’ sort of utility.
Cool. I assume those numbers are cumulative since last boot?
Since the process was created with fork().
Hate to tell you this, but you’ve just reinvented the wheel:
http://www.ussg.iu.edu/hypermail/linux/kernel/0508.1/0106.html
i looked at timertop and ran it. it only tracked when high precision timers caused wakeups.
this patch tracks things like poll/select as well which is a lot more useful for the specific use case of finding gtk applications that schedule timers to run more often.
You, sir, are a god.
A minor god, to be sure — somewhere between Pellor and Mnemosyne, I’d say — but still quite bodacious.
Looks good Ryan. Keep up the good work.
I appreciate this kind of work.
I have a question though. I wrote a peace of code for gnuplot where I have to care about stdin from the command-line interface and at the same time to look for the GUI events (clicks, moves, keys) coming from the ‘plot’ window. The main thread is a loop on a function called waitforinput() which is supposed to return the next character from stdin while processing the GUI events when they arrive. There is a second thread which is a usual GUI loop, and when it processes an event, it forwards it to the main one using a mutex’ed list. I ended up using select() on stdin with a timeout to periodically check for the contents of this list. So gnuplot falls in your category of “bad programs”, but it’s because I don’t know how else to do ! Do you have an idea ?
Timothée: the only real solution here is to use a single mainloop for X events and your stdin.
Toolkits like GTK will let you register a fd watch (like for stdin).
Otherwise, you can get the X socket fd from the display using the ConnectionNumber() call and add it to your stdin mainloop that way (this is what GDK does internally).
desrt: thanks !