Newly built GLib & Gtk+ API docs are online now. In the process, i fixed a long standing documentation feature request: searchable API docs. This turned out to be the most requested documentation feature at the GNOME booth LinuxTag 2007.
The documentation updates cover Gtk+-2.11.4, so they contain new features like: GtkBuildable, GtkBuilder, GtkScaleButton, GtkVolumeButton and GtkTooltip.
I just got back online after a full week of DSL-outage due to the Deutsche Telekom Strike. If you are awaiting response via email or bugzilla from me, please be patient, I’m doing my best to catch up. Feel free to poke me about urgent issues via IRC/IM though, or drop me a reminder if you haven’t heard back from me in another week.
As announced by Hallski, Sven, Mitch and me went to LinuxTag 2007 and operated the Imendio and GNOME booths. As usual, Sven did a great job nurturing random users approaching the booth, together with Mathias Hasselmann and Michael Köchling. I put more focus on the amazingly large program of the conference, of which I’ll give a short roundup here.
The Xen keynote was on Wednesday. Xen 3.0 comes with some interesting new features, it’ll introduce IO virtualization and supports new virtuallization technologies from Intel and AMD. One important lesson I took away from that session is that using virtualization aware drivers on the guest OS can boost performance from roughly 10% using generic virtualization techniques, to more than 90% of the ideal performance throughput (native host OS performance).
-
This day, there was also the ZODB3 talk. This is an object database which can be used completely independently from Zope, it provides a very nice interface in python to implement hierarchical object tree persistence, has ACID transactions with rollbacks and allows for doomed transactions. At the lower level it uses a stable subset of python’s pickle and supports multiple storage backends.
Thursday had a Linux forum where Thomas Gleixner discussed recent realtime work in the kernel. The low latency and preemption patches that went into the kernel over the 2.6.x series brought a number of positive side effects such as general responsiveness improvements on loaded systems and new debugging mechanisms. From the new debugging facilities and raised threading issue awareness amongst kernel developers, a good 1200 patches containing bug fixes and cleanups resulted. And development in this are has not come to a halt, current/future work includes getting rid of idle-state interrupts that do nothing by having a tick-less kernel that only wakes up every once in a while when actual work is due and cleaning up general jiffie dependent code in the kernel. Now what’s left to hope is that distributors get their act together and enable the low-latency preemption patches for their desktop kernels. The patches work, they are stable, and they provide a much better user experience. E.g. if you experience sluggish system behavior during crypto filesystem IO, or experience drop outs with your sound system/server, don’t blame it on the kernel people, blame it on your distribution not allowing time slice preemption.
-
The other talk I attended was held by Matthias Hopf about the Compiz+Beryl merger. The resulting effects he presented excited the crowd as usual, and then he talked some about ongoing developments like input event transformation. After the talk we had some more personal chatter about using the 32bit XRGB visual to add alpha channels to XWindows, and future X extensions to allow applications to notify the server about when to issue EXT_texture_from_pixmap (needed for flicker free composite support).
On Friday, Lennart gave a very good presentation on the state of PulseAudio. He described how it solves the vast majority of audio use cases and can in combination with libsydney finally put an end to the never-ending lack of a portable and usable audio API. Beyond the talk, Lennart and I used every spare minute during the LinuxTag days to discuss libsydneys new API. All in all, it looks like a suitable candidate to replace (or continue) the effort Stefan and I started with CSL (we later suspended the project assuming PortAudio would fulfil the role) to make sound backends transparent.
-
After lunch, Quim Gil presented present and future of the Maemo platform. The points I personally found most notable are:
* libhildon2 is going to become an upstream community project, using Gnome infrastructure like bugzilla, with Nokia providing the core developers.
* Future platform updates (applications and OS) should be possible via APT, so flashing becomes a secondary upgrade method.
* Nokia is currently collecting feature requests for the Maemo platform. They’ll be integrated into Nokias platform plans where possible, so if you have any input to provide, state it here: Maemo Roadmap Page.
* The Maemo versioning scheme now uses alphabetical letters to indicate versioning progress. The current/upcoming versions are:
- [B]ora: current platform (3.x);
- [C]hinook: next platform (4.x) based on Gtk+-2.10, comes with SDKs before launch;
- [D]iabolo: intended to keep API/ABI from here on, unless upstream also breaks;
- [E]lephanta: SBox2 might be available at this time.
* Nokia tries to open up as much of the Maemo platform as possible and they will try to reduce dependencies between opened software and closed platform components in the future. The reason that some programs stay closed anyway are: a) missing legal clearance or licensing for some code portions - opening up code in one division of the company might affect legal claims (IP) on closed components developed and supported in another division); b) code may be subject to hardware vendor NDAs.
* Nokia will embrace attempts such as getting Maemo to run on different hardware in the future, or running non-Maemo software on the N770/N800.
* Nokia intends to introduce abstraction layers for hardware specifics (N770 vs. N800 vs. future devices) where possible.
-
The closing talk for this day was the Gimp presentation by Simon Budig, titled “Pimp My Gimp”. Simon presenting new Gimp features has become somewhat of a LinuxTag tradition over the years, and as usual new Gimp features managed to excite the crowd, even though we also had the usual glitches like a perfectly tested new clone-tool refusing to work on stage. ;-)
Saturday morning started with a vivid presentation on DBus by Marcel Holtmann. Though no stunning new features were presented, he did a good job on introducing the overall architecture and getting the audience hooked up for DBus applications with his presentation and some simple example code.
-
After noon, there was a big podium discussion on preventive data mining of personalized records, currently planned to be realized in upcoming laws by the German government. While the forum was quite interesting and definitely necessary to have at a conference like this, the discussion didn’t present surprisingly new findings for anyone following matters already. The discussion was great however, mostly because of the strong involvement of a rather large audience. It was pointed out that there is a massive lack of public awareness for the incredible data mining hunger of the government organizations and certain big companies, and that this is one of the topics that are rather uneasy to educate the majority of the democratic republic on.
-
Later during the day, Jono Bacon gave his obligatory Ubuntu talk about how Ubuntu cares about user experience and about growing the community. Personally, I still feel that more involvement from Canonical in genuine upstream development such as e.g. the Gtk+ project would be better in the long run for both Canonical and the upstream eco systems.
-
Finally, there was the Open Source Press talk which started out on how CopyLeft used to be the natural state for information exchange since the beginning of mankind, until publishers came into existence in the 16th century and invented CopyRight restrictions to preserve their investments in terms of hardware and human resources. It then was suggested that in a completely networked future computer age, we’ll be back to a pure CopyLeft situation. The presenter didn’t really seem to apply his findings though, when in the second half of the session he assumed that all mass-digestible texts must be edited by publishers and thus unconditionally require copyright laws. He also didn’t want to acknowledge that books existed which are sold and published online or that large amounts of interesting text (>= 500 pages) could possibly be written by non-publishers. While this presentation showed that publishers do recognize and ponder about open content these days, it also made clear that they still have a lot to learn.
As a closing note, I’d like to say a big Thank You to the LinuxTag Orga-Team. All throughout the four days there was an exceptionally good program in five presentation rooms, with up to four additional forum presentations distributed across the booth areas. From our perspective overall event organization went pretty well, this also resonated in German media.
Gtk+ Patch Testing & Committing:
Recently i was caught on IRC and asked to look into a few outstanding Gtk+ bug reports. Some of them easy, some of them not so easy, e.g. one required a lengthy in-depth evaluation of interacting code while some of the others already had patches applied. That particular moment, I barely had any spare time to look into Gtk+ bugs and was actually on my way to the shower and then head downtown. We still managed to squeeze in and close the bunch of 3 or 4 bug reports in less than an hour however, because people actually helped me in applying, compiling, testing and committing the changes. While core maintainer approval was a necessity for some resolutions, there was much more work to be put into each report beyond giving the OK to the final patch versions. In particular the patch applying, adaptions to recent trunk, compiling it, running the test suite or test cases and committing is something that many developers who are not overly familiar with the code base can still do. This is a great way to learn and get into a project, and to help out an understaffed core team. It is a great way to contribute to your pet project and will help to make your visions and ideas about it come true.
You can do this for Gtk+: Patch testing & committing
Gtk+ has a volunteer task list up on the wiki. One of the open positions is the one i just described, and it’d much help the project if people signed up for it. Note that having SVN access can help but is not a requirement for all tasks, access rights can be granted for active contributors.
Next time you wonder why your Gtk+ bug report seems stalled, don’t ask why it’s being processed so slowly, ask yourself why you’re not helping the project already, so all submissions can be processed faster, including yours. ;-)
The Gtk+ project finally gets a structured way to absorb voluntary efforts. Here is my proposal for a Volunteer Task List on gtk-devel-list. The task list is currently being constructed in the Gnome Wiki: GTK+ Volunteer Tasks.
In other news, there’s been some discussion on width-for-height and height-for-width management around Gtk+ recently. So I summarized the approach taken by Rapicorn, which in a nutshell allows arbitrary constrains with multiple resizing iterations and employs dedicated strategies to stabilize the resizing process: Hysteresis problems in Gtk+ - Rapicorn size negotiation.
The generation rules for the Rapicorn website are now finally in place:
-> rapicorn.org <-
This should help to clear up some of the motivations behind the Rapicorn project, in particular how it relates to Gtk+ and why it aims at implementing features that are to some extend already covered by Gtk+ or other projects like GnomeCanvas.
Here is a small excerpt to wetten the appetite:
These days Gtk+ is [...] a very successful toolkit.
However, maintenance and continuous development of a project at this scope
and scale come at significant costs in terms of development flexibility
and compatibility. It is not anymore a suitable place for evolution of new
experimental GUI technologies and quick paradigm shifts. So radically new
toolkit approaches or design ideas now need to be explored elsewhere and
only if successful and applicable can reflect back onto Gtk+ development.
-
Rapicorn explores some approaches which are simply different from
established Gtk+ paradigms.
I will quickly roll up some of the interesting bits that happened around Beast in the last couple of weeks.
Stefan Westerfeld sat down and put up a collection of the various instruments and loops he is producing with Beast. Nicely described BSE files along with Ogg previews are available in his music collection: STW Music Archive.
Hanno Behrens had been fairly active in pushing the Beast project before the last release. In particular he has been stirring up the mailing list with feature requests. One of the things we managed to get in in response to these efforts is a list of commonly used musical tuning systems. Besides the already supported 12-TET which is the 12 note per octave equal temperament used in all western contemporary music, this includes further TET variants, Indian tuning, pentatonic tunings, meantone tunings and various well tempered tunings mostly intended for organs.
Hanno also published a very well written C64-retrospective article about Beast synthesis in the 20th Issue (german) of the Lotek64 Magazine. And additionally, the technical backgrounds for this article are described in the Beast wiki: SID Vicious (english).
The last Rapicorn snapshot had a couple build and compiler issues which could be fixed meanwhile, so here is the “real” 0.1.2 release:
rapicorn-0.1.2.tar.gz
It’s still a technology preview release with known issues, nevertheless useful to look at. The release NEWS:
Rapicorn 0.1.2:
* added Image support for PNG images.
* added HSlider, VSlider, Arrow, dot-grid.
* added ScrollArea.
* added Adjustments.
* added command system allowing on-click properties.
* added reset logic to recover from stale event situations.
* added requisition tuning to allow iterative size allocations.
* added event grabbing.
* added MainLoop.
* added Thread-per-Window paradigm.
* added Viewport abstraction of drawing backends.
* added color schemes.
* added simple function Evaluator for XML properties.
* added Markup to labels.
* added focus handling for Items.
* added TextEditor prototype.
* added Region to handle rectangle arrays.
* added partial screen updates.
* removed libcairo/libpixman dependency.
* overhauled Table shrinking logic.
* upgraded Birnet library.
Known Issues:
+ instabilities/hangs upon main loop exit.
+ possible build issues with g++ versions < 3.4.6.
One of the typical problems with implementing an allocator is that all sorts of memory failures in programs get attributed to the allocator. That’s because messing up heap memory somewhere in a program or library usually messes up the allocator state even if it is implemented correctly. For several months now, we have been trying to track down and fix a very nasty memory corruption issue in Beast (#340437), which could be hunted down to rare and racy triggering with GSlice enabled, but not with G_SLICE=always-malloc set. Valgrind’s memchecker didn’t help with this particular Beast bug and couldn’t possibly track all invalid GSlice uses.
So i sat down to implement a slice address and size validator to catch the most frequent GSlice misuses. The debugging validator consists of a flat fixed-size hashing tree with fairly big prime-sized nodes and binary searchable arrays to manage collisions in the hash buckets. This structure avoids circular dependencies with GSlice (like GHashTable and GTree would have it) and is still reasonably easy to implement. The hashing makes use of the approximately random distribution of allocator block addresses in the lower bits but fairly sequential distribution in the higher bits, so the structures are reasonably space efficient and can adapt in size incrementally with every new megabyte handed out by malloc(3)/memalign(3). This also keeps hash collisions on an acceptably low average for allocations within a 4GB address range. Using a single global structure keeps GSlice from scaling well across multiple threads, but i think that’s fair enough for a debugging mode validator. Quickly after running an early version of this validator, the cause of the above Beast bug was found and could be fixed.
The faulty code was mixing up GSList and GList nodes, which couldn’t be caught by the compiler because the use of object data and timeout data forces $ G_SLICE=debug-blocks testcase
GSlice: MemChecker: attempt to release block with invalid size: 0x80527e0 size=8 invalid-size=12
Aborted
void* casts and gives up type safety:
At this point, the validator is in GLib SVN and can be enabled with G_SLICE=debug-blocks. void *slist = g_slist_alloc(); // void* gives up type-safety
g_list_free (slist); // corruption: sizeof (GSList) != sizeof (GList)
Another issue that some people might run into with the recent GSlice code is:
The correct workaround for that is to add an early g_thread_init() call to your program. More details can be found in the gtk-devel email: bugs regarding late g_thread_init() calls. ***MEMORY-WARNING***: GSlice: g_thread_init() must be called before all
other GLib functions; memory corruption due to late invocation of
g_thread_init() has been detected; this program is likely to crash,
leak or unexpectedly abort soon...
It really has been a long way to get this release out of the door. It fixes some serious crashers that i intend to blog about later, but more importantly it fixes a security vulnerability issue, here’s the related advisory: artswrapper vulnerability 2006-2916
Upgrading from any older Beast version is thusly strongly recommended. I’ll not get into too many boring details here, so let me just say that Beast now supports different musical tuning systems and also ships with new and extended modules (BseQuantizer). All the g(l)ory details can be found in the original announcement:
ANNOUNCE: BEAST/BSE v0.7.1
Bad Behavior has blocked 177 access attempts in the last 7 days.