Getting multitouch to just work

There are many things going on in this field, from the recent merge of the xi2 branch into GTK+ master (yay!) to the ongoing effort in Xorg to support multitouch devices.

I’ve worked myself in a proposal to the evdev driver so it deals with multitouch devices as a set of XI2 devices (in GTK+ world, this means each finger would be represented by a GdkDevice), only one finger is able to send core events, while the others only get to send XI2 events, this seems pretty much in line with Peter’s thoughts on multitouch support

I also have a few patches in the xi2-playground branch that understand these semantics and basically make multitouch work out of the box.

So, if you have a multitouch device that’s understood by the linux kernel, these are the ingredients:

Voilá! xinput list should tell something like:

⎡ Virtual core pointer id=2 [master pointer (3)]
⎜ ↳ Virtual core XTEST pointer id=4 [slave pointer (2)]
⎜ ↳ N-Trig MultiTouch id=12 [slave pointer (2)]
⎜ ...
⎣ Virtual core keyboard id=3 [master keyboard (2)]
∼ N-Trig MultiTouch touchpoint 0 id=13 [floating slave]
∼ N-Trig MultiTouch touchpoint 1 id=14 [floating slave]
∼ N-Trig MultiTouch touchpoint 2 id=15 [floating slave]
∼ N-Trig MultiTouch touchpoint 3 id=16 [floating slave]

And the GTK+ tests in tests/multidevice/ should just work, providing at the same time a single-touch device feeling to applications unaware of XI2 or multiple devices.

If you find this subject interesting, don’t miss my guadec talk! where I’ll be talking about multitouch in GTK+ and what applications can do to support it.

Multi-touch support in Linux/Xorg/GTK+

So, for the first time ever (to my knowledge), the full multi-touch stack working on Linux:

Wee, multi-touch

This video features:

In my opinion, this is a quite important milestone, which reflects much work done lately in this camp, and from now on things can only get better!

And of course,

HP stands for “hijo de puta”

Ok, next blog post was supposed to be about multitouch and such as I promised, but that will have to wait a bit, let’s see why:

Today’s history is about N-trig, HP, and lots of idiotic behavior around. In order to have a better insight about multitouch, and seeing that the Thinkpad was aging, I decided to get a HP Touchsmart TX2 tablet. All in all, a nice computer, quite better than the old one, and there has been work lately on getting the N-Trig multitouch+stylus device working on linux, all fine! but wait…

WTF #1 (N-Trig)
After I started experimenting with the Linux multitouch input interface, I promptly realized I wasn’t getting multitouch events at all, why? turns out the N-Trig device had a firmware meant for Vista, and that I should install Windows 7 drivers in order to get the newer firmware that would allow multitouch events.

Yes, right, drivers and firmware are bundled in a single installer, so when you install such thing you must plug your laptop, remove it from the expansion base and cross your fingers (away from the screen of course), could something go wrong? of course, and it did for me, leaving me with a “N-Trig hardware not detected” message whenever I try to reinstall or uninstall the driver, so no reflashing will happen, leaving the touchscreen completely unusable under any OS.

In conclusion: Hardware manufacturers should stay away from hardware unless they hold a soldering iron, seriously, leave software to others.

WTF #2 (HP)
After this, the only feeling of relief I had was due to the laptop being under warranty, easy, ain’t it? It isn’t. After several long calls to their customer service, they still insist that I must purchase a recovery kit DVD set (I wiped the recovery partition out, no DVDs were shipped with the laptop) for 40€, so they can check remotely themselves (eek) that it’s actually a hardware issue. There are several problems with this:

  • According to Spanish law, enjoying any product warranty must imply no cost at all to the customer. I told them so and they dared me to sue them, I’m already looking into doing that.
  • They stated that they can only support a computer warranty with the pre-installed OS. However, in the booklet shipped with the laptop, it is mentioned that they don’t guarantee at all any shipped software. They’re just supposed to offer limited technical support the first 90 days (which already expired).

I’m currently trying to bypass phone customer service with the e-mail one, they at least seem more indifferent to me having other OSs, let’s see if I succeed.

In conclusion: looks like HP customer service’s only target is to cause grief and frustration, you don’t only deserve being sued, you deserve to die young, in pain and alone.


XInput2 GTK+
Sadly the only thing preventing me from sending a preliminar patch is polishing XInput 1 support, it was mostly readapted to GdkDeviceManager and the event handling refactor, but there are some glitches here and there. All this is now stalled by having the tablet functionality broken, if anyone wants to pick this up, please tell me (garnacho at #gtk+, etc) and I’ll try to help you through the code.

Likewise, non-X11 backends are completely untamed land, these need readapting to GdkDeviceManager and _gdk_windowing_* API changes, contributions there are more than welcome.

On Tracker stuff

Seeing that Martyn has updated his blog with some sweet tracker info, I figured I could do the same :)

The main feature I’ve been last working on (together with Martyn) is libtracker-miner, Which will ease the creation of data miners for tracker-store (that is, objects that extract useful info from applications and such and transform it into SPARQL, which is pushed into tracker-store).

The idea behind this is that one can develop both independent miners and plugins for the most popular applications which translate data to something Tracker can understand by implementing the TrackerMiner object.

This object also implements control logic, so the control of all available miners can be reduced to a single point, there is also a reworked tracker-status-icon which does precisely this, this is how it currently looks like:

There is also a TrackerMinerFS base class, which eases directory crawling, monitoring and other filesystem features. This is the base object for both applications and files miners.

XInput2 + GTK+
There’s much progress going on here, I empirically suck at blogging, but I promise I’ll make an update about this soon :)

Dublin Theming hackfest

So, today is officially the last day for the hackfest, it’s been a quite exciting week, with lots of discussions, ideas flowing around and crazy experiments, which had lead us to an agreement on how should a new theming infrastructure be.

People here have been working really hard to get different parts of the lower level infrastructure right, such as CSS theming, the GStylable interface, hit detection for non-rectangular shapes, etc… I’ve been mostly focusing on the replacement for GtkStyle, this is what has been achieved so far in that field:

  1. Current API limitations: GtkStyleContext would be able to store any kind of data, it’s kind of a hashtable with save/restore habilities, being able to store much more information that could be relevant to engines (goodbye detail string! goodbye widget peeping!), All drawing would be done on top of cairo, one important detail is that the engine wouldn’t get even get the widget pointer (quite necessary for third-party projects using GTK+ theming)
  2. Animations: These are focused to state changes and are mostly implicit if they affect the whole widget. The style context has to be told if they affect part of the widget though (a checkbox in a cellrenderer, for example). There can also be several animations for different state transitions running at the same time:

  3. Parallel prelight and checkbox animations running, slowed down for appreciation. Glitches will be fixed by correct properties inheritance

  4. Containers are able to pass placing information to children’s context: Think comboboxes, spinbuttons, notebook tabs, … Now painted elements can get a clue about how do they connect visually with other nearby/related elements. Of course engine implementations will be able to just ignore this information.

  5. Rounded pathbar

Nevermind the ugly rendered elements, that’s up to artists :). The code is in my github repo, in case anybody wants to give it a try.

Getting ready for the theming hackfest

There’s now just less than a week left for the theming hackfest! I’m quite looking forward for what may happen there, it looks really promising.

So, out of excitement, I’ve began dumping some ideas into a GTK+ git repo. Of course, this is just proof-of-concept code, years light from finished, etc… but hopefully it will help make further ideas start flowing :)

And now a meaningless screenshot!

(source code)

This is 9 separate boxes painted through the new API, it would allow containers to hand their children information about the placing context in a group, so the engine could know how to connect elements together. GtkStyleContext could contain any data, so it’s fully extensible.

More multitouchy fun

Sorry, nothing related to sex! Since my previous post, I didn’t actually hack much on getting MPX working, mostly blocking on the lack of ideas about a good API. That was until ImendioConf, where a talk with Richard about the subject made ideas start flowing, so here’s my late christmas present!

Besides making GDK use XInput 2.0, The main change is at the GtkWidget level, I’ve introduced GtkDeviceGroup and the “multidevice-event” signal. GtkDeviceGroup is just a container for GdkDevices, each widget can have several GtkDeviceGroups. The “multidevice-event” reports grouped motion events for the devices contained in a GtkDeviceGroup, it also provides hints about the latest event that happened and whether a device was just added,removed or updated.

The idea is, a widget would add devices to a group on button-press, enter-notify… remove them on button-release, leave-notify… and would get grouped motion events for all these devices in between. In the repo there are also a couple of test apps to show this working.

Obviously, there are quite some things left, I’ve collected missing stuff I’ve identified in the wiki, so if you feel curious about multitouch, compile xorg and help out!

And now of course, some videos of the examples :)

(Post) Guadec hack

No! this isn’t another “added tabs to $APP” [1], Instead I decided to spend my hacking time on getting MPX work with GTK+. Surprisingly, gdk is 99% prepared to handle multiple pointers, so besides adding support there for the new XInput method to handle device grabs, most of the remaining work should just be focused on making GtkWidgets deal sanely with multiple pointers (madness awaits there…). Here’s a “screencast” of my proof app, apologies for the quality, trying to record and use both the nipple thingy and the touchpad isn’t quite easy :)

GTK+ getting along with MPX

Also, I’ve spent one extra week in Istanbul and surroundings with Mitch and Pia, It has proven to be a really beautiful place full of life, returning to Turkey is a must.

The Marmara sea from Büyükada coast

[1] You caught me guys :), I was seriously concerned about your sanity for a while

Update: In case it isn’t clear, the patch isn’t really far from the “evil hack” state, I’ll just submit it to bugzilla as soon as there’s some reasonable base work in which other smaller patches can be developed incrementally.

Too far…

Almost everybody had some boss that was too exigent about how you spend your work time, but this time it got too far! This is the backtrace I’ve got from epiphany several times:

Program received signal SIGSEGV, Segmentation fault.
(gdb) bt
#0  ephy_node_db_is_immutable (db=0x854f2c0)
at /build/buildd/epiphany-browser-2.22.0/lib/ephy-node-db.c:174
#1  0x080a7ad9 in resolver_found_cb (resolver=0x8507240, interface=3,
protocol=GA_PROTOCOL_INET, name=0x951d5d8 "Richard Hult",
type=0x9229338 "_http._tcp", domain=0x9705e98 "local",
host_name=0x9166308 "Celery.local", address=0xbfa12944,
port=<value optimized out>, txt=0x883a960, flags=4, data=0x8547620)
at /build/buildd/epiphany-browser-2.22.0/src/bookmarks/ephy-bookmarks.c:979
#2  0xb7f86fbf in ga_signals_marshal_VOID__INT_ENUM_STRING_STRING_STRING_STRING_POINTER_INT_POINTER_INT () from /usr/lib/

Please, at least let me browse API docs and repositories, I promise I’ll behave!

Two new children in Cookieville

It’s been already more than 6 years since my first contribution to what then was called the ximian-setup-tools. Shortly after that, I was given the oportunity to maintain them, which has been quite rewarding during much time. But lately I’ve grown the urge to move on and hack on other things I nowadays consider more exciting or funny.

So, after 2.21.x I’ll consider both gnome-system-tools and liboobs orphan, left in the cold, crying for a new maintainer! If you want to add some emotion to your life, have these midnight wakeups because you forgot to upload that tarball or just think you had to be maintainer once in your life, please contact me!

If anyone wants to bite the bullet, I’ll of course be around to clear up doubts. Else, I think I’ll just be doing tarballs occassionally.