So, for the first time ever (to my knowledge), the full multi-touch stack working on Linux:
This video features:
- Linux 2.6.31 with the N-Trig driver (Plus some modifications of mine to have multitouch enabled with my touchscreen firmware)
- Xorg 7.5
- xf86-input-evdev-multitouch (By the ENAC people, plus some patch of mine to cope with devices that don’t provide tracking IDs for each tracking point)
- GTK+ from the xi2 branch
- Eye of GNOME (plus a small patch to handle multidevice events)
- Valen, the turtle
In my opinion, this is a quite important milestone, which reflects much work done lately in this camp, and from now on things can only get better!
And of course,
Isn’t a lot of this patented by Apple? 🙁
Ooh, very nice. Just need some good multi-touch hardware now… hmm, wonder how long it’ll take before someone has Linux running on Apple’s new toy 🙂
What’s valen the turtle =P
I love it! It makes me want to hack a multitouch panel into my netbook…
What hardware are you running that on, anyway?
Humans have been using multiple fingers to do things for..
quite a long time.
If Apple does indeed hold patents on such basic concepts as multi-finger interaction, then I count myself lucky that the gesture I have for them only really requires one.
If Ubuntu or Android reaches Just Works status on the iPad, I might actually consider getting one.
@Ernst “Isn’t a lot of this patented by Apple?”
That is Apple Marketing, don’t believe everything they tell. If you are interested in the facts, what is patented and what not have a look at this articles:
http://www.engadget.com/2009/01/28/apple-vs-palm-the-in-depth-analysis/
http://www.osnews.com/story/20876/Apple_vs_Palm_the_In-Depth_Analysis
http://www.osnews.com/story/20861/Apple_Granted_Patent_on_Multitouch_Gestures_More
@Carlos
Nice work. I can’t wait to see all these thing upstream.
Hi, for those who want to experiment with multi-touch and lack specific hardware, you could try with multiple mice, see also here:
http://ao2.it/en/blog/2010/01/19/poor-mans-multi-touch-using-multiple-mice-xorg
Not the same thing of a real multi-touch screen, but this is a quick way to test if software has at least basic support for multi-pointer/multi-touch interaction.
Regards,
Antonio
@Dmitrij: See the photo I’m manipulating in EOG 🙂
@Rob Sharp: This is an HP Touchsmart Tx2 with integrated N-Trig touchscreen, a reasonably cheap computer with MT technology, although actually I’m not that impressed by the hardware, hopefully things will be improving with further firmware versions
wow, very nice!!!
Hi Carlos,
Is the lag in the demo related to software or hardware (bad quality touch screen)?
Oh, and regarding patents, I would encourage people to ignore the software ones, since for a start they aren’t valid in the EU and we can argue for prior art where they are applicable. Regarding the hardware patents related to touchscreen technology, those are responsibility of the hardware manufacturers. And in fact, Apple does have a sort-of-monopoly on captative touch panels in the sense that their panels are light-years ahead regarding touch accuracy and precision.
@Jan, yes, it’s related to the hardware, it actually produces lots of nose for each screen contact, making pointers jump around below the fingertip, so the sensitivity was reduced in the Xorg input driver, I hope these issues are fixed in further N-Trig firmware versions.
And quite I agree, there’s much prior art in MT to just avoid doing anything related 🙂
Hi really great. Keep up your good work.
Now we just need kinetic scrolling built in to GTK+’s scroll containers 🙂
This is awesome stuff! Can’t wait for xinput 2 to land on my stable system so I can start hacking on it. (There are lots of little goodies I think should be pushed into Gnome asap).
With regards to patent fears, my personal approach would be to not use gestures; do it procedurally with a verlet cage physics simulation, where each touch event is just applying a force to the nearest point. (Should automatically produce beautifully smooth scaling / rotating stuff). That’s probably just because I over-think things.
How does the multitouch screen driver talk to XInput? Does it create a new pointer for each and every touch event, or is there something else going on? Can clients get more information, such as a picture of the finger / whatever it is that is affecting each pointer?
(And what kind of touch screen is it? Diffused Illumination, where it sees a detailed picture of whatever is on there; or FTIR, where it specifically sees fingers; or something else?).
Here’s a related goody for you: http://2dboy.com/2010/01/29/fingers-full-of-goo/
Awesome work!
Is there a way using a webcam to get multitouch to work in linux ona a computer that does not support multitouch?? I know there is for windows 7.
Sweet! 🙂
WOW, awesome! Well done!
Garnacho, you kick ass! Keep up the awesome work!
@Dylan, yeah, there are indeed plenty of things to do to make apps feel just good with MT :). Yes, ATM the X11 driver just creates a fixed number of subdevices, which later are assigned to a finger, these devices also appear in xinput –list, and can be assigned to a master pointer. Clients get to see each finger as a separate device, there are plans in the future to add blob events to X11, so clients would get to know the shape of the touch points. This device provides touchpoint rectangle length/width and orientation for sure. I think it uses capacitive technology, there’s not much info actually about N-Trig’s DuoSense.
@Aaron, there is I think, but might not be what you expect. There are tutorials to build a multitouch pad with a webcam, and using touchlib, I think there’s also a X11 driver in development, see http://www.youtube.com/watch?v=pQpr3W-YmcQ&feature=fvw
@Jono, @iFrade, @Code, thanks 🙂
I left a comment on the digg page of this article. But since I see more comments here, I’ll just post the link here too.
Actually, I think the first working multi touch stack on any desktop was on Linux. It was/is called MPX. It worked well before Microsoft’s “big ass table”… I forget its real name. Apple has only now come out with multi touch on a large(r) screen with it’s very absorbing iPad experience.
Check out this video on YouTube posted back in July 2007. Full multi touch working with full use of touch, keyboard and mouse simultaneously and even working for multiple users on the same interface. What I find very cool is that when the two people are painting in the same window, changing the color for one user does not affect the other user. Pretty awesome!
http://www.youtube.com/watch?v=Co3FONI6kHU
Remember, you saw it first on Linux 😉
Really cool carlos!!!
Interfaces with multi touch support will change the way we interact with all kinds of computers.
@shane, Well, MPX is a really important piece here, and there’s indeed much to thank to Peter Hutterer, but it’s not the full stack, back in 2007 there was no multitouch interface in the Linux kernel, and most applications don’t use the X APIs directly anyway.
What I mean with “full stack” is getting MT events right from the kernel, into Xorg (7.5 features MPX/Xinput2), down to a mainstream graphical toolkit (GTK+ in this case) and to a multitouch-enabled application.
Nice video !
The responce of your actions seems to be excellent !
Looking forward to see multitouch getting mature in Linux.
As somebody who purchased one of the first tx2 tablets, I’m really glad to see the pieces all coming together. I’ve been trying to test this on my machine, but so far I can’t get any touches (with stylus or fingers) to register. I know this isn’t the best avenue for support, but since this touches so many softwares I don’t know where to start.
What I’ve done: (this is under arch linux)
– Installed the new driver/firmware under windows 7.
– built and installed the evdev-multitouch driver linked above (with the linked patch)
– built and installed the xi2 branch of gtk2+ (including updating atk)
– patched and then built the hid-ntrig.ko driver (using make M=drivers/hid) and then copied this new module over the old one
– removed any linuxwacom stuff I had around
– moved my xorg.conf to xorg.conf.bak
– ran depmod and rebooted
Interestingly enough, my Xorg.0.log shows the device being recognized as an N-trig DuoSense, complete with “(type: MULTITOUCHSCREEN)” but still no response from any applications (including a patched eye of gnome)
Is there anything else you might have done to make events get recognized and used?
Posted this on the site that I belong to. This is a huge step in the right direction for ppl to migrate to Linux instead of Windows.
Well Done Carlos!!!!
Are you using the 2.59 firmware (the four finger touch version) from N-trig? I was trying out your source with it and found that the cursor jumps around a lot with that version. Did you experience this also?
@stephen, you might also need Xorg 7.5 (I think only F12 ships it at this moment, Lucid also ships it I think), having multitouchd from http://www.lii-enac.fr/en/projects/shareit/xorg.html eases testing by creating pointers for each touchpoint
@ayuthia, yes, I have that firmware installed and also experience this, there’s a hack to minimize it a bit in my evdev-multitouch patch, but it seems to me a firmware issue, someday I’ll have to try out the latest firmware, but I only read bad experiences about it on the internet.