Gtk3 vs HTML5

The last few weeks I’ve been working on an interesting new idea, hacking out a prototype.

The code is not really clean enough for public consumption yet, and a bunch of features are missing. However, its now at the stage where it can be demoed and evaluated.

I think the best way to introduce it is via a video: (original theora file)

Basically, its a backend for Gtk+ 3 that renders in a browser.

A more techincal description for the web geeks among us:

Each toplevel window is mapped to a canvas element, and the content in the windows is updated by streaming commands over a multipart/x-mixed-replace XMLHttpRequest that uses gzip Content-Encoding to compress the data. Window data is pushed as region copies (for scrolling) and image diffs. Images are sent as data: uris of uncompressed png data.

Input is gathered via dom events and sent to the server using websockets.

Right now this is Firefox 4 only, but it could be made to work in any browser with websockets.

Now, I want to know, Is this useful?

There are two basic ways to use this, you can either run your own apps on your own server and access it from anywhere (kinda like screen). Or you can put it on a public server that spawns a new instance of the app for every user (gimp on a webpage!).

If you had this technology, what cool stuff would you do with it? What apps would you run, and how would you use them?

175 Responses to “Gtk3 vs HTML5”

  1. Now where can I grab the source code? This is freaking awsome :)

  2. This is totally awesome! We’d love this for the abiword project.

  3. Could you try abiwidget? We’d love to embed abiword in a browser for our http://abicollab.net application :-)

  4. click says:

    Dope idea!

    I’d use this to integrate native apps into Chromium OS. Users spend 80% of their time on the internet, and as web standards improve that number is only going to increase. However, native apps are still necessary, and often better than web apps at certain tasks. Yet Chrome OS is forcing users to abandon native apps. I don’t need two machines; I want a single computer capable of doing everything. Right now, that means a traditional desktop interface with a browser installed. But this is a usability problem. While the browser remains a tool of the desktop, users are forced to distinguish between two interaction models: one for native apps, and one for web apps. With this technology we can bridge that divide and unify the browser and the desktop; provide both flexible webapps, and solid, high-performance native apps. All this in a single interface, without forcing users to switch context.

  5. movax says:

    This is madness. Really good hack.

  6. Bob Mahar says:

    The better question is “What wouldn’t I do with that!” ( Havoc: I like the possibility of Inception. :-) By the time get 5 levels deep of Netscapes, you’ll need a strong sedative indeed. IF the nested instances had to be daisy chained, that is. But it would be possible for the nested instances to directly post its contribution directly to the client. This eliminates the bucket brigade, and would presumably allow any level of nesting without a performance hit. )

    I guess the limitation is interaction with the client workstation. E.g. how does one upload a file to the hypothetical copy of GIMP embedded in my browser?

    Very cool!

  7. [...] links: http://blogs.gnome.org/alexl/2010/11/23/gtk3-vs-html5/ [...]

  8. halfline says:

    next up, clutter!

  9. ebassi says:

    @alex: this is awesome

    @havoc: I fear that the universe might implode if alex tried that

  10. Really awesome! Keep up your good work!

  11. Reeks says:

    Is there a possibility of running 3D OpenGL apps in this fashion with something like GtkGLExt?

  12. rms says:

    Interesting. I wonder if I can run emacs with this..

  13. [...] Pro Tweets RT @jstedfast: Really cool demo of gtk rendering in a browser: http://blogs.gnome.org/alexl/2010/11/23/gtk3-vs-html5/ migueldeicaza – Tue 23 Nov 23:21 previous [...]

  14. jcadam says:

    It is a real magic!
    Impressive!

  15. abadidea says:

    (Hermione’s voice) Oh, but Harry, that’s bloody BRILLIANT!

    No, really, I am *impressed.* Get a beta out the door and be a genetic donor to my offspring.

  16. lucasshrew says:

    Alex,

    I don’t actually got it, how is this different from using VNC or alike from a web browser client?

  17. jyf says:

    this seems like an X apps , the browser is an X server, and the server side apps are X client :]

  18. shazzner says:

    Very cool, so could you do something like do x forwarding through a browser?

  19. Gubatron says:

    If nobody did it before then you’ve just given many the idea of creating a VLC client that renders on HTML5, it’s be just a matter of re-implementing the protocol over XMLHttpRequest and creating all the javascript rendering routines for the canvas. Sick idea. Brilliant demo

  20. nucco says:

    Man, you’re dangerous!

  21. tonyb486 says:

    Yo dawg, I heard you liked gtk, so I ……..

  22. Wesley says:

    Wow! Great! Very nice work!

  23. says:

    That is seriously cool. Lots of potential. I always thought GTK and HTML had enough similarities to do something like this.

    (Tip: Type out your comments ahead of time and just scroll down… avoids filming all the typos. :p)

  24. foo says:

    Dude, it is about fucking time.

  25. MBro says:

    The cloud just got more desktopy. That is awesome work, keep coding.

  26. pankaj says:

    awesome !!!

  27. in a different way, this has been done, for a given value of “done”, in the form of pygtkweb, as a GSoC 2007 project. an absolute basic reimplementation of gobject.py (in python), combined with an absolute basic reimplementation of gdk.py (in python) in combination with basic reimplementations of gtk widgets (in python), combined also with a graphics layer that mapped onto HTML5 primitives, was passed through the pyjs compiler.

    the end-result was that, within the GSoC time-limit, luis pamirez was able to demonstrate 17 of the helloworld gtk examples up-and-running inside a web browser, as pure javascript, with the source still being in python.

    the advantage of this approach is that no server-side infrastructure is required to render the application.

  28. murrayy says:

    Is it themeable?

  29. Ben says:

    IMHO, this has the potential to be revolutionary.

    1) It provides the network transparency!

    2) ZERO install! No X server to configure/install, no *nix dependencies.

    3) Potential for excellent network latency characteristics!

    4) Solves the problems of X without requiring the dump of X applications?

    Let me say that this excites me, as you usher in a new era of network-based computing around a fundamentally different set of assumptions.

    This is incredible!

  30. There is only xul says:

    Welcome back in 2003:
    http://www.hevanet.com/acorbin/xul/top.xul

  31. emmanuel says:

    wow, this is really crazy amazing.

    question: how much does it slow down when you don’t connect to localhost but to a remote website (it’s not so much about bandwidth i guess, but about ping time…)?

  32. Yann says:

    Would it be possible to use the javascript bindings for it?

    Then, we could build an application with javascript/GTK and use it on an application server, without compilation.

    To go further:
    This application could let the user define some input forms in javascript/GTK, which would be render within the application.
    I am thinking of an application where some users define the input forms / flows for an other group of user.

    I think it would be fantastic!

  33. Andreas Klapperich says:

    Thank you for demonstration

  34. Anders says:

    VERY NEAT!

    I’ve been thinking about doing that for a long time – glad that someone else did it ;)

  35. Pjvandehaar says:

    How hard would it be to reverse this process (i.e., to implement a display-only web browser)?

  36. Yuri says:

    Waiting for someone to do this for QT … :-P
    But seriously, yes this is useful. If I wanted to access GUI apps on my own server from behind, say, a workplace firewall this might be the only way.

  37. drago01 says:

    I remember you talking about something like this on IRC a while ago ..

    Good to see that you actually implemented it and that it works that well, good work!

  38. Andy Wingo says:

    Very neat stuff, Alex. You are indeed a dangerous man :)

    I don’t know how I would use it though. The obvious use I guess is to expose in-house GTK+ apps to clients that aren’t running X…

    Can you select text? Does the clipboard work? You didn’t show very much keyboard interaction in the video.

    Just musing :) Happy hacking!

  39. Luca Bruno says:

    @Luke Leighton this has been done with wx too, the problem with that is the reimplementation of everything

    By the way, you just proved neither desktop will defeat web nor web will defeat desktop.

  40. Bob says:

    This is insane. Gives a whole new meaning to the term “web application.” Well, it doesn’t really, in the end this is exactly the same thing. But the fact that you can now run normal desktop GTK applications as web applications (possibly without modification) is just crazy.

  41. Andy Wingo says:

    In some ways this can compete with Adobe Flex, I think — from what I remember of its use, people use it when they want web apps with a more desktop-like feel.

  42. Mohammad Sadiq says:

    This is superb! I am sure it will go a long way in reducing the amount of bytes downloaded to render an HTML page.

    Keep up the good work!

    Yes.. even I am interested to see how this can be done with QT. QT and KDE4 object layout is very well organized, so I think it should be simple.

  43. Bruno says:

    Very cool, but I want to see it do text rendering and scrolling (at least) entirely in the browser to avoid latency.

  44. Duran says:

    With VLC you are literally just getting a big picture of the screen. It captures your click co-ordinates and forwards it over the wire. With this GTK to HTML thing, the GUI front-end is actually running on your computer so there is never any graphical lag et cetera.

  45. Morten says:

    Very cool demo! Have you tried a browser-within-browser example yet? :)

    I did a similar thing for Qt, but instead of using canvas I duplicated the widget hierarchy using DOM nodes. This means you can do interesting things like creating real Html text input fields instead of sending images.

    Shameless link to own project: http://labs.qt.nokia.com/2009/09/18/qt-in-the-cloud-with-qwebclient/

  46. This is insane! Will try it today.

  47. Ignacius says:

    This fucking awesome! You know you have created a monster?? Now all people are wanna want this. I can imagine a few use cases. A software project which let you try the program before installing it. Even useful for Linux distribution makers (think of a button on Ubuntu: try this program before installing it).

    I also can think of a pastebin-like service with a compiler and/or python interpreter where you can see the output of your code on the browser. Even for not needing to install a SDK for Maemo or similar projects, where the sdk resides on a remote server, and all you need to do is upload your source code.

    Not much related, but I think this also proves the need of a Client Side Window Decorations on Gtk+ itself :-)