The last few weeks I’ve been working on an interesting new idea, hacking out a prototype.
The code is not really clean enough for public consumption yet, and a bunch of features are missing. However, its now at the stage where it can be demoed and evaluated.
I think the best way to introduce it is via a video: (original theora file)
[vimeo width=”763″ height=”512″]http://vimeo.com/17132064[/vimeo]
Basically, its a backend for Gtk+ 3 that renders in a browser.
A more techincal description for the web geeks among us:
Each toplevel window is mapped to a canvas element, and the content in the windows is updated by streaming commands over a multipart/x-mixed-replace XMLHttpRequest that uses gzip Content-Encoding to compress the data. Window data is pushed as region copies (for scrolling) and image diffs. Images are sent as data: uris of uncompressed png data.
Input is gathered via dom events and sent to the server using websockets.
Right now this is Firefox 4 only, but it could be made to work in any browser with websockets.
Now, I want to know, Is this useful?
There are two basic ways to use this, you can either run your own apps on your own server and access it from anywhere (kinda like screen). Or you can put it on a public server that spawns a new instance of the app for every user (gimp on a webpage!).
If you had this technology, what cool stuff would you do with it? What apps would you run, and how would you use them?
wow! how awesome is that?
Pretty Awesome idea, let me know when the code is out
This have got to be one of the most awesome things in recent times!
Stunning! I’m a KDE fan but i like GNOME technologies too and this stuff is impressive. Good job!
Fun hack!
You forgot a picture of Xibit with a caption that reads “Yo dawg we heard you like toolkits so we put a toolkit in your browser so you can desktop while you browse!”. 😉
http://2.bp.blogspot.com/_xst6sdr7yAw/SKrgq3jWB2I/AAAAAAAAAPI/hUyiGj7qko4/s1600-h/xibit+cars.jpg
Awesome, I’ve thought something like a theme engine that outputs to HTML + js would be good, but was never going to implement it.
Interesting approach with the pngs, it I wonder if some things could use their web equivilents, maybe with svg you’d get a lot, like gradients for instance.
Interesting, considering some of the recent discussion over Wayland, and potential substitutes for X network transparency. If this was possible in just a few weeks, you’ve just demonstrated that adding remote-display to the toolkit itself is more feasible than I thought.
Although that said, can this handle (or be extended to handle) more complicated cases, like embedding a Clutter widget, or something like WebKitGTK?
How much bandwidth does this consume?
Reminds me of Qt for Google Native Client 🙂
http://labs.qt.nokia.com/2010/06/25/qt-for-google-native-client-preview/
That is awesome!
If you enable GL in Firefox, you’ve got a GPU-accelerated compositing window manager in there! 🙂
Please let us know what Web features (or browser implementation improvements) you’d need to make this even better!
Checkout the Wayland mailing list. One of the problems with Wayland is that you lose X network transparency. I’ve made some comments on the list the pretty much advocating what you are doing as the solution of network transparency under Wayland.
The trick to this is that you would always use it, even when running locally. That makes the apps network transparent. In this model Webkit basically replaces GTK3 as the app toolkit. Webkit runs like it does on embedded systems – no toolkit and everything draws with GL.
The stack looks like this:
app -> HTML5 -> webkit -> GL -> buffer
buffer -> Wayland -> screen
The world’s biggest network transparent app is the Internet. Slowly converting existing GTK apps to HTML5 will be great for Linux and cloud computing.
What immediately comes to mind is things like Mozilla’s Bespin, where they concocted a completely new text renderer in Canvas to build a browser-based code editor. With this, that kind of wheel-reinvention would be unnecessary, since you could just use gEdit. On the other hand, I suspect that the Bespin approach would be more performant, since it’s all client-side…
Ha! 🙂 How latency-sensitive is it?
One thing that would be cool is a GTK backend that draws to a PPAPI[1] surface instead of playing image games. It would even nicer if it could be compiled as a nativeclient[2] program to be safely embedded into the browser. In fact, Qt has a demo of this[3].
[1] http://code.google.com/p/ppapi/
[2] http://code.google.com/p/nativeclient/
[3] http://labs.qt.nokia.com/2010/06/25/qt-for-google-native-client-preview/
This is *very* useful. I want to deploy a gtk app to end-users in a trivial way on multiple operating systems and so far only nomachine’s NX is sort-of a solution.
this is awesome! where do i get the source code?
Yes, it’s useful!!! The best remote desktop solution I can think of.
Good work.
Woah! Pretty freaking amazing work! I am blown away
Alex, obviously you have to go recursive by running Firefox on this GTK port … there’s just no escaping that.
So my friend & I just watched this video – and he came to an awesome realization: You can revolutionize purchasing applications.
Imagine you want to try photoshop (and it was recoded in Gtk+3… lol). Instead of downloading a huge trial that’ll just delete itself in a week, why not demo it [b]in the user’s browser[/b]?!?
Please: release the source code, no matter if it’s a mess.
Very impressive. Nice work!
Pushing image diffs and region copies… this sounds like you implemented the X protocol over HTML5. Why make it GTK specific? I would think a VNC -> HTML5 proxy would be much more useful.
In either case an impressive demo, you should run gtk webkit inside this just for kicks.
this is very closely related to the pyjamas project:
http://pyjs.org
providing not only a general purpose python-to-javacript translator, but also runtime bindings to the MSHTML, Gecko, or Webkit engines.
i believe the pyGTK2 API was ported as well, so that apps written for GTK2 could run natively under pyjamas — additionally, though probably not still functional, i believe a window manager was even built that used the bindings from pyjamas-desktop.
Sockets.io would be a useful library to use, then even IE could get in on the fun!
http://socket.io/
“Now, I want to know, Is this useful?”
Yes. This is extremely useful for getting apps written for the desktop to run on the browser for remote access and virtualization.
I don’t know if there are any companies with GTK3 desktop apps that need this right now, but for the past six months I’ve been working on doing exactly the same thing for JazzScheme (http://jazzscheme.org/). JazzScheme has its own widget set and UI library that runs on top of Cairo. Instead of sending bitmap data across I’m providing a thin indirection layer on top of Cairo canvas and HTML 5 canvas (they’re almost identical in terms of drawing commands).
We’re using it for a pretty involved business app that needs a desktop version. Doing a web version any other way would have been a complete nightmare.
lol, we should just replace most of Tomboy Online with this 😛
The video never showed him typing into the demo, only into the gEdit window he had for saying what he was doing. Was that because it does not work? Or just something he forgot to do?
Otherwise totally awesome, though I can’t see how I would use it.
Which word is more awesome than awesome for this 🙂
impressive!
This has got to be one of the most amazing hacks I have seen all year.
This is absolutely brilliant Alex, congrats on one well executed hack!
Once the shock of this amazing hack started to settle, I could finally get my thoughts in order.
This is absolutely brilliant. It would let me run MonoDevelop over the Web, or export MonoDevelop to an iPad, or any other device.
Alex, this is magic at a whole new level. I can not wait to see this in the wild.
This actually has pretty substantial implications: during the whole flash vs. HTML5 debate, I’ve always argued that while Flash is already quite a complete programming language, HTML5 will behave more like modern operating systems with, “toolkits” built on top to aid the creation of genuine web-apps (think things like drag-‘n’-drop, keyboard input management, AJAX-y stuff etc., all in one toolkit). I never thought the first, “web toolkit” would be a full-on desktop toolkit that just displays in the browser.
Really, I think this should be pushed as much and as quickly as possible. You’ve not just demonstrated that HTML5 could replace Flash, you’ve pretty much done it code-wise, and even better, due to GTK+’s nature, you also provide webapp makers the option of making a desktop app available.
Very impressive! Can’t wait for the source!
My system does not like that .ogg file. MPlayer and totem both error out about 30s in. Lot’s of “Invalid frame duration value” messages.
That’s what Spice does to you I guess?
I bow to you, sir.
And next time, use a different color scheme or theme so I don’t confuse what parts run inside the browser and what parts don’t. 😉
This is the future of remote access. Instead of NX, VNC, or X11, pull up a browser and go. I can see great use when developing “Software as a service” as well.
That’s awesome. I’ve been expecting something like this to come out eventually, but did not expect it so soon. I didn’t expect the server-based approach, so I didn’t anticipate anything being born so full-baked in a single sitting 🙂 I kind of expected it to be somehow related to the whole ‘gnome 3 is javascript’ movement 🙂
Very very awesome. I can def. imagine using this, though I’m not sure exactly how yet. Looking forward to the source.
This looks awesome!
I think it will be awesome to use this to run Thunderbird in a browser. This way, you get the full featured e-mail client that you love, but you can have your data accessible from anywhere, and best of all, it will be hosted on your own server.
Obviously useful for RIAs!
This would be a nice replacement/addition for screenshots and screencasts. Very exciting work.
I’d love to use it on AbiCollab.net, so users don’t have to have AbiWord installed locally to edit their documents.
gtk3 vs html5 …
gtk3 winner, just because is there right now.
Great job is very important to firefox community something like this.
btw, is real ?