Web 3.8: the peace dividends release

I’m very happy about this release. Thanks to hard choices that we dared to make in the past we are now breaking new ground and giving GNOME some of the tools it needs to be the premier free software operating system. It’s been a long way since I spent an entire GUADEC porting good old epiphany to this newfangled thing called “WebKit”, and what a ride it has been.

There’s a lot to talk about, so let’s get started.

WebKit2

For the first time in this release, Web uses by default the all new multi-process architecture known as WebKit2. The advantages of such a design are well known by most, but here is a brief list of what this means for us:

  • Stability: we now run all the WebKit code in a different process (WebProcess) than the UI code (UIProcess). Most crashes, by far, happen inside WebKit, so this will make the application a lot more stable. When crashes happen, you’ll get a message explaining how to continue browsing, just a reload away. Note that for 3.8 we run all the web pages in the same WebProcess, just like Safari for OSX did in 5.1, but we plan to move to multiple WebProcesses in the near future.
  • Responsiveness: not only most crashes happen inside WebKit, but most of your CPU (and GPU!) time will generally also be spent there. By moving this processing away we allow the UI to be always ready to react to your input, so tab switching and scrolling are now super smooth. You’ll have to see it to believe it, but trust me when I say the improvement is enormous.
  • Plugins: in WebKit2 plugins run in their own process (PluginProcess). In general this makes the browser more robust against the frequent crashes and hiccups of some plugins, but for GNOME in particular it means something even better. Flash, as our more loyal users know well, has not worked natively in Web since we switched to GTK+3. The Adobe plugin is stuck in GTK+2, and since using both libraries at the same time in one process is forbidden we have been forced to rely on things like nspluginwrapper for a while. Well, not anymore: we compile our PluginProcess with GTK+2 precisely for this reason, so Flash works again out of the box.
  • Security: finally, by isolating all the web content from the application we reduce the possibility of malicious code having access to sensitive information. This will be greatly enhanced in the future with the sandboxing of the WebProcess, but 3.8 lays the first stone for a safer browsing.

Our WebKit2 port and Web backend have been more or less in continuous development for more than 2 years. This is the result of a lot of hard work by all the WebKit contributors, the Igalia WebKit team and the Web developers. Despite being a collective effort, like most free software, I want to thank Carlos García Campos in particular for leading the WebKit2GTK+ effort with so much energy. His hard work and perseverance were instrumental in making this happen, so the usual beverages or thank you notes are more than deserved next time you see him.

All this would be more than enough for a single release, but there is much more to see. Let’s keep going.

Private browsing

As advanced during the last WebKitGTK+ hackfest, Web 3.8 brings a straightforward way of launching a temporary and private browsing session, free from any data that could possibly identify you, that will leave no trace on the hard drive once you are done with it. Just go to the application menu and click “New Incognito Window”, you are all set.

private

Undo close tab

You closed a tab just to realize you actually needed to check one more thing in there. You pressed the wrong key. Selected the wrong menu item. Nevermind, in 3.8 Web will remember the tabs (and windows!) you have recently closed, and bringing them back to live is as easy as pressing Ctrl-Shift-T or selecting “Reopen Closed Tab” in the application menu.

UI polish

You want a shiny new search bar? We have it:

search

A handy new tab button in the main toolbar? Got it:

new-tab

Vastly improved Web App icons and titles? Yep:

facebook

Fancy HTML5 media controls perhaps? You bet:

after

Plus less clutter in the gear menu, improved theming, simplified Preferences dialog, and many more small UI tweaks.

3.10 and beyond

As usual we are already working on the next major release. For 3.10 and beyond you can look forward to:

  • Multiple WebProcesses. This is still under development, but at the very least we think we’ll be able to provide either the current “just one” model or the “one per tab” variant. Ideally also the “one per domain instance” that Chrome uses by default, but this one will be trickier. Stay tuned.
  • Deeper GNOME integration. Hooking into the new Shell search APIs or the Settings privacy controls is high in our TODO list.
  • Overview improvements. Animations? Access to recent history or recently closed tabs?

We think GNOME 3.8 is the best GNOME yet, and Web 3.8 is the best Web yet. We hope you’ll enjoy both, we definitely enjoyed doing our part.

Posted in Blogroll, General, webkit | 12 Comments

Theory and praxis of Netbanking

Do you know how when you get sick you suddenly cannot remember anything about your life but being sick in the past? With almost every memory gone, all your life is reduced to a series of very similar events that form a parallel reality that, as someone once said, you only get to experience for a short time. WebKitGTK+ hackfests often feel a bit like this, the difference being that this is the kind of disease that you look forward to as a child in order to skip school and stay at home playing video games. A couple dozen hackers sitting in a bright clean room, safe from the rainy weather, programming for hours on end until they have to be literally kicked out of the place. Lots of coffee. A blackboard full of tasks. Tortilla and beers, or pulpo if you are the sort of person that would eat an animal that can predict the outcome of football matches. The parallel life we live for a few days every December, in Igalia‘s Coruña headquarters.

Web Applications, improved

A lot of love went into making the Web Application experience better. We will now go out of our way to really find a good icon for the app, trying to use anything from Microsoft tiles, OGP icons, Apple Touch icons or normal favicons before falling back to the standard snapshot of the page content (which has been made nicer thank to the judicious use of rounded corners). We also try harder to find a proper title for the app, even hardcoding some popular ones that have really poor default page titles (hello, Facebook!). Add some small but much needed touches to the creation dialog itself, and we are good to go:

Incognito mode

I suspect there’s really not a lot to say about this that is not already well known, but anyway: need to launch a private instance of your browser in order to do sensitive tasks and minimize the risk of being tracked both on the internet and in your own computer? Well, now you can do it from the Web UI itself, just select the “New Incognito Window” option and you’ll be greeted with a dark themed window in a temporary profile that will be automatically deleted on exit. Your history and bookmarks will be copied (in read-only mode) to the new profile for your own convenience, but no passwords, cookies or other sensitive data will be carried over or stored. We are still working on some touches for this (a nice startup page, perhaps a more customized theme), but this is more than useful for now.

New search

A slick new UI for the search bar that matches the design of the other core applications and that does things like automatic case-sensitive search based on your input? Yes can do.

And more

And as usual, lots of other small details were fixed. Cleanups in our menus, fixes to the heuristics to decide where to open new tabs when Web is called from another application, support for exotic mouse buttons, the beginnings of an inline document view mode for PDF and the like, and lots of cleanups and nice refactorings to some of our internals.

Of course many other things happened throughout the hackfest, and I’ll be as anxious to read those blog posts with more updates as you are!

As always, a big thank you to all the sponsors and organizers that made the event possible, it’s always a pleasure to see old and new friends, hack the good hack, and live the WebKitGTK+ parallel life for a few days.

Posted in Blogroll, General | 10 Comments

On advertisement and international traveling

AdBlock, integrated

For a long time our adblock story was a bit underwhelming. We had an extension to do the job, but it was buggy and it would often either not block what it had to block or block just way too much (like, the entire page). So in the last cycle we pretty much rewrote the extension from scratch: it just works better, blocks what it has to block, and uses the adblock+ source filter files.

We just felt this was not enough though. Lots of people still found the wrong extension version, or were not aware there was an extension in the first place, so their experience was not as good as we hoped. So for 3.8 we are just going to bite the bullet, and integrate adblock into the browser itself. Last week we took the extension, merged it into Web, cleaned the the heck out of it, and added a simple entry point in our preferences to control it:

Of course there’s still some things that could be done, like a simple UI to edit the filter rules (for instance, to white-list a particular page where you do want to see ads), but for now we feel this is already a good improvement.

Boston Summit

In other news, I’m just a couple days away from jumping into a plane to attend this year’s Boston Summit. I’m looking forward to see old and new faces and continue the good discussions and work we started in GUADEC about the present and future of GNOME. On the selfish department, I hope to get hold of some lost designer to start fleshing out the 3.8 plans for Web, there’s so much to do.

Thanks to the GNOME Foundation for sponsoring my trip, and to Igalia for allowing me to attend the event!

Posted in Blogroll, General | 11 Comments

We are almost there: Web in 3.6.0

It’s that time of the year again where we have a look at what happened in the project in the last 6 months. In mere weeks it will be released with the shiny GNOME 3.6 (best.release.ever), but since release notes don’t write themselves here’s a sneak preview of all the new toys in Web-land.

The overview has landed

As Claudio already advanced the most noticeable difference in 3.6 is the first iteration of what we call “The Overview”. This is still an evolving design and implementation, but we felt that it was important to land a first version in time for this release that brought actual, solid improvements to the user experience.

What the overview currently does is easy to explain: we have changed the blank start page for a grid that holds your most visited pages. Your favorite pages are now easier to access, especially on touch devices, and we have put valuable screen real estate to work! But what about if some unwanted visitor ends up there? Fear not, if you need to you can remove them from the grid by clicking on the ‘X’ icon on the top right corner of each snapshot.

There’s more to come, but we feel this is already much better than it was, and fixes a long standing feature request for a better start page experience.

WebKit2, beta

It’s been a long time in the making, but the first bits of the WebKit2 support are already in the master branch. As we had anticipated for now it is available in beta form, which means we don’t build it by default and there’s still some functionality missing. That being said, it is really usable at this point, and doing the actual porting has helped us a lot in maturing the up and coming next generation of the WebKit framework. If you want to give it a shot, just build the browser with “–with-webkit2”, and you’ll notice:

  • Increased responsiveness (how amazing the scrolling is!) and stability.
  • Thanks to the OOP plugin support GTK 2.x plugins work out of the box again. That means Flash support without having to rely on nspluginwrapper.
  • Other things that are there, but invisible to users, like increased security or a new and improved API, built on top of all our experience with the classic WebKit.
You can look forward to having a WebKit2 powered Web by default in GNOME 3.8, but for now you are more than welcome to give us feedback about this Beta version.

Fullscreen mode, now with more full and more screen

For a long time Web’s fullscreen mode was somewhat awkward. You’d still get a toolbar, so it wasn’t really fullscreen, plus it would have a strange button embedded in it telling you how to go back to the safety of the vanilla mode. Probably this was fine for the time, but with more and more HTML5 games or presentations done inside browsers, it was about time we had way of making your browser show you the full content and nothing but the content. So we just did that:

Unit testing and code coverage

This is not sexy, but I think it’s really important. For a long time our test coverage was really poor. And when I say really poor, I mean completely non-existent. Some time ago we started to add a few tests here and there, but starting in this cycle we have taken this a lot more seriously and our test coverage has grown by leaps and bounds. We have unit tests for most major classes, they are executed automatically during distcheck (so you won’t release if you broke something), and lately we have added support for the code coverage infrastructure that is now available in gnome-common (thanks Philip!). As I write these lines our coverage is around 40%, which is not great but is a massive improvement considering where we were 6 months ago. These tests have helped us to catch a bunch of bugs (some really obscure and older than some good wines), prevented lots of regressions, and helped us to do some hairy refactorings with the confidence that we wouldn’t break absolutely everything.

lib/egg, you are an embarrassment

I think these three things (having unit tests, running them automatically before release, measuring your progress with code coverage) are very important, so I’m writing another post that just focuses on how to do this for a typical GNOME project.

Other bits and pieces

As usual there are lots of additional features and bugfixes, too numerous to mention: automatic session recovery, tab-less mode, Do Not Track, using SoupTLD to make the URL completion smarter (so it can automatically figure out that google.com is a URL, but foo.bar is a string you want to search), and many more.

Onwards GNOME 3.8

But the show must go on, and Igalians never sleep. We are already working towards the next major release: in 6 months we expect to graduate the WebKit2 version of Web out of Beta status (deprecating the classic WebKit version) and land the next iteration of the overview as major new features. But those are just the known knowns, the unknown unknowns is where it’s at. Until then, happy hacking.

Posted in Blogroll, General, webkit | 26 Comments

Next on GUADEC

GUADEC has been going on for a few days already, but it is far from over. This is a short post to make some noise about some upcoming BoFs & hackfests that might not receive as much publicity as the rest of the stuff, being outside of the “talks” track.

GNOME as an OS

As announced in our “A bright future for GNOME” talk Juanjo and myself will chair a BoF to talk about the future of GNOME. If you want to get things done and want our project to keep kicking ass for another 15 years (at least!) please come and talk with us next Monday 30th from 10AM, Room 2.1a. We intend to make this BoF productive, constructive and as little bikeshed-y as possible, your cooperation will be welcome!

Epiphany & WebKit hackfest

The might Epiphany and WebKit GNOME hackers will meet again in Room 2.1a on Tuesday 31st to hack, hack and hack. I have already committed to move the all-new Overview forward with Claudio, review the amazing new documentation submitted by Pierre-Yives Luiten, work with William on Data Sync or keep going on recently started secret projects. Some people have already approached me asking for various Ephy-related things (bug triaging, random bugs they want to see fixed), be there if you want to help!

Posted in Blogroll, General | 1 Comment

Do Not Track support in Epiphany

Twitter’s last Privacy Policy Update helpfully informs all users that they do now support the Do Not Track (DNT) browser setting, which aims to stop the collection of information at the user’s request (a collection which Twitter is actively engaged into).

Spurred by all this I sat down and added DNT support in Epiphany, which thankfully is an extremely simple spec to implement. It’s now in master, so anyone willing to enable just needs to go to the Privacy tab in Preferences and click:

Now the pages that choose to respect this setting (unfortunately not everyone does; by a long shot), should be able to detect your request. We can see that things are working in the donottrack.us page itself.

Note that the page claims our browser does not support the feature, yet it is enabled; this is because DNT being an HTTP header extension the only way for the page to tell you whether your browser supports it in theory is by having a hardcoded list of supported browsers, which does not include Epiphany. Oh well. Either way, enjoy your newly untraceable goodness, which should make its way into the next unstable release.

Posted in Blogroll, General, webkit | 8 Comments

Japan Freedom Hackers: Assemble!

Turns out I’ll get to spend the next two weeks in Tokyo, starting next Sunday. It will be third time I visit this weird and fascinating place, but I’m still excited to be there again.

Some time ago, in another trip, I proposed anyone who might be reading me to meet up and talk about all things GNOME or WebKit. Turns out I met some interesting people that way (hi everyone from Caixa Mágica!), so let’s try again: if you are reading this, are in Tokyo, and would like me to talk to your friends/colleagues/whatever about GNOME or WebKit I’d be happy to do so. We can also improvise a hackfest or anything else we can come up with. In exchange I only ask of you to show me around (always better with a local) and an unwavering commitment to freedom and justice.

Drop me a line at xan AT gnome DOT org, or leave a comment in this space.

Posted in Blogroll, General, webkit | Leave a comment

Summer of Code, Web 2012 vintage

This year’s Summer of Code has already started, and Epiphany has been lucky enough to get two students assigned. Let’s see who they are and what they’ll be working on.

William Ting, Data Synchronization

William is a last year student of Computer Science at the University of Texas, Austin. He’ll be helping to alleviate a paradigmatic first world problem: I use Epiphany from so many computers that all my data is scattered around and I suffer a permanent pseudo-memory loss condition.

The battle plan is easy. We’ll reuse Firefox’s excellent design for a data sync protocol plus their free-to-use servers (we assume that’s what they were hoping for!), and will integrate the feature right into our browser. Since all the specs and implementation are open GNOME could host in the future a sync.gnome.org instance, but for now we leave that out of the scope of our summer project. Hopefully by 3.6 you’ll be able to optionally enable this functionality, cruise through the web from your tablet, bookmark that hilarious XKCD comic strip, and have it show up in your good old laptop just like that.

I’ll be mentoring this project myself, which was initially proposed by Igalia’s very own Joanmarie Diggs.

Yann Soubeyrand, Anti-phishing Support

Yann is a first year student at the École Nationale Supérieure d’Informatique et Mathématiques Appliquées de Grenoble. His task will be to solve a complicated problem that Epiphany has suffered for a long time: for most users the information provided in the URL entry is not enough to judge whether the page they are visiting is safe. The SSL or certification data is useless for most people, and by showing scary warnings about things they don’t understand at best you’ll train them to just click through to get to the content. To make things worse, most of the time those warnings do not actually indicate someone is trying to scam you, just that apparently setting up web servers correctly is difficult. So all in all, while useful, the information we currently show in the UI is really not that great for the 99% out there.

With this in mind, we’ll try to do the following: using the Google Safebrowsing APIs we’ll try to request authoritative information about the potential “phishiness” of the pages you visit. If we get a warning through this channel we can be confident about there being a security threat, so we’ll show a big, clear message on top of the web content. No jargon about outdated certs or VeriSign trying to take over the planet, just “Listen, our best people tell us this page is almost certainly not safe. Let’s not go there.”. We think this will significantly increase the safety of the browser without violating the user’s privacy, since Google’s API do not require you to disclose the pages you are visiting to validate them (magic? no, science).

Sergio Villar will be mentoring this project.

That’s it, let’s roll

That’s it for now. We’ll make sure to keep you updated about these and other developments in GNOME’s very own web browser. Thanks to GNOME for choosing our proposals, to Google for sponsoring Summer of Code again, and of course to Igalia for its continued support for GNOME and for allowing us to spend our time mentoring the fearless next wave. Happy hacking!

Posted in Blogroll, General | 2 Comments

Web. It’s what’s for dinner.

GNOME 3.4 is around the corner, and with it a new version of its little web browser that could. This is for sure one of the most action packed releases in a long time, so let’s do a recap of the last 6 months of development.

New UI


The most obvious change at first glance is the refreshed UI, which was already covered in some detail in this blog post. Thanks to the hard work of the GNOME Design Team, the GNOME platform hackers and the Epiphany team we’ve got ourselves a completely new toolbar layout with elegant widgets (I particularly like the linked-style back/forward buttons), support for the new Shell application menu, the demise of our menubar and the debut of our ‘super-menu’ holding the less frequently used page specific actions. Not only the browser works better than ever, it now looks better than ever too.

New History

We have been talking about rewriting our history backend for many years now; to give just one example, as far back as 2007 we already had a Summer of Code project to try to fix some of its shortcomings, which were many. The old backend served us well for almost a decade, but it was showing its age: a difficult to understand legacy codebase, a bus factor of 1 and poorly scalable architecture that made storing more than a couple of weeks of browsing history a titanic task. Last year a crack team at Igalia was put together to fix this once and for all. Enter the new age:

  • Drop our XML based storage in favor of good ol’ SQLite. Many options were considered other than SQLite, and we could yet again change how things are done internally in the short term, but for the first step we decided to settle on a simple, trusted and performing technology.
  • Never block the UI. All actual history accesses are done in a special service thread which communicates its results when they are ready to the user interface. This way things will remain responsive and snappy, no matter how complex your queries are or how full to the brim your history is of non-stop reddit browsing.
  • Test it! User data is the most important thing the application handles, and losing your browsing history is the modern day lobotomy. Our new backend is thoroughly tested through unit tests, so your data is in good hands with us.

Thanks to the new backend we can now provide infinite history storage and instantaneous non-blocking search both in the URL entry and the history window, both long awaited features. Other than that, for 3.4 we tried to keep the UI as it was: one thing at a time. For 3.6, though, now that the we are beautiful on the inside, expect some surprises in how you interact with your history data.

New WebKit

As with every release, on time, we ship with the all new WebKitGTK+ 1.8. As usual there are way too many thing to list and they deserves their own blog post, but you can look forward to: the debut of the WebKit2 API (still experimental but with some modules already using it, like devhelp), WebGL, Accelerated Compositing, HTML5 History API, support for the last version of WebSockets, WebAudio, a rewritten favicon database class and loads and loads of bugfixes and improvements.

Less is more

A constant thought in my mind as a module maintainer is to focus our efforts in delivering the best experience we can given our available resources. I believe as a project, both GNOME and Epiphany, we are now facing the difficult choices successful software must go through at some point: stop trying to be everything for everybody, decide what you want to do and for whom, and try to do that really well.

Our new ongoing redesign is a great step in that direction, and I’m glad that we are finally focusing on what I think really matters. I believe 3.4 is our best release ever, with both cool new features and fixes for old major deficiencies, and things will only get better from here. And, a favorite pet peeve of mine, we did all that while massively cleaning up our codebase to make it cleaner and more hackable, a task without much glamour but a big payoff. Our last release, ignoring translations, icons and help files, comes with 214 modified files, 14,959 insertions and 24,341 deletions. For those keeping the score at home, that’s almost 10,000 lines less of code to maintain, read, patch and load!

Thanks!

Thanks to all the GNOME contributors that made possible this release, but a special thanks must go to Igalia for its continued support for Epiphany. We are not only the best WebKit consultancy around, but we are also putting our money where our mouth is by supporting web technologies in GNOME through its browser and beyond.

Work on 3.6 is already underway, so expect a lot more from your favorite webkittens 6 months from now. Until then, you can follow us on IRC, our mailing list, identi.ca, twitter, facebook or our new project page (wow!). Happy hacking!

Posted in Blogroll, General, webkit | 22 Comments

Epiphany marches on

Previously in this space we saw how the bright future of Epiphany looked like, and vague promises about incremental steps towards it were done. A month later, Epiphany 3.3.4 is out there, so let’s see how well we’ve done.

There’s a lot of new stuff here, so let’s go step by step.

Application menu

The application menu, accessible from its usual location in the Shell, holds actions that affect the entire application as opposed to the currently focused window or tab. You’ll need a fairly recent version of the Shell and gnome-settings-daemon (3.3.4 of both should do, when they are out) to get it working, otherwise the browser will fallback to a lonely “Application” entry in a now deserted menubar.

Also, notice that we now brand ourselves as “Web” in all user visible strings.

New toolbar

The bulk of the changes are here. As you can see the Back and Forward buttons have been visually merged, a fate shared by the location entry and the reload/stop button. The entire menubar is gone, being replaced by a “super menu” triggered by the funny looking button with a gear (more on this later). Everything else that used to be in the default toolbar layout is now gone, as is the ability to edit its contents, making the concept of a default layout more dramatic. Finally, we use a new style for the toolbar, making it seamlessly merge with the window decoration. We think it looks great!

Super menu

In the quest to save as much vertical space as possible in the default layout we have moved all the remaining actions of our menubar into a side “super menu”. Here will live actions related to the current page, although for the moment we have some visitors there en route to their new destination (like the Bookmarks menu, which will live in the new Overview).

The devil is in the details

A lot of other small tweaks and cleanups have happened, too many to mention. From a renewed floating statusbar (now shared with Nautilus), to spacing tweaks, to more thorough use of symbolic icons throughout the UI. Special thanks go to the Design Team, it’s a pleasure to work with them in both the small details and in the big picture re-designs.

Also, one benefit of having a renewed design focus is that it allows you to do this:

135 files changed, 14988 insertions(+), 26958 deletions(-)

Around 12,000 lines of code have been deleted since 3.3.2; the biggest chunk comes from the demise of EphyToolbarEditor and friends, but in other places we have just managed to do the same, or more, with less. This means more energy devoted to make Epiphany really good at what it should be doing, which is what every core GNOME application should aspire to do.

More to come

This is only the beginning, not the end. The Epiphany team will now continue full steam ahead to implement the new Overview, merge the new SQLite history backend, port our extension system to libpeas and many other exciting features, maybe including some surprise gift in the Web Application camp. Stay tuned to this space and, as usual, happy hacking!

Posted in Blogroll, General, webkit | 56 Comments