Desktop data/settings replication

In the last UDS, there were some talks about UbuntuOne, the technologies it uses, and how it could be well integrated into the Desktop. Also, there were discussions about how it could be integrated painlessly into upstream projects. So, here’s an idea on how this could be done.

First, it must be said that the easiest (and quickest) way of achieving UbuntuOne integration in Ubuntu would be to just patch/extend applications so that they supported accessing the UbuntuOne server, and have Ubuntu packages use that as default for users with UbuntuOne accounts. That would make most Ubuntu users happy, but it would not benefit at all users of other distributions, and worst, the upstream projects.

Now, if we look at the technologies being used in UbuntuOne, there is one awesome thing, called CouchDB, a project supported by the Apache Foundation, which provides databases (of JSON documents) that can be replicated (and 2-way synchonized) to other hosts. So, what if we had Linux Desktop applications use this for storage of files and settings?

couchdb-in-the-desktop

Well, what would happen is that we’d gain data / settings replication and synchronization for free. And also, if we could come up with standard formats / locations for common information (accounts, notes, mails, calendars, etc, etc), we’d also gain a shared storage for all applications to use, solving the problem of incompatible formats / locations used by similar free software applications.

And other advantages:

  • CouchDB knows already how to deal with conflicts, as this is included in the automatic replication / syncing features it provides.
  • While normal documents in CouchDB are JSON, you can attach any kind of file to any JSON document (even to empty JSON documents), so any kind of files can be stored. Also, it allows users to create as many databases as needed, so storage for different needs can be easily separated.
  • CouchDB provides a sort of revision history, so it could be used for nice stuff like Zeitgeist.
  • This, not being an Ubuntu-only solution, could benefit every Linux Desktop user.
  • UbuntuOne would be a service built on top of this that users can subscribe to. But others could just setup a CouchDB server on their home / company network and use that by just pointing their local CouchDB to their remote CouchDB replication server.

To continue my investigations/playing on this, I’m going to try writing a gvfs backend to manage files in the CouchDB instances. Once that’s done, applications could start just writing their files to couchdb://… URIs instead of file://… ones and enter the replication/synchronization world with just a single change. Next, a GConf/d-conf backend could be added for replicating/sync’ing settings, and so on.

14 thoughts on “Desktop data/settings replication”

  1. How does CouchDB handle authentication? It doesn’t have openid support on its own does it?

    So, you know what would be fun… getting the Tomboy/Snowy developers to take a good hard look at CouchDB. Do they feel like it solves their sync scenario adequately?

    Are you going to be going to GUADEC to talk about CouchDB?

    -jef

  2. I look forward to seeing your results.

    Make sure to test with syncing large files (300 MB +). I would hate to see something break because it tried to put a 1 GB json object into a string (think iso files), and ran out of ram.

  3. Isn’t this what conduit is for??

    Seems like a conduit provider for ubuntuone could do this and more?

    Plus then we could sync settings to a phone / usb drive …

  4. Don’t forget installed apps. Take apt’s list of installed applications and optionally add or remove them from different machines.

  5. ..because that’s the only thing Windows and OS X can’t do with such a service, because it requires a package manager and software repositories.

  7. I am interested by CouchDB for a few projects, but I am wondering about the footprint.
    From what I understand you need to run a CouchDB server on your machine (no embedded version) and communicate with it via an HTTP interface. This does not sound very lightweight. Have you made any measurement on this?

  9. Thanks for the post! Very interesting. Man, it was so good to see you at UDS. Hope the time frames between our next real life meeting are not 2 years again! (-:

  11. Pingback: couchdb-glib 0.1 « Rodrigo Moya
  14. Pingback: CouchDB contacts in Evolution « Rodrigo Moya

Leave a Reply

Your email address will not be published. Required fields are marked *