Tracker: Direct Access branch merged in master

A while back we had this bug from Bastien: 613255 – “Read-only, non-DBus, store access”. For the past 5 or 6 weeks we have been working on this. Initially the idea was just to do direct access, but once we got started, we realised that the libtracker-client API wasn’t really good enough and we would like to extend it. But we didn’t want the old API there either, so we came up with this new library to supersede libtracker-client. For now we package both, but all functions in libtracker-client are marked as deprecated at this point.

So what do we have now? Fundamentally we have ONE API for different backends using different technologies. To summarise:

1. D-Bus (libtracker-bus, backend)
2. Direct Access (libtracker-direct, backend)

D-Bus – Read/Write Access
Depending on the version, we either use FD passing (requires > 1.3.1) to avoid copious memory copies OR we use D-Bus glib marshalling which represents the worst performance you can get from Tracker (though it is still usable).

Direct Access – Read Only Access
This is based on a library we had internally in Tracker called libtracker-data. We merged some things to make this happen (like libtracker-db) but generally, we sit on top of this library in libtracker-direct.

Plugins?
The backends are dynamically loaded at run time depending on the client’s needs (i.e. if you only ever do SELECT type queries, you’ll use the direct-access backend).

How does the API look for libtracker-sparql?
The idea here was to facilitate all the old API needs and some new ones. What we wanted was less API bloat and to incorporate some of the things we had in the code base all spread out in multiple libraries into this libtracker-sparql. These things include:

• Connections – used in libtracker-client, we wanted some common way to get a connection to Tracker regardless of what backend was in use.
• Cursors – used in libtracker-db and wanted as a public API for some time, but not possible without WAL (Write Ahead Logging) in SQLite 3.7. Now we share the same API internally and externally.
• Builder – used in tracker-extract for building SPARQL queries for selecting/inserting data.
• Utilities – used in tracker-extract, the miners, etc. for escaping text used in SPARQL queries and some other common functionality.

Example
So this is what you might expect with the new API:

TrackerSparqlConnection *connection;
GError *error = NULL;
const gchar *query = "SELECT ?class WHERE { ?class tracker:writeback true }";

connection = tracker_sparql_connection_get (&error);

if (!connection) {
	g_printerr ("%s: %s\n", _("Could not establish a connection to Tracker"), error ? error->message : _("No error given"));
	g_clear_error (&error);
	return;
}

/* The NULL below is the GCancellable */
cursor = tracker_sparql_connection_query (connection, query, NULL, &error);

if (error) {
	g_printerr ("%s, %s\n", _("Could not query classes"), error->message);
	g_error_free (error);
	g_object_unref (connection);
	return;
}

if (!cursor) {
	g_print ("%s\n", _("No classes were found"));
} else {
	while (tracker_sparql_cursor_next (cursor, NULL, NULL)) {
		g_print ("%s\n", tracker_sparql_cursor_get_string (cursor, 1, NULL));
	}

	g_object_unref (cursor);
}

g_object_unref (connection);

So, now we have direct access. I have ported my tracker-needle experimental application to it and it seems faster than tracker-search-tool (which it aims to supersede). I will blog about tracker-needle later, but for now, direct-access is available in master for anyone interested to try it out! This has been a huge team effort with Jürg, Philip and Aleksander Morgado and myself involved. Try it out, any comments about the API are appreciated and it is documented quite nicely too.