On the track for 3.32

It happens sneakily, but there’s more things going on in the Tracker front than the occasional fallout. Yesterday 2.2.0-alpha1 was released, containing some notable changes.

On and off during the last year, I’ve been working on a massive rework of the SPARQL parser. The current parser was fairly solid, but hard to extend for some of the syntax in the SPARQL 1.1 spec. After multiple attempts and failures at implementing property paths, I convinced myself this was the way forward.

The main difference is that the previous parser was more of a serializer to SQL, just minimal state was preserved across the operation. The new parser does construct an expression tree so that nodes may be shuffled/reevaluated. This allows some sweet things:

• Property paths are a nice resource to write more idiomatic SPARQL, most property path operators are within reach now. There’s currently support for sequence paths:

  # Get all files in my homedir
  SELECT ?elem {
    ?elem nfo:belongsToContainer/nie:url 'file:///home/carlos'
  }
  

  
  And inverse paths:
  

  # Get all files in my homedir by inverting
  # the child to container relation
  SELECT ?elem {
    ?homedir nie:url 'file:///home/carlos' ;
             ^nfo:belongsToContainer ?elem
  }
  

  There’s harder ones like + and * that will require recursive selects, and there’s the negation (!) operator which is not possible to implement yet.

• We now have prepared statements! A TrackerSparqlStatement object was introduced, capable of holding a query with parameters which can be set/replaced prior to execution.

  conn = tracker_sparql_connection_get (NULL, NULL);
  stmt = tracker_sparql_connection_query_statement (conn,
                                                    "SELECT ?u { ?u fts:match ~term }",
                                                    NULL, NULL);
  
  tracker_sparql_statement_bind_string (stmt, "term", search_term);
  cursor = tracker_sparql_statement_execute (stmt, NULL, NULL);
  

  This is a long sought protection for injections. The object is cacheable and can service multiple cursors asynchronously, so it will also be an improvement for frequent queries.

• More concise SQL is generated at places, which brings slight improvements on SQLite query planning.

This also got the ideas churning towards future plans, the trend being a generic triple store as much sparql1.1 capable as possible. There’s also some ideas about better data isolation for Flatpak and sandboxes in general (seeing the currently supported approach didn’t catch on). Those will eventually happen in this or following cycles, but I’ll reserve that for other blog post.

An eye was kept on memory usage too (mostly unrealized ideas from the performance hackfest earlier this year), tracker-store has been made to automatically shutdown when unneeded (ideally most of the time, since it just takes care of updates and the unruly apps that use the bus connection), and tracker-miner-fs took over the functionality of tracker-miner-apps. That’s 2 processes less in your default session.

In general, we’re on the way to an exciting release, and there’s more to come!