Yesterday, Thomas rolled the 0.13 release of Storm, which can be downloaded from Launchpad. Storm is the object relational mapper for Python used by Launchpad and Landscape, so it is capable of supporting quite large scale applications. It is seven months since the last release, so there is a lot of improvements. Here are a few simple statistics:
|Tarball size (KB)
|Revisions in ancestry
So it is a fairly significant update by any of these metrics. Among the new features are:
- Infrastructure for tracing the SQL statements issued by Storm. Sample tracer implementations are provided to implement bounded statement run times and for logging statements (both features used for QA of Launchpad).
- A validation framework. The property constructors take a validator keyword argument, which should be a function taking arguments (object, attr_name, value) and return the value to set. If the function raises an exception, it can prevent a value from being set. By returning something different to its third argument it can transform values.
- The find() and ResultSet API has been extended to make it possible to generate queries that use GROUP BY and HAVING. The primary use case for result sets that contain an object plus some aggregates associated with that object.
- Some core parts of Storm have been accelerated through a C extension. This code is turned off by default, but can be enabled by defining the STORM_CEXTENSIONS environment variable to 1. While it is disabled by default, it is pretty stable. Barring any serious problems reported over the next release cycle, I’d expect it to be enabled by default for the next release.
- The minimum dependencies of the storm.zope.zstorm module have been reduced to just the zope.interface and transaction modules. This makes it easier to use the per-thread store management code and global transaction management outside of Zope apps (e.g. for integrating with Django).
It doesn’t include my Django integration code though, since that isn’t fully baked. I’ll post some more about that later.
For anyone that cares, the new series of Double the Fist is starting tonight at 9:30pm on ABC2 (and repeated tomorrow on ABC1 for those who don’t get ABC2). It has been a long time coming (4 years since the previous series), so will hopefully be worth it. I guess it will be available on the internet shortly after for those outside of Australia.
It is also good to see Roy and HG covering the Olympics again, even if it is only on the radio this time rather than television. The shows are being posted on the website after airing.
I’ve just finished the first day of the Ubuntu online services sprint in Orlando, Florida. I didn’t repeat last year’s trick of falling asleep at the airport, so the trip was only about 29 hours all up.
We’ve got a great team that I am looking forward to working with, so it’ll be interesting to see what we do over the next little while.
I’ve been playing around with Django a bit for work recently, which has been interesting to see what choices they’ve made differently to Zope 3. There were a few things that surprised me:
- The ORM and database layer defaults to autocommit mode rather than using transactions. This seems like an odd choice given that all the major free databases support transactions these days. While autocommit might work fine when a web application is under light use, it is a recipe for problems at higher loads. By using transactions that last for the duration of the request, the testing you do is more likely to help with the high load situations.
- While there is a middleware class to enable request-duration transactions, it only covers the database connection. There is no global transaction manager to coordinate multiple DB connections or other resources.
- The ORM appears to only support a single connection for a request. While this is the most common case and should be easy to code with, allowing an application to expand past this limit seems prudent.
- The tutorial promotes schema generation from Python models, which I feel is the wrong choice for any application that is likely to evolve over time (i.e. pretty much every application). I’ve written about this previously and believe that migration based schema management is a more workable solution.
- It poorly reinvents thread local storage in a few places. This isn’t too surprising for things that existed prior to Python 2.4, and probably isn’t a problem for its default mode of operation.
Other than these things I’ve noticed so far, it looks like a nice framework.
I’ve been doing a bit of work to make it easy to use Storm with Django. I posted some initial details on the mailing list. The initial code has been published on Launchpad but is not yet ready to merge. Some of the main details include:
- A middleware class that integrates the Zope global transaction manager (which requires just the zope.interface and transaction packages). There doesn’t appear to be any equivalent functionality in Django, and this made it possible to reuse the existing integration code (an approach that has been taken to use Storm with Pylons). It will also make it easier to take advantage of other future improvements (e.g. only committing stores that are used in a transaction, two phase commit).
- Stores can be configured through the application’s Django settings file, and are managed as long lived per-thread connections.
- A simple get_store(name) function is provided for accessing per-thread stores within view code.
What this doesn’t do yet is provide much integration with existing Django functionality (e.g. django.contrib.admin). I plan to try and get some of these bits working in the near future.