Category: Python

I presented a tutorial at linux.conf.au a couple of weeks ago on what there was for Python developers between CGI scripts and Django. That developers needn’t still be writing CGI scripts in 2014 (it happens) and there were frameworks that met your needs.

This tutorial introduces the microframework Flask, and shows of a whole bunch of things you can do with it, up to being a fully-fledged replacement for Django if you’re so inclined.

Video
Examples source

In other news, I am now a maintainer of Lettuce, a BDD framework for Python/Django. So expect a few more Lettuce related blog posts (if I stop ignoring my blog).

I’ve been working on the data collection part of my cycle route modelling. I’m hoping that I can, as a first output, put together a map of where people are cycling in Melbourne. A crowd-sourced view of the best places to cycle, if you will. Given I will probably be running this in the cloud ((your recommendations for cloud-based services please, must be able to run Flask and PostGIS and be super cheap)), I thought it was best to actually store the data in a GIS database, rather than lots and lots of flat files.

A quick Google turned up GeoAlchemy, which are GIS extensions for SQLAlchemy. Provides lots of the standard things you want to do as methods on fields, but this is only a limited set of what you can do with PostGIS. Since I’m going to be wanting to do things like binning data, I thought it was worth figuring out how hard it was to call other PostGIS methods.

GeoAlchemy supports subclassing to create new dialects, but you have to subclass 3 classes, and it’s basically a pain in the neck when you just want to extend the functionality of the PostGIS dialect. Probably what I should do is submit a pull request with the rest of the PostGIS API as extensions, but I’m lazy. Henceforth, for the second time this week I am employing monkey patching to get the job done (and for the second time this week, kittens cry).

Functions in GeoAlchemy require two things, a method stub saying how we collect the arguments and the return (look at geoalchemy.postgis.pg_functions) and a mapping from this to the SQL function. Since we only care about one dialect, we can make this easier on ourselves by combining these two things. Firstly we monkeypatch in the method stubs:

from geoalchemy.functions import BaseFunction
from geoalchemy.postgis import pg_functions

@monkeypatchclass(pg_functions)
class more_pg_functions:
    """
    Additional functions to support for PostGIS
    """

    class length_spheroid(BaseFunction):
        _method = 'ST_Length_Spheroid'

Note the _method attribute which isn’t something used anywhere else. We can then patch in support for this:

from geoalchemy.dialect import SpatialDialect

@monkeypatch(SpatialDialect)
def get_function(self, function_cls):
    """
    Add support for the _method attribute
    """

    try:
        return function_cls._method
    except AttributeError:
        return self.__super__get_function(function_cls)

The monkeypatching functions look like this:

def monkeypatch(*args):
    """
    Decorator to monkeypatch a function into class as a method
    """

    def inner(func):
        name = func.__name__

        for cls in args:
            old = getattr(cls, name)
            setattr(cls, '__super__{}'.format(name), old)

            setattr(cls, name, func)

    return inner


def monkeypatchclass(cls):
    """
    Decorator to monkeypatch a class as a baseclass of @cls
    """

    def inner(basecls):
        cls.__bases__ += (basecls,)

        return basecls

    return inner

Finally we can do queries like this:

>>> track = session.query(Track).get(1)
>>> session.scalar(track.points.length_spheroid('SPHEROID["WGS 84",6378137,298.257223563]'))
6791.87502950043

Code on GitHub.

I had to put together a small web app the other day, using SQLAlchemy and Flask. Because I hate writing code multiple times, when I can do things using a better way, I wanted to be able to serialise SQLAlchemy ORM objects straight to JSON.

I decided on an approach where taking a leaf out of Javascript, I would optionally implement a tojson() method on a class, which I would attempt to call from my JSONEncoder ((The tojson() method actually returns a Python dict understandable by JSONEncoder)).

It turns out to be relatively simple to extend SQLAlchemy’s declarative base class to add additional methods (we can also use this as an excuse to implement a general __repr__().

from sqlalchemy.ext.declarative import declarative_base as real_declarative_base

# Let's make this a class decorator
declarative_base = lambda cls: real_declarative_base(cls=cls)

@declarative_base
class Base(object):
    """
    Add some default properties and methods to the SQLAlchemy declarative base.
    """

    @property
    def columns(self):
        return [ c.name for c in self.__table__.columns ]

    @property
    def columnitems(self):
        return dict([ (c, getattr(self, c)) for c in self.columns ])

    def __repr__(self):
        return '{}({})'.format(self.__class__.__name__, self.columnitems)

    def tojson(self):
        return self.columnitems

We can then define our tables in the usual way:

class Client(Base):
    __tablename__ = 'client'

    ...

You can obviously replace any of the methods in your subclass, if you don’t want to serialise the whole thing. Bonus points for anyone who wants to extend this to serialise one-to-many relationships.

And what about calling the tojson() method? That’s easy, we can just provide our own JSONEncoder.

import json

class JSONEncoder(json.JSONEncoder):
    """
    Wrapper class to try calling an object's tojson() method. This allows
    us to JSONify objects coming from the ORM. Also handles dates and datetimes.
    """

    def default(self, obj):
        if isinstance(obj, datetime.date):
            return obj.isoformat()

        try:
            return obj.tojson()
        except AttributeError:
            return json.JSONEncoder.default(self, obj)

Cutting edge Flask provides a way to replace the default JSON encoder, but the version I got out of pip does not. This is relatively easy to work around though by replacing jsonify with our own version.

from flask import Flask

app = Flask(__name__)

def jsonify(*args, **kwargs):
    """
    Workaround for Flask's jsonify not allowing replacement of the JSONEncoder
    in my version of Flask.
    """

    return app.response_class(json.dumps(dict(*args, **kwargs),
                                         cls=JSONEncoder),
                              mimetype='application/json')

If you do have a newer Flask, where you don’t have to replace jsonify, you can also inherit from Flask’s JSONEncoder, which already handles things like datetimes for you.