Archive for the ‘Nerd’ Category

Great Minds Think Alike: Asynchronous Patterns

Monday, March 29th, 2010

Isn’t writing synchronous code nice?

function do_stuff () {
  do_thing_one ();
  do_thing_two ();
  do_thing_three ();
}

But synchronous is bad! Bad, bad, bad. So then came async. The simple pattern is to pass a callback to the function you are calling. It’s not as bad in JS because we just can do this:

function do_stuff () {
  do_thing_one ( function (result) {
    do_thing_two ( function (result) {
      do_thing_three ( function (result) {
      });
    });
  });
}

I’ve left out error handling. That really depends on the library.. But i imagine its messy. Now let’s see GIO style async.

function do_stuff () {
  do_thing_one_async ( function (ar) {
    var result = do_thing_one_finish (ar);
    do_thing_two_async ( function (ar) {
      var result = do_thing_two_finish (ar);
      do_thing_three_async (function (ar) {
        var result = do_thing_three_finish (ar);
      });
    });
  });
}

I like this a lot better than how i’d do it in python. But wouldn’t it be nice if you could write async code something like this?

var do_stuff = async (function () {
  var result = yield do_thing_one ();
  yield do_thing_two ();
  yield do_thing_three ();
});

Or even:

var do_stuff = async (function () {
  var result = yield do_thing_one ();
  yield do_thing_two ();
  try{
    yield do_thing_three ();
  } catch (e) {
    print("Exception handled");
  }
});

You can in python. You can in vala. And for JS? Well, I was going to say “now you can“. But while I was looking for a good Vala link, I noticed Alex already did something like this over a year ago. D’oh.

What would be really nice is if the async wrappers could be generated automatically by GI. I had a first stab at this by simply parsing the GIR xml with E4X and providing an alternative import mechanism (thanks for the suggestion jdahlin). However to get full coverage i’d have to consider interfaces and inspect every object that implements an interface as it lands in JavaScript land to ensure it is wrapped. Ew.

Sysadmin Team Meeting, Bugzilla

Tuesday, August 11th, 2009

Despite the constant threat of bike shedding, the sysadmin team will be hosting a small irc meeting (on #sysadmin) in the near future. Feel free to join in and tell us how much you love us <3. See here for more details, and here to help us choose between Friday and Sunday.

Minutes from the last meeting are here.

On a related note, one of the tasks to come out of the last meeting was to try and make bugzilla faster, or at least come up with a plan. That task kind of disappeared, because the bugzilla 3 porting work is now finished. Yes, as you may have seen, Bugzilla 3 is coming!!.

I’m not afraid of people using JHBuild

Wednesday, July 15th, 2009

Just a quick blog to point you at the external dependencies branch of JHBuild i’ve been working on.

The external deps branch aims to do 2 things:

  • Don’t jhbuild any modules that are already system installed
  • Add a new command to jhbuild to install system packages so they don’t need to be jhbuilded

It pretty much works, it needs massaging into shape to merge into master. Then we need to collect some meta-data:

  • Minimum versions. How do we know if the system installed glib is new enough? I added a minimum=”” annotation to the dependency information we already have.
  • Package names. Right now it’s a bit difficult to map jhbuild module names to package names. I’ve added aliases files for now, and some crude python to try and infer the aliases for Debian. Better ideas welcome.

I also need to get round to a PackageKit backend so it supports more than Debian/Ubuntu :]

If you want a jhbuild where ‘jhbuild build gedit’ only builds the least amount of stuff, help me :]

Playing with RDF

Friday, May 15th, 2009

I’ve been playing with the master branch of tracker and i’m loving it – it looks like its finally reached the stage where I won’t just turn it straight off after a fresh install.

It now brings GNOME an RDF store with a SPARQL interface. Powerful joo-joo, but kinda scary if you haven’t seen it before. Most conversations about it lead to words like graphs, triples, ontologies… My eyes start to gloss over.. I need to learn by doing. So i’ve been playing with writing some python wrappers to hide tracker and just provide a familiar pythonic interface.

Any object type that tracker knows about will be available in python via the wonders of introspection. All properties of the class are available, with docstrings explaining the intent of the property and its type. Obviously you can get, set and delete and do simple queries. And behind the scenes are SPARQL queries in all their glory. Theres a lot still to do, but enough done that I can synchronise my address book to Tracker with Conduit (see my tracker branch).

So far it looks something like this (but its subject to very rapid change):

import tralchemy
from tralchemy.nco import Contact

# add a contact
c = Contact.create()
c.fullname = "John Carr"
c.firstname = "John"
c.nickname = "Jc2k"
c.commit()

# find all the people called John
for c in Contact.get(firstname="John"):
  print c.uri, c.fullname

# subscribe to any contact changes
def callback(subjects):
  print subjects
Contact.notifications.connect("SubjectsAdded", callback)

# Will probably be just:
Contact.added.connect(callback)

While get() is a nice way to do simple queries, what if you wanted to do something a little more complicated. It always feels messy when you have SQL or SPARQL nested in other code. Existing SQL ORM tools are a great place to start at avoiding this, but i quite like the LINQ style generator-to-SPARQL. Something like:


q = Query(Contact.firstname for Contact in Store if Contact.nickname == 'Jc2k')

or


q = Query(c.firstname for c in Store if c is Contact and c.nickname == 'Jc2k')

Hmm decisions. Hope to implement similar abstractions in JavaScript, C# (via LINQ) and Vala (via Magic). Anyone wanna share their cloning tech?

Mercurial and Git

Sunday, May 3rd, 2009

Jelmer just pinged me to show me a new fangled Mercurial plugin. Thats right folks – pulling and pushing between Git and Mercurial! It uses the dulwich library that I worked on with Jelmer (was the basis for my “git serve” crack and is also used for pushing and pulling between Bazaar branches and Git with bzr-git).

Coolness.

I look forward to seeing some new dulwich patches (they are using a patched copy right now).

On git.gnome.org and GitHub

Monday, April 27th, 2009

When GNOME SVN was migrated to Git all your commits were attached to a generated e-mail address in the form username@src.gnome.org.

If you have a GitHub account you can go to your settings and add that e-mail address. Now all the old commits you made will be associated with your account when viewed on GitHub. If you’re a fan of the cute GitHub graphs, you’ll love this too.

Personal Catastrophic Fail Event

Thursday, January 8th, 2009

A couple of days ago I was pondering DVCS whilst wandering along an icy road. Not focusing on the ice proved disastrous. My right foot flew forward and i landed flat on my ****, much to the amusement of my fellow codethinkers. My head bounced off the tarmac and my elbow hit.. well who knows, but my elbow ended up pretty damn swollen. The day after brought little mercy – I seemed to have pulled muscles i didn’t even know I had and looking far left or right proved challenging, as did the simple act of putting my bag over my shoulder. Difficulty sleeping followed.

My point is, all this talk of DVCS caused me physical harm. Discussion considered harmful, lesson learned.

A couple of interesting points were raised about the git-serve bridge which I would like to answer, regardless of what infrastructure GNOME migrates to. I should have been more clear from the start, but the bulk of the work to implement the Git protocol is happening in an upstream project called dulwich. Theres no relation to Bazaar, and might be a nice library for anyone doing Git work in python (it doesnt need to spawn git processes to operate). The code there is based on some earlier work by James Westby and its current maintainer is Jelmer Vernooij. Any code for the git-serve mapping between Git and Bazaar is getting added to the bzr-git plugin, and again i’m working with Jelmer.

I’ve written some basic responses to the common gut reactions on the Bazaar wiki.

In the context of GNOME, a more interesting question was about how to merge branches from people not using GNOME infrastructure and a different VCS to the maintainer. First of all, why arent they using us? But one interesting thing i’d like to see is a code review system with tight VCS integration. For example, using dulwich to allow people to push a branch to review board and convert it into a review automatically. (The Git protocol will send only the commits that need to be merged). Anyone can push, no plugins or setup needed by the contributor. Then we can allow people to land a merge from review-board in the browser, or merge from review-board into their local branch or even request buildbot test the branch first. git:// is a protocol. We implement lots of other protocols in the freedesktop world, lots of them far more complicated, lots of them much less open. So this doesnt scare me. Its just an efficient way of swapping patches. And of course we can reuse bzrlib for supporting Bazaar.

Most points that have been raised have a valid counter-argument if you approach them with a “we can” attitude, but going over them would likely lead to more ice slipping. I will be at FOSDEM, be sure to bring a stick.

Even if we don’t use Bazaar (and dulwich) as the platform for our core source code hosting, we can still build lots of cool tools and services around GNOME with it. If you want to help out get in touch, or get in #gnome-bzr.

JHBuild and DVCS

Thursday, December 11th, 2008

A while ago I opened a bug asking to make it easier to use bzr-mirror.gnome.org with JHBuild. I proposed a patch which still needed some love, but wasn’t really sure how to finish it. It added a flag so you can say, “if there is a mirror of this for {bzr,git} users, use it”. Recently, thanks to zeenix pestering, I finished it. Of course he quickly identified some issues, which led to changes to the JHBuild code for svn, git and bzr. Among other things, SVN finally got tag support, and has a more consistent decision making path. And JHBuild-controlled-Git can handle branches a lot nicer now.

We really need to start cleaning the modulesets – one of the biggest problems we had is that the properties are getting misused. “../branches/foobar” is not a module, its part of a path! This is the route to pain, misery and suffering! As I find time I hope to clean this up, but for now the JHBuild mirror seems to be working.

To start using it, just set mirror_policy to “bzr” or “git” in your .jhbuildrc. You probably want a clean slate for this – it won’t doing anything clever in terms of taking over existing SVN checkouts. Another option is

module_mirror_policy = {
  "conduit": "bzr",
  "foobar": "git,
}

I hope to get JHBuild to automate git-svn a little here too (and automatically git-svn init) where appropriate.

DVCS for GNOME

Thursday, December 11th, 2008

The more I think about it the more I don’t want to pick a single DVCS for GNOME. It should be up to whoever is doing the work to use whatever tools they want. Unfortunately having each module choose its own hosting format has its own problems – do i need to check out gnome-super-cool from bzr.gnome.org or darcs.gnome.org? What if maintainer A likes cvs and maintainer B likes Git.

<snipped out the bit where i ramble about all the different options people have already suggested. me rambling FTYawn>

There is another approach. The Git protocol is quite easy. The Git pack container format is easy. The basic Git objects are easy. Who says i have to store Git packs in Git? As long as the client Git can grab packs and push packs it is happy. All other operations are local.

So lets take bzr-playground and run something that provides git:// and git+ssh:// access to the bazaar repositories. Push and pull. We’ll still have a central point from which we can view and checkout a GNOME module and the infra work can focus on a single platform. But the choice of tool i use to do my work is (partially at least) back down to me.

I am currently able to read and write the pack container format as well as speak a decent amount of the git-upload-pack and git-receive-pack protocols. It won’t be the first time i’ve serialized or deserialized git objects, so that won’t take very long. Then its just a case of storing things in Bazaar and maintaining a cache of sha1 to bazaar id. Sure, i’m trivializing here, but i am confident that this is doable.

I’m linking to the code I have so far (its an extension of bzr-git) as a gesture that this isnt total vaporware, but it’s not really anything you can try yet.

lp:~johncarr/bzr-git/git-serve

At this point im mainly interested in collecting +1′s and -1′s on the basic concept.

Dell Inspiron Mini 9 – First Thoughts

Wednesday, September 17th, 2008

I couldn’t resist the Dell Mini. Many confused looks from the ThinkPad corner. “What? Dells are cheap pieces of crap!”. Regardless, I’ve always had good experiences with Dell hardware (even if I do miss find myself missing a nub). So as soon as they became available on the Thursday 4th, I ordered. My new toy arrived on Thursday 11th.

Almost a week in, how is it going? Well, lets start with the things I can moan about:

  • No mobile broadband for me. I had hoped to pop in a standard card and my Three SIM, but even though the board is marked “WWAN” and the pads are there, the Mini PCI-E connector and aerial wires are missing.
  • Broadcom Wifi. Eww. It needed some proprietary blob, so only Hardy seemed to support it out of the box. Not quite true, stock Hardy has a bug so I needed to apt-get upgrade and reboot. I have a more permanent solution. I replaced it with an Intel card from ebay. £7, including postage. Much much better, and it even joins my local network 5s quicker :D
  • No sound. Had to edit /etc/modrobe.d/* to add options snd-hda-intel index=0 model=dell, then it did. Not sure why auto mode doesn’t work.
  • Battery doesnt have anyway to see the current charge status when not logged in (no charging LED, no LED strip on battery with a “press here for status” button).

Otherwise, I’m really happy. I’m writing this blog post on it, and as you can (hopefully!)  tell, the keyboard isn’t causing me too much pain. In fact, i’m adapting quite well and am currently sat in the dark not paying any attention to the keys im pecking away at. The sound quality good, the webcam is good, GNOME is nippy enough. Compiz is more than usable, and for once i’m not reaching to turn it off. As usual for my Dells, i’m pleased with the screen.

Ubuntu Hardy is your best bet to get the wifi work. Intrepid is also working just fine. Mandriva (GUADEC USB key) and Fedora 10 also boot, but won’t have wifi. Ubuntu Feisty and Fedora 9 wouldn’t boot. Atom doesn’t do x86_64, so have your 32bit media to hand.

It doesn’t come with a case, but I got a 9″ portable dvd player sleeve from Amazon which is perfect.

So far i’m pleased with my purchase. Low low low budget camera phone amateur Dell pron here.