Posts Tagged ‘Bazaar’

Personal Catastrophic Fail Event

Thursday, January 8th, 2009

A couple of days ago I was pondering DVCS whilst wandering along an icy road. Not focusing on the ice proved disastrous. My right foot flew forward and i landed flat on my ****, much to the amusement of my fellow codethinkers. My head bounced off the tarmac and my elbow hit.. well who knows, but my elbow ended up pretty damn swollen. The day after brought little mercy - I seemed to have pulled muscles i didn’t even know I had and looking far left or right proved challenging, as did the simple act of putting my bag over my shoulder. Difficulty sleeping followed.

My point is, all this talk of DVCS caused me physical harm. Discussion considered harmful, lesson learned.

A couple of interesting points were raised about the git-serve bridge which I would like to answer, regardless of what infrastructure GNOME migrates to. I should have been more clear from the start, but the bulk of the work to implement the Git protocol is happening in an upstream project called dulwich. Theres no relation to Bazaar, and might be a nice library for anyone doing Git work in python (it doesnt need to spawn git processes to operate). The code there is based on some earlier work by James Westby and its current maintainer is Jelmer Vernooij. Any code for the git-serve mapping between Git and Bazaar is getting added to the bzr-git plugin, and again i’m working with Jelmer.

I’ve written some basic responses to the common gut reactions on the Bazaar wiki.

In the context of GNOME, a more interesting question was about how to merge branches from people not using GNOME infrastructure and a different VCS to the maintainer. First of all, why arent they using us? But one interesting thing i’d like to see is a code review system with tight VCS integration. For example, using dulwich to allow people to push a branch to review board and convert it into a review automatically. (The Git protocol will send only the commits that need to be merged). Anyone can push, no plugins or setup needed by the contributor. Then we can allow people to land a merge from review-board in the browser, or merge from review-board into their local branch or even request buildbot test the branch first. git:// is a protocol. We implement lots of other protocols in the freedesktop world, lots of them far more complicated, lots of them much less open. So this doesnt scare me. Its just an efficient way of swapping patches. And of course we can reuse bzrlib for supporting Bazaar.

Most points that have been raised have a valid counter-argument if you approach them with a “we can” attitude, but going over them would likely lead to more ice slipping. I will be at FOSDEM, be sure to bring a stick.

Even if we don’t use Bazaar (and dulwich) as the platform for our core source code hosting, we can still build lots of cool tools and services around GNOME with it. If you want to help out get in touch, or get in #gnome-bzr.

Tags: , , ,
Posted in Nerd | 3 Comments »

JHBuild and DVCS

Thursday, December 11th, 2008

A while ago I opened a bug asking to make it easier to use bzr-mirror.gnome.org with JHBuild. I proposed a patch which still needed some love, but wasn’t really sure how to finish it. It added a flag so you can say, “if there is a mirror of this for {bzr,git} users, use it”. Recently, thanks to zeenix pestering, I finished it. Of course he quickly identified some issues, which led to changes to the JHBuild code for svn, git and bzr. Among other things, SVN finally got tag support, and has a more consistent decision making path. And JHBuild-controlled-Git can handle branches a lot nicer now.

We really need to start cleaning the modulesets - one of the biggest problems we had is that the properties are getting misused. “../branches/foobar” is not a module, its part of a path! This is the route to pain, misery and suffering! As I find time I hope to clean this up, but for now the JHBuild mirror seems to be working.

To start using it, just set mirror_policy to “bzr” or “git” in your .jhbuildrc. You probably want a clean slate for this - it won’t doing anything clever in terms of taking over existing SVN checkouts. Another option is

module_mirror_policy = {
  "conduit": "bzr",
  "foobar": "git,
}

I hope to get JHBuild to automate git-svn a little here too (and automatically git-svn init) where appropriate.

Tags: , , , ,
Posted in Nerd | 2 Comments »

DVCS for GNOME

Thursday, December 11th, 2008

The more I think about it the more I don’t want to pick a single DVCS for GNOME. It should be up to whoever is doing the work to use whatever tools they want. Unfortunately having each module choose its own hosting format has its own problems - do i need to check out gnome-super-cool from bzr.gnome.org or darcs.gnome.org? What if maintainer A likes cvs and maintainer B likes Git.

<snipped out the bit where i ramble about all the different options people have already suggested. me rambling FTYawn>

There is another approach. The Git protocol is quite easy. The Git pack container format is easy. The basic Git objects are easy. Who says i have to store Git packs in Git? As long as the client Git can grab packs and push packs it is happy. All other operations are local.

So lets take bzr-playground and run something that provides git:// and git+ssh:// access to the bazaar repositories. Push and pull. We’ll still have a central point from which we can view and checkout a GNOME module and the infra work can focus on a single platform. But the choice of tool i use to do my work is (partially at least) back down to me.

I am currently able to read and write the pack container format as well as speak a decent amount of the git-upload-pack and git-receive-pack protocols. It won’t be the first time i’ve serialized or deserialized git objects, so that won’t take very long. Then its just a case of storing things in Bazaar and maintaining a cache of sha1 to bazaar id. Sure, i’m trivializing here, but i am confident that this is doable.

I’m linking to the code I have so far (its an extension of bzr-git) as a gesture that this isnt total vaporware, but it’s not really anything you can try yet.

lp:~johncarr/bzr-git/git-serve

At this point im mainly interested in collecting +1’s and -1’s on the basic concept.

Tags: , , ,
Posted in Nerd | 36 Comments »

83% of statistics are made up on the spot

Monday, July 7th, 2008

Got up early for some healthy debate on the differences between Bazaar and Git at the DVCS Bof. Maybe the slot was changed when I wasn’t looking, but it turned into a “introduction to Git and Gitorious” presentation, with a packed audience. For a session meant to focus on DVCS in general, and how GNOME can move from SVN to $DVCS, it was a bit of a shame. BUT THEN…

There was a second smaller talk on DVCS that was slightly more productive. Neither side has managed to provide an action plan so far. When the Bazaar and the Git advocates can come to GNOME and say “we have a plan, this is how we are migrating and this is how we are going to fix all the systems that depend on GNOME”.. then we can talk. This whole issue needs some JFDI - we can’t cry/shout until our DVCS is picked, and then expect the sysadmins to take care of it.

On a lighter note, got some swag. Quite happy with the laptop bag, and the T-shirt is quite nice. Doesn’t match the codethink t-shirts though ;)

Now back to cleaning up the Git code so Rob doesn’t harm me.

Tags: , , ,
Posted in Uncategorized | 2 Comments »

Loggerhead + Search

Thursday, June 26th, 2008

TWIB blogged about a little something we have been working on with the Bazaar guys. The Bazaar mirror now has loggerhead and search, based on the bzr-search plugin. This lets it search over the whole history of trunk. When beuno finishes skinning it i’ll integrate it in to the main site. In the mean time, enjoy.

Oh, it also shows off something I badly explained the other day. See here. Revision 5 was committed by myself, but bzr-svn also saved the –author property, attributing that code to Jelmer correctly.

Tags: ,
Posted in Uncategorized | 1 Comment »

Is speed the only reason?

Thursday, June 26th, 2008

zeenix decided to quote me in a recent blog post. My remark was meant to be a light hearted jibe at our obsession with going over and over the same points but never getting anywhere. I failed, but it looks even worse out of context. So lets try this again.

We have ALL heard how Bazaar is so slow you might actually die of old age before you finish checking out your module, and that Git is so hard and voodoo ridden that you will erase your project from history and cause the universe to collapse if you use the wrong incantation.

Both of these arguments are nothing new. They are both very tired, and of questionable correctness in recent versions of both tools. Thats why I didn’t want to hear about them *again*. That doesn’t mean that they aren’t valid concerns, just that covering them again didn’t seem constructive.

As it happens, zeenix posted about how Git is fast. Now some interesting debate is happening over in the comments of that entry, so maybe my poor sense of humour did some good. Note that I never bothered to try and say Bazaar wasn’t slower because i thought it a waste of time and a good way to get flamed, but the people are now saying “hey, Bazaar is actually doing OK”. Yay.

In closing: people, please don’t get upset, i’d never waste my vodka over something like DVCS-wars! Now if this was vim vs emacs…

Tags: , ,
Posted in Nerd | 9 Comments »

The mirror man says

Thursday, June 26th, 2008

Federico wrote a nice post about Git. I just wanted to add a few things to it really.

  • If anyone wants to buy me beer for Bazaar mirror, Git mirror, or the Mercurial mirror that people are begging me for then please remember… Spirits, spirits, spirits. (They are especially good at making me forget, which is handy after setting up something like git mirror :P The stronger the better!) 
  • git-mirror.gnome.org does not make git.gnome.org any easier. It just makes it easier to try out git-svn on a GNOME project. (Some/all of these may be fixable with surgery, but the one way conversions will likely be quicker anyway).
    • The main reason is of course the polluted logs (filled with git-svn rev id metadata). I would resist any module having a Git repo with such ickyness in its history.
    • I didn’t have the data to provide git-svn with an authors lookup
    • What to do about the references to svn in git-mirror? All the branches have an svn/ prefix…
    • git-mirror ignores Bazaar metadata. Thats right folks, it says svn.gnome.org on the tin but the Bazaar users have already started upgrading GNOME to Bazaar with bzr-svn. Check out this. You’ll notice how commits made with Bazaar are somehow “better” than their SVN counterparts. When I apply someones patch with bzr-svn I log who it is from using the –author property. I want this preserved.
  • The git mirroring process uncovered bugs in git-svn, despite its heavy use by the core GNOME hackers. Even one on GTK+, which is meant to be the thing that everyone makes a mirror of! So we will likely break other git import tools too. This is true of any DVCS, so my point is really “its never going to be ‘easy’ to ditch SVN”.

Lets show a bit of even handedness and community and not make choosing $DVCS like chosing $EDITOR. Git people, please try out bzr-mirror. Especially merging and branching stuff. Unlike git-svn, you can use bzr-svn as though its a normal, full blown DVCS. Blog about your experiences, let the Bazaar developers what they need to work on. (I will spit my vodka in your eyes if all you come up with is speed). Likewise, the Bazaar folks should try out git-mirror and see if its as good as they say, and tell them which bits of the UI need fixing. What do you find yourself missing?

Mercurial people: Take me to your documentation! What is the standard way to make a Mercurial “mirror” of SVN?

In the next few days I should be taking the lid off a loggerhead instance for bzr-mirror. Of interest is the search, which uses a full text index created by the wonderful bzr-search plugin. It’s pretty impressive.

Tags: , , ,
Posted in Nerd | 6 Comments »

dvcs-mirror updates

Monday, June 23rd, 2008

As I’ve blogged before, the mirrors update in response to the svn-commits mailing list. This means that we don’t have to hammer 520 GNOME modules on a hourly basis. It also means we are nearly always up to date. I’ve tweaked the script that manages this:

  • Previously, the script would only do updates of existing modules. But now it will automatically mirror new modules the first time something hits the commits list for them, for both Bazaar and Git
    • For git, a new clone is created. It is repacked. Various config settings and symbolic references and internals are poked.
    • For bzr, we just need to run bzr svn-import. Its not needed, but I also create a full text search index of trunk for later secret fun.
  • Updates are now batched. Example: If 3 e-mails arrive together for tracker, that means that svn.gnome.org gets poked 3 times more than needed. So I batch updates together. Faster, cheaper, saner.

You git speed freaks will be pleased to know you can now clone over git://.

cd ~
git clone git://git-mirror.gnome.org/git/conduit
cd conduit
git svn init -s svn+ssh://svn.gnome.org/svn/conduit --prefix=origin/svn/
git svn rebase

Hope you enjoy the extra speed ;-

Tags: ,
Posted in Nerd | 4 Comments »

Bazaar: The shiny shinyness

Tuesday, June 17th, 2008

Did I get your attention? :-) This is just a shout out to let you all know about #gnome-bzr (on irc.gnome.org). If you use Bazaar for your GNOME project, or are thinking of using it, please pop in. You might just get a first look at some of the shiny, shiny things we have in the cooker and that the bzr team are helping us try out. I’ll also be able to help you out with any bzr-mirror related questions, and bzr-svn in general.

We have a few people from #bzr kicking around in there too, feel free to poke them with feature requests. Poke well, we don’t like letting them sleep too long..

Tags: ,
Posted in Nerd | Comments Off

Bazaar Love

Tuesday, May 20th, 2008

A few people have expressed their love for the Bazaar.

I love that when I need to throw something together quick its there waiting for me. Its commands are discoverable. It has a sane API that has led the awesome bzr-svn, and the rather delicious bzr-loom plugins. Where others have frustrated me, Bazaar has been there and has just worked.

If you want to join in the goodness, you can now grab a bazaar branch of any GNOME module trunk with ease, thanks to the bzr mirror I have been working on. It should normally be more up to date than launchpad, and is bzr-svn compatible - so you can push your changes back to svn.gnome.org. There are even some nifty instructions.

Tags:
Posted in Nerd | Comments Off

Bad Behavior has blocked 65 access attempts in the last 7 days.