June 26th, 2008 by John Carr
Federico wrote a nice post about Git. I just wanted to add a few things to it really.
- If anyone wants to buy me beer for Bazaar mirror, Git mirror, or the Mercurial mirror that people are begging me for then please remember… Spirits, spirits, spirits. (They are especially good at making me forget, which is handy after setting up something like git mirror
The stronger the better!)
- git-mirror.gnome.org does not make git.gnome.org any easier. It just makes it easier to try out git-svn on a GNOME project. (Some/all of these may be fixable with surgery, but the one way conversions will likely be quicker anyway).
- The main reason is of course the polluted logs (filled with git-svn rev id metadata). I would resist any module having a Git repo with such ickyness in its history.
- I didn’t have the data to provide git-svn with an authors lookup
- What to do about the references to svn in git-mirror? All the branches have an svn/ prefix…
- git-mirror ignores Bazaar metadata. Thats right folks, it says svn.gnome.org on the tin but the Bazaar users have already started upgrading GNOME to Bazaar with bzr-svn. Check out this. You’ll notice how commits made with Bazaar are somehow “better” than their SVN counterparts. When I apply someones patch with bzr-svn I log who it is from using the –author property. I want this preserved.
- The git mirroring process uncovered bugs in git-svn, despite its heavy use by the core GNOME hackers. Even one on GTK+, which is meant to be the thing that everyone makes a mirror of! So we will likely break other git import tools too. This is true of any DVCS, so my point is really “its never going to be ‘easy’ to ditch SVN”.
Lets show a bit of even handedness and community and not make choosing $DVCS like chosing $EDITOR. Git people, please try out bzr-mirror. Especially merging and branching stuff. Unlike git-svn, you can use bzr-svn as though its a normal, full blown DVCS. Blog about your experiences, let the Bazaar developers what they need to work on. (I will spit my vodka in your eyes if all you come up with is speed). Likewise, the Bazaar folks should try out git-mirror and see if its as good as they say, and tell them which bits of the UI need fixing. What do you find yourself missing?
Mercurial people: Take me to your documentation! What is the standard way to make a Mercurial “mirror” of SVN?
In the next few days I should be taking the lid off a loggerhead instance for bzr-mirror. Of interest is the search, which uses a full text index created by the wonderful bzr-search plugin. It’s pretty impressive.
Tags: Bazaar, Git, GNOME, Mercurial
Posted in Nerd | 6 Comments »
June 23rd, 2008 by John Carr
As I’ve blogged before, the mirrors update in response to the svn-commits mailing list. This means that we don’t have to hammer 520 GNOME modules on a hourly basis. It also means we are nearly always up to date. I’ve tweaked the script that manages this:
- Previously, the script would only do updates of existing modules. But now it will automatically mirror new modules the first time something hits the commits list for them, for both Bazaar and Git
- For git, a new clone is created. It is repacked. Various config settings and symbolic references and internals are poked.
- For bzr, we just need to run bzr svn-import. Its not needed, but I also create a full text search index of trunk for later secret fun.
- Updates are now batched. Example: If 3 e-mails arrive together for tracker, that means that svn.gnome.org gets poked 3 times more than needed. So I batch updates together. Faster, cheaper, saner.
You git speed freaks will be pleased to know you can now clone over git://.
cd ~
git clone git://git-mirror.gnome.org/git/conduit
cd conduit
git svn init -s svn+ssh://svn.gnome.org/svn/conduit --prefix=origin/svn/
git svn rebase
Hope you enjoy the extra speed ;-
Tags: Bazaar, Git
Posted in Nerd | 4 Comments »
June 21st, 2008 by John Carr
As most of you know, I run bzr-mirror.gnome.org in my spare time. This was to allow people to try out the awesomeness of Bazaar as easily as possible. As much as I love Bazaar, certain outside forces mean that I sometimes have to use Git. So a few weeks ago when no one had approached bkor to set up git-mirror.gnome.org, I decided I would.
The plan:
- Follow the example in the git-svn manual, 520 times
- Run the equivalent of “bzr pull” everytime a commit happens (yay svn-commits-list)
- Throw gitweb at it
- Profit
All in all it went OK. Here are some things I encountered.
- I converted conduit, f-spot, tomboy and a few larger modules like gtk+. Then I tried to clone them over http. Found out I have to call git-update-server-info in post-update hook. This is well documented, and no big deal, but a surprise as its not needed with Bazaar.
- My initial attempt was cut short by James, who spotted I was missing an important argument (–rewrite-root) in my git-svn cloning. I had to abort the import after 3 days. Rip up and retry. Or so we thought. After some talking in #git, we found that –rewrite-root http://anything works, making me wonder why the root URL is stored in the log at all.
- When I ran the import again I found some of the modules randomly aborted mid conversion. It was able to resume them manually – yay, because it was one of the bigger modules the first time (gimp).
- I also found that gitweb wasn’t showing the tags and branches. James saved me again, this time with a script from Tim Janik at Imendio. Its full of perplexing git incantations, but works perfectly. It has to run after each change to keep everything up to date.
- I couldn’t get git-svn to work on a clone. My hero James was MIA, and I was struggling. A trip to #git offered mixed results. There were patches, there were more perplexing git incantations. Nothing seemed to work. With Robs help we got it working, and when James returned we got it working with branches and tags too. Day was saved. HeroPoints++.
- My version of git-svn chokes on accerciser and fast-user-switch-applet. My hero might yet fix this one too, as his git-svn was able to convert the problematic tags (this is due to cvs2svn).
- Now the mirror is ticking along, but gtk+ won’t update properly. I can update it from a local rsync, but not over http. I get “Apache got a malformed URI: REPORT request failed on ‘/svn/gtk+/!svn/vcc/default’: Unusable URI: it does not refer to this repository at /usr/bin/git-svn line 3833″.
- gnomemm Uses a non standard layout (at least in GNOME terms) which git-svn doesn’t like that much, so I will have to special case it in my scripts.
- I still have to go back and repack all the repositories as git-svn generates packs that are excessively large, and git gc –prune –agressive isn’t enough (thanks to Pieter for this advice).
After a week of churning its nice to have my server back to normal load. I still have to try and skin gitweb so that it looks GNOMEy and there are a hundred and one other things bothering me, but now you guys can:
cd ~
git clone http://git-mirror.gnome.org/git/cheese
cd cheese
git svn init -s svn+ssh://svn.gnome.org/svn/cheese --prefix=origin/svn/
git svn rebase
I’m not sure what will happen if you pull from the mirror and from svn.gnome.org. That is the ideal use case IMO (should be faster for you as mirror does the conversion for you). If someone wants to try and let me know how it handles it…
Want a mirror for your $dvcs? Ping me, maybe i’ll give it a go.
Tags: Git, GNOME
Posted in Nerd | 9 Comments »
June 20th, 2008 by John Carr
Today is my last day working with the folks at Isotoma. It’s been a fun year since internet veterans Andy Theyers and Doug Winter took me under their wings, giving me the chance to code in python all day. Even better, I got to use Ubuntu. I’ve seen so many cool things while i’ve been here, and a bewildering variety of taste in music (we’ve found some gems). Squeezebox, I will miss you. Only a few hours left until I leave the loving embrace of the team. An event that shall be marked with modest beverages and an early night.
I’ll be watching you guys, when Forkd gets groups I think it would be perfect for the GNOME Cookbook people. And when its released, i’m looking forward to playing with Exotypes some more. Until then, keep on founding.
So to Monday, and from a daily commute to York to a daily commute to somewhere near Sheffield to work with Codethink. I’m really excited about working on the stuff Rob told me about. The next month is going to be really packed, and if a new job wasn’t enough I’ve got a LRL presentation to prepare for…
Tags: Codethink, Isotoma, LRL
Posted in Uncategorized | 4 Comments »
June 17th, 2008 by John Carr
Did I get your attention? 
This is just a shout out to let you all know about #gnome-bzr (on irc.gnome.org). If you use Bazaar for your GNOME project, or are thinking of using it, please pop in. You might just get a first look at some of the shiny, shiny things we have in the cooker and that the bzr team are helping us try out. I’ll also be able to help you out with any bzr-mirror related questions, and bzr-svn in general.
We have a few people from #bzr kicking around in there too, feel free to poke them with feature requests. Poke well, we don’t like letting them sleep too long..
Tags: Bazaar, GNOME
Posted in Nerd | Comments Off
June 7th, 2008 by John Carr
Lots of cool things going on. I’ve been name dropped by bkor and jcastro. A few weeks back I went in to York and met pvanhoof, learning much about Belgian politics The following Monday I met robtaylor. We talked about Git, libgitcore, wizbit and Conduit. A few days after that sabdfl called me! He tells me a lot of people have been talking about the Bazaar mirror of GNOME that i set up (bzr-mirror.gnome.org) and he wanted to talk about it.
Woot for meeting new people, and some of my heroes to boot, within GNOME and Ubuntu
Tags: GNOME, Ubuntu
Posted in General, Nerd | Comments Off
May 20th, 2008 by John Carr
A few people have expressed their love for the Bazaar.
I love that when I need to throw something together quick its there waiting for me. Its commands are discoverable. It has a sane API that has led the awesome bzr-svn, and the rather delicious bzr-loom plugins. Where others have frustrated me, Bazaar has been there and has just worked.
If you want to join in the goodness, you can now grab a bazaar branch of any GNOME module trunk with ease, thanks to the bzr mirror I have been working on. It should normally be more up to date than launchpad, and is bzr-svn compatible – so you can push your changes back to svn.gnome.org. There are even some nifty instructions.
Tags: Bazaar
Posted in Nerd | Comments Off
May 6th, 2008 by John Carr
Spent the evening playing with network sync with N810 as server, laptop as client. It’s lovely. Looking at the logs its hard to tell what is happening when I try it the other way around but I think I just need to make the hildon version of the GUI a bit more dynamic so it can cope with sections getting added and removed at runtime.
Thomas, I owe you beer
If you want to help out, 0.3.10 is available here, and i’ll see you in #conduit
Tags: Conduit, Maemo, N810
Posted in Nerd | Comments Off
May 5th, 2008 by John Carr
Sorry, no screenshots. Took a picture with my cameraphone, but it didn’t turn out very well.
If you want to give it a spin for yourself, the details are:
- Catalogue: Unrouted
- Web Address: http://apt.unrouted.co.uk
- Distribution: chinook
- Components: user
You will also need the Extras and Extras-devel repository.
Refresh your application list and install the conduit package. Among other things, you will get python2.5 and avahi. Its quite a big set of dependencies for now, it will get better when things stable up and we can split the conduit package into conduit-plugin-google etc.
There is a big bug screwing with a lot of dataproviders right now relating to icons. I’ll try to push a patch soon.
Tags: Conduit, Maemo, N810
Posted in Nerd | Comments Off
May 4th, 2008 by John Carr
So after a recent flurry of packaging, I turned my attention to Conduit on the N810. Most of the work is done, just a case of figuring out what bits are needed and pulling it together.
Most of the dependencies are taken care of. I needed to add the Extras and Extras-devel repositories to get python 2.5 and goocanvas. Right now python2.5-goocanvas doesn’t depend on libgoocanvas3, which IMO is a bug.
In the repositories I looked in, vobject and dateutil don’t seem to be available. These are needed for Conduit to go near VCard and iCal data. So I planned to grab the ubuntu/debian package and rebuild it for maemo. Not that easy, as I couldn’t find python-support or python-central. Damn. I’ve rolled my own python2.5-dateutil and python2.5-vobject and hope to get theses into extras or extras-devel shortly.
The Conduit package was relatively easy to fork from the Ubuntu PPA. I added a patch to set the default UI to Hildon instead of GTK, and then removed and tweaked dependencies. There are quite a few bugs to work out though:
- Need to reboot N810 to see Conduit in menu (Packaging bug; presume there is a macro to refresh this?)
- Conduit menu icon doesn’t work (Conduit bug; conduit/conduit script assumes bash.. which is bad).
- At the moment, I’m installing manually via ssh/apt/dpkg.. Need a one click install file (Packaging bug)
- Too big – remove manual and gtkui from package. (This needs changes in Conduit and Packaging, I think)
- Too many dataproviders are showing that won’t work on N810. Shouldn’t really show Tomboy unless tomboy is installed.
I hope to address all the important problems in the morning and send something out to the Conduit mailing list so we can get some proper testing done. A second post with screenshots and quick install link will follow.
Looks shiny. Thanks very much to the hard work of thomasvm for making this possible
Tags: Conduit
Posted in Nerd | Comments Off
