Olav Vitters

Regarding:

This was more annoying, since there is some script there in s.g.o’s guts that rejects any commit if the repo has an executable.

This is a simplification. Executable files are allowed. There is a pre-commit hook to disallow executable .po files, but that check is only done for .po files added/changed by the commit. It does not check for other .po files. If you see otherwise, please contact svnmaster@gnome.org. Normally developers shouldn’t hit this check btw (only when you’re changing the .po file).

Just wanted to respond via the planet on one thing:

It is getting increasingly hard to get new stuff into Gnome, and when someone approaches with something that is slightly controversial huge flamewars erupt. Consider Tracker and Empathy on the desktop-devel list and the recent “incident” on the gtk-devel list.

Ignoring Tracker (too long ago and I’d rather see that standard used instead of deciding between Tracker/Beagle), I don’t think the other examples are good. For one, the gtk-devel list was not regarding getting new stuff into GNOME. Further, the flamewar started because the person was clearly trolling (I had to resort to moderating!). The idea itself did not cause the flamewar.

Empathy has a few issues (not using gnome-keyring and the license). This doesn’t support your statement. If those are fixed, I’d give a +1 during the r-t meeting. From the last time we discussed Empathy, I felt these two items was the only reason Empathy wasn’t accepted.

Every release cycle we accept a few things into GNOME. We do ask for a certain quality, e.g. making releases on time and responding to the incoming bugs. Sometimes a proposed module only made one release on time. We do look at future possibilities of a certain module (good enough and shows promise is what I look for), but the basics shouldn’t be ignored.

Note that the distribution upgrade of svn.gnome.org has been completed. AFAICS everything is working. If you have issues, please ping me (bkor) or another person with ops in #sysadmin (irc.gnome.org).

PS: I do expect loads of commits now

This week I took a look at the moderation queue of a GNOME mailing list. There were loads of messages in it. There is a moderation team who looks at these queues and cleans it up, by discarding the spam and accepting the valid messages. The moderation queue of the mailing list I looked at had lots of similar spam messages over various days. To avoid newer type of spam messages, every day/hour (forgot how often) the Spamassassin rules are updated. These rules includes the ones from Sare. There is a big anti-spam gap in this as the new rules might not catch the things the moderators have classified as spam/ham.

To make the process more intelligent, I’ve added a patch to Mailman to allow moderators to use the discarded/accepted messages to train the Bayes classifier used by Spamassassin. The way it works is hackish, but very simple to implement. I’ve added a patch to our Mailman package which forwards all discarded and accepted messages to a special user. This user has a ~/.procmailrc file to divide these messages in two maildir folders. A script runs via cron to train sa-learn on the spam and ham folders. Sa-learn understands directories, avoiding the need to start sa-learn per spam/ham message.

Hopefully this will result in less spam messages for the moderators to classify.

A screenshot of the new functionality:

I’ve upgraded the Mailman version on mail.gnome.org to 2.1.10. In this version I redid the post-only patch. Basically, if the default action for new subscribers is to moderate them (done on e.g. metacity-devel-list), then members subscribed to post-only won’t be automatically accepted. In 2.1.10 Mailman supports something like post-only by default (see NEWS). I found it easier to patch this logic in than to change existing lists and to ensure new lists would get the post-only setting as well.

If you see any problems, please email gnome-sysadmin. Note that yesterdays email backlog wasn’t caused by the Mailman upgrade. I waited until it cleared.

A while ago I added some basic SSH public key checking to Mango. This for two reasons:

• Avoid email back and forth in case of copy/paste error
• To ensure the key is long enough

The key length checking was mostly a hack. I did that by checking if the number of bytes was larger than the number of bytes in a test SSH public key. I never actually knew how many bits the key had. The added benefit was that I figured out how the fingerprint is generated. I always wanted to determine the key length, but couldn’t be bothered. Of course, I could’ve used ssh-keygen, but I don’t like to start another process. It doesn’t actually matter to do a full check. This as the person providing the key wants it to work. We’ll probably get a mail when the public key was broken on purpose. Exception being the key length, as we don’t want too short keys (security issue).

Today I figured out how to determine the key length for RSA and DSA public keys. I’ll start with an example public key (wrapped for readability):

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAP8zwqYE675bpnYzui0pLNd2XyoB+
v4RtlK2QJ6+42w3VWREbDfeeUmvenLBzdcffs602WOuWB1DrbhjEv4CbABH/u
O89IMlC4h62wel7BfiQqEq6yKW0B+yqQxIsBQPhu8ID0gXrt0uhlPaHkqD1XR
WM9ywr5UP1K51cTPRZu8xQVtpCDMgppa/FwZTKY/+l3HXvu01/NAaNGMPOD3y
neIturzKi3x4f5Id65V1KD70B5YiCiJxFSevOcPx3yYYy+NQN52EBGLf76a78
1S2MiPcHxhoQtG8EPfAhWN3MmOnEd4iuy2IzHdyAK+LCp5Qtyy3mbKTBKSKQb
vrsm8jrjE= olav@bkor.dhs.org

The ssh-rsa is the key type; ssh-rsa for RSA and ssh-dss for DSA. After that there is a space and then the data (easily recognizable as base64 encoded). After the base64 data there is another space followed by a comment (contents doesn’t matter for SSH, you can change it with a text editor). The fingerprint of a key is nothing more than the MD5 hash of the base64 decoded data. The md5 hash of the example public key is 6f8c83c826ee51535a813756ff1bc9b5. The ssh-keygen program shows this with colons.

If you’ve compared a few public SSH keys, you probably noticed that the start of the base64 data is always the same (for the same key type). This is because the data itself again contains the key type (’ssh-rsa’). The data is encoded using a big endian number (4 bytes) providing the number of bytes of the string that follows. A hex editor will show 00 00 00 07 for the first 4 bytes. After these 4 bytes follows either ’ssh-rsa’ or ’ssh-dsa’. The rest of the data is key type specific.
For RSA, the key type is followed by 2 strings encoded using the same method. These strings actually represent ‘bignum’s. However, I don’t care about that. For DSA, 4 strings (bignums) follow after the key type, also encoded using the same method. For RSA you’ll want the 2nd bignum. For DSA the 1st bignum.

When you have such a bignum you’re very close to determine the key length. It is mostly just determining the number of bits needed for the bignum. You could do this by determining the number of bytes used by the bignum and multiplying it by 8. However, you have to deduct a few bits. This as the first byte causes the bignum to use more bits than it actually needs. For example, if the following are the first two bytes (shown as binary) out of the total 255 bytes used for the bignum:

00110011 11000010

You’ll note that the first two 0’s aren’t actually needed. So instead of calculating 255*8 (2040), you should subtract it by 2 bits. Resulting in a key length of 2038. This is what ssh-keygen will give you if you try the example public ssh keyfile shown above (make sure to unwrap the lines and remove the spaces in the base64 encoded data). Looking back, it actually is pretty easy.

Oh, and the reason why I want to know the exact key length is to add a check against blacklisted keys. For this I need to know the actual length of a key. I figured above out partly myself, partly by reading python-paramiko source (unfortunately I overlooked the public key reading part) and partly by trying to understand the openssh code. Running ssh-keygen would’ve been loads easier, but IMO nasty (especially when someone has multiple keys) and also not as interesting. Further, if you use Python instead of PHP (blergh!), just use python-paramiko.

As you could’ve read in my previous post, I plan to upgrade the svn.gnome.org server on:

This means that svn.gnome.org is NOT available at that time.

Suggest to do the following:

1. Use the Bzr mirror. Suggest to read the instructions on the wiki
2. Figure out Git. See the Git with GNOME instructions. Don’t forget: With Git you’ll need to do some work before the downtime (convert a SVN repos into a git-svn repos).

Note: Please *wait* with committing until *either* 10:00 UTC (hopefully;) *or* when it is announced as back up (#gnome-hackers on GIMPNet and devel-announce-list). This as I obviously will need to test it after it has been upgraded.

For those using a DVCS, I’d love to see a repeat of http://tinyurl.com/5yvgc3.

PS: Please subscribe to devel-announce-list.

Update: It is 8 Jun, not 9 (subject was correct, just not the text itself).

The server behind svn.gnome.org still runs the previous Ubuntu LTS (Dapper / 6.06). I want to upgrade this to the latest Ubuntu LTS, this being Hardy / 8.04. The upgrade itself should not take more than 30min, but the downtime will be longer than that (rsync’ing everything to another machine). I’ll setup another machine to handle SVN in case the distro upgrade the distro upgrade fails in unexpected ways.

I’m currently thinking of Sat 7 or Sun 8 Jun 2008. I prefer Sunday mornings (CEST / UTC +2) as hopefully not too many people are online. Further, the most important users of svn.gnome.org are developers and I expect less usage in the weekend. The weekend is more popular for translators though. It also isn’t just before an important release.

Above date is not fixed, I still have to check if there is someone standing by with physical access.

Anyone think above is a bad idea? Better suggestions regarding when to schedule this? Note: During the week it has to be in the evening (UTC +2).

Note: After the upgrade, the repository format has to be changed, but I’ll do that later. It shouldn’t be more than a few minutes per repository (due to converting it twice, first the whole repository, then disallowing commits, then doing the commits that have been made when the conversion was running). The repository format should allow things like svnsync, plus repository size should be smaller. See http://subversion.tigris.org/svn_1.4_releasenotes.html.

An answer regarding page loading problems:

Gave up on hoping that Gnome would serve my blog sanely – apparently asking for it is a denial of service attack. Duplicated it at http://www.go-oo.org/~michael/blog/index.atom.

This is not what I said or meant on IRC. There is a DDoS going on. Nothing too heavy. However, it really slows down the database, slowing down Bugzilla until a crawl. Due to the type of DDoS it is almost impossible to block (too many IPs, etc), resulting in unintended side-effects. Initially I thought I could add the workaround pretty quickly. However, was too difficult to do at UDS. I noted the delay in #sysadmin, so if you’ve asked there someone hopefully would’ve repeated in case I wasn’t around. I did inform the sysadmins regarding the config change (mailing list and IRC) causing these page load issues. Due to abuse avoidance (often if you make it known, seems others see a need to repeat/join) I did not inform anyone else. I do hope the DDoS doesn’t morph into something I cannot block.