Olav Vitters

Up to today LDAP replication was done using slurpd-style replication. In this setup the LDAP master pushes changes to the various slaves. For this to work the slave must be accessible by the master. This was setup using encryption (self signed certificates). This replication method is deprecated as of OpenLDAP 2.3 in favour of ’syncrepl’. OpenLDAP 2.4 doesn’t even have slurpd-style replication. Changing the replication method is important as I want to be able to upgrade to RHEL6 without being forced to do other changes at the same time.

The biggest drawback about slurpd is that it only replicates the changes. Meaning: If you have a new slave, you first have to stop the master, dump the database, then load this database on the slave, then start the slave and the master. Really annoying and during this time your master LDAP server is down!

With syncrepl-style replication the slaves contact the master. Slaves are now called consumers and the master is the provider. I don’t like consumers contacting the provider as I don’t want a security issue on some external host to cause security problems on the LDAP master db. I’ve tried to minimize the risk by using encryption (self signed certificates, server won’t accept any other certificates), using iptables, restricting the LDAP logon to just one user (to ensure brute forcing userPassword entries by logging in often will not work), not replicating the userPassword attribute, not allowing database modifications from external clients and some ACL restrictions.

There are two things I like about syncrepl.

1. After it connecting with the provider it checks the consistency of the database. At one point the SVN machine missed just one uid entry. That person couldn’t log in. With sycnrepl such a problem is not possible (ignoring possible config errors like ACLs).
2. You can restrict what is replicated to the slave using ACLs on the master.

Syncrepl has two ways of working, using refreshOnly it checks for LDAP changes every x minutes (configurable). This isn’t as nice as slurpd as I want all LDAP changes to be propagated instantly (even if the SSH sync script only runs once an hour). Fortunately it also has a refreshAndPersist, where it first checks and then waits for change notifications from the provider.

Tips to ensure a working syncrepl:

• Make sure the DNS entry points to the right server
• Make sure that DNS entry has not been manually added in /etc/hosts
• Make sure the userid running slapd can actually access the various certificate files
• If you test using ldapsearch, make sure /etc/openldap/ldap.conf has a TLS_CACERT entry
• Make sure the TLS_CACERT file can actually be read by the user running ldapsearch
• slapd (etc) -d 1023 contains the debug messages, you won’t find them anywhere else (by default)
• Kill any running nscd and wipe it off the system
• etc etc etc

Really excited that possibly we’ll have a Akademy and GUADEC at the same place next year. I’m starting to see KDE at this GUADEC already:

Yeah, not entirely the same, but it is still 2008, not 2009.

There will be a Bazaar demo today at 15.30 in the Medium room. This wasn’t on the schedule before. They’ll demonstrate not only how to use Bazaar (using the Bzr mirror), but also show a few new features (o.a. search). Another item is a playground server, allowing GNOME people with SSH keys to store their own branches. If you are interested in DVCS and/or Bazaar, suggest to attend this demo. It will be an open demo, feel free to ask them to demonstrate whatever you’d like to see. I’d love questions about how Bazaar could make your workflow easier (supporting what you do easily).

As preparation for next years combined event, I already noticed one KDE dude at GUADEC:

He said he’d participate in the GNOME release team meeting as well (starting in 20min).

I’m planning to stay at the following hotel: Istanbul Hotel Ipek Palas. Some other crazy Dutch guys will be staying there too (so you don’t have to prepare a map or something.. just rely on them ). Need a roommate though, anyone interested? I’ll arrive Sun 6 and leave Sat 12. If so, mail me, or post below. The hotel has free wifi btw.

<saned> hello. I'm wondering whether http://l10n.gnome.org/ has been defaced...
<saned> it says "Damned Lies about GNOME"
<saned> is it some kind of joke?

Weird. I think that just thinking about something like that being possible is very weird. Planet GNOME is one of the aggregators which shows every post, always (a full feed is often requested). The only exceptions (AFAIK) being: bugs, bugs and bugs (either planetplanet, GNOME server or when there is a server problem with the feed).

So to question the openness of GNOME Planet.. weird. I thought pgo had more trust than this. Wonder what can be done to change this impression.

I also don’t agree with:

Furthermore censorship on a service provided by those who consider themselves advocates of freedom and liberty is hypocritical to say the least.

Start a personal attack on a GNOME service and you can forget about mentioning freedom. Furthermore, freedom/liberty is to be able to announce something, not about being able to do that anywhere you please. Meaning: if GNOME provides a service, GNOME decides on what is allowed (but don’t forget about the beginning of this blog).
Lastly, I don’t see GNOME as an advocacy of freedom and liberty (just free software).

Why is Git for some only the one perfect DVCS and no discussion is possible? From Git is the only DVCS caring about performance incl outdated benchmarks, to the ‘all DVCS’ systems are hard’ and lastly ’switching VCS systems is easy’. Regarding the latter, who is going to do the conversion? I am fully willing to do Bazaar (perhaps Mercurial.. I’d first would have to investigate that). But Git? Even a ‘am I capable?’ would dismiss me from doing that. Anyway, from following the stuff before the CVS->SVN switch, to fixing loads of things afterwards, switching a VCS system not at all as easy as some make it out to be. Just the under estimation makes me worry.
When I see something which is broken, I cannot just leave it without trying to fix it (aside from lacking time and if I am able to, but ‘able’ is pretty easy with root access). From DNS stuff, to LDAP, accounts process (Mango), SVN repos creation, etc. Although I’d really love if more GNOME sysadmins would be active… anyway, if Git is considered to be the only option and the attitude regarding the ease of switching stays the same, I do not want to have the ability to fix whatever was forgotten or broken.

I wanted to rewrite Mango in Python for a while now, but couldn’t determine which framework to use. Mostly as these frameworks either do way too much or just not enough. Shaunm (and some others in #gnome-hackers) was so nice to convince me that Django was a really good framework. This as I hate frameworks. I want to ignore most of it. My basic needs were CSRF protection (not that I cannot do it myself, but I want to ensure they’ve thought about security), session, something like fastcgi/scgi (the Mango Python stuff needs to run under a different user) and templates etc should be optional.

For the rewrite I’ve decided to use the Bzr mirror. This while still publishing the branch on svn.gnome.org and having the ability to merge both ways (from trunk to django and django to trunk.. at one point). The initial setup for the tools I needed took some time, hopefully this will be resolved soon. This as Mandriva doesn’t have a bzr-svn package yet. I started off with bzr-svn.

Installing bzr-svn is pretty easy (only needed to push to svn.gnome.org, pulling can be done via bzr-mirror):

mkdir -p ~/.bazaar/plugins/
cd ~/.bazaar/plugins
bzr branch http://people.samba.org/bzr/jelmer/bzr-svn/bzr.dev/ svn

However, that plugin wants the right Subversion version and 1.5 wasn’t packaged for Mandriva Cooker yet. Compiling Subversion failed initially, as it somehow wanted to link to the installed subversion libraries (this while I did not have the 1.4 svn devel package installed). After I figured that out, there was another issue as the Subversion API changed with 1.5. The workaround for now is to remove the assertion on line 944 in subversion/libsvn_ra/ra_loader.c (or wait for the new bzr-svn plugin).

After getting the tools working, the rest is a bit magical. I followed the GNOME Bazaar instructions and branched Mango trunk locally in a django branch. A bzr push svn+ssh://svn.gnome.org/mango/trunk would be enough to push it back to GNOME (you need to specify the push URL once). However, I wanted a branch in SVN. The push command currently (?) will not create new branches, but there is a svn-push which does.

The command I used to create the new ‘upstream’ branch was:

$ bzr svn-push svn+ssh://svn.gnome.org/svn/mango/branches/django

The result from svn-commits-list:

Author: ovitters
Date: Sat Jun 21 17:58:09 2008
New Revision: 205
URL: http://svn.gnome.org/viewvc/mango?rev=205&view=rev

Log:
        Try and port the mess to Python using the Django framework.
        * .bzrignore: Ignore the django directory.
        * mango/manage.py: Standard Django script to manage the Django
        project.
                * mango/models.py: Describes the Database (needs a bit of work).
        * mango/settings.py: Django settings and the config.xml file.
        * mango/urls.py: Maps URLs to functions which will be called.
        * mango/views.py: Handles turning an http request into an http
        response (application/xml).

Added:
   branches/django/   (props changed)
      - copied from r204, /trunk/
   branches/django/.bzrignore
   branches/django/mango/
   branches/django/mango/manage.py   (contents, props changed)
   branches/django/mango/models.py
   branches/django/mango/settings.py
   branches/django/mango/urls.py
   branches/django/mango/views.py
Modified:
   branches/django/ChangeLog

From above email it is clear it copied (branched) the trunk and not just put new files into that Django branch. The resulting test Django installation works pretty similar as the current PHP Mango version:

Oh, and some way too early screenshots:
Current trunk (PHP)




Django branch

It seems nice, but that is just because I am sharing the XSLT templates. It doesn’t do anything useful yet. For instance, you can clearly see the missing ‘tabs’ and that it doesn’t filter the account requests. A Django LDAP model would also be nice (has a really nice abstraction for models), but was designed with database in mind unfortunately.

SUbversion 1.5.0 has been released. I plan to upgrade svn.gnome.org when 1.5.1 is released. See the announcement for full 1.5 details. Things I like are:

• Merge tracking
  
  There are still a few known issues with this. This is why I want to wait until 1.5.1.
• Sparse checkouts
• Relative URLs in svn:externals
  
  Allows stuff like /svn/someotherproject/trunk. Avoids the http vs svn+ssh.
• Memory leak fixed
  
  There was a memory leak which caused problems for bzr-svn and others.

SVN dump+reload script?
I still have to dump/reload all GNOME repositories for the server to use the SVN 1.4 repos format. I’d appreciate if someone has a smart script for this to minimize any downtime (by doing the dump+reload in two stages, first stage dump+reload, 2nd stage disables permissions, dumps+reloads any commits done during the 1st stage, then moves over the 1.4 SVN in place, lastly enabling access again.. this per repository).