muellis blog – Page 3 – …und alle so “Yeaahh!”

Talking at PET-CON 2017.2 in Hamburg, Germany

A few weeks ago, I was fortunate enough to talk at the 7th Privacy Enhancing Techniques Conference (PET-CON 2017.2) in Hamburg, Germany. It’s a teeny tiny academic event with a dozen or so experts in the field of privacy.

The talks were quite technical, involving things like machine learning over logs or secure multi-party computation. I talked about how I think that the best technical solution does not necessarily enable the people to be more private, simply because the people might not be able to make use of the tool properly. A concern that’s generally shared in the academic community. Yet, the methodology to create and assess the effectiveness of a design is not very elaborated. I guess we need to invest more brain power into creating models, metrics, and tools for enabling people to do safer computing.

So I’m happy to have gone and to have had the opportunity of discussing the issues I’m seeing. Likewise, I find it very interesting to see where the people are currently headed towards.

Talked at mrmcd 2017 in Darmstadt, Germany

I attended this year’s MRMCD in Darmstadt, Germany. I attended a few times in the past and I think this year’s edition was not as successful as the last ones. The venue changed this year, what probably contributed to some more chaos than usual and hence things not running as smoothly as they did. I assume it will be better next year, when people know how to operate the venue. Although all tickets were sold during the presale phase, it felt smaller than in the last years. In fairness, though, the venue was also bigger this year. The schedule had some interesting talks, but I didn’t really get around to attend many, because I was busy preparing my own shows (yeah, should’ve done that before…).

I had two talks at this conference. The first was on playing the children’s game “battleship” securely (video). That means with cryptography. Lennart and I explained how concepts such as commitment schemes, zero knowledge proofs of knowledge, oblivious transfer, secure multiparty computation and Yao’s protocol can be used to play that game without a trusted third party. The problem, in short, is to a) make sure that the other party’s ships are placed correctly and b) to make sure the other party answers correctly. Of course, if you get hold of the placements of the ships these problems are trivial. But your opponent doesn’t like you to know about the placements. Then a trusted third party would solve that problem trivially. But let’s assume we don’t have such a party. Also, we want to decentralise things, so let’s come up with a solution that involves two players only.

The second problem can be solved with a commitment. A commitment is a statement about a something you’ve chosen but that doesn’t reveal the choice itself nor allows for changing ones mind later. Think of a letter in a closed envelope that you hand over. The receiver doesn’t know what’s written in the letter and the sender cannot change the content anymore. Once the receiver is curious, they can open the envelope. This analogy isn’t the best and I’m sure there’s better real-world concepts to compare to commitment schemes. Anyway, for battleship, you can make the other party commit to the placement of the ships. Then, when the battle starts, you have the other party open the commitment for the field that you’re shooting. You can easily check whether the commitment verifies correctly in order to determine whether you hit a ship or water.

The other problem is the correct placement of the ships, e.g. no ships shall be adjacent, exactly ten ships, exactly one five-field ship, etc. You could easily wait until the end of the game and then check whether everything was placed correctly. But that wouldn’t be (cryptographic) fun. Let’s assume one round of shooting is expensive and you want to make sure to only engage if the other party indeed follows the rules. Now it’s getting a bit crazy, because we need to perform a calculation without learning anything else than “the ships are correctly placed”. That’s a classic zero knowledge problem. And I think it’s best explained with the magic door in a cave.

Even worse, we need to somehow make sure that we cannot change our placement afterwards. There is a brain melting concept of secure multi-party computation which allows you to do exactly that. You can execute a function without knowing what you’re doing. Crazy. I won’t be able to explain how it works in a single blog post and I also don’t intend to, because others are much better in doing that than I could ever be. The gist of the protocol is, that you model your functionality as a Boolean circuit and assign random values to represent “0” or “1” for each wire. You then build the truth table for each gate and replace the values of the table (zeros and ones) with an encryption under both the random value for the first input wire and the random value for the second input wire. The idea now is that the evaluator can only decrypt one value in the truth table given the input keys. There are many more details to care about but eventually you have a series of encrypted, or garbled, gates and you need the relevant keys in order to evaluate it. You can’t tell from the keys you get whether it represents a “0” or a “1”. Hence you can evaluate without knowing the other party’s input.

My other talk was about a probable successor of Return Oriented Programming: Data Oriented Programming (video). In Return Oriented Programming (ROP) and its variants like JOP the aim is to diverge the original control flow in order to make the program execute the attacker’s functionality. This, however, can probably be thwarted by Control Flow Integrity. In its simplest form, it checks on every branch whether it is legit. Think of a database with a list of addresses which are allowed to a list of other addresses. Of course, real-world implementations are more clever. Anyway, let’s assume that we’ll have a hard time exploiting our target with ROP, because we cannot change the CFG of the program. If our attack doesn’t change the CFG, though, we should be safe for anything that detects its modification. That’s the central idea of DOP.

Although I’m not super excited about this year’s edition, I’m looking forward to seeing the next year’s event. I hope it’s going to be a bit more organised; including myself 😉

Talking at FrOSCon 2017

I attended my first FrOSCon in St. Augusting, Germany. It’s one of the bigger Free Software events in Germany. Supposedly, the Chemnitzer LinuxTage is one of the few events which are bigger than FrOSCon. I thought it’s time for me to attend this event, so I went.

I was scheduled for two talks. One in the very first slot and one in the very last slot. So, to some extent, I was opening and closing the conference 😉 But the official keynote was, to my surprise, performed by Karen. She keynoted the conferences with her “big heart” talk. He told her story about her wanting to find out what software her pacemaker runs. Of course, it was an endless quest with no success. She described herself as a cyborg because of the machinery that is linked up to her body. She researched the security of devices such as pacemakers and found devastating results. In fact, software is deployed in many critical parts with people having no clue how the impact will be if the software is being attacked. She described the honeymoon effect and projected it to the security aspects of deployed software. She described it as a time in which no vulnerabilities are known. But once a vulnerability has been found, the number of known vulnerability increases exponentially. She found a study which shows that Free Software responds better to found vulnerabilities than proprietary systems. She said she went from thinking “Open Source was cool” to “Open Source is essential” because it responds much better in case of security breaches. She cautioned us to be careful with the Internet of Things™, because it will lead to people being connected without the people even knowing. All software has bugs, she said, but with Software Freedom we are able to do something about the situation. It’s been an enjoyable talk and I recommend watching the video.

Another interesting talk was given by Raffa about open data in public transport. Open data, especially in trip planning, can give us better results, he said, because personal preferences can be respected better. But also competition will become tougher if the data is free which might lead to better products. My personal argument in favour of open data is that it would allow offline routing rather than having to connect to the Internet. Some public transportation companies have freed their data, like the companies in Berlin, Ulm, Rhein Neckar, and Rhein Sieg, which is, funnily enough, the local company responsible for the public transport in the area of the event. However, some companies are still hesitant. The reasons are manifold. One is that they don’t want to deal with complaints about wrongly displayed data or simply outdated data that the third party didn’t bother to update. Also abuse is a concern. What would abuse even mean in this context? Well, some companies are afraid that the data is not only being used for trip planning but for finding out how the companies work or what their financial situation is, e.g. by inferring information from the data.

Andreas Schreiber talked about the complications of Open Source in Science. He works at DLR, which is a publicly funded research institute. Software is important to the DLR. 1500 people develop software which costs around 150 Million EUR per year and makes them probably the biggest software house in Germany, he said. As they are producing as releasing software they got in trouble with licensing issues. For example, they released software which was not open source although they thought it was. They also used software themselves which they may not have been entitled to use. Their CIO eventually issued a warning regarding the use and release of Open Source which made the speaker offer workshops and knowledge databases for issues around open source. They created a brochure which they intend to distribute to other institutes, too, because they tend to get more requests for this kind of information from the outside than from the inside of their organisation. I found interesting that the problems, according to the participants of their workshops, are that monetising won’t work, that building a community is hard, and that it costs more time to do “open source” than not which is demotivating. It’s been interesting to learn about issues involved in both consuming and producing open source software.

As I’ve mentioned, I was booked for two events, a talk and a workshop. My workshop was about signing OpenPGP keys. I held a small presentation and ranted, some times a bit unfairly, about the current state of affairs. I showed how people do it as of now and how I think we can do better than that. It’s been the first slot in this conference and the audience was small, albeit larger than expected. We even got to suggest improvements in Gentoo’s packaging, so I consider it a success. My talk (slides) was about how GNOME advances the security of desktop systems. The audience was super engaged and I felt I couldn’t focus so much on other things I only touched upon. But the discussion showed that people do care about a usable desktop. We were talking a lot about dialogues and modal prompts and how they do not contribute to the security of a system. I claimed that they exist because they were cheap for the app developer to do. But we at GNOME, I said, try or at least should try to avoid those as much as possible and we find other ways of enabling the app to capture the user’s intent. I’m surprised that we had such a lively discussion in the very last slot of the conference.

I’m happy to have attended the event and to meet surprisingly many GNOME people! It’s surprisingly close to Frankfurt and Cologne both of which have good connections via plane or train. With around 1800 attendees it’s quite big although the many tracks and rooms make it feel less crowded.

GUADEC 2017

It’s summer and it’s GUADEC time! This year’s GUADEC took place in Manchester, England. It was surprisingly less bad for that location 😉 The organisers deserve a big round of applause for having pulled the event off. After having organised last year’s GUADEC I have first hands experience running such an event. So a big “thank you” to the team from England 🙂

The venue was a big and modern university and the accommodation was neatly located a few footsteps from the lecture hall. That’s especially nice for the typical English weather 😉 We got to live in the student dorms and I’m a bit jealous of today’s student to be able to live in such a comfortable place.

I attended a few talks from the list, among them was Christian Hergert reporting on The State of Builder which was a bit scattered and not very well structured for beginners like me. I guess was meant to be more of a showing off new features instead of a structured walk through the design and thoughts behind the project. I knew the project existed but I never really got around to work with it so I was a bit put off. But I took that for a good opportunity for installing the latest Flatpaked application 🙂

I liked Simon’s talk on enabling users to modify the software they are running. Essentially, you can click a button in the application and it’ll fire up an IDE where you can change code and hit “play” to run the new version. Amazing. Software Freedom at its best. He demoed a prototype and I think it’s got potential. I really like the idea of the user being able to tinker around easily. Especially given that the status quo is jhbuild. That’s a nice tool, but it proves to be hard for people to make good use of it. I hope we will see something like this being used in the future.

Federico was telling us about the efforts to make use of the Rust language for GNOME. The gist is, essentially, that you better start with leaf functions of your app or library rather than a central function in your architecture. I then tried to find leaf functions with the help of the compiler, but I failed. I tried Egypt but I wasn’t patient enough to make proper use of the generated dot file in order to identify leaf functions. Maybe I should give cflow a try next time.

I used the BoF days to dip a little bit into Rust. It’s always helpful to have a bunch of smart hackers around. That’s what I like about these kind of events. You get to know and talk to very smart people. I also tried to catch up with my very talented student and discuss the changes we’d like to see.

Thanks to the GNOME Foundation for sponsoring my travel and to the local team for having organised a successful event!

OpenSuSE Conference 2017 Nuremberg, Germany

As last year I was honoured to be invited to OpenSuSE Conference in Nuremberg, Germany.

The event has grown and I felt a relaxed yet productive atmosphere when entering the venue. Just a few minutes after I arrived I hooked up with interesting people with even more interesting discussions. It was very nice to get together with all the Free Software friends I made over the last years. It was also pleasent to see the event becoming bigger and bigger. I take that as a sign that our community grows although it might also just be consolidation of events.

The organision team provided everything from Brazilian BBQ to perfect weather 🙂 The schedule hosted interesting sessions, including mine of course 😉 I had a small workshop on signing OpenPGP keys and I made people use GNOME Keysign. 😉 It was successful in the sense that we were able to shake out a few bugs to make the application more robust. I also realised that networks are not very permissive nowadays. More precisely, the WiFi blocked mDNS traffic preventing the apps from finding each other 🙁 One design goal of the app was to not have to rely on an Internet connection. But if the networks prevent clients from talking to each other then I think we need to go via the Internet in order to transfer files locally :/ Fortunately, we are working on an Internet transport. Stay tuned for further posts on this issue.

Oh, and we even had a GNOME stand full of amazing stuff.

Talking at GPN 2017 in Karlsruhe, Germany

Although the GPN is an annual event, I haven’t managed to go very often. Last time has already been a while. It’s a pity, because the event is very cute. The location is just amazing which makes being there really fun. It’s a museum hosting many things from our digital world. If you visit only one thing in Karlsruhe, go and visit it. In fact, we tried to organise a small excursion during GUADEC last year. Bloomberg also has an article about the event.

I could only stay one day, but I opened the conference with my talk on building a more secure operating system without sacrificing usability which, of course, was a GNOME related talk. The room was packed and people had to sit on the floor. Based on the feedback I think the people liked to be explained what challenges are to be solved in order to ship more secure systems to more people. You can find the slides here. In case you want to practise your German, you can watch the video here.

The schedule had a few other gems, too. My favourite was Loeschi talking about the upcoming Smart Meter Gateway situation in Germany and how it compares to the rest of Europe. The talk about QR Codes was also nicely done and explained quite well how they work. I hope to be able to attend the event more often 🙂 Especially because I wish the Free Software and the “hacker” people would mingle a bit more.

GNOME Keysign 0.9 released

Oh boy, it’s been a while that we’ve released GNOME Keysign 0.9. We changed quite a few things since last time I’ve reported but the most visible change are the new widgets which I already announced last time. Now it should be much easier to make changes with the GUI and experiment with designs.

Other changes include less visible things like the ability to run the program in a VM. We use gtksink now which not only reduces the amount of code we have to maintain but also makes it easier for us to maintain compatibility with different display servers. Similarly, we don’t use the v4l2src but rather a autovideosrc hoping that it will be more compatible with other platforms.

If you want to try to new version, the instructions in the README should get you going:

pip install --user 'git+https://github.com/GNOME-Keysign/gnome-keysign.git#egg=gnome-keysign'

Alternatively, you may try the Debian or openSuSE package. The Flatpak is still work in progress as we still need to figure out how to work with GnuPG running on the host.

The future will bring exciting changes, too. I plan for i18n support and more Python 3 compatibility.

Installing a “full” disk encrypted Ubuntu 16.04 Hetzner server

I needed to install a server multiple times recently. Fully remotely, via the network. In this case, the machines stood at Hetzner, a relatively large German hoster with competitive prices. Once you buy a machine, they boot it into a rescue image that they deliver via PXE. You can log in and start an installer or do whatever you want with the machine.

The installation itself can be as easy as clicking a button in their Web interface. The manual install with their installer is almost as easily performed. You will get a minimal (Ubuntu) installation with the SSH keys or password of your choice deployed.

While having such an easy way to (re-)install the system is great, I’d rather want to have as much of my data encrypted as possible. I came up with a series of commands to execute in order to have an encrypted system at the end. I have put the “full” in the title in quotes, because I dislike the term “full disk encryption”. Mainly because it makes you believe that the disk will be fully encrypted, but it is not. Currently, you have to leave parts unencrypted in order to decrypt the rest. We probably don’t care so much about the confidentiality there, but we would like the contents of our boot partition to be somewhat integrity protected. Anyway, the following shows how I managed to install an Ubuntu with the root partition on LUKS and RAID.

Note: This procedure will disable your machine from booting on its own, because someone will need to unlock the root partition.

shred --size=1M  /dev/sda* /dev/sdb*
installimage -n bitbox -r yes  -l 1 -p swap:swap:48G,/boot:ext3:1G,/mnt/disk:btrfs:128G,/:btrfs:all  -K /root/.ssh/robot_user_keys   -i /root/.oldroot/nfs/install/../images/Ubuntu-1604-xenial-64-minimal.tar.gz


## For some weird reason, Hetzner puts swap space in the RAID.
#mdadm --stop /dev/md0
#mdadm --remove /dev/md0
#mkswap /dev/sda1
#mkswap /dev/sdb1

mount /dev/md3 /mnt
btrfs subvolume snapshot -r /mnt/ /mnt/@root-initial-snapshot-ro

mkdir /tmp/disk
mount /dev/md2 /tmp/disk
btrfs send /mnt/@root-initial-snapshot-ro | btrfs receive -v /tmp/disk/ 
umount /mnt/

luksformat -t btrfs  /dev/md3 
cryptsetup luksOpen /dev/md3 cryptedmd3

mount /dev/mapper/cryptedmd3  /mnt/

btrfs send /tmp/disk/@root-initial-snapshot-ro | btrfs receive -v /mnt/
btrfs subvolume snapshot /mnt/@root-initial-snapshot-ro /mnt/@

btrfs subvolume create /mnt/@home
btrfs subvolume create /mnt/@var
btrfs subvolume create /mnt/@images


blkid -o export /dev/mapper/cryptedmd3  | grep UUID=
sed -i  's,.*md/3.*,,'   /mnt/@/etc/fstab
echo  /dev/mapper/cryptedmd3   /     btrfs defaults,subvol=@,noatime,compress=lzo  0  0  | tee -a /mnt/@/etc/fstab
echo  /dev/mapper/cryptedmd3   /home btrfs defaults,subvol=@home,compress=lzo,relatime,nodiratime  0  0  | tee -a /mnt/@/etc/fstab

umount /mnt/
mount /dev/mapper/cryptedmd3 -osubvol=@ /mnt/

mount /dev/md1 /mnt/boot

mv /mnt//run/lock /tmp/
chroot-prepare /mnt/; chroot /mnt


passwd

echo  "termcapinfo xterm* ti@:te@" | tee -a /etc/screenrc
sed "s/UMASK[[:space:]]\+022/UMASK   027/" -i /etc/login.defs  
#echo   install floppy /bin/false  | tee -a    /etc/modprobe.d/blacklist
#echo "blacklist floppy" | tee /etc/modprobe.d/blacklist-floppy.conf

# Hrm. for some reason, even with crypttab present, update-initramfs does not include cryptsetup in the initrd except when we force it:
# https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1256730
# echo "export CRYPTSETUP=y" | tee /usr/share/initramfs-tools/conf-hooks.d/forcecryptsetup



echo   cryptedmd3 $(blkid -o export /dev/md3  | grep UUID=) none luks     | tee  -a  /etc/crypttab
# echo   swap   /dev/md0   /dev/urandom   swap,cipher=aes-cbc-essiv:sha256  | tee  -a  /etc/crypttab


apt-get update
apt-get install -y cryptsetup
apt-get install -y busybox dropbear


mkdir -p /etc/initramfs-tools/root/.ssh/
chmod ug=rwX,o=   /etc/initramfs-tools/root/.ssh/
dropbearkey -t rsa -f /etc/initramfs-tools/root/.ssh/id_rsa.dropbear

/usr/lib/dropbear/dropbearconvert dropbear openssh \
        /etc/initramfs-tools/root/.ssh/id_rsa.dropbear \
        /etc/initramfs-tools/root/.ssh/id_rsa

dropbearkey -y -f /etc/initramfs-tools/root/.ssh/id_rsa.dropbear | \
        grep "^ssh-rsa " > /etc/initramfs-tools/root/.ssh/id_rsa.pub



cat /etc/initramfs-tools/root/.ssh/id_rsa.pub >> /etc/initramfs-tools/root/.ssh/authorized_keys

cat /etc/initramfs-tools/root/.ssh/id_rsa

 
update-initramfs -u -k all
update-grub2

exit

umount -l /mnt
mount /dev/mapper/cryptedmd3 /mnt/
btrfs subvolume snapshot -r /mnt/@ /mnt/@root-after-install

umount -l /mnt

Let’s walk through it.

shred --size=1M /dev/sda* /dev/sdb*

I was under the impression that results are a bit more deterministic if I blow away the partition table before starting. This is probably optional.

installimage -n somehostname -r yes -l 1 -p swap:swap:48G,/boot:ext3:1G,/mnt/disk:btrfs:128G,/:btrfs:all -K /root/.ssh/robot_user_keys -i /root/.oldroot/nfs/install/../images/Ubuntu-1604-xenial-64-minimal.tar.gz

This is Hetzner’s install script. You can look at the script here. It’s setting up some hostname, a level 1 RAID, some partitions (btrfs), and SSH keys. Note that my intention is to use dm-raid here and not btrfs raid, mainly because I trust the former more. Also, last time I checked, btrfs’ raid would not perform well, because it used the PID to determine which disk to hit.

mdadm --stop /dev/md0 mdadm --remove /dev/md0 mkswap /dev/sda1 mkswap /dev/sdb1

If you don’t want your swap space to be in the RAID, remove the array and reformat the partitions. I was told that there are instances in which it makes sense to have a raided swap. I guess it depends on what you want to achieve…

mount /dev/md3 /mnt btrfs subvolume snapshot -r /mnt/ /mnt/@root-initial-snapshot-ro

mkdir /tmp/disk mount /dev/md2 /tmp/disk btrfs send /mnt/@root-initial-snapshot-ro | btrfs receive -v /tmp/disk/ umount /mnt/

Now we first snapshot the freshly installed image not only in case anything breaks and we need to restore, but also we need to copy our data off, set LUKS up, and copy the data back. We could also try some in-place trickery, but it would require more scripts and magic dust.

luksformat -t btrfs /dev/md3 cryptsetup luksOpen /dev/md3 cryptedmd3 mount /dev/mapper/cryptedmd3 /mnt/

Here we set the newly encrypted drive up. Remember your passphrase. You will need it as often as you want to reboot the machine. You could think of using pwgen (or similar) to produce a very very long password and save it encryptedly on a machine that you will use when babysitting the boot of the server.

btrfs send /tmp/disk/@root-initial-snapshot-ro | btrfs receive -v /mnt/ btrfs subvolume snapshot /mnt/@root-initial-snapshot-ro /mnt/@

Do not, I repeat, do NOT use btrfs add because the btrfs driver had a bug. The rescue image may use a fixed driver now, but be warned. Unfortunately, I forgot the details, but it involved the superblock being confused about the number of devices used for the array. I needed to re-set the number of devices before systemd would be happy about booting the machine.

btrfs subvolume create /mnt/@home btrfs subvolume create /mnt/@var btrfs subvolume create /mnt/@images

We create some volumes at our discretion. It’s up to you how you want to partition your device. My intention is to be able to backup the home directories without also backing up the system files. The images subvolume might become a non-COW storage for virtual machine images.

blkid -o export /dev/mapper/cryptedmd3 | grep UUID= sed -i 's,.*md/3.*,,' /mnt/@/etc/fstab echo /dev/mapper/cryptedmd3 / btrfs defaults,subvol=@,noatime,compress=lzo 0 0 | tee -a /mnt/@/etc/fstab echo /dev/mapper/cryptedmd3 /home btrfs defaults,subvol=@home,compress=lzo,relatime,nodiratime 0 0 | tee -a /mnt/@/etc/fstab

We need to tell the system where to find our root partition. You should probably use the UUID= notation for identifying the device, but I used the device path here, because I wanted to eliminate a certain class of errors when trying to make it work. Because of the btrfs bug mentioned above I had to find out why systemd wouldn’t mount the root partition. It was a painful process with very little help from debugging or logging output. Anyway, I wanted to make sure that systemd attempts to take exactly that device and not something that might have changed.

Let me state the problem again: The initrd successfully mounted the root partition and handed control over to systemd. Systemd then wanted to ensure that the root partition is mounted. Due to the bug mentioned above it thought the root partition was not ready so it was stuck on mounting the root partition. Despite systemd itself being loaded from that very partition. Very confusing. And I found it surprising to be unable to tell systemd to start openssh as early as possible. There are a few discussions on the Internet but I couldn’t find any satisfying solution. Is it that uncommon to want the emergency mode to spawn an SSHd in order to be able to investigate the situation?

umount /mnt/ mount /dev/mapper/cryptedmd3 -osubvol=@ /mnt/


mount /dev/md1 /mnt/boot

mv /mnt//run/lock /tmp/ chroot-prepare /mnt/; chroot /mnt

Now we mount the actual root partition of our new system and enter its environment. We need to move the /run/lock directory out of the way to make chroot-prepare happy.

passwd

We start by creating a password for the root user, just in case.

echo "termcapinfo xterm* ti@:te@" | tee -a /etc/screenrc sed "s/UMASK[[:space:]]\+022/UMASK 027/" -i /etc/login.defs #echo install floppy /bin/false | tee -a /etc/modprobe.d/blacklist #echo "blacklist floppy" | tee /etc/modprobe.d/blacklist-floppy.conf

Adjust some of the configuration to your liking. I want to be able to scroll in my screen sessions and I think having a more restrictive umask by default is good.

echo "export CRYPTSETUP=y" | tee /usr/share/initramfs-tools/conf-hooks.d/forcecryptsetup

Unless bug 1256730 is resolved, you might want to make sure that mkinitramfs includes everything that is needed in order to decrypt your partition. Please scroll down a little bit to check how to find out whether cryptsetup is indeed in your initramdisk.

echo cryptedmd3 $(blkid -o export /dev/md3 | grep UUID=) none luks | tee -a /etc/crypttab # echo swap /dev/md0 /dev/urandom swap,cipher=aes-cbc-essiv:sha256 | tee -a /etc/crypttab

In order for the initramdisk to know where to find which devices, we populate /etc/crypttab with the name of our desired mapping, its source, and some options.

apt-get update apt-get install -y cryptsetup apt-get install -y busybox dropbear

Now, in order for the boot process to be able to decrypt our encrypted disk, we need to have the cryptsetup package installed. We also install busybox and dropbear to be able to log into the boot process via SSH. The installation should print you some warnings or hints as to how to further proceed in order to be able to decrypt your disk during the boot process. You will probably find some more information in /usr/share/doc/cryptsetup/README.remote.gz.

mkdir -p /etc/initramfs-tools/root/.ssh/ chmod ug=rwX,o= /etc/initramfs-tools/root/.ssh/ dropbearkey -t rsa -f /etc/initramfs-tools/root/.ssh/id_rsa.dropbear


/usr/lib/dropbear/dropbearconvert dropbear openssh \

        /etc/initramfs-tools/root/.ssh/id_rsa.dropbear \

        /etc/initramfs-tools/root/.ssh/id_rsa
dropbearkey -y -f /etc/initramfs-tools/root/.ssh/id_rsa.dropbear | \

        grep "^ssh-rsa " > /etc/initramfs-tools/root/.ssh/id_rsa.pub
cat /etc/initramfs-tools/root/.ssh/id_rsa.pub >> /etc/initramfs-tools/root/.ssh/authorized_keys

cat /etc/initramfs-tools/root/.ssh/id_rsa

Essentially, we generate a SSH keypair, convert it for use with openssh, leave the public portion in the initramdisk so that we can authenticate, and print out the private part which you better save on the machine that you want to use to unlock the server.

update-initramfs -u -k all update-grub2

Now we need to regenerate the initramdisk so that it includes all the tools and scripts to be able decrypt the device. We also need to update the boot loader so that includes the necessary Linux parameters for finding the root partition.

exit


umount -l /mnt

mount /dev/mapper/cryptedmd3 /mnt/

btrfs subvolume snapshot -r /mnt/@ /mnt/@root-after-install

umount -l /mnt

we leave the chroot and take a snapshot of the modified system just in case… You might now think about whether you want your boot and swap parition to be in a RAID and act accordingly. Then you can try to reboot your system. You should be able to SSH into the machine with the private key you hopefully saved. Maybe you use a small script like this:

cat ~/.server_boot_secret | ssh -o UserKnownHostsFile=~/.ssh/server-boot.known -i ~/.ssh/id_server_boot root@server "cat - >/lib/cryptsetup/passfifo"

If that does not work so well, double check whether the initramdisk contains everything necessary to decrypt the device. I used something like

zcat /boot/initrd.img-4.4.0-47-generic > /tmp/inird.cpio mkdir /tmp/initrd cd /tmp/initrd cpio -idmv < ../inird.cpio find . -name '*crypt*'

If there is no cryptsetup binary, something went wrong. Double check the paragraph above about forcing mkinitramfs to include cryptsetup.

With these instructions, I was able to install a new machine with an encrypted root partition within a few minutes. I hope you will be able to as well. Let me know if you think anything needs to be adapted to make it work with more modern version of either Ubuntu or the Hetzner install script.

Update: Ivan F. Villanueva B. from wikical sent this:

I have followed the instructions but installing Ubuntu 18.04 bionic by using::

installimage -n myhostname -r yes -l 1 -p swap:swap:32G,/boot:ext3:1G,/mnt/disk:btrfs:128G,/:btrfs: all -K /root/.ssh/robot_user_keys -i /root/.oldroot/nfs/install/../images/Ubuntu-1804-bionic-64-minimal. tar.gz

After the chroot command, the file /etc/resolv.conf were a symlink to a
non-existent file. I solved it by::

rm /etc/resolv.conf
echo “8.8.8.8” > /etc/resolv.conf

After that, `apt update` works.

Recent versions of dropbear works differently. Before generating the
initramfs, you need to::

cat /etc/initramfs-tools/root/.ssh/id_rsa.pub >> /etc/dropbear-initramfs/authorized_keys

See https://unix.stackexchange.com/q/411945

Also, ssh -o UserKnownHostsFile=~/.ssh/server-boot.known -i ~/.ssh/mykey root@5.9.78.44 “echo -ne \”mypassword\” >/lib/cryptsetup/passfifo”

Attended FOSDEM 2017

Unsurprisingly, the biggest European Free Software event happened in Brussels, Belgium a g a i n . I’m talking about FOSDEM, of course. It’s a fixed entry in many peoples calendar and always a good excuse to visit Brussels 🙂

I’m a bit late to report on what talks I managed to see as others have already covered some of the talks, but I still want to add some observations.

Richard Brown from SuSE talked about dinosaurs and resurrecting them (video). It was more about containerised apps than actual dinosaurs, though. The general theme was about repeating mistakes that we might or should have learned in the past. He started by mentioning that the Windows DLL Hell was a nightmare. You needed to test your application with each and every version combination of every possible library. The DLLs did not necessarily have ABI compatibility so it was very cumbersome to test. Windows 2000 brought Side-by-Side assembly, which is some form of DLL containerisation, he said. It uses separate memory space for each app and its DLLs. Programs can ship “private” DLLs in their application directory so you don’t necessarily break other apps with your DLL carrying the same name. This approach, however, still has issues: Security wise each app needs to update their libraries themselves rather than have them updated. So each app needed to build and ship their own updater which is not trivial to do. Legally it’s also interesting, he said, because bundling these DLLs may impose restrictions. Last but not least, you have to have the same DLL potentially multiple times on disk, because each app may ship the same DLL.

The contemporary software distribution model has its problems, too, he said. Compatibility with various distros is an issue, because each distro is slightly different. Each distribution also has their own pace of change which may be incompatible with the application in question, e.g. the distros may decide to ship an older version because they have tested it more. Different distributions have different libraries and versions thereof. Also, each distribution has different toolsets to package applications up for their environment. Application developers, however, don’t want to care about these details.

Containerised applications solve these issues. Maybe. He mentioned Flatpak, snappy, and Appimage. The latter is the oldest technology dating all the way back to 2003. The solutions have in common that they bundle the app and run it in some kind of container or sandbox. From his criteria, the compatibility issue is solved, because the libraries are in the bundles. Portability is solved, because all dependencies are shipped in the bundle. And the pace of change is up to the app developer.

The containerisations, though, make assumptions of a common standard base provided by the distributions. According to him, such a common standard base does not exist in a practical sense, though. With containerised apps, he said, we might be repeating history. He explained that we might get a security nightmare because each app needs to update their dependencies themselves. The question also begs whether all the libraries can actually be bundled and shipped. App developers are picking up the responsibilities that distros used to have. You still have to test everything on each distro just to be sure that your base dependencies still work correctly, he said. He sees distributions as part of the solution to these problems. He thinks that a rolling release might solve the issues we’re trying to solve with containserised apps. A rolling release can ship new releases of applications very quickly. The distribution still uses their tools for the common problems like maintenance, security, and legal stuff.

In a lightning talk, David talked about “practical TPM 2.0 usage”. He showed how to generate a signing key, sign a document with it, and verify the signature. He said that Microsoft mandated TPM2.0 for Windows 10 Mobile and that it is a cryptographic processor rather than an accelerator. TPM2.0 is different from TPM1.2 in various ways, he said. For example, the 2.0 can do ECC (P256 and BN256) and SHA-256. But it’s also “algorithm agile” which means that you can add algorithms without having to change the specification. He sees three main usages: Platform integrity like secure boot and trusted boot, disk encryption where the TPM stores and controls access to the key, and Digital Restriction Management by verifying code signatures. In order to use the TPM you have two options, he said. IBM or Intel have developed some tools. IBM doesn’t have a “resource manager” according to the specification. Like a multiplexer. Intel does have such a resource manager and they are working on putting that into Linux. However, Intel has less tools, he said, although it’s wasn’t entirely clear to me what he was referring to. He mentioned that his employer, Facebook, uses TPMs for platform attestation.

Hanno talked the security on the Linux desktop. He referred to the issues Chris Evans exposed a few weeks ago.
He wanted to make the audience angry, he said. But not at him, I suppose because he considers himself to be the messenger only. The basic problem is an unfortunate agglomeration of bugs or behaviours. It starts with the browser automatically downloading files into the users downloads folder, i.e. without asking the user. Then there is Tracker which indexes files that you add to your home directory. Such as the download folder. And then there are buggy (read: vulnerable) implementations of file parsers.

He also referred to Carlos’ comment about bugs being bugs and no problem being found except bugs being bugs. Hanno’s point, as far as I could make it out, was that a project of the size as tracker, especially with that number of dependencies that you don’t control, cannot make sure that there will be not yet another bug that can be exploited. That’s quite fatalistic but probably not too far from reality. It’s not just a Tracker issue, though, he said. KDE has Baloo and everybody wants to have thumbnails of the files in your folders. He reiterated that automatic downloads AND automatic indexing creates a huge attack surface. And that the indexers support a vast variety of file formats by using many libraries of varying quality. While Tracker quickly adopted sandboxing, he said, KDE hasn’t.

He mentioned other exploit mitigation techniques such as ASLR or CFI. With ASLR, he said, the idea is to load code and data into random addresses in memory. This mitigates exploits, because they cannot reliably target valid code in memory. A least that’s the idea. You need to compile the code with -fpic and -pie, he said. Linux distribution have been slow in adopting ASLR though. Ubuntu has introduced it with 16.10, Feora with 23, and Debian is WIP. OpenSuSE has it for a few packages only. It should be the default, he said. Windows, on the other hand, has it since Vista. They also explore and experiment with more modern mitigations like CFI. Yet another approach is to avoid the C language, because “[it] is full of memory corruptions”. Rust comes to mind as an alternative. GStreamer already supports plugins in Rust, he said. He concluded that fixing all these bugs, like Carlos seemed to be wanting, is very hard. Not only because GStreamer is very prone to memory corruption due to the amount of complicated formats it parses. He mentioned fuzzing as a viable strategy to shake out bugs and he found many bugs in a few days. He mentioned that probably to make do so more of that ourselves. I’m working on it. More to posted separately.

The next talk was about testing TLS implementations. For the last year or so I began investigating TLS issues myself and I was wishing for a TLS testing framework. Now I learned about an implementation. Hubert Karlo introduced his “tls fuzzer” which is a bit of a misnomer, because it actually doesn’t perform any fuzzing. He said that TLS was complex and that it has 326 official ciphersuites, 4 PKI cryptosystems, 16 signature-hash pairs, and many more countable things that make the test matrix grow fast. There is a lot of state to be maintained, he said. He presented his tool which takes care of TLS specifics but allows you to define your own payloads and modifications to them. For example, with a few lines of code you can define a client to open a TLS connection and to use a GCM ciphersuite for collecting the nonces. He claims to have found more than 20 issues in NSS, GnuTLS, and OpenSSL. I’m curious to play around with it and maybe hook it up with Scapy’s fuzzing facilities.

Another TLS related talk was given by Fridolin who showed us a TLS Linux Kernel module implementation. The advantages are manyfold he said. Obviously, establishing the connection should be cheaper in terms of computation because the context does not need to be switched so often. Others are already using a kernel implementation of TLS, he said. He mentioned that Solaris has a kssl socket and that netflix uses a modified sendfile() for TLS on BSD. His implementation has been evaluated by Facebook, he said. The implementation leaves the handshaking still to user space and cares about the symmetric encryption only.

Compared to other FOSDEMs, I was able to actually see a few talks, although I was impressed by the number of people I randomly bumped into and who kept me from attending more talks 😉 The size of FOSDEM is its cause and solution to problems. A good thing about it was that I could bribe something to cook up a Debian package for GNOME Keysign so that, hopefully, 200 people don’t have to queue up and do weird things :o)

GNOME Keysign 0.8

I’ve just release GNOME Keysign 0.8. It’s an exciting step towards a more mature codebase with less cruft and pieces of code moved to places where they should be more discoverable. To get the app, we have a tarball as usual, or an experimental flatpak (see below). Also notice that the repository has changed. The new URL should be more discoverable and cause less confusion. I will take down the old URL soon. Also note that this release will not be compatible with older releases. So you cannot find older clients on the network.
One problem that existed was when you selected a key and then pushed the “back” button, the UI would stall an unpleasantly long time. The actual problem is Python’s HTTPd implementation using select() with a relatively long interval instead of, say, doing things asynchronously. The interval is now shorter which increases the number of times the polling loop is executed but should make the UI more responsive. I wonder whether it makes sense to investigate hooking up the GLib Mainloop with Python’s SocketServer…

Another fix went into the HTTP client side which you could stall with a non reacting keyserver, i.e. when the HTTP request was simply not answered. Because the download is not done asynchronously as it should, the UI waits for the completion of the download. The current mitigation is to let the HTTP request time out.

A new thing is a popup when an uncaught exception happens. It’s copy and pasted from MyPaint and works by setting Python’s sys.excepthook.

You can also now switch the screen on which the fullscreen barcode is being shown. Once you have selected a key, you get the barcode displayed. If you click it it will cover your whole screen. If you are hooked up to a projector you might want to make sure that the barcode is shown on the bigger screen. Now you can press the left or right key to “move” the barcode. I needed to work around a bug in GTK which seems to prevent gtk_window_fullscreen_on_monitor () from working.

Finally, a new GPG abstraction consolidates all the required functionality into one module rather than having the required functionality spread around various modules. I named it “gpgmh” for “gpg made hard” which is a pun on “gpgme”, “gpg made easy”. The new module will also allow to use the real™ gpg module instead of the gpg executable wrapper provided by monkeysign. We cannot, however, switch to the library just yet, because it needs gpgme 1.8 which is too recent for current distros (well, Debian and Ubuntu). So we have to wait until we can depend on it.

If you want to try the application, you can now get the Flatpak from here. It should be possible to install the app with a command like flatpak --user install --from http://muelli.cryptobitch.de/tmp/2017-01-29-keysign.flatpakref. You can also grab the bundle if you want. Please note that the flatpak is very experimental. It would be surprising if anything but showing the UI actually worked. There are several issues we still need to work out. One is to send an email from within the sandbox and the other is re-use an existing gpg agent from the existing user session inside the sandbox. Gpg is behaving a bit weirdly there. Just having the agent’s socket available inside the sandbox does not seem to be enough to make it work. We need to investigate what’s going on there.

The future brings other exciting changes, too. We have a new UI in preparation which should be much more appealing. Here is what it will look like: