Benchmarking raw user mode disk access for VirtualBox

I am working with a virtual GNU/Linux system, because the machine I’m working with must run a Windows on its bare metal.

I thought I wanted to give raw disk access to the guest, but it turns out, that it is not very easy in Windows to give permanent permissions to a regular user. You can give permissions using the semi-official subinacl like so:

C:\WINDOWS\system32>"C:\Program Files (x86)\Windows Resource Kits\Tools\subinacl.exe" /noverbose /file \\.\physicaldrive1 /display=sddl

+File \\.\physicaldrive1
/sddl=O:BAG:SYD:(A;;FX;;;WD)(A;;FA;;;SY)(A;;FA;;;BA)(A;;FX;;;RC)

Elapsed Time: 00 00:00:00
Done: 1, Modified 0, Failed 0, Syntax errors 0
Last Done : \\.\physicaldrive1

C:\WINDOWS\system32> "C:\Program Files (x86)\Windows Resource Kits\Tools\subinacl.exe" /noverbose /file \\.\physicaldrive1 "/sddl=O:BUG:SYD:(A;;FA;;;WD)(A;;FA;;;SY)(A;;FA;;;BA)(A;;FX;;;RC)"

Elapsed Time: 00 00:00:00
Done: 1, Modified 1, Failed 0, Syntax errors 0
Last Done : \\.\physicaldrive1

C:\WINDOWS\system32> "C:\Program Files (x86)\Windows Resource Kits\Tools\subinacl.exe" /file \\.\physicaldrive1 /display=sddl /display=owner /display=primarygroup

+File \\.\physicaldrive1
/sddl=O:BUG:SYD:(A;;FA;;;WD)(A;;FA;;;SY)(A;;FA;;;BA)(A;;FX;;;RC)

+File \\.\physicaldrive1
/owner =builtin\users

+File \\.\physicaldrive1
/primary group =system

Elapsed Time: 00 00:00:00
Done: 1, Modified 0, Failed 0, Syntax errors 0
Last Done : \\.\physicaldrive1

I needed to figure out that the language the ACLs are written in, is SDDL and how to give my current user or group all permissions. I failed doing that so I opted for giving all access to every entity known to the system… You can see the relevant SDDL in the listing above.

But that change will only survive until the next reboot. To make the software permanent, an unofficial tool called dskacl can theoretically be used. Apparently it tries to write special values to the Windows Registry. Although I found my way through the documentation, I couldn’t make it work. It actually failed so hard on me that even Windows could not see the drive itself. So make a good contingency plan before even trying to make it work. It’s not really meant for attached disks, but rather external disks via USB. Anyway, I thought I’d have to redo the above mentioned step on every boot.

The question I had was, whether it’s actually worth it. I assumed that it would be a speed up to write directly to the harddrive without having to go through Windows and VirtualBox’ VDI layer before hitting the disk.

So I measured my typical workload: compiling Chromium. As it is what I’m working on, I want compilations to happen as quickly as possible.

My technique was the following:


export CCACHE_DISABLE=1
rm -rf out
./build/gyp_chromium
time ninja -C out/Debug chrome

I did a handul of runs to compensate for some irregularities that might happen on the host (or in fact, on the guest…)

When writing on an Ext4 straight onto the disk, I get the following results:

real 61m59.895s
user 322m52.832s
sys 46m49.268s

real 61m25.565s
user 318m40.680s
sys 46m7.608s

real 58m59.939s
user 320m36.500s
sys 46m28.336s

Having an Ext4 filesystem in a VDI container on an NTFS partition yields the following results:

real 60m50.360s
user 322m18.184s
sys 47m3.588s

real 57m30.324s
user 318m48.752s
sys 46m52.016s

real 63m29.179s
user 328m55.004s
sys 48m4.692s

I couldn’t test shared folders, because either the NTFS or in fact the vboxfs don’t support operations needed for the compilation. I guess it’s VirtualBoxes fault though…

My interpretation of the results:

Writing directly to the disk seems to be marginally slower than going through the VDI container. At best, there is no significant deviation from writing to the container. I decided that it’s not worth to write straight to disk. Going through the VDI container and through Windows is fine. Especially with all the risks involved such as Windows not being able to see the drive at all.

I acknowledge that my data is a bit flawed. It is likely that you cannot generalise my findings for any other workload. The method is also questionable as I didn’t flush caches or took care of anything disturbing my measurements. If you do measure yourself, I’m interested in getting the results.

Imaging RAM using Windd, /dev/fmem or QEmu

While using FireWire to obtain RAM is pretty convenient and powerful, one might still consider using a different method to capture RAM. A reason might be the absence of a FireWire port or driver, or simply the fact that the necessary cable or software is missing. Another more interesting scenario is the analysis of a systems behaviour, i.e. how does the memory differ between a locked and an unlocked workstation.

Windd

Using FireWire is not the only method to get the RAM. If you want to image Windows’ RAM, you could consider running program, i.e. Windd, on the victims machine which simply dumps all memory to the disk. You might have some problems running it though:

Error: InstallDriver cannot start service (Err=0x00000002)
Error: Cannot open \\.\win32dd

This not very helpful error message wants you to move the .sys file to %WINDIR%\SYSTEM32 and search and delete windd references in the Registry.

When you try to use Windd using a normal, non administrator account, you’ll find that elevated privileges are needed to run the executable:

    -> Error: win32dd requires Administrator privileges

If faced with a PC which is logged in with a non administrative account, which is typically found in a business environment, the Windows runas command needs to be used to run Win32dd with the rights of either a local or domain administrator account:

runas /user:Administrator "Win32dd.exe /f c:\images\memorydump"

When using runas you’ll notice that a full path is needed to be specified for creating the image file. Without specifiying a path, Windd saves the RAM image file in to the %WINDIR%\system32 directory instead of the current directory as you would have expected.

The fmem Kernel Module

Although Linux provides a /dev/mem file, one cannot read the physical RAM through it. In order for Linux to expose the RAM via a file, one can load the fmem Kernel module and use dd to obtain the contents of the physical memory address space, e.g. to obtain one gigabyte of memory dd if=/dev/fmem of=memdump bs=1048576 count=1024.
I tried using cat to dump the RAM but found that it read past the end of the physical memory. This is a known bug with fmem which the author notes in the README file provided with the module.

In order to be able to load the module, it needs to be downloaded and compiled, doing a normal untar, make, and run the supplied run.sh script to install, first. These steps are pretty straight forward.

Using QEmu to save RAM

QEmu is a virtualisation solution that allows to dump the RAM of the guest into a file unsing the pmemsave command. I think this deserves attention because it can be used to prepare attacks using the FireWire technique. That technique allows patching the memory on a running (Windows) system which in turn can be used to, say, unlock a password protected workstation. While unlocking has been done for a Windows XP SP2 system, no publicly available tools do that for a later Windows system.

With QEmu, we are able to run a target system, say Windows XP SP3, and get two memory dumps: The first while the machine is locked, and the second right after unlocking. By looking at the difference, we might be able to tell which data has to be modified in what way in order to remotely unlock the machine. This technique can also be used to, say, disable a Firewall, elevate privileges or even inject new processes into the running system. It is noteworthy that this method is Operating System independent for both, the host and the guest, because QEmu is free software which run on many platforms and can itself run many Operating Systems.

Reading RAM using FireWire

Recently, I had to obtain the Random Access Memory (RAM) of a Windows XP SP3 system. The RAM is a storage for (program) data but unlike a harddisk it is around a million times faster and volatile. That is, its contents vanishes within a few seconds after the machine is powered off. But as there is data of the running kernel and almost every running program in the RAM, it contains valuable information about the state of a machine, including passwords, running processes, open connections or registry data.

However, manually obtaining the RAM contents usually involves altering the state of the machine, because the system has to either run a special program (and running programs modifies the state) or to load a special FireWire driver (and loading drivers off the disk into memory alters the machines state). Even hibernating the system modifies the machine because files are written, memory might get compressed, etc. It is possible to obtain the contents of the RAM by cooling the chips down and quickly putting them into a system which keeps refreshing the data so that it can be read with an own system. This method, however, requires a lot of preparation, skill and fortune because every second matters. But even if you manage to save the RAM, you might get into trouble because the examined system will still have a CPU cache which might keep the system up and running, potentially noticing the loss of RAM and changing contents of the disk (i.e. erasing). Since a Windows (Vista) Kernel is around 9.9MiB in size it would not necessarily fit into a 2MiB cache which modern CPUs have, but a Linux (2.6.29) Kernel can be 1.7MiB in size which would fit perfectly. Other, smaller, Kernels such as L4 or QNX exist.

I, however, went with the FireWire method just because I thought it’s fun to snarf other machines memory using just a cable 🙂

FireWire was invented in 1995 by Apple for high-speed data transmission. By design, it is able to read and write directly to the host machines memory contents. This feature can be exploited to dump the machines memory.

However, to dump Windows’ memory via FireWire, one needs to convince Windows to be eligible to do so by pretending to be, i.e. a proprietary and expensive audio player called “iPod”.

By using existing tools it is relatively easy to dump Windows’ memory.
One needs to download and build pythonraw1394 and the necessary dependencies.
Then, the proper tools, which basically configure the Linux host to be an iPod and dump the victims machine’s memory must be run.
Following script should do all the necessary steps and finally save a 2GB memory dump in a temporary directory.
It is known to work on an Ubuntu 9.10 Linux system.

#!/bin/bash
 
DIR=/tmp/$RANDOM
PYTHONRAW=http://www.storm.net.nz/static/files/pythonraw1394-1.0.tar.gz
LOCALPORT=0
REMOTEPORT=1
NODE=0
 
mkdir -p $DIR && cd $DIR &&
  wget --continue -O- $PYTHONRAW | tar xvf - &&
  cd pythonraw1394/ &&
  sudo apt-get install -y libraw1394 libraw1394-dev swig python-dev build-essential &&
  sed -i 's/python2\.3/python2\.6/g' Makefile &&
  make &&
  sudo modprobe raw1394 &&
  sudo chgrp $USER /dev/raw1934 &&
  ./businfo &&
  ./romtool -g $LOCALPORT $DIR/localport.csr &&
  ./romtool -s $LOCALPORT ipod.csr &&
  ./1394memimage $REMOTEPORT $NODE $DIR/memdump 0 2G &&
  echo Success, please read the memory at $DIR/memdump || echo Failure

(I’ve just finished packaging that up for Ubuntu)

As for mitigation, the best would be to have no FireWire ports 😉 But it’d be stupid thing to rip off your FireWire ports or putting lots of glue into it, because FireWire is a nice technology that you want to use for your external harddrives, cameras, networking, etc. So simply deactivate the driver if you don’t need it! A plugged in device can thus not make use of the functionality. For Linux, rmmod ohci1394 should be sufficient. To make this permanent, you might want to add “blacklist ohci1394” to your /etc/modprobe.d/blacklist. If you then want to use some FireWire device, simply modprobe ohci1394.

By using the above mentioned script, we successfully obtained the contents of a Windows XP SP3 machine in a temporary directory.
In our case, it took around 500 seconds to capture 1024 MiB RAM. Interestingly enough, when reading more RAM than installed in the victims machine, the program puts 0xffs in the dump file. We rebooted the machine from Windows into Linux and read the RAM. Hence, reading RAM from a machine running Linux (2.6.31) works fine. We found it interesting that we still were able to read memory of the Windows system which we have shutdown properly. So Windows left artefacts which we were able read.

Note that the machine whose RAM we read ran a Linux Kernel which did not make use of the most recent FireWire stack, simply because it has not been officially released yet. It will be interesting to see, how and under which circumstances the new FireWire stack, called Juju, allows the RAM to be read and written.

For completeness, we also tried reading the RAM of a Laptop running MacOS and it worked equally well.

muelli@xbox:/tmp/pythonraw1394$ ionice -c 3 ./1394memimage 0 1 /tmp/windows.mem 0-1G
1394memimage v1.0 Adam Boileau, 2006.
Init firewire, port 0 node 1
Reading 0x3fd00000 (1045504KiB) at 2027 KiB/s...
1073741824 bytes read
Elapsed time 517.51 seconds
Writing metadata and hashes...
muelli@xbox:/tmp/pythonraw1394$ volatility ident -f windows.mem
              Image Name: windows.mem
              Image Type: Service Pack 3
                 VM Type: nopae
                     DTB: 0x39000
                Datetime: Mon Mar 22 19:06:42 2010
muelli@xbox:/tmp/pythonraw1394$

It might be interesting for future research to remotely change Windows’ memory to, say, unlock a password protected workstation, change configuration of a firewall or even inject new processes. Although we consider it to be very interesting to build a tiny appliance that does the memory dump of the victims machine, we could not find anything on the Internet. Given that products on the forensic market are pretty expensive, building such a machine might be very rewarding.

Creative Commons Attribution-ShareAlike 3.0 Unported
This work by Muelli is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported.