Recently, the EFF launched an initiative to collect and analyze Carrier IQ profiles, which dictate exactly what information is collected by the Carrier IQ application, and where and when it is submitted to the carrier. Jered Wierzbicki and Peter Eckersley published a great tool that converts the profiles, which are WBXML-encoded blobs, to human-readable XML.
However, not all Carrier IQ profiles are stored in this format on disk. Instead, they may be kept inside a file named “archive.img”. In this post, I describe how I reverse engineered the format used to store these profiles in order to create a tool to extract them.
First, taking a look at the header of a sample archive.img revealed that it’s a whefs filesystem:
$ hd archive.img | head -n 5
00000000 a0 00 00 07 d9 a0 00 00 00 0c a0 00 00 00 01 a0 |................|
00000010 00 00 00 20 a0 00 17 e1 d9 90 00 26 77 68 65 66 |... .......&whef|
00000020 73 20 76 65 72 73 69 6f 6e 20 32 30 30 39 31 32 |s version 200912|
00000030 30 31 20 77 69 74 68 20 33 32 2d 62 69 74 20 49 |01 with 32-bit I|
00000040 44 73 a0 00 00 00 80 a0 00 00 2a 71 a0 00 00 01 |Ds........*q....|
Whefs is a public domain embedded filesystem. After downloading and building libwhefs and its associated tools (which took some minor tweaking of the source code), it was possible to list the files in the filesystem.
$ whefs-ls archive.img
List of file entries:
Node ID: Size: Timestamp: (YMD) Name:
2 1536 2011.12.17 11:58:08 COV48Y3W
3 965 2011.11.02 05:26:37 9LS1142UY2
4 185 1970.01.01 19:00:30 9LS1O12Z
5 0 2011.11.02 05:30:36 R3AJJMOQ8S
Total: 2686 bytes
4 of 505 total inodes listed.
One of the files in particular was interesting because it appeared to have a sequence of magic bytes in its header:
$ whefs-cat archive.img 9LS1142UY2 | hd | head -n 3
00000000 80 1a 06 00 5a 4c 51 49 90 0d 4a 5f dc de 56 64 |....ZLQI..J_..Vd|
00000010 64 54 53 64 54 66 a2 50 6a 2e 0e 0a 0c 1b 03 04 |dTSdTf.Pj.......|
00000020 51 3d 27 35 55 26 05 17 1f 13 17 19 5d 3c 16 f4 |Q='5U&......]<..|
The "ZLQI" tag, backwards for "IQLZ", seemed to be a clue that this was a compressed format ("IQ" as in "Carrier IQ", and "LZ" as in "Lempel-Ziv"). I remembered from my previous reverse engineering of the Carrier IQ native ARM libraries that there were a couple functions that seemed related to compressing and decompressing data. After pulling up IDA, they jumped out at me immediately: CIQ_LZReadHeader and CIQ_LZUncompress. At a glance, these functions appeared to implement some sort of custom compression.
To make sure I didn't spend too much time on a dead end, I decided to first see if these functions actually did what I suspected. The easiest way to confirm this was to write a small application that linked against libiq_service.so and made calls directly into the Carrier IQ library, which allowed me to test these functions without having to figure out exactly what they did. I cross-compiled this program for ARM, and ran it on my Android phone. Sure enough, after a bit of tinkering to get the invocation of these functions right, I passed the compressed data to the CIQ_LZUncompress function and my program spit out a fully decompressed profile!
At this point, I decided to fully reverse engineer the decompression algorithm used by Carrier IQ in order to create my own tool that can decompress these archives in order to extract the profiles, which can then be converted to human-readable form using the IQIQ tool. I won't bore you with the details, but needless to say, it involved several hours of staring at ARM assembly in IDA, lots of bit-shifting, and a bunch of frustrating math mistakes on my part that were hard to pin down. After completing the reversing, the result looks a lot like the LZSS algorithm (or similar). The results of this work have been merged with the EFF's IQIQ tool, which now includes my new tool, "IQunpack". IQunpack takes a path to an "archive.img" file, and automatically extracts and decompresses any Carrier IQ profiles contained inside. The full source tree can be found here.
This entry was posted on Sunday, December 25th, 2011 at 3:58 pm and is filed under Android. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.