Thursday, October 8, 2015

Identifying your Userdata Partition

In case you only want to image one partition

All blog posts to date
For starters, I'm sorry I haven't posted anything in a long time.  And really, there is a main reason.  You, the reader, may also be able to help.

The reason I haven't posted in a long time is because I have not come up with many great posting topics.  I look for several qualifications in a topic ...

  • A topic which which is consistent with other topics on my blog
  • A topic which is not overly well covered and known on the Internet
  • A topic which somebody with some good Linux and mobile knowledge can do.  In other words, a topic which does not require a mobile forensics expert to do.

So if you have a topic you would like me to cover, please reach out to me.  Crowd-sourcing is very popular these days!

Anyways, this topic comes from a question I field often.  A lot of people reach out to me wanting help imaging specifically the userdata partition.  In my post on live imaging an Android device, I said to reach out to me to find the exact block to image if you want to image userdata.

This posts presents a general way to find the right block to image.  This post is NOT meant to be a replacement for reaching out to me with imaging questions.  If you are working on imaging your Android device and are having issues or have questions, go ahead and send me an e-mail.

Imaging your entire device vs Imaging your Userdata partition
First, do you want to image the entire device, do you want to image just your userdata partition, or do you not know?

If you create an image of your entire device, the result will be a file representing your entire internal storage.  This image file will contain your boot image, recovery image, radio software, system partition, userdata partition, and depending upon the make and model can have a lot more blocks too.  Check out a full image of a Samsung Galaxy S5 sometime if you want to be overwhelmed with partitions and blocks!

If you create an image of your userdata partition, you will have a file representing one partition of the device from beginning to end.  This partition is the "userdata" partition, which contains evidence of user activity.  It contains call and SMS records, contacts, user-installed apps, app data, settings, and so-on-and-so-forth.  In most newer phones, it also is likely to contain photos and videos and other user-generated files unless an external SD card is present.

And if you want to image an external SD card, I would recommend a good ol' fashioned write blocker and traditional forensic techniques.  If you need more information on this, reach out to me.  My skill set is not restricted to Android forensics!

If you do not know which type of image you want to create, go for the full image.  It takes more space because the image file contains all the device partitions instead of just one.  The full image will give you a greater insight into the device - and you may learn a thing or two about how Android devices work!

How to image the entire device
If you've not checked out my post on live imaging an Android device, go ahead and check it out.  This current post will probably not make a whole lot of sense without the knowledge on the live imaging page.

I said on my live imaging post that you should image a block on the device and the command you enter via adb shell to the device looks something like this ...
dd if=/dev/block/mmcblk0 | busybox nc -l -p 8888

This line reads all of the contents of the block /dev/block/mmcblk0 and passes it to the computer via netcat.  And as you saw on the post, this command only works if the device is rooted and busybox is installed.

The block /dev/block/mmcblk0 in most cases refers to the very first sector of the device.  By reading this block, you read the entirety of the device's internal storage.  Imaging this block gets you a physical image of the entire device.

Note: I did not say in all cases the block /dev/block/mmcblk0 refers to the very first sector of the device.  All it takes is one device to have a different naming convention to make a liar out of me if I said the block /dev/block/mmcblk0 is the first sector of the device in all Android devices.

How to image the userdata partition
If you only want to image the one partition, you need to know what block to read.  We do this with the mount command.

Open an adb shell to the device.  Type the following:

You will get several lines of output.  Each of these lines represents mounted partitions.  On my Nexus 5, I receive the following as output of the mount command:
rootfs / rootfs ro,seclabel,relatime 0 0
tmpfs /dev tmpfs rw,seclabel,nosuid,relatime,mode=755 0 0
devpts /dev/pts devpts rw,seclabel,relatime,mode=600 0 0
proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,seclabel,relatime 0 0
selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
none /acct cgroup rw,relatime,cpuacct 0 0
none /sys/fs/cgroup tmpfs rw,seclabel,relatime,mode=750,gid=1000 0 0
tmpfs /mnt/asec tmpfs rw,seclabel,relatime,mode=755,gid=1000 0 0
tmpfs /mnt/obb tmpfs rw,seclabel,relatime,mode=755,gid=1000 0 0
none /dev/cpuctl cgroup rw,relatime,cpu 0 0
/dev/block/platform/msm_sdcc.1/by-name/system /system ext4 ro,seclabel,relatime,data=ordered 0 0
/dev/block/platform/msm_sdcc.1/by-name/userdata /data ext4 rw,seclabel,nosuid,nodev,noatime,nomblk_io_submit,noauto_da_alloc,errors=panic,data=ordered 0 0
/dev/block/platform/msm_sdcc.1/by-name/cache /cache ext4 rw,seclabel,nosuid,nodev,noatime,nomblk_io_submit,noauto_da_alloc,errors=panic,data=ordered 0 0
/dev/block/platform/msm_sdcc.1/by-name/persist /persist ext4 rw,seclabel,nosuid,nodev,relatime,nomblk_io_submit,nodelalloc,errors=panic,data=ordered 0 0
/dev/block/platform/msm_sdcc.1/by-name/modem /firmware vfat ro,context=u:object_r:firmware_file:s0,relatime,uid=1000,gid=1000,fmask=0337,dmask=0227,codepage=cp437,iocharset=iso8859-1,shortname=lower,errors=remount-ro 0 0
/dev/fuse /mnt/shell/emulated fuse rw,nosuid,nodev,noexec,relatime,user_id=1023,group_id=1023,default_permissions,allow_other 0 0

Look specifically at the following line:
/dev/block/platform/msm_sdcc.1/by-name/userdata /data ext4 rw,seclabel,nosuid,nodev,noatime,nomblk_io_submit,noauto_da_alloc,errors=panic,data=ordered 0 0

This line indicates that the block /dev/block/platform/msm_sdcc.1/by-name/userdata is mounted read/write (as opposed to read only) at the directory /data and the partition is formatted ext4.

That means the block /dev/block/platform/msm_sdcc.1/by-name/userdata is my userdata block.  I can substitute /dev/block/platform/msm_sdcc.1/by-name/userdata for /dev/block/mmcblk0 in my imaging command.

Now I fully understand that typing /dev/block/platform/msm_sdcc.1/by-name/userdata is a bit cumbersome.  There is a shortcut.  I can enter the following command (and you might need to be root):
ls -l /dev/block/platform/msm_sdcc.1/by-name/userdata

I get the following output:
lrwxrwxrwx root     root              1971-03-18 07:17 userdata -> /dev/block/mmcblk0p28

You may notice a timestamp from the year 1971.  That is nothing to be concerned over - I am in fact not a time traveler.  That timestamp is an artifact of Linux timestamp convention.

This output indicates that the block /dev/block/platform/msm_sdcc.1/by-name/userdata is a reference to /dev/block/mmcblk028.

So, I can substitute /dev/block/mmcblk0p28 for /dev/block/mmcblk0 in my imaging command.
If I want to image just the user partition, the imaging command I enter in an adb shell to the device would look like this:
dd if=/dev/block/mmcblk0p28 | busybox nc -l -p 8888

I hope this all makes sense.  Feel free to reach out to me if you need further assistance.

And now some frequently asked questions
Seeing as this post started with answering a FAQ, I will answer a couple more FAQs.

"Can I use the live imaging technique in Windows?"
Not without some serious hacking around.  The technique I show is a Linux technique that also works on Mac.  This is because the technique relies upon the netcat utility.  Netcat is on both Linux and Mac; it is not on Windows.

Now maybe you are familiar with Cywgin.  Cygwin allows you to run Linux commands in Windows.  I have never tried this imaging technique out on a Cygwin-enabled Windows computer using the Cygwin version of netcat.  If anybody out there tries this out, let me know!  I'd love to know the results.

"I found this specific rooting technique for this specific device.  I have not tested it out yet.  Will it work?"
Sometimes I get asked if a rooting technique works on a certain phone.  And I always look into it and will provide the best answer I can.  The trick is, whatever the phone is, I probably do not have one conveniently laying around to try the technique out myself.

The issue with rooting is sometimes you can accidentally wipe device data you plan to image.  Example: the most common method for rooting Nexus devices is to unlock the bootloader and install ClockworkMod and root through CWM.  This technique works great except for one minor little detail: as a security feature, unlocking the bootloader on a Nexus device wipes all user data on the device.  I would not exactly call that forensically sound.

Similarly, some techniques on newer Samsung phones can result in flipping the warranty bit.  If you root your Samsung phone and then see the following in the bootloader interface:

Then the warranty on the device is void.  If you have a problem with the device, service centers are likely to turn it away.  That may or may not matter to you.  However, without that warranty bit, the device can never have a Knox container.  Again, this may or may not matter to you.  However, I would generally avoid flipping the warranty bit.

So when I get these questions, I research what I can to determine if your rooting technique is likely to safely work.  But as I said on my live imaging post, I am not liable if you harm your device.

"I am interested in a career in digital forensics.  How can I start?"
First, this is a fun career!  It is challenging because technology changes all the time, but there are great rewarding opportunities if the curious minds out there.  If you are thinking of university education in the field, I can point out some great programs - and there are new programs opening every year it seems because we need digital forensics experts.
But if you are not sure and want to explore around before truly going down this career path, I can give you a few pointers ...
  • Learn command line.  I cannot stress this enough.  In any area of computer science / technology, being comfortable in a command line interface is such an asset
  • Learn Linux.  Whether you install Ubuntu and use it to browse the web or you go low level with Linux and do some fun hacks, Linux is good to know.
  • Download the free tools.  Try FTK Imager and Autopsy.  Learn how they work.  Try examining your own computer.  Try using a Linux live environment by booting from disk and image your drive onto an external.  Just do anything you can to learn using the tools you already have.
  • Read.  There is a great academic community in digital forensics publishing some cutting-edge research.  But before getting into the new stuff, read about the basics.  I can point you in the direction of some good papers.  If you are thinking about a BS or MS in this field, you will certainly read academic papers and do research of your own, so reading academic work beforehand can only help.
  • Mobile device forensics specific:  hack around with your own phone.  Or preferably, if you have an old phone, don't sell it - hack around with it.  
  • Android forensics specific:  Root the device and learn how it works.  Flash ClockworkMod and adb into the device in recovery mode.  Just hack around with the phone and don't be afraid to take risks, especially if it is not your day-to-day phone.  And don't hack around with somebody else's phone without their permission!  That can get you into trouble!
  • Write.  In any field, the ability to do the work and obtain results is of course of great importance.  What so often sets apart leaders in a field is the ability to effectively communicate work and results.  Our universal way of communicating in just about any field is writing.  So write your work in a journal as you go.  Practice writing reports.  If you do not have a template to work from, create your own template.  Polish your writing style.  I know this bullet point sounds on the un-technical side but I cannot stress the importance of writing and effective communications enough.

"Do you consult?"
Yes.  If you have a need for digital forensics services (Android or not, mobile or not) or other Android/mobile related needs and you feel based off this blog I am a person you would like to work on the problem, e-mail me.  I do not take every case as I do this in my side time, but I take side work from time to time.  I can discuss privately how I work, needs you have, and my terms.

"Can you do speaking engagements / guest lectures?"
Yes, if scheduling works.  I have done guest lectures and simple Q&As at universities, including remotely over Skype.  I enjoy connecting with the academic community.


  • If you are not sure whether to image a device's entire internal storage or just the userdata partition, image the entire device
  • Use the mount command to find the block to image if you plan to image just the userdata partition
Questions, comments, suggestions, or experiences?  Leave a comment below, or send me an email.

Thursday, April 2, 2015

Why not load ClockworkMod or TWRP to image a device?

Alternate recovery modes are useful, but maybe not for imaging a device

All blog posts to date
This post comes from a question I've fielded a few times.  A lot of people, like me, who hack around with their Android devices install a custom recovery mode in place of the default recovery mode.  So the question is, why not install ClockworkMod or another custom recovery mode on a device and use the custom recovery mode as a method to image the device?

So, let's get some terminology out of the way because I threw a lot of information in that first paragraph which you may or may not understand.  Part of the Android architecture is recovery mode.  The user can boot into recovery mode instead of Android in order to perform some basic recovery operations, including installing official updates and factory resets.  Recovery mode is a small operating system with a basic kernel.  The source is available online if you would like to browse.

Users can install a custom recovery mode, such as ClockworkMod or Team Win Recovery Project (TWRP).  These custom recovery modes are also operating systems and the user can boot into this alternate recovery mode instead of Android in order to access recovery functionality.  Extra functionality within these alternate recovery modes allow users to backup their devices, recover from backup, install non-official operating systems, and other advanced features.

Recovery mode runs by default as root.  Recovery options must run as root or else they would not have the proper permissions to execute.  In the stock recovery mode, the user cannot use these root privileges to image the device.  However, the alternate recovery modes allow the user to access a full shell as root privileges via ADB.  As I mentioned in my post on imaging a device, you need a root exploit to image the device.  You can treat an alternate recovery mode as a root exploit, then boot into recovery mode and image the device while the Android operating system is not even running.

This all sounds great, right?  So do I recommend install ClockworkMod or TWRP in place of the default recovery mode in order to image a device?  It depends upon why you want to image the device. In this post, I'll show how to install alternate recovery modes, how to image a device using an alternate recovery mode, and why this method of imaging may or may not be appropriate.

How to install an alternate recovery mode
Installing an alternate recovery mode is very device specific.  Honestly, the best way to figure out how to install an alternate recovery mode is to Google it.  Here are a few search terms:

  • clockworkmod nexus 5
  • clockworkmod galaxy s5
  • twrp htc one
For example, here is how to install ClockworkMod on a Nexus 5.  Note that unlocking the bootloader results in all user data gone.  Note and remember for later in this post.

Here is a guide for more advanced users for installing TWRP on a Galaxy S4.

In short, I can't guide how to install ClockworkMod or TWRP on each and every device.  It is a device specific project.  Search online to find how to install an alternate recovery mode on your own device, and you can always contact me for help.

How to image in recovery mode
Now let's say you've got ClockworkMod installed on your device.  My personal device is a Nexus 5, so I'll use my phone as a guide device.

Disclaimer:  you can cause big problems on your device if you do some of these steps wrong.  I am not liable for any damages done to any devices as a result of reading this page or any other pages on this blog.

First, boot the device into recovery.  If you're not sure of how to do that, search online.  If your device is booted into Android and you have adb enabled, you can try the following command

adb -d reboot recovery
Your device should reboot into recovery.  Then if you enter
adb devices
you should get a response along the lines of ...
03xxxxxxxxxxxx17        recovery
If you get a strange response with a bunch of question marks, try entering the following
adb kill-server
sudo adb start-server
And then try adb devices again and see if you get a better response.

In my post on imaging Android devices
, I said you need the following three things
  1. Data connection between the computer and the device
  2. Exploit
  3. Imaging command
If you've followed along so far, we've got all three satisfied
  1. You can communicate with the device in an alternate recovery mode over ADB
  2. The alternate recovery mode gives full root access to the device
  3. Imaging command below
Next, you'll need netcat, or nc, in the recovery mode.  I mentioned before that recovery mode is its own operating system.  As an alternate recovery mode allows a shell to the device, the alternate recovery mode operating system contains commands the user can use.  ClockworkMod and TWRP should both have netcat installed as they both should include busybox.  To make sure, open an ADB shell to the device in recovery mode and type the following:
busybox nc
If the output includes something along these lines:
Usage: nc [-iN] [-wN] [-l] [-p PORT] [-f FILE|IPADDR PORT] [-e PROG]
Then you are set.  If not, contact me and I'll help you get netcat installed.

Now, just like on my
post on live imaging an Android device, we'll image the device using netcat.  I'll go over the instructions again.  Note, I'm basically just copying and pasting from the post on live imaging.

To image the device, you need to do some commands in two different sessions: one shell session to the device, and one shell session to your computer. Open up a terminal window and adb into your device.  Then open up a new terminal window (it will open as a shell to your computer, not your phone) and navigate to the directory where you intend to store your image. Note: if you create the image in a volume formatted FAT32, the maximum file size is 4 gigabytes, so imaging the device would require splitting the file. For ease sake, I suggest imaging to a volume formatted ext or NTFS. Also, make sure the volume has enough space for the device image, which will be as large as the device's storage. For my phone, I need 32 gigabytes of storage to image.

Now, in the shell to your computer in the directory of your choosing, type the following:
adb forward tcp:8888 tcp:8888
This command allows adb to communicate via netcat on port 8888.

Next, in the shell to your phone, type the following:

dd if=/dev/block/mmcblk0 | busybox nc -l -p 8888

This command reads the contents of /dev/block/mmcblk0 (the head block of my device) and writes it via port 8888 across adb using netcat.

Finally, back in the shell to the computer, type the following:

nc 8888 > device_image.dd

This command saves the output of the contents across port 8888 (which will be the results of reading /dev/block/mmcblk0 on the device, or the complete image of the device) to the file device_image.dd.

If there's no errors, you are imaging the device. The window will “freeze”, or not allow any more commands because it is busy executing this command. When the imaging process is done, you will be able to type commands into this shell window again. To confirm, open a new terminal window, navigate to the directory where you are saving the image, and type ls -l. This will get a file listing, including file size. If the size of your file is increasing, you are successfully imaging your device.

A quick note on crypto.  If the device's data is encrypted, the image will be of encrypted junk.  You are imaging the device while the data partition is inactive.  If the image appears to be encrypted, you'll need to acquire data while the device is live.

Now that I've explained how to image a device using recovery mode, I'll go over whether or not it is a good idea.

Why recovery mode to image a device can be a good idea

Digital forensics is a big community containing many realms.  Digital forensics can be involved in law enforcement, federal government, research, and IT security.  Digital forensics can also be practiced by individuals wanting to learn more about how their devices operate.  What I'm saying here is the purpose of imaging the device dictates whether or not using recovery mode to image your device is a good idea.

If you are an advanced Android user, you may very well have an alternate recovery mode on your device.  I have ClockworkMod or TWRP on all of my personal devices.  If you are imaging your device in order to do some research on some files or a process, then using an alternate recovery mode is a great idea.  You can image the device while it is not booted into Android, the data partition is not actively in use while imaging.

If you are a security researcher working on mobile work, I definitely recommend installing a custom recovery mode (unless doing so interferes with research).  You may have good uses for some of the advanced functionality which the alternate recovery mode offers.  So if I've described your job and you need to image a device, I recommend the method on this page.

If you are imaging a device where the device is evidence in a case and the device already has an alternative recovery mode loaded, then you can use the alternative recovery mode to image the device.  In doing so, the device is not booted into Android so no user data on the device is changing or even loaded when imaging.  I say this paragraph with a caveat: users can write their own alternative recovery modes, and it is conceivable that an advanced user could bake some special sauce into recovery mode to, say, wipe the device.  While I find such a scenario unlikely, it is conceivable.  And this caveat works as a perfect segue into my next section.

Why recovery mode to image a device can be a bad idea
I said before that the purpose of imaging the device determines whether or not installing a custom recovery mode is a good idea.  I'm going to detail why installing the custom recovery mode can be a bad idea.

If you are imaging a device where the device is evidence in a case and the device does not already have an alternative recovery mode loaded, I definitely do not recommend installing an alternative recovery mode.  If you do choose to install one, you are doing so at your own risk.  There are two reasons why I do not recommend this action.

  1. Installing an alternative recovery mode involves overwriting the previous recovery mode.  If it appears that the stock recovery mode is installed, it is most likely not a problem to overwrite the recovery mode.  Users by default cannot access the recovery mode to store data there.  I would not call replacing a stock recovery mode (which you can download easily from the Internet) a material change.  However, it is conceivable that an advanced user could write their own recovery mode to include some extra data or functionality and make it appear identical to the stock recovery mode.  In the unlikely but possible case where you have such a device, overwriting recovery mode means overwriting important data.
  2. And this one is just a minor detail.  Installing a new recovery mode often involves factory resetting the device, or wiping all user data.  As in, deleting all case data from the device.  This process is what I would call an RGE, or a Resume Generating Event.  Minor detail, right?
So to sum up the above points, if you are imaging a device because you need to retain the data on the device as you found it, I absolutely do not recommend installing an alternative recovery mode to image the device.

  • Alternative recovery modes, such as ClockworkMod or TWRP, are useful for Android hackers like myself
  • Alternative recovery modes provide the user a root shell so imaging the device using an alternative recovery mode is a very similar process to live imaging a device
  • It may or may not be recommended to use this method to image a device.  If it is essential to maintain the data on the device as you found it, such as in an investigation, do not use this method for imaging a device.  Live image the device or use a commercial forensic tool
Questions, comments, suggestions, or experiences?  Any RGEs you care to (or not care to) share?  Leave a comment below, or send me an email.

Monday, February 9, 2015

Facebook for Android Artifacts

A Cache of Personal and Communication Information

All blog posts to date
First, a disclaimer.  This post will detail lots of artifacts on the Facebook for Android app which can be useful from a forensic perspective.  These findings regard personal information about the user and the user's communications with contacts.  My goal with this post is to educate, inform, and possibly assist people working on cases involving Facebook data on mobile devices.  My goal is not to scare the reader away from using Facebook.

I am not writing this post with the goal of getting you, the reader, to remove Facebook from your phone.  I use Facebook on my phone, so I obviously am not too worried about the incredible amount of personal information stored in an unprotected manner on my phone.

So ... if this post disappears at any point, go ahead and assume I received a cease and desist note from Mark Zuckerberg!

Now ... back to the post.  I use Facebook.  My wife occasionally uses Facebook.  Her friends and my friends use Facebook.  My siblings, parents, cousins, aunts, and uncles use Facebook.  I have a grandmother who uses Facebook.  I have a friend from grad school who has a Facebook page for her cat.

Facebook launched over ten years ago as a collegiate social network and this mega-popular website revolutionized social networking.  Facebook evolved from a set of unconnected profiles to a place to share status updates and thoughts of the day to one of the largest (if not the largest) collection of photographs of people in the world.  I'm now of the age that whenever I log onto Facebook, I swear Facebook is nothing except a website for parents to upload cute picture of their kids.

As with any other new technology, Facebook can also attract criminal activity.  Facebook stalking is a real thing and can lead to in-life stalking and worse (even if this video makes it look humorous).  In Facebook's early days, some universities used Facebook photos of underage students drinking as evidence of university policy violations.  Facebook posts have admitted as evidence in criminal trials before and depending upon applicable state law, Facebook posts and messages may be admissible in court.

If you are examining an image of an Android phone for a criminal case and Facebook is installed, there may be good reason to examine data associated with the Facebook app.  This post will detail some of the Facebook data that can be stored on the device and how to interpret it.  And again, please don't uninstall Facebook just because some guy on some Android forensics blog said the Facebook app is creepy!

Where is the data?

First, how do you access this Facebook data?  The Facebook app is an app, so app data is protected by permissions.  I'd recommend reading my previous post on viewing SQLite Databases before diving into this post.  In short, you'll need a rooted device or an image of a device to access Facebook data.

There are two Facebook apps that I'll detail in this post.  The first is the main Facebook app.  The package name is com.facebook.katana, so the data associated with this app will be stored in the data partition in the directory data/com.facebook.katana.  The version of the app on my device is  The second is the messenger app.  The package name is com.facebook.orca, so the data associated with this app will be stored in the data partition in the directory data/com.facebook.orca.  The version of the app on my device is  Depending upon the version of Facebook installed on the device, data may be slightly different than what I present in this post.  If you have any questions about where data is, you can always contact me.

So yes, if you install both the main app and the messenger app on your device, you have a killer whale (orca) and Michonne's sword from the Walking Dead (katana).

If you've already imaged the device you are investigating, go ahead and copy these directories away from the image to your forensic computer.

Information about Facebook Friends

First, we'll look at the com.facebook.katana app, or the main Facebook app.  Check out the directory com.facebook.katana/databases.  This directory predictably stores database files.

In my previous post on viewing SQLite Databases, I showed how to open a SQLite database file to browse data.  Explore the file contacts_db2.  This file stores a database of Facebook friends.  Within the file is a table called contacts.  There are several columns in this table to be aware of:
  • first_name:  self explanatory
  • last_name:  self explanatory
  • display_name:  self explanatory
  • small_picture_url:  A URL to a small version the user's profile picture.  More on that later.
  • big_picture_url:  A URL to a big version the user's profile picture.  More on that later.
  • huge_picture_url:  A URL to a huge version the user's profile picture.  More on that later.
  • communication_rank:  A number representing how often the user communicates with this particular contact.  This number is calculated using some Facebook formula.  Communications include messages, posts, likes, comments, etc.  A 0 in this column means no communication.  The higher the number, the more communication.  From a forensic perspective, this number is a way of determining how often the user interacts with another user.
  • is_messenger_user:  A true/false field.  True indicates that the user uses a mobile messenger app (such as the com.facebook.orca app for Android).
  • data:  A long string of data describing user profile information.  More on this later
  • bday_day:  Birthday.
  • bday_month:  Birthday.
For some of the points above, I indicated that I would discuss more later.  It is later now.

There are entries for small_picture_url, big_picture_url, and huge_picture_url.  Here is what a huge_picture_url string looks like for a friend of mine:  https://fbcdn-<redacted>_n.jpg?oh=<more_redacted>eea.  (I redacted most of the URL for privacy reasons.)  And when I entered the URL into a browser, I found this image:

(I chose this specific friend of mine for the sake of anonymity.  No face in this Facebook profile picture).  Yes, this friend of mine is an Oregon Ducks fan.  Don't be too hard on him after the college football national championship game.

Notice how there is no protection, no encryption, no login required to access these Facebook photos.  While there is no public index page that I am aware of to associate a URL with a user, it still bears mentioning that photos are stored without protection online.

There is an entry above for "data".  I said that this is a blob of user data text.  Here is what one of the blobs looks like (with redactions):
{"contactId":"Y2<redacted>k2","profileFbid":"62<redacted>09","graphApiWriteId":"contact_20<redacted>96","name":{"firstName":"<redacted>","lastName":"<redacted>","displayName":"<redacted>"},"phoneticName":{},"smallPictureUrl":"https://fbcdn-profile-a.<redacted>a40","bigPictureUrl":"https://fbcdn-profile-a.<redacted>26e","hugePictureUrl":"https://fbcdn-profile-a.<redacted>eea","smallPictureSize":160,"bigPictureSize":320,"hugePictureSize":466,"communicationRank":0.03445798,"withTaggingRank":0.3325288,"phones":[{"id":"62978<redacted>259","label":"Mobile","displayNumber":"(6xx) 9xx-xxxx","universalNumber":"+16xx9xxxxxx","isVerified":true}],"nameSearchTokens":["<redacted>","<redacted>"],"canMessage":true,"isMobilePushable":"YES","isMessengerUser":true,"messengerInstallTime":1417438579000,"isMemorialized":false,"isOnViewerContactList":true,"addedTime":1419017431000,"friendshipStatus":"ARE_FRIENDS","subscribeStatus":"IS_SUBSCRIBED","contactType":"USER","timelineCoverPhoto":{"focus":{"x":0.5,"y":0.39435146443515},"photo":{"image_midres":{"uri":"https://fbcdn-sphotos-h-a.<redacted>201","width":320,"height":179},"image_lowres":{"uri":"https://fbcdn-sphotos-h-a.<redacted>817","width":500,"height":281}}},"nameEntries":[],"birthdayDay":<redacted>,"birthdayMonth":<redacted>,"cityName":"<redacted>, Ohio","isPartial":false}
Obviously this blob is hard to read, but it is a nice treasure trove of useful information about the individual.  I'll space this out to make it a little more readable:
contactId:  Y2<redacted>k2
profileFbid:  62<redacted>09
graphApiWriteId:  contact_20<redacted>96
   firstName:  <redacted>
   lastName:  <redacted>
   displayName:  <redacted>
smallPictureUrl:  https://fbcdn-profile-a.<redacted>a40
bigPictureUrl:  https://fbcdn-profile-a.<redacted>26e
hugePictureUrl:  https://fbcdn-profile-a.<redacted>eea
smallPictureSize:  160
bigPictureSize:  320
hugePictureSize:  466
communicationRank:  0.03445798
withTaggingRank:  0.3325288
   id:  62978<redacted>259
   label:  Mobile
   displayNumber:  (6xx) 9xx-xxxx
   universalNumber:  +16xx9xxxxxx
   isVerified:  true
nameSearchTokens:  ["<redacted>","<redacted>"]
canMessage:  true
isMobilePushable:  YES
isMessengerUser:  true
messengerInstallTime:  1417438579000
isMemorialized:  false
isOnViewerContactList:  true
addedTime:  1419017431000
friendshipStatus:  ARE_FRIENDS
subscribeStatus:  IS_SUBSCRIBED
contactType:  USER
      x:  0.5
      y:  0.39435146443515
      uri:  https://fbcdn-sphotos-h-a.<redacted>201
      width:  320
      height:  179
      uri:  https://fbcdn-sphotos-h-a.<redacted>817
      width:  500
      height:  281
nameEntries:  []
birthdayDay:  <redacted>
birthdayMonth:  <redacted>
cityName:  <redacted>, Ohio
isPartial:  false
The entry for a contact's "data", as you can see, can contain all kinds of personal information, ranging from birthday to cell phone number, and I've even seen people's addresses in this entry before.  Two takeaways: one, be careful what you put online, and two, all of this sensitive information is stored on your phone without encryption.

Facebook Messages

Facebook has the ability to send private messages to other users.  These messages are stored on Facebook's servers, and they also can be stored on your phone.  The file com.facebook.katana/databases/threads_db2 stores messages the user has sent and received, and they are all stored in the table messages.  As before, I'll point out columns of interest.
text: the actual text of the message
sender:  the user who sent the message.  You can use this column to tell if the message was sent or received
timestamp_ms:  the date and time of the message in epoch time
attachments: any attachments with the message.  The attachment may include a link to a photo
coordinates: if the user sent the message using a mobile device and allowed access to device location, the location of the device when the message was sent.
source: whether the message came from a computer or a device or any other source.

Here is an example of the sender field:  {""email"":""20<redacted>"",""user_key"":""FACEBOOK:20<redacted>86"",""name"":""Mark Lohrum""}.  This field is formatted similarly to the data field in the contacts table as I mentioned above.  You can see a field for email, which is basically the numerical user ID  You can try sending an email to this address from your GMail; for me, the message forwarded to my email address where I receive Facebook notifications.  But you can see my name in the sender field, so you know that the message in this entry is from me.

You probably noticed above an entry for coordinates.  This entry stores latitude and longitude as reported by the device at the time the message was sent.  Yes, you can determine where a person was, or where their device was, when a message was sent.  That can be rather useful information because you have determined where the device was when a message was sent at a specific time.  If you can be sure that the user and not another individual was holding the device and sending the message, then you know where the person was at a specific time when sending a message.  Note, on Android it is very easy to spoof location.

Cached Images

The Facebook app stores a whole lot of data on the device.  Much of this data is cached images.

For example, on my device, there is a file com.facebook.katana/cache/image//v2.ols100.1/99/8vNUdrezcgt0__oST83Rc5g0QIE.cnt.  (I don't know what the .cnt extension means, but all of the cached images have this extension.)  Obviously there is no context in this filename what the file is, but the file was 102 KB so I was interested.  Here is what the file looks like in a Hex editor:

You can see that the file header includes JFIF, so clearly this is a JPG file.  I renamed the file to include a .jpg at the end and opened it as an image and here is what I found:

Yes, I am a big football fan.

Now how useful are these cached images?  To be honest, not horribly.  These are images from the timeline that my device saved.  In other words, these are public pictures that a user posted online.  It is not horribly useful, just interesting.

That's all the data I'll cover for now from the com.facebook.katana app.  If there's anything else you would like me to cover, comment or contact me and I'll take a look.

com.facebook.orca data

The com.facebook.orca app is just a messenger app.  Basically there is also a threads_db2 file within the databases directory just like with com.facebook.katana.  These database files store basically the same information, so I won't cover it again.  The important thing to know is that if the com.facebook.orca app is present, the user uses Facebook messenger for Android.

That is all I will cover for now.  Did I cover everything that Facebook stores?  No.  Here's a few more artifacts worth noting that the app stores:
  • Facebook posts by the user
  • Facebook pictures and videos uploaded by the user
  • Places the user has been
Now something I haven't covered yet is important.  The device stores a lot of data, but Facebook is ultimately a cloud service, meaning all Facebook data is ultimately stored on a remote server.  If you are in law enforcement and you need data associated with a user from Facebook's servers and you have a court order allowing access to these records, there is an avenue to get this.  Check out this link for more information.  I am not law enforcement so I have no personal experience in this avenue, but I do know this avenue exists if needed.

  • Facebook stores lots of data on Android devices if the user uses Facebook
  • Private messages and personal friend information can be retrieved from the device in an investigation
  • There exists a method for law enforcement to retrieve Facebook records should they be needed.  The procedure requires a court order
Questions, comments, suggestions, or experiences?  Walking Dead or college football fan chat?  Leave a comment below, or send me an email.

Monday, January 5, 2015

Viewing SQLite Databases

How to view user data

All blog posts to date
Android apps by default use SQLite databases to store user data.  When you think Android apps, you may think about Netflix and Pandora and ask yourself why you care about user data associated with those apps.  The reason you care about user data associated with apps is because nearly all user interaction with the device is user interaction with an app.  Have you ever made a phone call on an Android phone?  The phone dialer is an app, and that app stores call logs.  Have you ever sent a text message on an Android phone?  The SMS interface is an app, and the app stores text message history.

SQLite is a lite implementation of a Structured Query Language (SQL) database.  I could go into detail about how SQL works and how the file is formed, but I'll skip the low level details and instead focus on Android relevant SQL information.  This post will cover two main topics:

  • The locations of SQLite databases, including a few keys ones
  • How to read a SQLite database from an Android device

Finding a SQLite database
First, Android security prohibits users from accessing the userdata partition, which is where SQLite databases storing user data associated with apps are stored.  You either need an image of the device (and you can create an image using my post on live imaging an Android device) or you need a rooted device.  In this post, I am working from an image of a device.

Android by default stores user data in the /userdata partition in the directory /data.  The below screenshot is from a screenshot of FTK Imager looking at the data directory.

(Note:  some older devices store userdata in a separate location.  I've seen both older Samsung and Motorola devices that have a partition called dbdata.  This partition would store user databases.  But newer devices are pretty standard at this point.  Look to the userdata partition in the data directory first.)

You'll see that within the data directory are directories containing package names.  The directory air.WatchESPN stores user data associated with the WatchESPN app.  The directory stores data associated with the YouTube app.

In fact, let's drill into YouTube for now.  The below screenshot is from the directory /(userdata partition)/data/  

Within this directory, you'll see a directory called databases.  Within the databases directory below ...

.... you can see 14 files.  These files represent the databases associated with YouTube.  I'm willing to bet that the file history.db contains YouTube history.  I extracted this history.db file from the image and opened it in a SQLite browser (more on how to do that below), and I see a table called "suggestions" containing five columns (and in parentheses what I interpret each column to represent):
  • _id (an auto-generated id for each entry in the database)
  • display1 (the stored search suggestion based off user input)
  • display2 (contains nothing)
  • query (the actual text the I typed into the YouTube app to search for a video)
  • date (epoch date / time stamp of the time I searched for a given video)
Here is a sample row from this database:
"1","hobbit battle of five armies trailer",,"hobbit battle of five armies trailer","1406574949951"
  • _id in the above example is "1", which I'm going to say is the least recent search in this database
  • display1 is "hobbit battle of five armies trailer" is the stored search suggestion based off a search I made.  Apparently I was interested in seeing the Hobbit movie in December 2014 but as of my writing this I have not seen the movie
  • display2 is blank
  • query is the actual text, which is "hobbit battle of five armies trailer"
  • date is stored as epoch time.  1406574949951 correlates to Mon, 28 Jul 2014 19:15:49 GMT, which I converted using an online epoch time stamp converter
Well there you go.  This database file appears to contain the user's YouTube search history, along with the date and time of each search.  Nifty, huh?

I could deep dive into a few other databaes on the device, but instead I'd rather let you, the reader, explore.  So ... if you have an Android device image to explore, check out the following files:
  • /(userdata partition)/data/
  • /(userdata partition)/data/
  • /(userdata partition)/data/
  • /(userdata partition)/data/ (a bit difficult to parse through this one but incredibly useful)
  • /(userdata partition)/data/
  • /(userdata partition)/data/
  • /(userdata partition)/data/ (also check out the directory /(userdata partition)/data/
  • /(userdata partition)/data/explore the entire com.facebook.katana/databases directory if the Facebook app is installed
  • /(userdata partition)/data/explore the entire com.facebook.orca/databases directory if the Facebook Messanger app is installed
As always, if you have any specific questions about how to read user data, such as text messages, call logs, calendar entries, or more, do not hesitate to contact me.

Reading a SQLite database file
In the above section, I showed where SQLite databases may be on the device.  You can find and extract a SQLite database either from a rooted device or from an image of a device.  Now, how do you actually explore the database file?  It is actually incredibly easy.

First, you'll need the SQLite database file extracted away from your image or you rooted device and stored on your computer in a location you will remember.

There are a few Windows and Linux applications to read SQLite databases.  Autopsy also includes SQLite functionality, so if you are using Autopsy to examine an image, you can use the built-in SQLite plugin.  You can read more about using Autopsy to analyze Android images on a previous post on Autopsy.

My personal favorite method of reading SQLite databases is a plugin for Firefox.  I will demonstrate this plugin tool in this post, but if requested I can show other tools also.  The SQLite Manager is a free download for the Firefox web browser.  If you browse to this page page in firefox you can download and add the add-on to your browser.

NOTE: The SQLite Manager add-on is a developer tool.  The intended audience is developers, not forensic examiners.  This tool has the ability to both read and write to SQLite database files.  Now we are working with an extracted version of the file.  This file is extracted from an image or from a device, so the original is intact even if the file you extracted is altered.  If you choose to use the SQLite Manager tool and need to prove that you have not altered data, it would be wise to create an MD5 or SHA hash of the the database file in the original image before extracting and take another hash of the extracted file on your computer after you examine it in SQLite Manager.  Or, if you are in such an environment where using a non-forensic tool is unacceptable, I would personally recommend opening the image of the device in Autopsy and using the built-in SQLite reader to read your database files.

Once the SQLite Manager add-on is installed, open your Firefox browser and go to the add-ons list.  The SQLite Manager will be there.

Click SQLite Manager.  If you have used SQLite Manager before, you may get this annoying pop-up:

I always click on Cancel because if you click OK you open the last opened database file.  Once the add-on is active, click the open icon.  If you browse over the icon, the text "Connect Database hovers." In the choose file window, make sure you view all files instead of just SQLite files because most Android SQLite database files have the extension .db.  Browse to where you are storing your extracted SQLite database file and open it.  SQLite Manager will now show a list of all tables associated with the database.  In the below screenshot, I opened /(userdata partition)/data/, which stores SMS and MMS messages.

On the left side of the interface is a clickable list of tables.  Click on one.  I will be exploring the table sms.  The table then opens up so you can see the data stored in this database file's table.

As you can see, I've blacked out data, but there are three columns in the screenshot of interest:
  • thread_id (The thread, or conversation.  This ID is a number which references the table "threads" in the same database)
  • address (The phone number with which I am texting)
  • date (The date in epoch time of the message)
Out of the current screenshot is another column called "body".  This column stores the actual text of the message.  There are other columns which indicate the status of the message (draft, sent, received, etc) and if the message has been read or not (only applicable to "received" messages).

SQLite Manager allows you to export a table.  Check out the drop down menu Table -> Export Table.  You can export the table you are currently exploring as a .CSV file, which you can open in Excel if you prefer that interface.

And that's how to explore a SQLite database.  It is straightforward.  Now have fun browsing around user data!

  • Android stores user data from apps, including phone logs and SMS, in SQLite databases
  • You need to pull SQLite files from an image of a device or a rooted device in order to see the contents
  • SQLite database files can be viewed using free viewers
Questions, comments, suggestions, or experiences?  Hobbit movie reviews?  Leave a comment below, or send me an email.