Tuesday, February 4, 2020

The Iowa Caucus App

A Rant

All blog posts to date
Introduction Acquisition Analysis
Introduction Imaging an Android Device Examining the image
Picking a Toolkit Live imaging an Android device Some hidden artifacts in a physical image
Why not load ClockworkMod or TWRP to image a device? Using Autopsy to examine an Android image
Identifying your Userdata Partition Some artifacts in the /data/system/ directory
Some non-root methods to learn about a device Viewing SQLite Databases
A quick note on imaging newer Android devices Facebook for Android Artifacts
Using Windows to Live Image an Android device Interpreting data from apps
Obtaining all files in the data partition without a physical image Waze for Android forensics
Magnet Forensics App Simulator
App Reversing Other Topics
Reverse Engineering an Android App File The differences between a physical image and a logical extraction
Fun with Apktool Dirty cow
Deep dive into an app Imaging and examining an Android car stereo
Unpacking boot and recovery kernels
I am a football nut and follow stats like it is my job. Something I've found in common between my nerdiness with technology, movies, video games and sports, is they require some degree of dedication in following stories, along with a high degree of analysis to predict what is going to happen next.

So it should come as little surprise that I am, at least to a degree, interested in politics. I'm not a political super-nerd, I'm no hyper opinionated thought leader, I don't get into Twitter battles, and I follow no political ideology as if it were a cult. That's not my game. But the stories in the news, the polling data, the strategy of campaigns, and the back-stabbing associated with politics all require the same mindset to follow as sports and pop culture.

Note: this post will be about a political story, but it will not be of political opinion. I don't care to publish to the whole world my political opinions for two reasons. One, I don't want to lose readers, and two, much of my viewership is outside of the United States where other political stories are more relevant.

So here we go. A post on an Android forensics blog about the political world, with potentially world event altering consequences. (I never thought I'd say that one).

What happened last night in Iowa?
As I post this, about 24 hours ago the caucus sites in Iowa closed. It is the beginning of the presidential election cycle, occurring every four years. Come early November, the citizens of the United States will flock to polling places to cast their votes for President of the United States of America. And though November is nine months away, the whole process has started. Because before citizens can vote, there must be a nominee. And last night in Iowa, the first state in the nation to have a say in selecting a nominee, there were caucuses to determine the state's choices for Republican and Democratic nominees for president.

(A caucus, if you are unaware, is different from a primary. A primary vote is like any other vote where you put select your candidate of choice on a ballot. A caucus is more of a community event and involves actual social interaction as part of the process. And each caucus has a party official to manage the event and report the vote tally. I'm not going to get too much into the caucus process because it isn't relevant here, but it is a fun process.)

In general, the caucus results should be known quickly. But not until nearly 24 hours after the caucus sites closed did Iowa finally had some results. And why is that?

A mobile app.

The Democratic Party of Iowa elected to try some new technology to get the caucus results faster. Each caucus leader would download an app to their mobile phones and use the app to report the caucus results. Those results would be gathered to a central server and tabulated. The idea is the whole world could know nearly instantaneously who won the Iowa Democratic caucus. And I'll give some credit where credit is due, that is a novel idea. But the devil is in the implementation.

As you may well have heard, that mobile app was an utter disaster. The app crashed, those results either did not come at all to the central server or came delayed, and the results were declared "inconsistent". Over a day later, we still don't know who won Iowa.

What went wrong?
Not a whole lot is actually known about the app, so I'm just going to report what I know. Please comment and correct me if I get any of this wrong.

The developer of the app is a small firm in Washington, DC. The app was not widely distributed, so I don't have access to it. I've heard it was developed for both iOS and Android, though I can't confirm this.

The Democratic Party's cyber security chief saw problems coming and advised not going forward with the app. That advice was not heeded, and as anyone advising on cyber risk knows, that is a common outcome.

The app is not widely available, has not been made open source, the security (and more specifically cryptography) algorithms used are not known, the general process for authenticating is not known, nobody knows what data the app stores on the phone, and the security community at large has not vetted this app. I'm going to go through some of these main points in detail, and lest we forget the importance of the outcome of this caucus (more on that later). But before that, here is nearly accurate footage of me hearing about this fiasco:

Open source
I am an open source proponent in general. When we are talking about anything security related, open source is important because it invites scrutiny from the security community. The encrypted chat apps Telegram and Signal are open source, and the cryptography for WickR is open source, and I am sure there are many other such open source encrypted call and chat apps.

Now I am not saying the developer should have open sourced the app. They presumably spent a lot of man-hours on the app. However, at least documenting the cryptography used would be good security practice. Even for the closed-source encrypted chat apps, the developers generally have white papers detailing cryptography algorithms used. I know of no such for the Iowa Democratic Caucus app. If this information were made available, the security community could have reviewed the information and raised concerns.

I've heard the method for even downloading the app is rather convoluted. So I have no idea what the method is for authenticating a user.

The user should be the Democratic party official reporting on the results from the caucus. How does the user authenticate that they are genuinely the one reporting data? I presume there is a password. Is there a second form of authentication, such as the Google Authenticator app? How are passwords secured? Can the authentication session be hijacked, allowing an attacker to masquerade as the official and feed false results?

We don't know the answer to any of these questions. And that is a problem. Lest we forget about the importance of accurately reporting results (more on this later).

I've heard the app was independently vetted, but we've not heard by what security firm and what tests were attempted. We've also heard it was not publicly vetted by the US government.

App vetting is a big deal. The process involves attempting various network attacks, on-device attacks, and attacks tailored to the exact app use case. What was the vetting process? Now I don't expect anyone to publish flaws that were found and fixed as that would not be good security practice, but it would be nice to say who vetted it and what they attempted.

Possible scenarios
As of me writing this post, we still don't have results. So I'm going to go over some best and worst case scenarios that could happen, and I'm also going to put on the hypothetical hat here for some really bizarre and awful scenarios that could happen.

For a best case scenario, the caucus results are counted accurately using a backup method involving physical records, which appears to be the case right now. The result is the winner is officially announced, but late. And that really is a problem because in politics, timing is everything. At this point, all the Democratic candidates have shifted their efforts away from Iowa and are off to New Hampshire for the upcoming primary. Whoever ultimately wins Iowa has been denied their prime-time moment to give a nationally televised, dramatic speech to a massive national audience watching the Iowa results. Instead, there will be a delayed announcement for the winner who will set up an impromptu speech and campaign party in New Hampshire, with a smaller national audience who only associates the Iowa primary with a failed mobile app. All those volunteers from the winning campaign in Iowa have been denied their right for a well-earned party complete with personal gratitude from the winning candidate.

But let's imagine some worst case scenarios here. As a forensics guy, I have developed a knack for thinking worst case.

There's the obvious worst case of inaccurate results. Between unvetted security in transit and unknown authentication methods, there could be inaccurate results fed to the central server. The wrong candidate gives a rousing speech to supporters and a massive national television audience. The actual winner never knew that he or she was cheated out of a win in Iowa and the television spotlight, along with all the campaign momentum. And this should serve as a warning for why physical records in voting are so important.

But there's another worst case scenario. And this is where that forensic mindset is important. Let's say there was no physical records from the caucuses, or those were destroyed. The caucus result data never made it from the app to the central server. All those apps on all those devices stored the only known sources of caucus results. And to make this more complicated, all those mobile phones used by party officials are personally owned. In this worst case scenario, the only way to get caucus data would be to acquire the data from personally owned phones. Would these officials voluntarily hand over their devices to be imaged? Can they be seized? What if the device is stolen, lost, or damaged? Those caucus results could be gone forever. Nobody would ever know who actually won Iowa. Again, a warning for why physical records in voting are so important.

Overall, it does look like we're getting the best case scenario. We'll still figure out who won Iowa. And everybody learned a thing or two about this uncharted territory of reporting vote results via app.

Regardless, there already is a major result. We've all heard about foreign interference in the 2016 election, with the goal being to sow doubt and distrust into the US election process. Well, there's already doubt and distrust in the 2020 election, and it required no Russian influence. Unless you count the vodka we all need after this debacle.

The bigger picture
There's a bigger picture to all of this that needs to be considered, which is really the point of this post. And that is engineering consequences.

Automotive engineers design cars. They may imagine new elegant or comfortable features, or more aerodynamic designs to add an extra tenth of a mile per gallon. But automotive engineers never forget that they are designing vehicles that carry people at speeds exceeding 60 miles per hour. That is a weighty concern. Poor engineering in cars has led to deaths and will continue to, but good engineering, which is exceedingly more prevalent than poor engineering, has led to incredible safety features that have saved far more lives.

Medical engineers design surgery equipment and devices. These keep people alive during surgery. These devices must be sterile. They must function or else a patient can die. In an open heart surgery, there is crucial equipment necessary to maintain circulation through the body. People have died due to poorly engineered products. But exceedingly more have lived longer lives due to well engineered devices.

In software engineering, we so often lose sight of real world consequences. A job of mine before entering the world of forensics involved writing an asset tracking system for a mid-sized manufacturing company. It rarely dawned on me that actual company products were being shipped from site to site and to end customers. That thought rarely occurred to anyone on the team. We were so focused on developing a usable system that on occasion our manager would have us walk through the factory to see the manufacturing process and speak with the factory employees about the need for better asset tracking.

Digital forensics is one of the few disciplines within technology where real life circumstances are intrinsically interwoven. If your realm is incident response, the real life ramification of your work is identifying the theft of perhaps millions of dollars worth of corporate intellectual property and preventing future attacks. If you are a criminal investigator, the real life ramification is the life and liberty of a suspect. There is a story I heard once about a teenage girl who ran away. Her mother called the police who examined the home computer and found evidence of the girl chatting with a "boyfriend", who turned out to be an adult male and a child abductor. The real world ramification of this work was catching the man en route to a meet with the girl, and protecting the girl from exploitation at the hands of a truly sick individual.

In this case of the Iowa caucus app, the real world ramification is the election of the leader of the free world. Whoever wins the Iowa Democratic race gains incredible momentum going into future state races. That winner can very well be the nominee for the party and run against the current President of the United States. The winner of that election will be the commander in chief of the greatest fighting force the world has ever known, at the head of the world's largest economy, will have a greater influence on world affairs than any other person on the planet, and will have the entire world's attention with every action, every inaction, and every word.

Do you really want to leave all of that to a mobile app?

  • Vet your apps
  • Open source, or at least document as well as you can, your crypto for the security community to review
  • Never forget real life ramifications of digital work

Questions, comments? Leave a comment below, or send me an email. And if there are any politically charged nonsense comments, they'll be deleted.

Saturday, April 27, 2019

Magnet Forensics App Simulator

And Avengers

All blog posts to date
Introduction Acquisition Analysis
Introduction Imaging an Android Device Examining the image
Picking a Toolkit Live imaging an Android device Some hidden artifacts in a physical image
Why not load ClockworkMod or TWRP to image a device? Using Autopsy to examine an Android image
Identifying your Userdata Partition Some artifacts in the /data/system/ directory
Some non-root methods to learn about a device Viewing SQLite Databases
A quick note on imaging newer Android devices Facebook for Android Artifacts
Using Windows to Live Image an Android device Interpreting data from apps
Obtaining all files in the data partition without a physical image Waze for Android forensics
Magnet Forensics App Simulator
App Reversing Other Topics
Reverse Engineering an Android App File The differences between a physical image and a logical extraction
Fun with Apktool Dirty cow
Deep dive into an app Imaging and examining an Android car stereo
Unpacking boot and recovery kernels
I'm back! Sorry it's been a while. No, I did not turn to dust, nor did I get stuck in the quantum realm. And I also didn't violate the Sokovia Accords. Instead, I am now a dad! My wife and I have a little daughter now which means my spare time is gone and replaced instead with the joys of parenthood.

Being a dad, I will always try to guide her in a good direction. Maybe I will be like Howard Stark and encourage her to be pursue the newest technology at every step. Maybe I will be like Odin and raise her into royalty. She already is a princess. Just as long as I don't become a celestial, or an evil living planet.

And also ... there's this movie that just hit the theaters.

Avengers: Endgame. The result of 11 years of building up this cinematic universe, which has become an unprecedented cultural phenomenon. Already rewriting the box office record books. Have I seen it? Of course not! I'm a dad. I don't have time for movies anymore! So don't spoil it for me because I will see it sometime in the near future.

So the point of this post. Magnet Forensics just released a new tool called the App Simulator. This post will walk through using the tool and why such a thing is useful.

But as for me. New forensics tool?

And do I want to use it?

Magnet Forensics App Simulator
The app simulator is essentially a virtualized Android device, running in Oracle VirtualBox, with automated methods to emulate an app, and its data as retrieved from an image, in the way the user saw it. Magnet has developed a clean, intuitive, and streamlined method, which is why I am detailing their tool specifically.

So let's say you are an analyst for Strategic Homeland Intervention Enforcement and Logistics Division (SHIELD) and you are analyzing a device belong to a Hydra operative. You have successfully imaged the phone. Great. You have acquired all the data on the device.

Now you are analyzing the data and you come across an app. You suspect the app may contain useful data. However, the data files and difficult to analyze. In this example, the app is WhatsApp, the messaging app. How do you expect to find meaningful data out of this?

The above is a hex view of the main database for WhatsApp. And it is encrypted. I see no meaning of any kind there. But if I ask one character, I know what he'll say of what he sees.

"I am Groot!" Very helpful there, thanks.

Here's the thing. I don't know off the top of my head how to decrypt a WhatsApp database. My laptop was made by Dell. It is not Wakandan. It does not have the processing power to brute force an encrypted database file.

And I could reverse engineer the WhatsApp application and deduce the decryption logic. But that would take going through possibly millions of lines of code. That is frustrating. And heavens help me if I had this guy's anger problems:

So one way I can examine the WhatsApp data is to emulate it. I extract the associated WhatsApp data, along with the app itself, and install it on an Android Virtual Machine (VM). The VM acts just like an Android device. It will install the app, accept the data, and behave just as if it were the Hydra operative's phone. The WhatsApp app file knows how to decrypt all this encrypted data, just as it does on the phone itself. Then I can scroll through the installed instance of WhatsApp and see the data just as the Hydra operative saw it. And that right there is the very point of the App Simulator by Magnet Forensics. Pretty nifty, huh?

How to use the tool
First, go download the tool. It is free! It takes registering an email address.

Follow the download and install instructions. You have to download two different files and follow two different installations. You first install Oracle VirtualBox, and then you install an Android VM.

Once all done, open your Magnet App Simulator and follow the instructions. It will first ask you to run the Android VM. Make sure that is online. Here is what the emulator looks like.

It looks very similar to any normal instance of Android. This emulator runs Android 8.1.

Then pick out your Android install file (WhatsApp.apk, found in /data/app of the image of my device):

And then add associated data. For me, I extracted /data/data/com.whatsapp for Application Data, and /sdcard/WhatsApp for SD Card data.

And then hit next. And in a snap, you'll be ready to analyze.

In no time WhatsApp gets installed.

And then all the data gets loaded into WhatsApp in the proper location and given the proper permissions, all without you having to do a thing. When all done, you can browse through the app data, decrypted and readable, and use it all as part of your investigation into Hydra activities.

And another nice feature. You can create a snapshot. That is, save the state of your VM. That way if you want to manipulate some data for any reason, you can restore to a known state. It is about like having your own time loop.

Now this technique is actually not that new. I have been emulating app data like this for many years and have often considered doing a blog post on the technique. However, my process is extremely manual. Instead of a nice user interface, it takes using Android Debug Bridge to push all the files in the right place and change permissions. And it historically relied upon the Android Virtual Device included with the Android Software Development Kit. If you've ever used that, you know how slow and clunky that is. Going from my old method to the new Magnet tool is about like going from this:

to this:

And that's about all I have for this tool. It is excellent and should be part of your examination kit, along with a gauntlet to hold the Infinity Stones should the need arise. As for what I do next, I'm not sure. Just remember, in all things in life, Maximum Effort!

Oh wait, wrong universe.

  • Go ahead and download the Magnet Forensics App Simulator.
  • The app simulator can load up an app's data straight from an acquired image and present it the same way the user saw it before acquisition.
  • If data is encrypted or otherwise obfuscated, this can be a workaround. Instead of figuring out the hard way how to interpret data, let the app do what it was designed to do, but in your own virtualized environment instead of a suspect's device.

Questions, comments? Favorite Avengers movie (NO SPOILERS)? Leave a comment below, or send me an email.

Friday, April 6, 2018

Obtaining all files in the data partition without a physical image

And why it can be a good call

All blog posts to date
Introduction Acquisition Analysis
Introduction Imaging an Android Device Examining the image
Picking a Toolkit Live imaging an Android device Some hidden artifacts in a physical image
Why not load ClockworkMod or TWRP to image a device? Using Autopsy to examine an Android image
Identifying your Userdata Partition Some artifacts in the /data/system/ directory
Some non-root methods to learn about a device Viewing SQLite Databases
A quick note on imaging newer Android devices Facebook for Android Artifacts
Using Windows to Live Image an Android device Interpreting data from apps
Obtaining all files in the data partition without a physical image Waze for Android forensics
Magnet Forensics App Simulator
App Reversing Other Topics
Reverse Engineering an Android App File The differences between a physical image and a logical extraction
Fun with Apktool Dirty cow
Deep dive into an app Imaging and examining an Android car stereo
Unpacking boot and recovery kernels
In my previous post, I made a major announcement. Spoiler alert, I'm going to be a dad!
I have since learned, well, a lot. If I were to boil down everything I've learned so far, it would be as follows:
  • The words "crib", "pack 'n play", and "bassinet" are not synonymous.
  • Strollers are in fact fashion items for parents, not vehicle for children, and are priced accordingly.
  • There are many brands of baby bottles. And they are different. And your baby decides which one he or she likes, regardless of how many of a different brand you have purchased.
  • Baby monitor systems are nothing short of security camera marvels. And as a security person, I am leery of every claim made regarding wireless security.
  • Kids don't come pre-programmed to sleep through the night. And there is no golden guide to expediting the learning process.
  • Consignment shops are great places to buy baby stuff. So long as you don't think about drool from other children.
  • A bouncer is not just a big guy outside of a bar.
  • A rocker is not just a cool guy with a guitar.
  • Rockers either go left to right or front to back. And babies only want one or the other, regardless of which one you have already purchased. Or sometimes they want both.
  • Car seats are like food. They have expiration dates. And you don't want to buy them used.

So as you can imagine, this has been quite the learning curve! I wish somebody would write a good guide to this whole parenting thing meant for us computer guys, except I know us computer guys never read manuals. Unless they are short.
So on that note, here's a nice short post on obtaining all the user data files from an Android device instead of a physical image. And why you may do such a thing.

Why would I want to do such a thing?
So in what scenario would you want to skip the process of physically imaging an Android device in lieu of a file system dump? Well first, what exactly am I talking about?

A physical image is a file (or set of files) representing the beginning to end of a storage volume, such as a hard drive, or in this case, an Android device. It contains every bit of data, stored just as stored on the device itself. Done properly, a physical image is an authentic duplicate of the device in question.

A file system dump is a copy of all the files in a specific volume. It contains just files. No deleted records, no slack space, just files.

In Android, all user data is stored in a partition called userdata. If you only had to pick out one area of the device to image, you would pick userdata, every single day. That is where all databases, user activity, installed apps, all app data, and logs are stored. And assuming there is no external SD card, it is also where photos, videos, and other media files are stored. So this post is about acquiring all the files from userdata. Not slack, just files.

So why would we do that? Here are some possible reasons:
  • You are in a rush
  • This is not a criminal case, it is only for research
  • You already obtained a physical image but are having a hard time parsing a file system (more on that later)

How do we do it?
Now that I've explained what we are doing and why, let's get it done. First, just like in obtaining a physical image, your device needs to be rooted, and you also need busybox installed. Hook up your rooted Android device to your Linux or Mac computer.

Now we are going to use multiple terminals the way we do in live imaging a phone. Open up two terminals.

Terminal 1 will be for interacting with the device. We are going to port forward, effectively making a network connection between the computer and the device over the USB cable. We will create the archive file and pass it over that USB cable to the computer.

Port forward to create that network.
adb forward tcp:8888 tcp:8888
Open up an adb shell with your phone using the following command:
adb shell
Once shelled in, gain root.
Then we will begin making the archive.
busybox tar -cvz /data | busybox nc -l -p 8888
If all goes well, the screen should freeze. This command copies all of /data into an archive file and passes it over the USB wired network connection to whoever is listening. Which leads us to Terminal 2.

Terminal 2 will be for interacting with the computer and receiving the archive file. First, go to the directory you want the archive to arrive.
cd /path/to/desired/location
Now use netcat to listen to the USB network connection and receive the archive.
nc 8888 > data.tar.gz
If all goes well, the screen should freeze. This command listens to local port 8888, as set up previously with the adb forward command, and passes whatever comes out to the file data.tar.gz.

You can optionally open a third terminal for monitoring progress. Navigate to the same location as terminal 2 and use ls -l to check the file size.

When all is done, you will have the file data.tar.gz, containing all the user files. No slack, just files.

And finally, a note on f2fs, or Flash Friendly File System. f2fs is an alternate file system to ext4. f2fs seems to perform far faster than ext4 and may someday be the default on Android devices. As is, only a few devices have f2fs support. But I know this is problematic because I have received several emails asking for f2fs assistance. To that, I say two things. One, thank you for letting me know. This post is largely because of calls for f2fs support. Such emails keep me posted on what all issues are out there. And two, f2fs support forensically is currently not good.

FTK Imager does not have f2fs support as of right now. Neither does Autopsy. Some of the pricey tools have f2fs support, but the free tools are lacking. There is Linux support for f2fs, but I've had limited success.

So, if you are trying to image device where userdata is formatted f2fs, my suggestion is as follows. Use this post to obtain a file system dump of the data partition. Then if you would like, obtain a physical image for searching through slack.

  • You can obtain a file system dump of an Android device's data partition, and in some circumstances, this is just what you are looking for.
  • The process is similar to obtaining a physical image.
  • Without good f2fs support, consider this process if you are working with an f2fs-enabled device.

Questions, comments? Dad advice? Leave a comment below, or send me an email.

Thursday, February 22, 2018

Deep dive into an app

And an announcement

All blog posts to date
Introduction Acquisition Analysis
Introduction Imaging an Android Device Examining the image
Picking a Toolkit Live imaging an Android device Some hidden artifacts in a physical image
Why not load ClockworkMod or TWRP to image a device? Using Autopsy to examine an Android image
Identifying your Userdata Partition Some artifacts in the /data/system/ directory
Some non-root methods to learn about a device Viewing SQLite Databases
A quick note on imaging newer Android devices Facebook for Android Artifacts
Using Windows to Live Image an Android device Interpreting data from apps
Obtaining all files in the data partition without a physical image Waze for Android forensics
Magnet Forensics App Simulator
App Reversing Other Topics
Reverse Engineering an Android App File The differences between a physical image and a logical extraction
Fun with Apktool Dirty cow
Deep dive into an app Imaging and examining an Android car stereo
Unpacking boot and recovery kernels
First, I have a major announcement to make regarding this myself and this blog going into the future. The announcement is at the end of the post. If you don't want to read a whole bunch about reversing, feel free to skim over this post and get to the bottom.

I've done posts before on reverse engineering apps. I've always covered the commands, so I wanted to do something different. I wanted to do some actual reversing and post the findings.  Sounds like fun, right?

Well, yes and no. Yes, reversing is fun (and frustrating). No, reversing may not always be, say, legal. Which leads me to a problem. I would have loved to reverse a major well known app and post the findings here. The problem is something nice called an "End User License Agreement" (EULA). An EULA is a contract between the user of software and the producer of the software. Normally you just skip over the EULA, but I decided to read some on Android apps. Here's a nice little sample from an actual EULA from a major app by a major publisher, under a section called "GENERAL PROHIBITIONS":
Attempt to decipher, decompile, dissassemble or reverse engineer any of the software used to provide the Services or Collective Content.
Basically, no reverse engineering or we'll send our lawyers after you. Lawyers, they just take all the fun out of life, don't they?  (I have multiple lawyers in the family and I guarantee they are laughing right now).

So seeing as this blog is called "Free Android Forensics" and there are absolutely no ads, I do not make any money here. I have a PayPal link and I use donations there to maintain my equipment. I would rather not ask for donations to pay my legal bills.

So, scratch that plan. Instead, I decided to reverse ...

Yes, this post is going to be on reversing a Solitaire app. You know, the one player card game packaged with every Windows operating system ever. Well, I found an open source version of Solitaire for Android on a free and open git site. I downloaded it, played it, pulled the compiled APK from my tablet, and reversed it.  Why open source?  So that way I can't get in any trouble for reversing it! The source is already online.

Here's also why. I want to show the process of reversing, some basic findings, and some makeups of an app. We've all played Solitaire, we all know the makeup of a deck of cards and the game logic, so I figured this would be a good app to show reversing. Everybody should be familiar with Solitaire, so when I show some reversed app source, the logic should hopefully make some sense.

Getting the APK
So let's pretend I didn't download source code and compile an Android app. Let's pretend I am pulling an app I installed on my device from the Google Play Store to my computer to reverse it. First step, get the APK (the actual app file) from the Android device to my computer. And I'll show a lazy way.

Download ES File Explorer (or some other file manager of your choice) to your device. ES File Explorer allows you to copy installed APKs to your SD card. I opened ES File Explorer, went to the app manager, and backed up the Solitaire app. It appeared in a directory on my device at /sdcard/backups. Then I plugged the device into my computer and retrieved the APK using adb.
adb pull /sdcard/backups
Within the pulled files was the Solitaire app. That was easy.

Getting the Android manifest and app version
My first step in reversing an app is always to obtain the manifest and the version of the app. The manifest is the unique name of the app. The version number is just as it sounds. When you have the manifest and the version of an app, you have a unique way of differentiating that app from others. There are many Solitaire apps out there, but only one with a given package name and version number. I have two ways, two different tools to obtain the manifest and version.

First, aapt. If you have the Android SDK installed on your computer, navigate to the build-tools directory and look for the file aapt. Open your terminal and type the following:
/path/to/aapt l -a /path/to/apk > /path/to/manifest.txt
That command runs the tool aapt against the apk file and sends the output to the file manifest.txt. Open the txt file and look for the following:
A: android:versionName(0x0101021c)="1.12.2" (Raw: "1.12.2")A: package="com.exubero.solitaire" (Raw: "com.exubero.solitaire")
These two lines describe the version number (1.12.2) and package name (com.exubero.solitaire).

Second method, apktool. I've done a post on apktool. Feel free to read over it. I downloaded the newest version of apktool, placed it in the same directory as the Solitaire APK, and ran it against the app using the following command:
java -jar apktool_2.3.1.jar d solitaire.apk
The result is a directory called "solitaire" containing a fully dumped version of the app. There is the smali code (Java byte code), all the xml files within the app, all the native libraries used, and all the images/videos/audio clips/any other resource file required.

Within the directory "solitaire" there is a file AndroidManifest.xml. The package name is there. There also is a file called apktool.yml. The version is there.

By resources, I mean media files, custom fonts, dictionaries, or other extraneous files required to present the app to the user. We can get to resources one of two ways. First, the apk is just a zip file. We can unzip the apk and retrieve resources. Second, we can use apktool as described above to view resources.

I used apktool. Within the solitaire directory is a directory "res" and within that "drawable". I'll highlight a few interesting images for this demonstration. First, a single image containing the font on all the black cards:

There's a red one also. Next, the suit icons.

Here's the logo on a red king card.

There's similar for other face cards. And finally, the actual logo for the app you would see in your device launch tray.

Solitaire is a simple app. Complicated apps could have way more images. There also could be audio and video.

There also is a directory within res called "layout". This directory contains xmls describing any given screen the user could see. If you've ever written an Android app in either the old Eclipse or the current Android Studio, you may have played with the app layout editor. When you create a layout for a screen, you actually are creating an xml file. Here is an entry in the file stats.xml, which describes the stats screen you can see of your game usage.
<TextView android:textSize="20.0sp" android:gravity="center" android:id="@id/text_best_time" android:focusable="true" android:layout_width="fill_parent" android:layout_height="wrap_content" android:text="@string/solitaire_layout_text_text" />
This line describes a text box called "text_best_time", which would presumably display the best playing time the user has ever achieved in Solitaire.

Note on the xml files. The compilation process of an app also compiles the xml files. If you want to see the decompiled xml files from an APK, you have to use apktool. If you unzip the APK and try to look at any xml file, including the manifest, you get junk.

Source code
apktool decompiles the apk to smali, which is java byte code. It is tougher to read than actual java. So let's decompile the app to java. I am using Santoku, a custom version of Linux developed for free by the company NowSecure which contains built-in mobile forensics tools. It contains dex2jar and jd-gui. I'll describe both as I use them for the Solitaire app. You also can Google these tools and download them. They can run in Linux and Windows. I prefer Linux personally. I also previously wrote a post on this process.

First, let's use dex2jar. Within an Android apk file is a file called classes.dex. That file is the compiled source code. It is the actual executable part of an app. It contains all the logic that makes the app run and codes the app's behaviors. dex2jar converts the classes.dex into a Java JAR (Java Archive) file, which you can decompile using your Java decompiler of choice. The syntax for dex2jar is as follows:

d2j-dex2jar solitaire.apk 
The result is a file called solitaire-dex2jar.jar. Note, you also can unzip the apk, retrieve the file classes.dex, and run dex2jar against classes.dex.

Now we use a decompiler to view the resultant jar file. I use JD GUI as it is built in to Santoku. The line to open the jar in JD GUI is as follows:
jd-gui solitaire-dex2jar.jar
Then the tool JD GUI opens up to view the Java source. Pretty nifty.

Here are some quick findings. First, the coder defined what a card is. So there is an object called "Card."

As you can see, the developer defined all four suits (hearts, diamonds, clubs, and spades) and the face value cards (jack, queen, king, and ace).

Above are public functions getSuit and getValue. Other classes can call these functions within the class Card to do logic based on what card it is.

Here is logic that is part of dealing. There are seven stacks of cards to move around in order to reveal the hidden cards and ultimately move all the cards into four stacks of face values. The logic above has seven card "anchors" where a card can be added.

Speaking of dealing, there's shuffling. The deck is shuffled three times to ensure randomness.

And above is the actual shuffling logic.

The above shows some important functions. As you know in Solitaire, when you are moving cards among the seven stacks, you can put a red card onto a black or a black onto a red. But you can not put a black onto a black or a red onto a red. You see a function called "isSuitsOppositeColours". And the objective is to get four stacks of the face values - all thirteen hearts in a stack, all thirteen diamonds, etc. So there is a function called "isSuitsEqual". These functions are called to compare two cards to see if they can be stacked on top of each other for a given context.

When the user wins, the above edits a file to update the statistics. In the app's directory within /data/data, there is a file SolitairePreferences.xml in the shared_prefs directory. The code above has an integer called j. This gets the fastest time as stored in the SolitairePreferences.xml file. The line as follows:
if ((j == -1 || (this.elapsed < j))
checks to see if the win just now took less time than the fastest as recorded in the xml file. Or if the fastest recorded is -1, that means the win just now was the first win.

My tablet is rooted, so I was able to retrieve the app's data files. Here is what that xml file looks like:
<?xml version='1.0' encoding='utf-8' standalone='yes' ?><map>    <boolean name="SolitaireDealThree" value="false" />    <boolean name="PlayedBefore" value="true" />    <boolean name="SolitaireSaveValid" value="false" />    <int name="SolitaireNormalDeal1Score" value="0" />    <int name="LastType" value="1" />    <int name="SolitaireNormalDeal1Attempts" value="19" />    <int name="SolitaireNormalDeal3Score" value="0" />    <int name="SolitaireNormalDeal1Wins" value="10" />    <int name="SolitaireNormalDeal1Time" value="213768" /></map>
The entry "SolitaireNormalDeal1Time" is the fastest time (in milliseconds). So if the value this.elapsed, or the time required to win the game in milliseconds, is smaller than 213768, the new value of SolitaireNormalDeal1Time will be the current time.elapsed.

If you remember above, I mentioned there was a stats.xml file which describes the layout of the stats screen. The logic above fills in the best time time. The integer k is the fastest time as read from the SolitairePreferences.xml file. The value i1 will be the number of minutes and n will be seconds. This logic can take the fastest time from the xml file (213768) and turn it into minutes and seconds (3 minutes 33 seconds).

And there's a screenshot of my stats view. Don't judge.

And there's lots more you could find by reversing the app. Reversing is a rabbit hole. It always keeps getting deeper.

Big picture
So admittedly reversing a game is not very useful. If you are conducting an investigation and you spend your time reversing somebody' solitaire game, you are probably wasting a lot of time.

Here's the thing. If you are investigating a device and you see some data associated with an app that could be interesting but you cannot explain the meaning of that data, you may need to do some reversing. The processes, tool, and techniques in this post can work on any app out there. You can take an APK file, reverse it to source using dex2jar and a decompiler such as jd-gui, and then go hunting through the Java to find logic that creates said interesting data. You can use apktool to get the xml files to compare against the Java source code. Then you can figure out what that data means. Just expect way harder to find results and way more time spent than in a simple Solitaire app. The methods are straight forward. The analysis can be quite advanced. And frustrating.

So here's my announcement, and it is a major one.  I often talk about my wife, and sometimes my cat. That's my family. Well, that's all changing. My wife is pregnant! We're going to be parents!

So what does that mean going forward? Well, lots.  Everything in my life is going to be changing. What does that mean for this blog? I do this blog in my free time. My free time is right now spent approximately as follows:
  • playing video games
  • hacking around with phones and computers
  • playing video games
  • going for bike rides and runs
  • playing video games
  • watching football
  • playing video games
  • spending time with my wife which often involves ...
  • playing video games
  • working on my car
  • playing video games
I suspect that free time is soon going to look more like the following:
  • changing diapers
  • freaking out about everything sharp or poisonous in my house thus baby proofing everything
  • changing diapers
  • attempting to get an hour or two of sleep
  • changing diapers
  • cleaning up barf
  • changing diapers
  • checking on a crying child which often will lead to ... 
  • changing diapers
  • taking pictures of absolutely everything
  • changing diapers.
Point is, I may not have as much time to devote to this blog as normal going forward. No I'm not retiring, no this blog is not going away, and yes you'll still be able to reach me for questions. I just will probably be around a bit less.

Exciting news!

  • There are many tools out there for reverse engineering Android apps
  • Check out the app's code and correlate findings with files in the res/ directory and data files created by the user
  • Code review can help make sense of user-generated files
  • The process for reverse engineering a simple app is the same as a complicated app
Questions, comments? Good parenting advice? Leave a comment below, or send me an email.