Unit-testing Android Apps: Netbeans and Robolectric

Testing for Android Apps is a bit of a mess. The Android Test Framework dismally fails to make a clear distinction between unit tests, functional/integration tests and UI tests, and assumes that all tests will be run as an Android application on a device or emulator (AVD) as Dalvik code. For unit testing this is extremely clunky. It means that unit tests must be written as a standalone Android project, built as an APK and deployed to the device running a build of the application under test, where it takes over the running of the application to instrument it for testing. This is quite a slow process, and slowness is the very opposite of what we want for unit tests.

All I wanted, though, was to run (quickly!) some simple unit tests against domain classes. There might be some Android mocks needed where my application classes rub up against the SDK, but that's just accidental.

Then, too, there is (currently) a bug in the Ant build script for the Android Test Framework that means that, out of the box, tests cannot be built or run using Ant. I neither know, nor do I care, whether the test might be runnable if I were using Eclipse; I'm a diehard Netbeans user, and Netbeans uses Ant (or Maven, and possibly soon, Gradle) rather than relying on a parallel build system of its own. I think this is one of Netbeans's key strengths. Along with the still-evolving NBAndroid Module and Netbeans's superb out-of-the-box handling of XML files, I've been finding it an absolute pleasure to use for Android development. But unit-testing remained an itch that I had to scratch.

The upshot of all this is that I could not find a satisfactory -- or, indeed, a working -- way to write and run unit tests for the Android project I'm working on.

Down into a Rabbit Hole I went!

Emerging a couple of days later, here's the solution I came up with... It's probably not the best solution ever. It's certainly not definitive. But it Works For Me.

I'm using Robolectric -- a framework that modifies Android platform methods on the fly and provides shadow classes for Android platform classes so that unit test can be run in a standard JVM on my development machine.

Predictably, it was not all Plain Sailing!

The Robolectric documentation is pretty sparse, and very strongly oriented towards IntelliJ and/or Maven-and-Eclipse usage (didn't investigate too closely, don't care!) If I'm going to be shafted into using Maven, then my unit testing is so slow that I might just as well stick with using the standard Android Way of Testing, so I wanted to avoid that. Additionally, this brings the advantage that I can use JUnit-4 style tests (with a caveat - see below.)

The first step is to create a standard Java Project in Netbeans. We already have a directory structure like

...project-root
       |
       +---android-main-project
       |
       +---other-project-related-dirs

so I created my unit-test project on the same level as the main project:


...project-root
       |
       +---android-main-project
       |
       +---other-project-related-dirs

       |

       +---unit-test-project

In Netbeans add at least the following libraries as dependencies for your test project:

• Your main project's bin/classes directory - otherwise the test-project cannot find the classes you want to test.
• JUnit
• Any mocking library you might choose to use - PowerMock looks pretty good, though I haven't used it beyond tyre-kicking yet.
• android.jar for the SDK version you have as your targetSdkVersion in the AndroidManifest of your main project. This is important, since the Robolectric framework uses a custom test-runner (which we'll slightly modify as described below) that examines your AndroidManifest to figure out where to find resources.

The order you specify the libraries is critical: android.jar must come after junit in the classpath for your unit-tests.

Now only one more things remains: telling Robolectric where to find your main project's manifest file and resources...

In your test-project, create a custom test-runner like

net.mikro2nd.android.junit;

import java.io.File;

import org.junit.runners.model.*;
import com.xtremelabs.robolectric.RobolectricConfig;
import com.xtremelabs.robolectric.RobolectricTestRunner;

public class MyTestRunner extends RobolectricTestRunner {
    public MyTestRunner( final Class<?> testClass )
        throws InitializationError
    {
        super( testClass, new RobolectricConfig(
            new File( "../android-main-project" )
        ));
    }
}


Of course you should now annotate your test classes using

    @RunWith( MyTestRunner.class )
    public class DeviceIdTest{
        ...
    }


Now you can "Test" your test project to run all unit tests, select just a single unit-test class to run, or even debug unit-tests without leaving Netbeans at all. And you can run them in a CI server.

Slightly less convenient than unit-testing a non-Android project, but still way better than the alternatives (no testing, Eclipse or Maven, or the bloody slow Android Test Framework.)

My thanks to Justin Schultz who's Eclipse Robolectric Example put me on the right track. All I've done is adapt his hard work for Netbeans use.