Testing on Android part 1: Unit testing

Hello fellas! The last article received a lot of positive feedbacks and questions. I noticed that most of the questions was related to tests, specially about unit testing on Android. I admit that testing could be tricky at first. There are too many option and approaches to choose from and one could get lost trying to pick the right one.

In this series we’ll talk about:

  • Different kinds of tests and when to use then
    • Unit tests
    • Integration tests
  • The most common frameworks available on Android
    • JUnit4, Robolectric, Espresso, Mockito and Hamcrest Matchers
  • How to setup the test environment
  • How to create unit and instrumental tests
  • How to run the tests

On this article we’ll talk about testing concepts and concentrate on Unit Testing. In the next part we’ll analise the Integration Tests, paying close attention to UI testing.

The Testing process

Testing holds a special place on development process. It could be implemented at the project’s earliest stages by unit testing modules and small parts of code. Later, the modules integration and the UI could be tested. When the project reaches some maturity, it’s time to test and optimize the performance and the whole system.

  1. Unit test modules and small parts of code
  2. Test the integration between modules, classes, UI, etc.
  3. Test and optimize system and performance

On this text we’ll skip the performance and system tests. It is a complex topic that deserves its own article.

Unit testing

If you’re a developer for some time, you’re at least heard about Unit Testing. It is a concept that has been around for a very long time and it is part of the workflow of every serious project. Although, sometimes it is left aside, especially by beginners.

In plain and simple english, the main goal to unit testing is to test small parts of code. A modern approach would be to isolate this code and test it on a controlled environment.

There are two basic unit testing approaches : Sociable and Solitary.  The sociable test could be considered a classical approach and the solitary is the current modern paradigm.

  • Sociable test: Relies on real collaborators. The test results are tied to the return given the tests object’s collaborators. If there some issue with them, the test will fail.
  • Solitary test: Replace the collaborators for Mocks, allowing complete control over them.  The test concentrates only on the code being tested. It doesn’t matter if the collaborator’s code contains issues.
Sociable and Solitary Unit Testing
Sociable and Solitary Unit Testing

Ideally we need to detach the operation, method, class or whatever part that we’re testing from its collaborators, replacing them for mocks.

Replacing the collaborators by mocks allows us to focus on the code tested, ignoring its partners. It doesn’t matter if the collaborator object contains errors or if it was even implemented because mocks gives the possibility to specify exactly the result to return.

Mocks are really useful and you’ll use it most of the time. Although there will be some situation where will be interesting to let you unit testing rely on its real collaborators. The challenge with this approach is that it could be hard to find errors since the problem could be within the unit collaborators.

To avoid this scenarios it is advisable to test the collaborators first or use a more advanced technic, like Mockito.spy that intercepts the collaborator, taking control of a specific operation.

  • Isolate and test small pieces of code on a controlled environment.
  • Replace collaborators for mocks.
  • Mocks allows a complete control over the object.

Unit testing on Android

There are different ways to unit test your code on Android, but the standard one is by using JUnit, a Java framework that has been around for quite some time. It is currently on version 4 and it is very simple to use and setup.

Android provide us with its own mocking system, but it needs a lot of boilerplate to use. Hence, the most efficient way to use mocks on Android SDK is with Mockito. I don’t have words to describe how awesome and life saver is Mockito, but trust me, you’ll learn to love it.

JUnit4 is a incredible tool that will help us a lot, but it lacks some Android resources and that could create some issues during tests. This framework is Java based, thus a lot of Android SDK special resources aren’t implemented by default.

We’ll compensate that by using Robolectric, another magic framework. It provides a version of the Android SDK that runs directly on JVM, giving us a lot of extra speed and allowing access to Android exclusive resources.

Robolectric is a pretty complex framework and this explanation only scratches its surface. I advise you to take some time reading the documentation to fully comprehend its possibilities.

At last but not least, we’ll need Hamcrest matchers to filter and control our tests correctly. According to the official documentation, it is “a tool that allows you to pick out precisely the aspect under test and describe the values it should have, to a controlled level of precision”.

These four framework are my weapons of choice to do unit testing on Android. You could do it differently, but this setup will get you going. Now let’s prepare our environment.

Setup on Android Studio

  1. On your apps build.gradle
dependencies {
    //...
    testCompile 'junit:junit:4.12'
    testCompile 'org.hamcrest:hamcrest-library:1.1'
    testCompile "org.robolectric:robolectric:3.0"
    testCompile 'org.mockito:mockito-core:1.10.19'
}

2. Create this folder structure: /src/test/java/ [package-name]

3. Click on “Build Variants” and in “Test Artifacts” select “Unit Tests

Selection Unit Tests
Selecting Unit Tests in Android Studio

Creating a Test configuration

  1. Click “Edit Configurations…”
Click on "Edit Configurations"
Click on “Edit Configurations…”

2. Click on “+” and select JUnit

 Click on "+" and select JUnit
Click on “+” and select JUnit

3. Give it a name and in “Tesk Kind” select “All in Package”

Give it a nem and In "Test Kind" select "All in Package"
Give it a nem and In “Test Kind” select “All in Package”

4. To run Robolectric you need to select “$MODULE_DIR$” in “Working Directory”

in "Working Directory" select "$MODULE_DIR$"
To run Robolectric you need to select “$MODULE_DIR$” in “Working Directory”

Building our Unit Tests

In our tests we’ll use a app developed using MVP architecture pattern. It consists of a basic note taker with only one Activity working as a passive View, a Presenter working as a middle man and a Model containing all data business logic. We’ll consider that you domains all concepts adopted on the application, so we’ll concentrate only on the tests. Fell free to clone the repository

Testing the database

Let’s consider that we just created our SQLiteOpenHelper class. Now we want to test it to see if it is working correctly before proceeding.

Our test class should be inserted in the test package created earlier.

@RunWith(RobolectricGradleTestRunner.class)
// To use Robolectric you'll need to setup some constants.
// Change it according to your needs.
@Config(constants = BuildConfig.class, sdk = 21, manifest = "/src/main/AndroidManifest.xml")
public class DBSchemaTest {

    private Context context;
    private DBSchema helper;
}

Now we need to setup our tests correctly. We’ll use the @Before annotation to setup. The annotated method will be called every time before a test is executed. If you need to cleanup your code after the test execution, use the @After annotation.

To create our SQLiteOpenHelper we need to inject a Context object. We’ll need a real Context to allow SQLite operations. Robolectric give us an Application Context that lives only during tests executions.

@Before
public void setup(){
    context = RuntimeEnvironment.application;
    helper = new DBSchema(context);
}

First we’ll test if the database is being created correctly.

@Test
public void testDBCreated(){
    DBSchema helper = new DBSchema(context);
    SQLiteDatabase db = helper.getWritableDatabase();
    // Verify is the DB is opening correctly
    assertTrue("DB didn't open", db.isOpen());
    db.close();
}

If you run the test you’ll see that it is ok.

Now let’s verify if all SQLite columns were implemented. But first we’ll modify our setup and add a little cleanup.

private Context context;
private DBSchema helper;
private SQLiteDatabase db;

@Before
public void setup(){
    context = RuntimeEnvironment.application;
    helper = new DBSchema(context);
    db = helper.getReadableDatabase();
}

@After
public void cleanup(){
    db.close();
}

@Test
public void testDBCols() {
    Cursor c = db.query(DBSchema.TABLE_NOTES, null, null, null, null, null, null);
    assertNotNull( c );

    String[] cols = c.getColumnNames();
    assertThat("Column not implemented: " + DBSchema.TB_NOTES.DATE,
            cols, hasItemInArray(DBSchema.TB_NOTES.DATE));
    assertThat("Column not implemented: " + DBSchema.TB_NOTES.ID,
            cols, hasItemInArray(DBSchema.TB_NOTES.ID));
    assertThat("Column not implemented: " + DBSchema.TB_NOTES.NOTE,
            cols, hasItemInArray(DBSchema.TB_NOTES.NOTE));

    c.close();
}

Our lest check on the database will be if it is being deleted correctly.

@Test
public void testDBDelete(){
    assertTrue(context.deleteDatabase(DBSchema.DB_NAME));
}

Testing DAO

To test our DAO insert a new class inside the test package created earlier. Name it as you want. We’ll need to setup our test environment first, paying special attention to the mock objects.

You’ll see that setup the mocks is a little cumbersome. Specially considering the code’s simplicity. Although, when you project is more complex this knowledge will be very handy.

First, let’s setup all Mocks and test only the getNote() method.

@RunWith(RobolectricGradleTestRunner.class)
// To use Robolectric you need to setup some constants.
// Change it according to your needs.
@Config(constants = BuildConfig.class, sdk = 21, manifest = "/src/main/AndroidManifest.xml")
public class DAOTest {

    private DAO dao;
    private DAO daoWMocks;
    private SQLiteDatabase dbM;
    private Cursor cM;
    private Note noteMock;

    @Before
    public void setup() {
        Context context = RuntimeEnvironment.application;
        dao = new DAO(context);

        setupNoteMock();
        daoWMocks = new DAO(context, setupHelperMock() );
    }

    /**
     * Setup a DBSchema mock
     * @return  Returns the mocked obj
     */
    private DBSchema setupHelperMock(){
        // create the mocks
        DBSchema helperM = Mockito.mock(DBSchema.class);
        dbM = Mockito.mock(SQLiteDatabase.class);

        // Define method's results for the mock obj
        when(helperM.getReadableDatabase()).thenReturn(dbM);
        when(helperM.getWritableDatabase()).thenReturn(dbM);
        return helperM;
    }

    /**
     * Setup a mock Note to be used in the tests
     */
    private final String noteText = "MockNote";
    private final String noteDate = "00/00/00";
    private final int noteId = 111;
    private void setupNoteMock() {
        noteMock = mock(Note.class);
        when(noteMock.getText()).thenReturn(noteText);
        when(noteMock.getDate()).thenReturn(noteDate);
        when(noteMock.getId()).thenReturn(noteId);
    }

    /**
     * Setup a mock Cursor
     * that returns a given Note data
     */
    private void setupMockCursor(Note noteToRet){
        // create the mock cursor
        cM = Mockito.mock(Cursor.class);
        // define method's return
        when(cM.moveToFirst()).thenReturn(true);
        int idId = 0;
        int idNote = 1;
        int idDate = 2;
        // define method's return
        when(cM.getColumnIndexOrThrow(DBSchema.TB_NOTES.ID)).thenReturn(idId);
        when(cM.getColumnIndexOrThrow(DBSchema.TB_NOTES.NOTE)).thenReturn(idNote);
        when(cM.getColumnIndexOrThrow(DBSchema.TB_NOTES.DATE)).thenReturn(idDate);

        // define method's return
        String nText = noteToRet.getText();
        String date = noteToRet.getDate();
        int id = noteToRet.getId();
        when(cM.getInt(idId)).thenReturn(id);
        when(cM.getString(idNote)).thenReturn(nText);
        when(cM.getString(idDate)).thenReturn(date);
    }

    /**
     * Setup a mockDB query search.
     * @param noteMockId    Note id to search
     * @param cursor        Query result
     */
    private void setupQueryMock(int noteMockId, Cursor cursor) {
        String[] selArgs = new String[] { Integer.toString(noteMockId)};
        // Defining DB query return
        when(dbM.query(
                anyString(),
                (String[]) isNull(),
                anyString(),
                aryEq(selArgs),
                anyString(),
                anyString(),
                anyString()
        )).thenReturn(cursor);
    }

    /**
     * Testing DAO with a db mock
     */
    @Test
    public void getNoteTestMock(){

        setupMockCursor(noteMock);
        setupQueryMock(noteMock.getId(), cM);

        Note note = daoWMocks.getNote(noteMock.getId());
        // Verify if specified method was called
        verify(cM).moveToFirst();
        assertNotNull(note);
        assertEquals(note.getId(), noteMock.getId());
        assertEquals(note.getText(), noteMock.getText());
        assertEquals(note.getDate(), noteMock.getDate());
        // Verify if specified method was called
        verify(cM).close();
        verify(dbM).close();
    }

    /**
     * Testing DAO with a db mock
     */
    @Test
    public void getNoteTestMockFail(){
        setupQueryMock(noteMock.getId(), null);
        Note note = daoWMocks.getNote(noteMock.getId());
        // result should be null
        assertNull(note);
        // should close db
        verify(dbM).close();
    }
}

If you run the test you will see that all works correctly. Let’s try something different now. Our next test will rely on a real DBSchema class and it will be possible to deal with real DB data.

private Note noteReal;
private void setupNoteReal() {
    noteReal = new Note();
    noteReal.setText("RealNote");
    noteReal.setDate("00/00/00");
}

/**
 * Testing DAO with a REAL db
 */
@Test
public void insertNoteTest() {
    setupNoteReal();
    Note noteInserted = dao.insertNote(noteReal);
    assertNotNull(noteInserted);
    assertEquals(noteReal.getText(), noteInserted.getText());
}

/**
 * Testing DAO with a REAL db
 */
@Test
public void getNoteTest() {
    setupNoteReal();
    Note noteInserted = dao.insertNote(noteReal);

    Note note = dao.getNote(noteInserted.getId());
    assertNotNull(note);
    assertEquals(note.getText(), noteReal.getText());
}

/**
 * Testing DAO with a REAL db
 */
@Test
public void noteListTest() {
    ArrayList<String> noteTexts = new ArrayList<>();
    noteTexts.add( "note1" );
    noteTexts.add( "note2" );
    noteTexts.add( "note3" );

    for( int i=0; i<noteTexts.size(); i++){
        Note note = new Note(noteTexts.get(i), "00/00/00");
        dao.insertNote(note);
    }

    ArrayList<Note> notes = dao.getAllNotes();
    assertNotNull( notes );
    assertEquals(notes.size(), noteTexts.size());
}

/**
 * Testing DAO with a REAL db
 */
@Test
public void deleteNoteTest() {
    setupNoteReal();
    Note noteInserted = dao.insertNote(noteReal);

    long delResult = dao.deleteNote( noteInserted );
    assertEquals(1, delResult);
}

Testing Activity with Robolectric

Our Mainactivity.class is really simple. Since we’re using MVP architecture the Activity is as passive as it can be and don’t have much code to test. Although, I believe some of you would appreciate to know that it is possible to test an Activity with great control using Robolectric. The following code is a demonstration of the framework’s power.

You should pay attention to the ActivityController. This class is responsible to control the Activity’s lifecycle, giving you the opportunity to easily test different scenarios.

@RunWith(RobolectricGradleTestRunner.class)
@Config(constants = BuildConfig.class, sdk = 21, manifest = "/src/main/AndroidManifest.xml")
public class MainActivityTest {
    private MainActivity activity;
    private ActivityController<MainActivity> controller;
    private MVP_Main.ProvidedPresenterOps presenterOps;
    private StateMaintainer stateMaintainer;


    @Before
    public void before(){
        controller = Robolectric.buildActivity(MainActivity.class);
        presenterOps = Mockito.mock(MVP_Main.ProvidedPresenterOps.class);
        stateMaintainer = Mockito.mock(StateMaintainer.class);
    }

    // testing Activity onCreate
    @Test
    public void testOnCreate(){
        activity = controller.get();
        
        when(stateMaintainer.firstTimeIn())
                .thenReturn(true);
        activity.mStateMaintainer = stateMaintainer;

        controller.create();

        Mockito.verify(stateMaintainer, VerificationModeFactory.atLeast(2))
                .put(Mockito.any(MainPresenter.class));
        Mockito.verify(stateMaintainer, VerificationModeFactory.atLeast(2))
                .put(Mockito.any(MainModel.class));

        assertNotNull(activity.mPresenter);

    }

    // testing Activity onCreate during a reconfiguration
    @Test
    public void testOnCreateReconfig(){
        controller = Robolectric.buildActivity(MainActivity.class);
        activity = controller.get();

        when(stateMaintainer.firstTimeIn())
                .thenReturn(false);
        when(stateMaintainer.get(MainPresenter.class.getName()))
                .thenReturn(presenterOps);
        activity.mStateMaintainer = stateMaintainer;

        Bundle savedInstanceState = new Bundle();
        controller
                .create(savedInstanceState)
                .start()
                .restoreInstanceState(savedInstanceState)
                .postCreate(savedInstanceState)
                .resume()
                .visible();

        verify(stateMaintainer).get(MainPresenter.class.getName());
        verify(presenterOps).setView(any(MVP_Main.RequiredViewOps.class));

    }

    // testing Activity onDestroy
    @Test
    public void testOnDestroy(){
        controller.create();
        activity = controller.get();
        activity.mPresenter = presenterOps;

        controller.destroy();
        Mockito.verify(presenterOps).onDestroy(Mockito.anyBoolean());

    }
}

Conclusion

We discussed a lot about Unit testing and the tools available on Android, although the reality is that we’ve just scratched the surface. There are lots of useful tools out there and even the ones that we talked about have many resources that we chose to ignore. Although, you have a good starting point.

On the next part we’ll talk about Integration testing with focus on UI. We’ll use Espresso, AndroidJunit and many tolls already discussed here. I hope to see you soon.

References


Also published on Medium.

Leave a Reply

Your email address will not be published. Required fields are marked *