Logcat and JUnit: An Unstoppable Combination for Android Tests

Published on February 08, 2019/Last edited on February 08, 2019/4 min read

A stack of cards featuring an Android robot icon and a list with options labeled A, B, C, and D. The background has a gradient from orange to pink with circular wave patterns.

AUTHOR

Julian Contreras

Android Engineer at Braze

At Braze, we do our due diligence to make sure that everything within our control is working in top shape for our customers. One critical element that we’re always testing? Our Braze Android Software Development Kit (SDK). Our SDKs are a crucial part of the Braze platform; and we aim for bug-free code through very high code coverage. Bug-free code within our Android SDK is hugely important to us, not only because we ship our SDK to hundreds of millions of devices, but also because a clean code means a more seamless experience on our customers’ end. When we do find a bug in the SDK code, that means we have to ship out an update, then our customers need to upgrade, put out a new version of their app, and have all of their end users then need to download that update. Ultimately, that means that unwanted bugs can be in the wild for a very long time.

Aiming for bug-free means lots of tests and even more time to respond to failing tests. Below is a snippet of the test report from our base SDK module.

Test summary showing 902 tests, 0 failures, a duration of 22.999 seconds, and a 100% success rate. Details for the package "com.appboy" include 95 tests, 0 failures, a duration of 4.223 seconds, and a success rate of 100%.

When writing a new feature, it's important to know exactly how and why a test failed as quickly as possible. If your tests regularly write to the Logcat (Android's system message tool), you can debug by finding where in your logs the failure occurred. However, in tests Logcat can be very verbose.

A screenshot of a terminal displaying log messages related to application activities, including notifications and registration processes. The text includes various status updates and error messages.

At Braze, we use JUnit's TestRule to automate the process of retrieving relevant logcat output from when a test fails.

A Simple Test

The following is a simple util method used in our code, with a log message on the first line:

A code snippet in Java that defines a method to check if a string is null or empty. The method logs a message and returns a boolean value.

Now, if we modify the method’s corresponding test to fail, like this:

Code snippet showing a JUnit test method that checks if a string is null or empty, with assertions for both true and false conditions.

Then, predictably, we get the following output:

Error log from a Java application, showing stack trace and method calls related to a failed assertion in a test case.

However, let's say that we wanted to record our original logcat output during the test along with the original test failure information. With some special logic, it would ideally look like the following:

A screenshot of a Java error message indicating an assertion failure in a testing framework, with details about the error and a stack trace. The message includes timestamps and method calls related to the failure.

Much cleaner! And we got the important log message included. This improved test failure message will be visible in Jenkins, making debugging easier after test failures.

Getting the Logcat

To read the logcat, we first obtain an instance of the Runtime and run some logcat commands. There are, however, a few things to consider:

1. Since the logcat has unfiltered output from any other apps on the device, we want to only obtain logs matching our test's process ID.

2. We only want logs relevant to our test. They conveniently all start similarly and include the name of the test.

Code snippet showing a Java method for retrieving relevant logs after a test starts, including error handling and process execution. The code features comments explaining its functionality.

The method works by aggregating any lines that match our process ID and occur after the “TestRunner: start” line. We’ll also only run this method when needed, immediately after a test fails. To accomplish this, we’ll need to utilize TestRules.

Using JUnit

JUnit's TestRule is a very powerful tool that allows for fine control over test execution. I'll be skipping over basic JUnit step setup in this post, but you can check out this link for more information.

To standardize our test classes, we have a base test class which houses our JUnit specific logic, like TestRules:

Code snippet showing a Java class definition for a test base in Android, including an annotation and a rule for capturing logs on test failures.

Writing a TestRule

In JUnit4, tests are handled by the test runner as statements that get evaluated. You can think of statement evaluation as just running the test. A trivial TestRule example simply evaluates the input statement and returns it.

Code snippet showing a Java method that overrides a base class method to return a new Statement object. The evaluate method calls the base class's evaluate method.

After a TestRule evaluates a statement, it passes that statement to any other TestRules present, forming a chain. You can read the RuleChain javadoc if you’re interested in how this works under the hood!

Behind-the-Scenes Look at TestRule

When a test fails in JUnit, it's actually just throwing an AssertionError that gets caught by the test runner. Below is the source for JUnit Assert:

Code snippet in Java that defines a method called "fail" which throws an AssertionError if the input message is null or contains a specific message.

If we want to repackage an authentic test failure with a custom message, we’ll have to raise an exception somewhere else.

Custom TestRule

So now that our logcat capture method is ready, the next step is to create a custom TestRule that will invoke it when our test fails.

Code snippet showing a Java method that handles exceptions and logs error messages, including stack trace information. The method overrides a statement application and evaluates throwables.

The Final Product

This is how the test runner would display our modified test from earlier. Note that the message at the top contains our custom information like the class of the original error and the logcat output: