At Braze, we do our due diligence to make sure that everything within our control is working in top shape for our customers. One critical element that we’re always testing? Our Braze Android Software Development Kit (SDK). Our SDKs are a crucial part of the Braze platform; and we aim for bug-free code through very high code coverage. Bug-free code within our Android SDK is hugely important to us, not only because we ship our SDK to hundreds of millions of devices, but also because a clean code means a more seamless experience on our customers’ end. When we do find a bug in the SDK code, that means we have to ship out an update, then our customers need to upgrade, put out a new version of their app, and have all of their end users then need to download that update. Ultimately, that means that unwanted bugs can be in the wild for a very long time.

Aiming for bug-free means lots of tests and even more time to respond to failing tests. Below is a snippet of the test report from our base SDK module.

When writing a new feature, it's important to know exactly how and why a test failed as quickly as possible. If your tests regularly write to the Logcat (Android's system message tool), you can debug by finding where in your logs the failure occurred. However, in tests Logcat can be very verbose.

At Braze, we use JUnit's TestRule to automate the process of retrieving relevant logcat output from when a test fails.

A Simple Test

The following is a simple util method used in our code, with a log message on the first line:

Now, if we modify the method’s corresponding test to fail, like this:

Then, predictably, we get the following output:

However, let's say that we wanted to record our original logcat output during the test along with the original test failure information. With some special logic, it would ideally look like the following:

Much cleaner! And we got the important log message included. This improved test failure message will be visible in Jenkins, making debugging easier after test failures.

Getting the Logcat

To read the logcat, we first obtain an instance of the Runtime and run some logcat commands. There are, however, a few things to consider:

1. Since the logcat has unfiltered output from any other apps on the device, we want to only obtain logs matching our test's process ID.

2. We only want logs relevant to our test. They conveniently all start similarly and include the name of the test.

The method works by aggregating any lines that match our process ID and occur after the “TestRunner: start” line. We’ll also only run this method when needed, immediately after a test fails. To accomplish this, we’ll need to utilize TestRules.

Using JUnit

JUnit's TestRule is a very powerful tool that allows for fine control over test execution. I'll be skipping over basic JUnit step setup in this post, but you can check out this link for more information.

To standardize our test classes, we have a base test class which houses our JUnit specific logic, like TestRules:

Writing a TestRule

In JUnit4, tests are handled by the test runner as statements that get evaluated. You can think of statement evaluation as just running the test. A trivial TestRule example simply evaluates the input statement and returns it.

After a TestRule evaluates a statement, it passes that statement to any other TestRules present, forming a chain. You can read the RuleChain javadoc if you’re interested in how this works under the hood!

Behind-the-Scenes Look at TestRule

When a test fails in JUnit, it's actually just throwing an AssertionError that gets caught by the test runner. Below is the source for JUnit Assert:

If we want to repackage an authentic test failure with a custom message, we’ll have to raise an exception somewhere else.

Custom TestRule

So now that our logcat capture method is ready, the next step is to create a custom TestRule that will invoke it when our test fails.

The Final Product

This is how the test runner would display our modified test from earlier. Note that the message at the top contains our custom information like the class of the original error and the logcat output:

Happy testing!

Interested in working at Braze? Check out our current job openings!