case studies

Berlin-based Kolibri Games achieved success with mobile titles such as Idle Miner Tycoon, Idle Factory Tycoon, Idle Restaurant Tycoon and others, but wanted to roll out new features that could increase revenue without upsetting the apple cart. With Braze and Looker, they built a segment-driven testing approach that demonstrated the value of testing.

If you’ve spent any time exploring the world of mobile gaming, you’ve likely run across the “incremental” genre. These titles, also sometimes referred to as “idle games,” require minimal interaction from the player (often just repeatedly clicking on a given screen) but leverage a stepwise reward system that drives continued gameplay and inspires intense involvement that can lead to lost sleep and that “just five more minutes” feeling—all very good things from a game designer’s POV.

Since 2016, Berlin-based publisher Kolibri Games has been crushing it in this sector of the mobile gaming space. Started in a student housing apartment without external financing, Kolibri struck a nerve with its first title, Idle Miner Tycoon, which has been downloaded over 100 million times. Their follow-up, Idle Factory Tycoon, crossed the 5 million-download threshold even faster than its predecessor, demonstrating once again that Kolibri knew how to excel with these sorts of incremental business simulations.

They also knew, however, that the success in their chosen genre can’t be sustained if you rest on your laurels. Rolling out new features post-release to drive continued revenue is a key part of Kolibri’s business model, but it can also be a risky proposition, and the possibility that a new feature might unbalance the game or upset customers is very real. For Kolibri, this means that understanding how their audience might react to new offers or gameplay elements is an imperative part of their post-release approach.

By leveraging Braze and Braze Alloys technology partner Looker, a leading data analytics platform, Kolibri found the key that would help them conquer the uncertainty of feature rollout through segmentation and testing.

Breaking Down the Audience with Braze Canvas

The first step in the process was to segment the Kolibri Games audience. Segmentation creates groups based on user behavior, and is an important step for any brand that understands a one-size-fits-all approach isn’t going to fit the bill. When it came time to trial a new feature, Kolibri created segments through the Braze Canvas customer journey management feature, divvying up their users into test and control groups, with only the former slated to receive a particular update as part of pre-release testing.

Next, Kolibri made use of silent push notifications to inform those in the testing group of the new feature. Rather than utilizing push as a means of messaging, Kolibri got creative and used it to transfer data to a client and inform them of their status as part of a certain segment. These users can then interact with the element being tested (or choose not to do so, if that’s their preference), thereby generating further behavioral data to inform Kolibri’s design decisions.

Looker Locks in Data Analysis

One the testing process gets underway and the data starts rolling in, it can be challenging at times to tell when a new feature is being positively received and having a concrete impact on revenue. That’s where Looker comes in. Looker serves as a conduit between the pools of data collected through Braze and the analysis of that data that can inform tactics and strategy.

In Kolibri’s case, Looker helped to create a clear picture of how users inside and outside of the company’s test segments interacted with Tycoon products. Which features draw a positive user response? What does their implementation do to user retention rates and session length? How do these changes compare to the behavior of control groups who haven’t been exposed to the new feature? By diving into the numbers that answer these questions via Looker, Kolibri was able to accurately assess the likely success of various features before rolling them out to the audience at-large.

Kolibri Clocks a High Score in New Revenue

This all sounds great in theory, but how did this data-driven testing approach work in practice? In one instance, the company sent out a new offer to 300,000 members of a test group. Their response allowed Kolibri to project with 97% confidence that the test group generated 15% more revenue than the control group. Best of all, with the guesswork taken out of the process, Kolibri is able to consistently work toward improving its successful titles without widely implementing changes that could potentially hurt the user experience or their business as a whole.

“By taking advantage of Braze and Looker together, we’ve been able to be much more intentional about assessing the impact of every new feature in one of our games,” said Daniel Reichert, Product Director at Kolibri. “The result has been extraordinary—allowing us to effectively mitigate risk while maximizing profit at the same time.”

Final Thoughts

Crafting a great game is all about finding the features that click with an audience. Thanks to Braze and Looker, Kolibri has that process down to a science, and has been able to generate quantifiable results from their comprehensive, data-driven testing strategy.

For more on how to analyze, interpret, and strategize around customer engagement, check out the Braze Humanity in Action eBook.

Adam Swiderski

Adam Swiderski is a writer, editor, musician, surfer, and recovering professional nerd from Queens, New York. He's interested in the meeting of technology and humanity, and disinterested in zombie apocalypse.