Reinforcement learning: How to create an AI decisioning system that learns

Marketing campaigns are made up of dozens of unique variables, from the copy, imagery, and type of offer to the channel and send day and time. And any marketer worth their salt wants to be able to measure the impact of each of these factors on customer engagement outcomes so that they can further iterate and improve their overall performance. But to do that, teams have historically had to rely on complicated, time-consuming methods like A/B and multivariate testing.

Now, however, there’s a new, more effective option. By leveraging AI decisioning platforms like BrazeAI Decisioning Studio™ that learn via a type of machine learning called reinforcement learning, brands can accomplish the same goal much more efficiently.

The pitfalls of traditional A/B testing and multivariate testing

Brands typically face two challenges with these types of experiments. First, this kind of testing only helps us learn about the effects of each variant independently; the variables can’t be grouped in any way. As a result, it’s hard to learn about the sort of general trends that drive performance and develop new marketing content that capitalizes on winning strategies.

Second, the more variants you want to test, the greater your sample size needs to be. Let’s say you have 100 variants you want to compare. To do that, you’d have to subdivide your audience into 100 segments, leaving you with a very low sample size and making it difficult to generate meaningful, actionable insights.

Third, results from one A/B test may not apply to your next marketing campaign. Let’s say you run a campaign in the summer and get a certain result. It’s entirely possible that your customer base acts completely different in the winter, making it difficult to generalize from those earlier findings.

How AI decisioning helps brands overcome the limitations of traditional A/B and multivariate testing

Using the sort of grouping logic common in decision trees, AI decisioning can run through various scenarios and compare their outcomes, drawing general conclusions and allowing multiple tests to be run simultaneously behind the scenes. Instead of having to run through each test seqentially to eliminate poor-performing campaign variants one by one, brands can quickly identify and group together less impactful variants versus stronger combinations of variables. For instance, AI decisioning can compare different campaign images, test the impact of adding or excluding emojis, and see what difference using a dollar off or a percent discount has on the campaign’s performance.

In AI decisioning systems, efficiently parameterizing your options allows the model to learn quickly from the different options. Parameterization is somewhere between an art and a science and the best way to do it is often to leverage a team of experts that have experience with effective parameterization strategies for your industry or use case.

Best practices for AI decisioning

Whether you’re considering AI decisioning or getting started with it, here are some ways to see the most value from this type of marketing.

Get started by testing out the backlog of hypotheses you’ve always wanted to explore

There’s no need to come up with entirely new strategies when implementing AI decisioning. Simply start with some ideas you’ve been curious about, like personalizing send times or channels for each individual customer. Any test you’ve been wanting to run at the 1:1 level can work.

Don’t stop after your first experiment; keep drilling down

The more that teams are able to experiment with new and different approaches—to see what works and dig deeper iteratively—the more efficient your system will become.

For instance, you could begin by exploring whether customers respond better to percent-off discounts versus dollar-off discounts, then throw out some of the dollar-off tests that don’t end up moving the needle and then deepen the analysis by comparing the impact of different discount offers.

Don’t go overboard trying to test everything

Not all marketing variables are created equally. From our experience, the most influential marketing KPIs include:

• The hero image of the campaign
• The subject line (in emails)
• The call to action
• The type of offer
• The frequency
• The send day and time

Other variables, such as the background color of an image or the placement of a disclaimer, aren’t likely to make that much of a difference.

Don’t waste time trying to engineer the perfect combination of factors to test

As I mentioned before, as long as you start with questions you’re curious about and make sure you test out the key factors we’ve found make a difference (listed above), you’ll be setting yourself up for success.

The great thing about reinforcement learning is that even if a particular option isn’t that effective, you won’t end up wasting that much time, money, or resources testing it out. The AI decisioning system will learn fairly quickly whether it’s having an impact and, if it’s not, it will stop showing that option to customers.

Recognize that there aren’t all-or-nothing winners or losers; you’re always learning and optimizing

With A/B testing, you set a specific goal, such as to determine whether blue call-to-action buttons outperform other colors. Once you find out the answer, you may declare victory and be done with the experiment.

But that’s not the case with AI decisioning. AI decisioning is an always-on practice.

As you acquire more customers and they advance through their customer journeys, AI decisioning is continuously learning what’s working—and what’s not—for each individual. This type of testing, learning, and iterating also adapts as circumstances change, whether that’s due to seasonality or shifts in customer behavior driven by other factors, such as what’s happening with the economy.

Conduct a quarterly regular review of the variables you’re testing

Do some housekeeping and take a look at the top 80% of factors that are creating value and consider eliminating the bottom 20% of options that are not creating value and replacing those with new, clever ideas.

The reality is, even if you don’t have any new ideas to test out, the model doesn’t need to send traffic to underperforming campaign variants to keep testing and exploring them further.

You’ll get the most value when you work with a brand that offers AI decisioning services

Self-service, off-the-shelf AI decisioning systems won’t help you customize the model to work in ways that best support your business. As part of the BrazeAI Decisioning Studioᵀᴹ, we offer forward-deployed data scientists and technical support that tailor our AI decisioning system so that it recognizes the metrics and data that matter for your brand and acts in ways that create the most value for your organization.

For instance, when one streaming services brand was using the BrazeAI Decisioning Studioᵀᴹ to test various offers off—such as $10 off for two months, $15 off for three months, and so forth—we found that customers were responding to the dollar amount more than they were to the total value. So even though $15 off for three months is worth $45 in savings, customers were more likely to take action when receiving $20 off for one month. Based on that insight, we helped the brand improve their marketing performance and save the overall business more money.

See how we help major brands across entertainment, travel and hospitality, financial services, and more make smarter marketing decisions, faster with the BrazeAI Decisioning Studioᵀᴹ.

Forward-Looking Statements

This blog post contains “forward-looking statements” within the meaning of the “safe harbor” provisions of the Private Securities Litigation Reform Act of 1995, including but not limited to, statements regarding the performance of and expected benefits from Braze and its products and features, including without limitation BrazeAI Decisioning Studio™. These forward-looking statements are based on the current assumptions, expectations and beliefs of Braze, and are subject to substantial risks, uncertainties and changes in circumstances that may cause actual results, performance or achievements to be materially different from any future results, performance or achievements expressed or implied by the forward-looking statements. Further information on potential factors that could affect Braze results are included in the Braze Quarterly Report on Form 10-Q for the fiscal quarter ended October 31, 2025, filed with the U.S. Securities and Exchange Commission on December 10, 2025, and the other public filings of Braze with the U.S. Securities and Exchange Commission. The forward-looking statements included in this blog post represent the views of Braze only as of the date of this blog post, and Braze assumes no obligation, and does not intend to update these forward-looking statements, except as required by law.