Experiment Step

Experiment Steps allow you to test multiple Canvas paths against each other and a control group at any point in the user journey. These steps will enable you to track path performance to make informed decisions about your Canvas journey.

A Canvas Experiment Step will randomly assign users to different paths (or an optional control group) you create. Portions of the audience will be assigned to different paths according to percentages you select, allowing you to test different messages or paths against each other and determine which is most effective. After launching, analytics will allow you to track performance and see whether results differ across the different paths to help you determine which path should get what proportion of users (or all of them!).

Experiment Step

Create an Experiment Step

To create an Experiment Step, add a step to your Canvas. Then, using the dropdown at the top of the new step, select Experiment.

In the default Experiment Step shown below, there are two default paths, Path 1 and Path 2, with 50% of the audience being sent down each path. Click on the Experiment Step itself to expand the Experiment Settings panel, and you’ll see the configuration options for the step.

Step 1: Choose The Number of Paths and Audience Distribution

Experiment Settings

You can add up to 4 paths by clicking Add Path and an optional control group by checking Add a Control Group. Using the percentage boxes above each path, you can specify the percentage of the audience that should go to each path and the control group. The provided percentages must add up to 100% to proceed. If you want to quickly set all the available paths (and control) to the same percentage, click Distribute Paths Evenly.

Optionally, you can add a description to explain to others what this Experiment Step intends to test or include other information that might be helpful to note.

Step 2: Select How Long To Track Conversions

The Experiment Step will record users who enter each step and convert while in the assigned path. This step will track only the Primary Conversion event of the Canvas. Use the input box at the bottom of the panel to enter how many days (between 1 and 30) you’d like this Experiment Step to track conversions.

Step 3: Create Paths

Lastly, you must build your downstream paths. Select Done and return to the Canvas builder. Click Plus under each path to begin creating journeys using the usual Canvas tools as you see fit, and launch the Canvas when you are ready.

Keep in mind that once created, paths and their downstream steps cannot be removed from a Canvas. However, once launched, you can modify the audience distribution across paths as you see fit. For example, if a day after launching a Canvas, you conclude that one path is superior to the rest based on the analytics, you can set that path to 100% and the others to 0%. Or, depending on your need, you can continue sending users down multiple paths.

Tracking Performance

Each path will display statistics in the Canvas Analytics view, just like any Canvas step. Additionally, clicking on the Experiment Step from Canvas Analytics will open a detailed table identical to Analyze Variants to compare detailed performance and conversion statistics across paths. You can also export the table via CSV and compare percent changes for metrics of interest relative to the path (or control) you select.

New Stuff!