Testing for optimization

Welcome to the sixth in our new series of blog posts that deliver practical advice for getting the most from the Sitecore® Experience Platform™ (XP). I’ll be sharing advice and best practices for testing, optimizing, and personalizing your marketing campaigns so you can deliver contextual customer experiences.

In this session, I’ll show you how testing with the machine learning capabilities of Sitecore XP informs personalization, and how this benefits your site optimization. Watch the video on this topic by clicking over to our testing and optimization toolkit page.

Tests are getting smarter

I’ve talked a lot about optimization in the last few of these posts. But I’m really excited about this one. Testing has evolved a lot since the early days of primitive reports and hours of manual wrangling. Now you can run tests with machine learning in the background and gain much more insight about where to optimize further.



Let’s say you’re running a test on a component with different variants. You want to figure out which variant delivers the most business value, based on the trailing value per visit.

Sitecore XP uses machine learning to evaluate test results and detect new audience segments to whom you can deliver personalized experiences.

The graph on the above left represents all the visitors—a jumble of dots. Add machine learning, and you get the graph on the right. Why is that exciting? Because it uncovers and tells you which newly detected audience segments may prefer (based on higher trailing value per visit) other variants than the test “winner,” essentially finding new audiences for whom you can personalize experiences. (I explain more about this below.)

In this example, the test returned three new valuable audience segments that you can start personalizing for immediately.

Using machine learning for personalization

Knowing the winning variant is one thing. But what about the other variants? What if you had visitors who took completely different journeys through your site, wanted to buy stuff, but couldn’t because the components weren’t relevant to them?

As long as you’ve told Sitecore XP which engagements matter to you, it’ll know to look out for rogue variants with hidden value for your business.

In the example above, Sitecore’s machine learning algorithms suggest you add two extra segments: One for visitors from Germany, and one for males. It’s done that because it has seen that both segments tend to carry a high Engagement Value, even though your winning variant had a deceptively mixed audience. In other words, it’s spotted something valuable that you would have missed.

With bespoke results and recommendations like these, you know exactly what to personalize. More important, you know why and for whom.

Knowing when to test and when to personalize

Sometimes it makes sense to personalize a component before you’ve tested it. It’s the quickest action when you want to act on known audience needs or intents. 

So while a test with machine learning will help you figure out which component best suits an audience segment’s needs, you might not always need it. If you have a strong understanding of what your male German audience needs, personalize around that immediately. You can always confirm, or further refine, through later testing.

Your business, your results

With machine learning, you learn more about your audience with every test. You also get to know your most valuable audience members—the ones worth personalizing for. Optimize your site for them, and you’ll never look back.

Thanks for reading. For more on how Sitecore uses machine learning in its testing functionality, click over to our testing and optimization toolkit page.


Lars Birkholm Petersen is Sitecore’s VP of Business Optimization. Follow him on Twitter @LarsBirkholm