• Not Answered

How these Test Strategies function in A/B Testing

In Sitecore 8 there are couple of Test Strategy and I wanted to understand these and I am unable to find any documentation..

Test Strategies in Sitecore 8

  1. Subgroup Round Robin
  2. Sticky
  3. Round Robin
  4. Random

Of these Subgroup Round Robin is default in Sitecore 8.

Can anyone give some brief or link/document which has details about these ?

3 Replies

  • I was just going to ask the same question. Especially because "Subgroup Round Robin Sticky" is the default for me which sounds quite ambiguous.

    Also side question. When creating a test we go through 3 tabs full of configuration options. Yet Test Strategy is not there. So after creating the test, we seem to need to find our way to the Test Lab for just this one config option. Any idea if it's being deprecated or if its support is just lagging behind other features?
  • In reply to Daniel DeLay:

    Have you guys got some clarity on these terms?
  • In reply to Mayank Thapak:

    Hi all,
    I did get some clarity from Sitecore on this. Pasted below.

    1) The marketer users are not expected to change the test strategy, so it is not available in the UI. The most appropriate strategy that will provide the required exposure for the test was selected and set as default.

    2) The strategy can be selected if the test was created manually (by creating a new test under the "/sitecore/system/Marketing Control Panel/Test Lab" item)

    3) Default content strategy can be set in the following setting from the Include\ContentTesting\Sitecore.ContentTesting.config file:
    <!-- CONTENT TESTING - TEST STRATEGY
    The default content test strategy.
    Default value: /sitecore/system/Settings/Content Testing/Test Strategies/Subgroup Round Robin Sticky
    -->
    <setting name="ContentTesting.TestStrategy" value="/sitecore/system/Settings/Content Testing/Test Strategies/Subgroup Round Robin Sticky" />


    4) Regarding the "Round Robin Sticky" and "Subgroup Round Robin Sticky" strategies:

    Both strategies use round robin (choosing all elements in a group equally in some rational order) approach.
    The difference between the strategies is that the subgroup round robin strategy evaluates any personalization rules being tested to ensure we do not assign an experience to the user that includes a rule which they don’t match, so they wouldn’t see the intended experience.

    For example, the "Round Robin Sticky" works like this for two versions of the item with one personalization rule applied to the page:

    content version 1, rule applied
    content version 1, default rule (no rule applied)
    content version 2, rule applied
    content version 2, default rule (no rule applied)

    However, the "content version 1, rule applied" and "content version 1, default rule (no rule applied)" will look the same if the visitor does not match personalization rule condition.
    Therefore, the subgroup round robin strategy will not display variant with a rule applied to the subgroup of visitors who does not meet the rule condition.