• Not Answered

A/B page testing in Sitecore 8.2 – why not showing previous versions on site publish?

1. Submit two versions – version 1 with Header A; version 2 with Header B;

2. Create page test with two versions – this displays as it should when reviewing (version 1 has Header A; version 2 has Header B); 

3. Review and start test;

4. Publish site as per Sitecore's recommendation:

https://doc.sitecore.net/sitecore_experience_platform/analyzing_and_reporting/experience_optimization__content_testing/setting_up_tests/preview_and_start_a_content_test

1. To start a test, in the Preview and start test dialog box, click Start test.
2. To publish the test to your website, you must perform a site publish using the Publish wizard. This makes the test go live. Open the Publish wizard and select Smart publish.

5. When I check the A/B test, version 1 now shows Header B; version 2 shows Header B.

Why has it – on site publish – overridden version 1 to show Header B although version 1 was set to show Header A?

7 Replies

  • Brad - First thing I'd check is to see if any caching is set that could cache the header. I know my CM environment disables caching so that could explain why it worked in the CM environment but not in the CD.

    Also, are you sure you are seeing a different version? In other words, is there content that is changing elsewhere on the page?
  • In reply to Scott Freeman:

    Hi Scott,

    Thanks for replying so quickly – it's greatly appreciated.

    Yes, I am seeing a different version when previewing before the test. When I preview version 1, the heading content reads 'Header A'. When I preview version 2, the content heading reads 'Header B'. This is acting as it should.

    When I click 'publish site' then go back to preview versions 1 and 2, both versions 1 and 2 have 'Header B'.

    Please note that I'm sticking to a page test even though this basic content testing. Why? In basic content testing, when I preview and test, both versions 1 and 2 show 'Header B', even though version 1 has been set up with 'Header A'.

    Make sense? More than happy to explain more if need be.

    In regards to caching? I'm assuming these are what you're referring to?

    https://sitecorebasics.wordpress.com/2011/10/30/basics-of-html-caching/

    http://sitecoreblog.alexshyba.com/how_to_verify_html_cache_is_working/

  • In reply to Brad Davison:

    Correct - look at the rendering parameters of the header. I would bet that it is cached. See explanation below:

    CM environments typically disable caching as it can cause issues with editing content and not seeing the changes. This would explain why the preview's work as expected.

    CD environments should always cache for performance

    In your scenario, you are hitting a url: loveabtesting.com/except-when-it-doesnt-work
    The page loads Header B - and the cache is stored as Header B. All future calls to this page will show Header B due to the cache setting

    Another possibility is that you are using sticky testing strategy. This will cause a visitor to always see the same version of the page. In my development environment, I set the default testing strategy to random.

    Since we are talking header of a website, I imagine that caching is very likely as loading the items for the navigation is a performance hit so most add caching.

  • In reply to Scott Freeman:

    Okay, that sounds about right. I'm going to do some further testing tomorrow. If I get stuck – and thank you Scott for reaching out – then I'll post again on this channel. If I get stuck – given that I'm relatively new to Sitecore – then I'll provide more feedback on this forum.
  • In reply to Scott Freeman:

    Hi Scott, I'm still having no luck unfortunately. I've tested the CD, CM and preview environments – same result. Any other ideas?
  • In reply to Brad Davison:

    Hi Brad, I'm having a similar issue when using page tests that the variation never triggers and it always shows the most recent version. Components tests on the other hand work just fine. Did you find a solution in the end?
  • In reply to Erik ter Beeke:

    Same issue here... Works great on CM, but only shows latest version on CD, where it matters. Has anyone figured out any ideas to resolve this.  Thanks