Sitecore 7 and Language Fallback

Earlier this year, I blogged in a series on Language Fallback where I leveraged Alex Shyba's Partial Language Fallback Module.

The various posts discussed all aspects of fallback, theory and things to consider, useful tools, and enhancements. Here are some quick links to those posts:

Post 1 - Introduction to Language Fallback
Post 2 - How to Install and Configure Alex Shyba's Partial Language Fallback Module
Post 3 - Enforcing Language Version Presence, Out-Of-The-Box
Post 4 - Enforcing Language Version Presence, Customizations
Post 5 - Using Fallback With The Dictionary
Post 6 - Using Fallback With The Advanced Database Crawler
Post 7 - Defaulting Language Versions For Fallback
Post 8 - Language Migration Tool
Post 9 - Verndale's Language Fallback Reporting Tool
Post 10 - Additional Miscellaneous Topics for Multilingual and Language Fallback Implementation

Much of my experience with all of the above were with Sitecore versions 6.5 and 6.6, though. With Sitecore 7 being the norm now, and the impending Sitecore 8 on the horizon, I figured it was about time to run some tests and make sure all of this still worked on the latest publicly released version: Sitecore 7.2 Update 2 (revision 140526).

I am happy to announce that with the exception of the Advanced Database Crawler post, everything here is still relevant and works like a charm.

As per the "How to Install and Configure" post, you will need to use the source code of Alex’s fallback module (which is included in the marketplace) to compile the project against the version of the Sitecore Kernel and Client dlls that you will be using. Once you do that, drop your fallback dll into your main project bin and you are all set.

As mentioned above, the glaring exception here is the Advanced Database Crawler situation. As most people reading this will know, the release of Sitecore 7 was the death knell of ADC. RIP ADC…

One of the most important overhauls in Sitecore 7 is the Sitecore.Search enhancements. It is a huge change and there is a ton of information you need to absorb. With the exponential increase in flexibility comes a responsibility to really get to know the options, and I recommend spending quality time reading not only the official Sitecore documentation like the Search and Indexing guide and Search Scaling guide for 7, but also the plethora of blog posts that various members of the community have been churning out.

I'm not going to go into any more detail about Search here, but will include in the github files my SearchHelper that replaces some of the ADC GetItems methods with Linq to Sitecore functionality.

What I will talk about is the replacement for enabling fallback with Lucene indexes. If you read my post about fallback with ADC, you will have seen that I addressed fallback at the time of index. I felt this was the simplest way to deal with it.

To refresh the scenario: Say you have an ‘About Us’ page and all of the text is in the en-US language. You also have an es-US language version, but have not overridden any content on it - it falls back to en-US. When viewing the site in es-US, the ‘About Us’ page displays the en-US content. The user searches for the word 'About' in the global search and gets no results. This is because the search crawler, by default, knows nothing about fallback and to it, the ‘About Us’ item simply has no content, and therefore no content is entered into the index for it.

I had added to the crawler logic in ADC to look for the fallback language version of each item field that was null and to check if that fallback language version had a value. If so, then I added that value to the index for the language falling back instead of null. It worked well.

Now I had to figure out how to replicate the concept with Sitecore 7, without the benefit of having the source code at my fingertips as I had with ADC. The sheer amount of exposed logic and features available with search in Sitecore 7 added to the effort, but I finally found the setting I was looking for in the Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config file. In here is the following:

<!-- DOCUMENT BUILDER

Allows you to override the document builder. The document builder class processes all the fields in the Sitecore items and prepares

the data for storage in the index.

You can override the document builder to modify how the data is prepared, and to apply any additional logic that you may require.

-->

<documentBuilderType>Sitecore.ContentSearch.LuceneProvider.LuceneDocumentBuilder, Sitecore.ContentSearch.LuceneProvider</documentBuilderType>

I reflected this class and after spending time reviewing the various methods, I identified that the AddField(IIndexableDataField field) method was the one that I needed to override. I copied the logic and added in my own steps that are better than the implementation that I originally did with ADC, as I realized I could take advantage of the Partial Language Fallback's method: FallbackLanguageManager.ReadFallbackValue. This method will take care of the recursions automatically (in the case of chained fallback), because the Partial Language Fallback module additionally overrides the StandardValues provider.

So essentially what you need to do for this to work right is to:

Add a class that implements LuceneDocumentBuilder and overrides the AddField method, leveraging the FallbackLanguageManager.ReadFallbackValue method
Update the documentBuilderType attribute in the config (or better yet patch it in your Fallback.config file)
Use Lucene as your index (obviously)

I don't yet have a solution for SOLR but I'm currently working a project that uses SOLR so I'm sure I'll have that over the next few months.

I will paste the source code here for the AddField method, but I have also added it to the git repository (linked below) and you can download not only the fallback config with the patch and CustomLuceneDocumentBuilder.cs file, but also the example of the SearchHelper method that I have so far (still a lot more that I want to add to that helper class though!)


using Sitecore.ContentSearch.LuceneProvider;
using Sitecore.Data.Items;
using Sitecore.Data.Fields;
using Sitecore.SharedSource.PartialLanguageFallback.Extensions;
using Sitecore.SharedSource.PartialLanguageFallback.Managers;
public class CustomLuceneDocumentBuilder : LuceneDocumentBuilder
    {
        private readonly LuceneSearchFieldConfiguration defaultTextField = new LuceneSearchFieldConfiguration("NO", "TOKENIZED", "NO", 1f);
 
        public CustomLuceneDocumentBuilder(IIndexable indexable, IProviderUpdateContext context)
          : base(indexable, context)
        {
        }
 
        public override void AddField(IIndexableDataField field)
        {
            object fieldValue = this.Index.Configuration.FieldReaders.GetFieldValue(field);
 
            //UPDATED, Added By Verndale for Fallback
            //<!--ADDED FOR FALLBACK DEMO-->
            if (fieldValue == null || fieldValue == "")
            {
                // Get the Sitecore field for the Indexable Data Field (which is more generic) that was passed in
                // If the field is valid for fallback, then use the ReadFallbackValue method to try and get a value
                Sitecore.Data.Fields.Field thisField = (Sitecore.Data.Fields.Field)(field as SitecoreItemDataField);
                if (thisField.ValidForFallback())
                {
                    // ReadFallbackValue will get the fallback item for the current item 
                    // and will try to get the field value for it using fallbackItem[field.ID]
                    // Merely calling fallbackItem[field.ID] triggers the GetStandardValue method
                    // which has been overridden in the standard values provider override FallbackLanguageProvider
                    // which will in turn call ReadFallbackValue recursively until it finds a value or reaches a language that doesn't fallback 
                    fieldValue = FallbackLanguageManager.ReadFallbackValue(thisField, thisField.Item);
                }
            }
 
            string name = field.Name;
            LuceneSearchFieldConfiguration fieldSettings = this.Index.Configuration.FieldMap.GetFieldConfiguration(field) as LuceneSearchFieldConfiguration;
            if (fieldSettings == null || fieldValue == null)
                return;
            float boost = BoostingManager.ResolveFieldBoosting(field);
            if (IndexOperationsHelper.IsTextField(field))
            {
                LuceneSearchFieldConfiguration fieldConfiguration = this.Index.Configuration.FieldMap.GetFieldConfiguration("_content") as LuceneSearchFieldConfiguration;
                this.AddField("_content", fieldValue, fieldConfiguration ?? this.defaultTextField, 0.0f);
            }
            this.AddField(name, fieldValue, fieldSettings, boost);
        }
    }

Code can be found here:

https://github.com/Verndale-Corp/Sitecore-7-Fallback-Lucene-Indexing

Corey Burnett

Have you created a solution for SOLR yet?
lsolana lsolana

Hi Elizabeth. Thanks for great posts! One question... did you succeed with SOLR configuration?
Sander Schutten

Hi Elizabeth, thank you for this extensive write-up for the Partial Language Fallback solution. As i'm working on a Sitecore 8 implementation i've done a little experimentation with the solution. I did manage to get it working by following your instructions and also some comments on a few forums. But i've stumbled upon an issue with the Experience Editor, which i don't know if it's by-design or because of Sitecore 8 changes. I hope you can give me some insights.. If i edit some content on a page via the Experience Editor which has been "falling back", i modify the "fallback content". I expected that a new language-version would've been added if i save the modified content and no fallback anymore. Also i've found that because of Sitecore 8's versioned layout's the components added to placeholders don't "fallback". thanks in advance, Sander Schutten
Sander Schutten

Hi Elizabeth, thank you for this extensive write-up for the Partial Language Fallback solution. As i'm working on a Sitecore 8 implementation i've done a little experimentation with the solution. I did manage to get it working by following your instructions and also some comments on a few forums. But i've stumbled upon an issue with the Experience Editor, which i don't know if it's by-design or because of Sitecore 8 changes. I hope you can give me some insights.. If i edit some content on a page via the Experience Editor which has been "falling back", i modify the "fallback content". I expected that a new language-version would've been added if i save the modified content and no fallback anymore. Also i've found that because of Sitecore 8's versioned layout's the components added to placeholders don't "fallback". thanks in advance, Sander Schutten