• Not Answered

xDB support for Azure DocumentDB through their protocol support for MongoDB

Azure has released (in preview at the moment) a protocol compatibility layer for MongoDB to their DocumentDB offering.

https://azure.microsoft.com/en-us/documentation/articles/documentdb-protocol-mongodb/

Does Sitecore supports running xDB on DocumentDB through this compatibility layer?

12 Replies

  • Hi there, yes indeed, it could be very interesting!

    For the moment, this service "Azure DocumentDB With Protocol Support for MongoDB" is in Preview. You can ask the Azure DocumentDB Team from the Azure portal to be part of this preview. Then you will receive a Getting Started guide to use it. This document describe how to use that by connecting with the MongoDB C# Driver:

    Quotation of the document:

    All DocumentDB accounts require both authentication and secure communication via SSL.  While the MongoDB connection string URI format supports an ssl=true query string parameter, working with the MongoDB C# driver requires use of the MongoClientSettings object when creating a MongoClient.  Given the account information above, the following code snippet shows how to connect to the account and work with the “Tasks” database.

               MongoClientSettings settings = new MongoClientSettings();
    
               settings.Server = new MongoServerAddress("docdbm1.documents.azure.com", 10250);
    
               settings.UseSsl = true;
    
               settings.SslSettings = new SslSettings();
    
               settings.SslSettings.EnabledSslProtocols = SslProtocols.Tls12;
    
               MongoIdentity identity = new MongoInternalIdentity("Tasks", "docdbm1");
    
               MongoIdentityEvidence evidence = new PasswordEvidence("<primary master key>");
    
               settings.Credentials = new List<MongoCredential>()
    
               {
    
                   new MongoCredential("SCRAM-SHA-1", identity, evidence)
    
               };
    
               MongoClient client = new MongoClient(settings);
    
               var database = client.GetDatabase("Tasks",);

    Does anyone know how we could override the default MongoClient creation/management in/by Sitecore? Does it exist a pipeline for that?

    Whith that it could be easy to target Azure DocumentDB for the 4 Sitecore MongoDB: analytics, tracking.live, tracking.history and tracking.contact.

    Thank you for your feedback,

  • In reply to Mathieu Benoit:

    Hi there, after some investigation here are the steps I have found to support Azure DocumentDB as a MongoDB provider (but there is some issue currently):

    (Setup : Sitecore 8.1 update 2 with MongoDB.Driver 1.10.0.62)

    • Create a custom UpdateMongoDriverSettingsProcessor
    using System.Security.Authentication;
    using MongoDB.Driver;
    using Sitecore.Analytics.Pipelines.UpdateMongoDriverSettings;
    
    namespace MyNamespace
    {
        public class UpdateMongoDriverSettingsProcessor : global::Sitecore.Analytics.Pipelines.UpdateMongoDriverSettings.UpdateMongoDriverSettingsProcessor
        {
            public override void UpdateSettings(UpdateMongoDriverSettingsArgs args)
            {
                if (args.MongoSettings.Server != null
                    && !string.IsNullOrEmpty(args.MongoSettings.Server.Host)
                    && args.MongoSettings.Server.Host.EndsWith(".documents.azure.com"))
                {
                    args.MongoSettings.SslSettings = new SslSettings();
                    args.MongoSettings.SslSettings.EnabledSslProtocols = SslProtocols.Tls12;
                }
            }
        }
    }

     

    • Add this processor to the associated Sitecore pipeline
    <configuration xmlns:patch="www.sitecore.net/xmlconfig">
      <sitecore>
        <pipelines>
          <updateMongoDriverSettings>
            <processor type="MyNamespace.UpdateMongoDriverSettingsProcessor, MyDllName" /> 
    </updateMongoDriverSettings>
    </pipelines>
    </sitecore>
    </configuration>
    • Define mongodb connectionstrings as follow for analytics, tracking.live, tracking.history and tracking.contact
    <add name="<keyname>" connectionString="mongodb://<username>:<password>@<host>:<port>/<databasename>?ssl=true;authMechanism=SCRAM-SHA-1" />

    After that, Sitecore is connected to this new MongoDB provider and I'm able to start my website after publishing these new changes. 

    I'm able to see 1 collection for analytics and tracking.live BUT after that I have an error page:

    Tracker.Current is not initialized
    
    [InvalidOperationException: Tracker.Current is not initialized]
       Sitecore.Analytics.Pipelines.StartAnalytics.StartTracking.Process(PipelineArgs args) +304
       (Object , Object[] ) +74
       Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args) +484
       Sitecore.Analytics.Pipelines.StartAnalytics.StartAnalyticsPipeline.Run() +293
       Sitecore.Mvc.Analytics.Pipelines.MvcEvents.RequestBegin.StartTracking.Process(RequestBeginArgs args) +139
       (Object , Object[] ) +74
       Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args) +484
       Sitecore.Mvc.Pipelines.PipelineService.RunPipeline(String pipelineName, TArgs args) +184
       Sitecore.Mvc.Routing.RouteHttpHandler.BeginProcessRequest(HttpContext context, AsyncCallback cb, Object extraData) +74
       System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +921
       System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +137

    After looking at the logs, here is the details about 4 differents exception:

    Exception: MongoDB.Driver.MongoQueryException
    Message: QueryFailure flag was Syntax error, incorrect syntax near '84'. (response was { "_t" : "OKResponse", "ok" : 0, "$err" : "Syntax error, incorrect syntax near '84'.", "code" : 9, "errmsg" : "Syntax error, incorrect syntax near '84'." }).
    Source: MongoDB.Driver
       at MongoDB.Driver.Internal.MongoReplyMessage`1.ReadHeaderFrom(BsonBuffer buffer)
       at MongoDB.Driver.Internal.MongoConnection.ReceiveMessage[TDocument](BsonBinaryReaderSettings readerSettings, IBsonSerializer serializer, IBsonSerializationOptions serializationOptions)
       at MongoDB.Driver.Operations.QueryOperation`1.GetFirstBatch(IConnectionProvider connectionProvider)
       at MongoDB.Driver.Operations.QueryOperation`1.Execute(IConnectionProvider connectionProvider)
       at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
       at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
       at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
       at Sitecore.Analytics.Automation.Data.MongoDbAutomationProvider.GetContactIdsToProcessByWakeUpTime()
       at Sitecore.Analytics.Automation.AutomationWorker.Process()
       at Sitecore.Analytics.Automation.AutomationWorker.<Wakeup>b__6(Object state)

    Exception: MongoDB.Driver.MongoQueryException
    Message: QueryFailure flag was Syntax error, incorrect syntax near '10'. (response was { "_t" : "OKResponse", "ok" : 0, "$err" : "Syntax error, incorrect syntax near '10'.", "code" : 9, "errmsg" : "Syntax error, incorrect syntax near '10'." }).
    Source: MongoDB.Driver
       at MongoDB.Driver.Internal.MongoReplyMessage`1.ReadHeaderFrom(BsonBuffer buffer)
       at MongoDB.Driver.Internal.MongoConnection.ReceiveMessage[TDocument](BsonBinaryReaderSettings readerSettings, IBsonSerializer serializer, IBsonSerializationOptions serializationOptions)
       at MongoDB.Driver.Operations.CommandOperation`1.Execute(MongoConnection connection)
       at MongoDB.Driver.Operations.BulkUnmixedWriteOperationBase.ExecuteBatch(MongoConnection connection, Batch`1 batch, Int32 originalIndex)
       at MongoDB.Driver.Operations.BulkUnmixedWriteOperationBase.Execute(MongoConnection connection)
       at MongoDB.Driver.Operations.UpdateOpcodeOperationEmulator.Execute(MongoConnection connection)
       at MongoDB.Driver.Operations.UpdateOpcodeOperation.Execute(MongoConnection connection)
       at MongoDB.Driver.MongoCollection.Update(IMongoQuery query, IMongoUpdate update, MongoUpdateOptions options)
       at MongoDB.Driver.MongoCollection.Save(Type nominalType, Object document, MongoInsertOptions options)
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbCollection.<>c__DisplayClass3c.<Save>b__3b()
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbCollection.Execute(Action action, ExceptionBehavior exceptionBehavior)
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbCollection.Save(Object value)
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbDictionary.Store(Object value)

    Exception: MongoDB.Driver.MongoQueryException
    Message: QueryFailure flag was Syntax error, incorrect syntax near '10'. (response was { "_t" : "OKResponse", "ok" : 0, "$err" : "Syntax error, incorrect syntax near '10'.", "code" : 9, "errmsg" : "Syntax error, incorrect syntax near '10'." }).
    Source: MongoDB.Driver
       at MongoDB.Driver.Internal.MongoReplyMessage`1.ReadHeaderFrom(BsonBuffer buffer)
       at MongoDB.Driver.Internal.MongoConnection.ReceiveMessage[TDocument](BsonBinaryReaderSettings readerSettings, IBsonSerializer serializer, IBsonSerializationOptions serializationOptions)
       at MongoDB.Driver.Operations.QueryOperation`1.GetFirstBatch(IConnectionProvider connectionProvider)
       at MongoDB.Driver.Operations.QueryOperation`1.Execute(IConnectionProvider connectionProvider)
       at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)
       at MongoDB.Driver.MongoCollection.FindOneAs[TDocument](FindOneArgs args)
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbCollection.<>c__DisplayClass29`1.<FindOneAs>b__28()
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbCollection.Execute(Action action, ExceptionBehavior exceptionBehavior)
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbCollection.FindOneAs[TDocument](IMongoQuery query, ExceptionBehavior exceptionBehavior)
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbContactStorage.LoadContactIdentifier(ID id)
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbDataAdapterProvider.LoadContactIdentifierValue(ID contactId)
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbDataAdapterProvider.TryLoadContact(ID id, IContactFactory factory, LeaseOwner leaseOwner, TimeSpan leaseDuration)
       at Sitecore.Analytics.Data.ContactRepository.TryLoadContact(Guid id, LeaseOwner leaseOwner, TimeSpan leaseDuration)
       at Sitecore.Analytics.Tracking.ContactManager.TryLoadContact(Guid contactId, Int32 lockDurationMinutes)
       at Sitecore.Analytics.Pipelines.EnsureSessionContext.LoadContact.Process(InitializeTrackerArgs args)
       at (Object , Object[] )
       at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
       at Sitecore.Analytics.Pipelines.EnsureSessionContext.EnsureSessionContextPipeline.Run(InitializeTrackerArgs args)
       at Sitecore.Analytics.DefaultTracker.EnsureSessionContext()
       at Sitecore.Analytics.Pipelines.CreateTracker.GetTracker.Process(CreateTrackerArgs args)
       at (Object , Object[] )
       at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
       at Sitecore.Analytics.Tracker.Initialize()

    14284 22:22:00 ERROR Exception when executing agent aggregation/aggregator
    Exception: System.IO.EndOfStreamException
    Source: MongoDB.Bson
       at MongoDB.Bson.IO.ByteBufferFactory.LoadFrom(Stream stream)
       at MongoDB.Driver.Internal.MongoConnection.ReceiveMessage[TDocument](BsonBinaryReaderSettings readerSettings, IBsonSerializer serializer, IBsonSerializationOptions serializationOptions)
       at MongoDB.Driver.Operations.QueryOperation`1.GetFirstBatch(IConnectionProvider connectionProvider)
       at MongoDB.Driver.Operations.QueryOperation`1.Execute(IConnectionProvider connectionProvider)
       at Sitecore.Analytics.Data.MongoDb.ProcessingPool.MongoDbProcessingPool.GetCandidates()
       at Sitecore.Analytics.Data.MongoDb.ProcessingPool.MongoDbProcessingPool.TryCheckOut()
       at Sitecore.Analytics.Processing.ProcessingPool.ProcessingPoolScheduler`1.TryGetNext(ItemBatch`1& batch)
       at Sitecore.Analytics.Aggregation.InteractionBatchAggregationAgent.Execute()
       at Sitecore.Analytics.Core.BackgroundService.Run()

    I sent some details to the Azure DocumentDB team to have some feedback to see if they are aware about some query limitations.

    I'll keep you posted.

    Mathieu.

  • In reply to Mathieu Benoit:

    Hi there,

    I received a quick answer from the Azure DocumentDB team, here are some information to know:

    « Thanks for participating in the preview and thanks for submitting this issue.  We have a fix coming out for a bug with .NE, so that’s the likely culprit here.  With that said, we’ll have a look at this and I’ll get back to you with our findings (and I’ll let you know when the .NE fix has been deployed). »

    and

    « We don’t currently support GUID queries (we’re working on it, no exact timeline at this point).On a related note, we have limited support for BSON type queries (ObjectID and ISO_Date today), which we’re also hard at work on expanding.  I’ll keep you posted on our progress. »

    Keep in touch for the future versions of this preview ;)

    Mathieu.

  • In reply to Mathieu Benoit:

    Do you happen to have this working yet? Has Microsoft released the required things which are needed for DocumentDB to work with Sitecore?
  • In reply to Steven Hayes:

    Hi there,

    Azure DocumentDB team recenlty pushed out a new update which includes a fix for query by GUID.

    Previous errors don't happen anymore for the moment.

    In the detailed log files, I have now this error (I sent it to the Azure DocumentDB team) in different places:

    Nested Exception

    Exception: System.Net.Sockets.SocketException

    Message: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

    Source: System

       at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)

       at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)

    I received this feedback from them:

    We have a timeout fix in the works, I’ll ping you when it gets deployed.

    I will let you know when I have some updates about that.

    FYI, here is an interesting Support Matrix link for this Preview Service:

    Regards,

    Mathieu.

  • In reply to Mathieu Benoit:

    Hi Mathieu

    I just completed a new install of Sitecore 8.1 Update 3 (with MongoDB driver 1.10.0.62) configured against Azure DocumentDB with protocol support for MongoDB.

    Once configured, I see that the databases are being created within the account, plus the collections expected are present

      

     

    However, analytics and tracking data is still not reaching the storage. I am still seeing one of the 4 query errors being logged that you mention above. Let's hope the MS DocumentDB team have this one in line for a fix.

    ERROR Exception when executing agent aggregation/aggregator
    Exception: MongoDB.Driver.MongoQueryException
    Message: QueryFailure flag was Syntax error, incorrect syntax near '10'. (response was { "_t" : "OKMongoResponse", "ok" : 0, "code" : 9, "errmsg" : "Syntax error, incorrect syntax near '10'.", "$err" : "Syntax error, incorrect syntax near '10'." }).
    Source: MongoDB.Driver
       at MongoDB.Driver.Internal.MongoReplyMessage`1.ReadHeaderFrom(BsonBuffer buffer)
       at MongoDB.Driver.Internal.MongoConnection.ReceiveMessage[TDocument](BsonBinaryReaderSettings readerSettings, IBsonSerializer serializer, IBsonSerializationOptions serializationOptions)
       at MongoDB.Driver.Operations.CommandOperation`1.Execute(MongoConnection connection)
       at MongoDB.Driver.MongoCollection.RunCommandAs[TCommandResult](IMongoCommand command, ReadPreference readPreference, IBsonSerializer resultSerializer, IBsonSerializationOptions resultSerializationOptions)
       at MongoDB.Driver.MongoCollection.RunCommandAs[TCommandResult](IMongoCommand command, ReadPreference readPreference)
       at MongoDB.Driver.MongoCollection.FindAndModify(FindAndModifyArgs args)
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbCollection.<>c__DisplayClass2c.<FindAndModify>b__2b()
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbCollection.Execute(Action action, ExceptionBehavior exceptionBehavior)
       at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbCollection.FindAndModify(IMongoQuery query, IMongoSortBy sortBy, IMongoUpdate update, Boolean returnNew, Boolean upsert)
       at Sitecore.Analytics.Data.MongoDb.ProcessingPool.MongoDbProcessingPool.TryCheckOut()
       at Sitecore.Analytics.Processing.ProcessingPool.ProcessingPoolScheduler`1.TryGetNext(ItemBatch`1& batch)
       at Sitecore.Analytics.Aggregation.InteractionBatchAggregationAgent.Execute()
       at Sitecore.Analytics.Core.BackgroundService.Run()

    Regards

    Chris

  • In reply to Chris Lowndes:

    Hi Chris (and Mathieu),

    Whats your actual experience with xDB on DocumentDB? Did you (or MS DocumentDB team :)) manage to get it up and running without errors?

    Thanks,
    Mark
  • In reply to Mark de Bruijne:

    Hi Mark,

    It's better version after version, I just tried with Sitecore 8.2 (initial release) and the current version of the Azure DocumentDB (support for mongodb) service and I now just got 1 error in the logs:

    Message: Exceeded max attempt count when trying to lock contact
    Source: Sitecore.Analytics.MongoDB
    at Sitecore.Analytics.Data.DataAccess.MongoDb.MongoDbDataAdapterProvider.TryLoadContact(ID id, IContactFactory factory, LeaseOwner leaseOwner, TimeSpan leaseDuration)
    at Sitecore.Analytics.Data.ContactRepository.TryLoadContact(Guid id, LeaseOwner leaseOwner, TimeSpan leaseDuration)
    at Sitecore.Analytics.Tracking.ContactManager.TryLoadContact(Guid contactId, Int32 lockDurationMinutes, Boolean exclusive)
    at Sitecore.Analytics.Pipelines.EnsureSessionContext.LoadContact.Process(InitializeTrackerArgs args)
    at (Object , Object[] )
    at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
    at Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain)
    at Sitecore.Analytics.Pipelines.EnsureSessionContext.EnsureSessionContextPipeline.Run(InitializeTrackerArgs args)
    at Sitecore.Analytics.DefaultTracker.EnsureSessionContext()
    at Sitecore.Analytics.Pipelines.CreateTracker.GetTracker.Process(CreateTrackerArgs args)
    at (Object , Object[] )
    at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
    at Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain)
    at Sitecore.Analytics.Tracker.Initialize()

    I have just tried loading the default homepage. Complementary tests need to be done by playing with tracking and reporting.

    Thank you,
    Mathieu
  • In reply to Mathieu Benoit:

    Hi All,

    Just wondering if the errors on this have been resolved with 8.2 Update-1 or Update-2 ?

    Looking at below Perficient blog post seems to suggest it will work (but i am a little cynical, so would like to see if anyone has put it on production)

    blogs.perficient.com/.../

    Also, i believe that it will be supported in 8.2 Update-3, which i belive is due out sometime First Half 2017. Its appearing in the Suported documentation list, so that would suggest it won't be long before it is supported.

    kb.sitecore.net/.../682999

    Thanks

    Pete
  • In reply to Pete Newall:

    Still the same error on the TryLoadContact method, with version 8.2 update-2
  • I am also trying to get CosmosDB (rebranded DocumentDB) running on a Sitecore 8.1 Update 1 install but am running into the error below. I correctly set up the TLS 1.2 authentication and Sitecore connected with the databases and automatically created all the collections but all the xDB features have various errors in the UI and within the log file I see the follow error repeated over time. As best I can tell this is simply the MongoDB driver in 8.1 being incompatible CosmosDB but others here were getting different errors.

    3460 20:04:19 ERROR Exception when executing agent aggregation/historyWorker
    Exception: System.InvalidCastException
    Message: Unable to cast object of type 'MongoDB.Bson.BsonInt32' to type 'MongoDB.Bson.BsonInt64'.
    Source: Sitecore.Analytics.MongoDB
    at Sitecore.Analytics.RangeScheduler.MongoDbRangeMap2.BuildRangeRecord(BsonDocument document)
    at Sitecore.Analytics.RangeScheduler.MongoDbRangeMap2.TryLockRange(WorkerHandle worker, DateTime leaseExpiration, RangeHandle& range)
    at Sitecore.Analytics.Processing.RangeScheduler.RangeMapWorkScheduler3`1.TryLockRange(WorkScheduler`1& context)
    at Sitecore.Analytics.Processing.RangeScheduler.RangeMapWorkScheduler3`1.GetNextRange(WorkScheduler`1& context, Boolean returning)
    at Sitecore.Analytics.Processing.RangeScheduler.RangeMapWorkScheduler3`1.TryGetNext(ItemBatch`1& item)
    at Sitecore.Analytics.Processing.WorkSchedulerWithBacklog`1.TryGetNext(ItemBatch`1& batch)
    at Sitecore.Analytics.Aggregation.Data.Processing.InteractionBatchHistoryWorker.Execute()
    at Sitecore.Analytics.Core.BackgroundService.Run()

    6176 20:04:23 ERROR Exception when executing agent aggregation/rebuildAgent
    Exception: System.InvalidOperationException
    Message: Cannot schedule a new processing task before the previous task is finished
    Source: Sitecore.Kernel
    at Sitecore.Diagnostics.Assert.IsFalse(Boolean condition, String message)
    at Sitecore.Analytics.Aggregation.History.HistoryTaskManager.StartProcessingHistory(Nullable`1 cutoff, Boolean copyDefinition)
    at Sitecore.Analytics.Aggregation.History.InteractionHistoryAggregatorManager.Start(List`1 targetProvidersConfigPaths)
    at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
    at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
    at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
    at Sitecore.Analytics.Aggregation.History.ReportingStorageManager.ProcessRebuild()
    at Sitecore.Analytics.Aggregation.Data.Processing.RebuildAgent.Execute()
    at Sitecore.Analytics.Core.BackgroundService.Run()
  • In reply to Christopher Morisette:

    At this time, DocumentDB and Cosmos DB are not officially supported. Support for this will be coming in the future, but it is not available yet.