LibrarySites.Banner

All About Jobs in the Sitecore ASP.NET CMS

While Sitecore is always hiring, this blog post is not about employment, but about the job management facilities in the ASP.NET web content management system (CMS). You can use Sitecore jobs to invoke long-running process in background threads that you can monitor in a foreground thread or from Sitecore Rocks. Jobs often involve administrative functions such as system maintenance. For more information about background processes, see the blog post All About Sitecore Scheduling Agents and Tasks (Sitecore uses jobs to invoke agents). For more information about Sitecore Rocks, see the blog post Sitecore Differentiating Factors Blog Series: Sitecore Rocks. To see additional deep-dives into various topics, see the index of blog posts All About the Sitecore ASP.NET CMS.

Introduction

You can use the job infrastructure to perform background tasks triggered by a variety of mechanisms. The example described in this blog post invokes a class that imports data in a comma-separated values (CSV) file to create or update items in Sitecore. I could imagine triggering such a process in a variety of ways, including:

  • Using an agent (scheduled process) as described in the blog post linked in the introduction to this post
  • Interactively through the user interface
  • Due to a file system trigger, such as a file appearing in a subdirectory monitored by an ASP.NET HTTP module
  • Based on something calling a web service on the Sitecore instance

Each of these triggers might require slightly different configuration. For example, you might want to increase the priority of the thread when running the import through the user interface. The job infrastructure supports these types of requirements.

How Jobs Differ from Method Calls

While creating a job eventually results in calling a method on an object, several things about jobs differ from standard methods calls:

  • Sitecore invokes the <job> pipeline defined in the Web.config file to process each job. Like any other pipeline, you can add, remove, and replace processors in the <job> pipeline, and you can pass parameters to processors using the configuration factory. For more information about pipelines, see the blog post All About Pipelines in the Sitecore ASP.NET CMS. For more information about the configuration factory, see the blog post The Sitecore ASP.NET CMS Configuration Factory.
  • Sitecore invokes jobs in separate threads. Your code continues to run after you invoke the job, which runs in the background. For long-running operations, this can be important to prevent thread blocking, allowing you to update the user interface and preventing potential browser and server timeouts.
  • Jobs provide infrastructure that allows you to monitor progress of the background thread, and can persist information about jobs for some time.
  • Jobs make it easy to set thread priority for a process.
  • Sitecore will not run multiple instances of a job concurrently unless you enable concurrency.

Important Classes for Working with Jobs

There are three primary classes involved in creating jobs:

  • The Sitecore.Jobs.Job class represents a job.
  • The Sitecore.Jobs.JobOptions class represents configuration information about a job, such as the class and method to invoke and options such as thread priority. You pass an instance of this type to the constructor when you create a Sitecore.Jobs.Job.
  • The Sitecore.Jobs.JobManager static class initiates and manages jobs. To initiate a job, you pass an instance of the Sitecore.Jobs.Job or Sitecore.Jobs.JobOptions class to the Start() method of the Sitecore.Jobs.JobManager static class (in the latter case, Sitecore creates the Sitecore.Jobs.Job object for you, and returns that job).

Of these, the Sitecore.Jobs.JobOptions class has the most properties of interest. The constructor for the Sitecore.Jobs.JobOptions class sets the most important of these properties. Specify these arguments to the constructor of the Sitecore.Jobs.JobOptions class to define these properties:

  • JobName: The first constructor argument defines a name for the job. You typically access the name in user interfaces that monitor job progress, but Sitecore also uses the job name to determine whether an instance of a job is already running. Unless you want to allow concurrent instances of a single job (in which case you should also set the AtomicExecution property to false as described below), you should name your jobs consistently. For instance, include the namespace, class, and the name of the method associated with the job in a specific format, and potentially include values for specific parameters passed to the method.
  • Category: The second constructor argument defines a category for the job. You might access the category in user interfaces that monitor job progress.
  • SiteName: The third constructor argument specifies the name of the managed site in which to run the job. For more information about managed sites, see the blog post Managed Web Sites in the Sitecore ASP.NET CMS.
  • Method: The fourth constructor argument specifies the object that contains the method to invoke (Sitecore sets the Method property based on this constructor argument and the subsequent two constructor arguments).
  • Method (continued): The fifth constructor argument specifies the name of the method to invoke.
  • Method (continued): Optionally, the sixth constructor argument specifies an array of objects to pass to the method to invoke.

Additional interesting properties of the Sitecore.Jobs.JobOptions class include:

  • AfterLife: Length of time Sitecore maintains information about the job after its completion. Defaults to 1 minute.
  • AtomicExecution: Disable (true) or enable (false) concurrent instances of a single named job. Defaults to true.
  • ContextUser:  Context user for the duration of the job. Defaults to null (the job runs as the existing context user).
  • EnableSecurity: Enable (true) or disable (false) Sitecore security for the duration of the job. Defaults to true.
  • Priority: Thread priority. Defaults to System.Threading.ThreadPriority.BelowNormal. For a little more information about thread priority, see the blog post Set the Priority for a Scheduled Agent in the Sitecore ASP.NET CMS.
  • WriteToLog: Controls whether Sitecore logs invocation of the job. You can also check this property to determine whether your method should write messages to the log. Sitecore Defaults to true.

Updating Job Status

You can access the Sitecore.Job.JobOptions object associated with that job through the Options property through the Options property of that job (Sitecore.Context.Job.JobOptions). You can use the Status property (of type Sitecore.Job.JobStatus) of the Sitecore.Jobs.Job class to store information about the state of a job. The Sitecore.Job.JobStatus class exposes a Messages property (of type System.Collections.Specialized.StringCollection) to which you can add status messages and a Processed property (a long integer) indicating the number of units of work processed. You can also set the Total property to indicate the number of units of work the job will process. From within the method invoked by the job, you can access the Sitecore.Jobs.Job object through the Sitecore.Context.Job property. For example, in a job that imports content, you might set the Total property (another long integer) to the number of items that the job expects to import, write information about each item that you import to the Messages collection, and increment the Processed property to indicate the number of items imported thus far. Because the job method might not always run as part of a job, you should check Sitecore.Context.Job against null before accessing it. You can use code such as the following for these purposes:

//TODO: populate csvImportItems with an array of data items to import
  
// if this method is running in the context of a job
// store the total number of items to import
if (Sitecore.Context.Job != null)
{
  Sitecore.Context.Job.Status.Total = csvImportItems.Length;
}
  
// iterate each of the items to import
foreach(Sitecore.Sharedsource.Data.CSVImporter.CSVImportItem importItem
  in csvImportItems)
{
  // if this method is running in the context of a job
  // report the item this process is about to import
  if (Sitecore.Context.Job != null)
  {
    Sitecore.Context.Job.Status.Messages.Add(
      "Creating " + importItem.Path);
  }
  
  //TODO: create item based on importItem
  
  // if this method is running in the context of a job
  // increment the counter of units processed
  // and report the item this process just imported
  if (Sitecore.Context.Job != null)
  {
    Sitecore.Context.Job.Status.Processed++;
    Sitecore.Context.Job.Status.Messages.Add(
      "Imported " + importItem.Path);
  }
}

Starting a Job

Sorry, the actual code I used to import is far too long to describe in this blog post, which is already too long. Here is some code that starts a job to import content (depends on the System.IO namespace):

// CSV file containing data to import, one row per item
FileInfo csvFile = new FileInfo("C:\\temp\\importme.csv");
  
// database in which to create/update items
Sitecore.Data.Database db = Sitecore.Configuration.Factory.GetDatabase(
  "master");
Sitecore.Diagnostics.Assert.IsNotNull(db, "db");
  
// object that imports data (contains method to run as a job)
Sitecore.Sharedsource.Data.CSVImporter csvImporter =
  new Sitecore.Sharedsource.Data.CSVImporter();
    
// import all items in the CSV file
Sitecore.Sharedsource.Data.CSVImporter.CSVImportOptions csvImportOptions
  = new Sitecore.Sharedsource.Data.CSVImporter.CSVImportOptions(
    db.GetRootItem());
      
// require a context site rather than hard-coding a site name
Sitecore.Diagnostics.Assert.IsNotNull(
  Sitecore.Context.Site, 
  "context site");
    
// unique name for the job  
string jobName = csvImporter.GetType() + "_Import_" + csvFile.FullName;
  
// arguments to the Import() method
object[] args = new object[] { csvImportOptions, csvFile };
  
// job configuration
Sitecore.Jobs.JobOptions jobOptions = new Sitecore.Jobs.JobOptions(
  jobName,                            // identifies the job
  "Automated import",                 // categoriezes jobs
  Sitecore.Context.Site.Name,         // context site for job
  csvImporter,                        // object containing method
  "Import",                           // method to invoke
  args)                               // arguments to method
{
  AfterLife = TimeSpan.FromHours(1),  // keep job data for one hour
  EnableSecurity = false,             // run without a security context
};
  
// invoke the job
Sitecore.Jobs.Job = Sitecore.Jobs.JobManager.Start(jobOptions);
  
//TODO: monitor the job

Monitoring Jobs

Once you have the job, you can use its Status property to report information about its progress. If you need your UI to post back while the job runs, you can call the ToString() method of the Handle property of the Sitecore.Jobs.Job, store that value somewhere, create an instance of Sitecore.Handle by passing that value to the constructor for that class, and retrieve the corresponding job by passing that handle to the GetJob() method of the Sitecore.Jobs.JobManager static class.

You can access information about active and queued jobs through the GetJobs() method of the Sitecore.Jobs.JobManager static class. To monitor jobs using Sitecore Rocks, right-click on the connection, select Tools from the context menu that appears, and then click Job Viewer from the submenu that appears.

Limitations of Job Method Arguments and Results

You can use the job infrastructure to invoke any method in any class, although:

  • That method should typically return void, as I cannot see how you would access the return value of the method.
  • I do not think you can pass null as the value of an argument to the method, as reflection would not be able to identify the signature of the method.
  • I do not think you access values populated by the method into arguments passed by value (such as string, int, enum, and struct) rather than by reference (such as object and all of its derivatives).
  • I do not see how you can use the ref keyword to force passing of an argument by reference rather than by value.

If your method returns complex results, you should probably create a specific class to represent those results, pass an instance of that class to the method, and populate that object within the method. If you find it convenient, you might use the CustomData property of the Sitecore.Job.JobOptions class to pass an object. In the called method, you can access this property through Sitecore.Context.Job.Options.CustomData, but then your method could become dependent on the job infrastructure, where it might not always run as a job.

The <job> Pipeline

As mentioned previously, to process a job, Sitecore invokes the <job> pipeline defined by the /configuration/sitecore/pipelines/job element in the Web.config file. Processors in this pipeline accept an argument of type Sitecore.Job.JobArgs. In Sitecore CMS 6.5.0 rev. 111230, the <job> pipeline contains the following processors in the order listed. All of these processors in the <job> pipeline are in the Sitecore.Jobs.JobRunner class; the titles indicate method names within that class.

  • SignalStart: Writes a message indicating job initiation to the Sitecore log if the WriteToLog property of the Sitecore.Jobs.JobOptions object passed to the constructor of the Sitecore.Jobs.Job object and exposed by the Options property of that object) is true (for more information about Sitecore logging, see the blog post All About Logging with the Sitecore ASP.NET CMS); adds a message indicating job initiation to the Messages property of the Sitecore.Jobs.Job regardless of the WriteToLog property
  • SetPriority: Sets the priority of the thread running the job to that specified by the Priority property of the relevant Sitecore.Job.JobOptions object (defaults to System.Threading.ThreadPriority.BelowNormal)
  • SetSecurity: Enables or disables Sitecore security based on the EnableSecurity property of the relevant Sitecore.Job.JobOptions object, effectively causing the job to run as the context user or a Sitecore administrator
  • SetContextUser: Sets the context user to the user specified by the ContextUser property of the relevant Sitecore.Job.JobOptions object (if defined), causing the job to run as that user unless EnableSecurity is false.
  • InitializeContext: Sets the context site to that specified by the SiteName property of the relevant Sitecore.Job.JobOptions object, and sets the Sitecore.Context.Job property for the durration of the job
  • RunMethod: Invokes the method specified on the object specified when creating the Sitecore.Jobs.Job object
  • ResetSecurity: Resets the security context previously defined by the SetSecurity processor
  • SignalEnd: Writes a message indicating job completion and the number of units processed to the Sitecore log if the WriteToLog property of the relevant Sitecore.Jobs.JobOptions object is true; adds a message indicating job initiation to the Messages property regardless of the WriteToLog property

The ResetSecurity processor corresponds to the SetSecurity processor, but no ResetContextUser processor exists corresponding to the SetContextUser processor. I actually think ResetSecurity is unnecessary because I assume that when the job completes, the thread ends, so no additional logic occurs in that security context, but I filed Sitecore support case 363064 to confirm.

Conclusion

Sorry for the very long post that does not even cover everything it could include about jobs. To share any additional information about jobs or ask any relevant questions, please comment on this blog post.