Dynamics CRM and Azure Scheduler – The feedback loop

In my last article I showed how to build a simple command line exe, that could be deployed as an Azure WebJob. Furthermore I showed how to pass parameters to the WebJob, parameters that can be used to perform specific actions. For me this is enough information to stop the proof of concept, however that would not be fair to you.

In this article I’ll describe what you actually can do when you combine all the techniques I demonstrated in this series of articles.

Using the Azure Job Scheduler and Azure WebJobs in combination with Dynamics CRM offers you the possibility to build long running batch jobs that you can schedule to run at any given time.

In this article I’ll describe how you can set up a generic batch mechanism with an integrated feedback loop.

Batchjob entity model

In CRM we have to define two entities:

  • BatchJob
    This entity contains all information needed to define a batch job. The WebJob command, the additional parameters required to run it, the start and end date of the series of batch jobs and the interval on which the job should be run.
    Furthermore the entity contains a status field (needed to trigger the functionality) and a message field to store any error message or information message.
  • BatchJobMessage
    This entity will be used to store the results of a (running) batch job. The BatchJobMessage has a lookup to the BatchJob via the BatchJobId. Furthermore the entity contains fields like ExecutionStatus (running, completed, error) and a message field to store the information or error messages.

On the batchJob entity we define a plugin which we register on the update message (async execution). The plugin checks if the status has been changed (e.g. from “concept” to “start”). In case the status has been changed, the plugin creates a new Azure Scheduler Job (using the WebJobCommand, the additional parameters. etc).
One of the parameters that is passed with the WebJobCommand is the BatchJobId. The batchJobId will be used to write back status messages to the BatchJobMessage Entity, creating a feedback loop!

The Azure Web Job uses the BatchJobId and the additional parameters to perform its job. The BatchJobId will be used to write status information to the BatchJobMessage entity. The Job itself is created as an ordinary command line executable that can be fully developed and debugged locally.

In the article “Dynamics CRM and Azure Scheduler – Breakthrough” I posted a code sample on how to use a REST call to create an Azure Scheduler Job. In the previous article I described how to create an Azure WebJob and how to pass parameters to it.
You also can find all the information in that article you need to specify the correct url required to run the Azure Web Job.

Once the Azure Scheduler Job is created, the status in the BatchJob entity is updated to “execution”.

When the Azure Web Job completes it execution it writes a message to the BatchJobMessage entity using the BatchJobId to identify it. On the BatchJobMessage entity a plugin is registered on the POST CREATE / POST UPDATE message.
Within that plugin a check is done on the BatchJob entity, in which is checked if the executed Web Job was the last in the series of Jobs that had to be performed. If that is the case, the status of the BatchJob is updated to “completed”.

What remains to be done is creating a view on the BatchJob entity in which the jobs are shown with the current status. A grid containing BatchJobMessages on the BatchJob entity page shows all the relevant information.

This described in a nutshell how to create a generic batch job mechanism using the Aure Job Scheduler and Azure Web Jobs.