#Crm2Crm – part 7: Bidirectional replication up-and-running

Welcome back in the final episode in this series of articles on building a replication mechanism within Dynamics CRM. In the previous article
I implemented a concept of a message pump.

The goal of the message pump is to process the messages created by the actions performed on the source entity which are captured by a plugin registered on the messages:

  • Create
  • Update
  • Delete
  • SetState
  • SetStateDynamicEntity

Initially I registered the steps as asynchronous plugin steps, in the end I had to register the steps as synchronous steps. More on that later.

In the first version of the message pump I was able to replicate all actions done on the source entity to the destination entity.

1way

Once this scenario turned out to be working, I registered the same plugin on the destination entity. This resulted in the following scenario.

2wayasync

Running the bidirection approach resulted in an infinite loop, as an action conducted on either the source or destination entity resulted into a replication storm. Despite a mechanism I implemented that should prevent these endless loops.

The mechanism I implemented was the following

As part of the replication I pass an Id that I store in the jcrm_entitysourceid field. This way I can identify what the source is of the action on the record.
As soon as the action is done, the plugin on the entity fires.

In the plugin I implemented a check to verify if the jcrm_sourceentityid field was passed. In case the jcrm_sourceentityid contained a value, the field is updated to null. The update results in the plugin to fire again.
In the second run of the plugin I verify if the incoming field on the target is null. If this is the case, then I bail out.

A rock solid mechanism, however it does not work on an asynchronous plugin step. The reason it doesn’t work is that the steps are not executed in a sequence but at two different moments. The solution for this behaviour was registering the plugin steps as synchronous plugin steps.

This resulted in the following scenario: A full working version of a bidirectional replication mechanism.

2waysync

When running the message pump (I tricked it to run in an infinite loop), you can see the message flow from the source entity to the destination entity and vice versa. Resulting in a chit chat log like this:

Polling...
 - Check for new messages

Polling...
 - Check for new messages

Polling...
 - Check for new messages
 - Process Create in entity: jcrm_destinationentity for id: 42ebe1b6-48d4-e611-8425-00155d021400
 - Remove message 43ebe1b6-48d4-e611-8425-00155d021400

Polling...
 - Check for new messages

Polling...
 - Check for new messages
 - Process Update in entity: jcrm_destinationentity for id: 42ebe1b6-48d4-e611-8425-00155d021400
 - Remove message 44ebe1b6-48d4-e611-8425-00155d021400

Polling...
 - Check for new messages
 - Process Update in entity: jcrm_destinationentity for id: 42ebe1b6-48d4-e611-8425-00155d021400
 - Remove message c8c31ec0-48d4-e611-8425-00155d021400

Polling...
 - Check for new messages

Polling...
 - Check for new messages
 - Process Update in entity: jcrm_dataentity for id: 42ebe1b6-48d4-e611-8425-00155d021400
 - Remove message 72171acc-48d4-e611-8425-00155d021400

Polling...
 - Check for new messages

Polling...
 - Check for new messages
 - Process Update in entity: jcrm_dataentity for id: 42ebe1b6-48d4-e611-8425-00155d021400
 - Remove message df677ad3-48d4-e611-8425-00155d021400
 - Process SetStateDynamicEntity in entity: jcrm_dataentity for id: 42ebe1b6-48d4-e611-8425-00155d021400
 - Remove message e0677ad3-48d4-e611-8425-00155d021400

Polling...
 - Check for new messages

Polling...
 - Check for new messages

Polling...
 - Check for new messages
 - Process Update in entity: jcrm_destinationentity for id: 42ebe1b6-48d4-e611-8425-00155d021400
 - Remove message 93fc86e8-48d4-e611-8425-00155d021400
 - Process SetStateDynamicEntity in entity: jcrm_destinationentity for id: 42ebe1b6-48d4-e611-8425-00155d021400
 - Remove message 94fc86e8-48d4-e611-8425-00155d021400

Polling...
 - Check for new messages
 - Process Delete in entity: jcrm_destinationentity for id: 42ebe1b6-48d4-e611-8425-00155d021400
 - Remove message 95fc86e8-48d4-e611-8425-00155d021400

Polling...
 - Check for new messages
 - Process Delete in entity: jcrm_dataentity for id: 42ebe1b6-48d4-e611-8425-00155d021400
 - entity Id 42ebe1b6-48d4-e611-8425-00155d021400 does not exist! Ignoring...
 - Remove message 96fc86e8-48d4-e611-8425-00155d021400

Polling...
 - Check for new messages

Lessons learned

In the end it turned out that building a replication mechanism is not that difficult, it’s a matter of abstract thinking.
You need to think on how to get the data from the source to the destination without any loss of information. In part 5 I implemented a custom serialization and deserialization class that runs in a CRM sandbox.
Last but not least, I foresaw in a mechanism that prevents a replication storm by passing an additional field in the data to indicate the source of the data.

In order to make this solution usable (or commercially viable) the following things need to be done:

  • Implement a UI in which you can specify what entities need to be replication.
  • Implement a dynamic provisioning mechanism that registers/deregisters plugin steps on the fly.
  • Implement an entity in which you can specify the mapping (from what entity to what entity).
  • Implement robust error handling and logging.
  • Implement the message pump as a service (windows, azure, custom workflow action).

For your convenience, I enclosed the solution and the Visual Studio project. File Attachment: JourneyIntoCrm.Replication.zip (48 KB)

I hope you enjoyed this series as much as I did.

Leave a Reply

Your email address will not be published. Required fields are marked *