Mirroring 2.0 – Overview

This is the first post in a series about mirroring 2.0. In this first post I will give an overview of the design for the mirroring functionality in CMS 6.

Background

Although the previous mirroring worked fine it has some limitations (especially when it comes to large amount of data). Below is a list of issues that we often received feedback about regarding the previous implementation of mirroring:

• Responsiveness for site decreased when a mirroring job was executed (both for source and destination sites).
• Failures during mirroring of huge site structure, due to either OutOfMemoryException or web service communication problems.
• A single failure stops the whole mirroring process.
• Poor status and error messages.

With that in mind some of the design goals we set up for the new mirroring implementation were:

• Scalability – There should be no upper limit of how much content the mirror will handle. Of course it will take some time to mirror a huge site but the time should be linearly proportional to the amount of data.
• The mirroring job should not execute in the same process as the web application.
• It should be possible to continue the mirroring job for certain types of errors.
• It should be possible to get detailed information from ongoing jobs.

Design

The design we came up with conceptually looks like:

And a process flow for a mirroring job looks like:

1. A scheduled job (or a user in CMS admin mode) starts the mirroring job and sends a request to the mirroring source service to start mirroring.
2. The source service initializes itself with settings from the site and returns a message to the sender that it has started working with the job.
3. The source service starts to export pages and files in packages to the target service.
4. The target service starts importing the data and responds with status messages to the source service for each package.
5. When the target service is finished it sends a message to its site to allow it to clear invalidated cache entries.
6. The source service persist the mirroring status to the site’s database.

During execution, a monitoring client has the chance to follow the whole flow in real-time.

In the picture above the source and target services runs in separate processes outside the web site application. Both source and target services are instances of MirroringService which we have chosen to implement as an IIS hosted WCF service. However the services have no dependency to web context so it would be easy to host them for example in a Windows Service instead.

Upcoming posts will go into more detail about various aspects about mirroring.