Azure Service Bus Topology migration with NServiceBus

  1. Azure Service Bus Topologies
  2. Azure Service Bus Topologies with Forwarding
  3. Azure Service Bus Topologies of NServiceBus
  4. Azure Service Bus Topology migration with NServiceBus (current)

In the last post I explained the complexity we introduced for our customers when we created multiple topologies without providing a smooth migration path. In this post I’ll walk you through the online migration path that we created to allow our customers to transition their existing NServiceBus endpoints using the EndpointorientedTopology to the ForwardingTopology piece by piece.

At Particular Software, we strive as much as possible to support our customers and allow them to migrate existing code, patterns or systems as smooth as possible to newer versions of our components. Whenever we can we favor an online migration approach. Let’s face it our customers have mission-critical systems and big-bang solutions where everything has to be taken offline, migrated and redeployed is probably not going to make our customers happy. So when we started brainstorming a solution for migrating customers running on the EndpointorientedTopology to the ForwardingTopology we had three main requirements:

  • Migration should not be a big-bang and thus be aligned with whatever cadence our customers will deploy into production
  • Migration of NServiceBus endpoints should be possible in any order to avoid unnecessary complexity of deployment to production. By any order, we wanted to make sure that it did not matter whether the subscribers were migrated first or the publishers were migrated first
  • Subscribers should be safeguarded from duplicates during the migration period

To fulfill the above requirements we came up with a hybrid topology that bridges the gap between the EndpointorientedTopology and the ForwardingTopology by running a hybrid mode of those two topologies. Since our customers move from the EndpointorientedTopology to the ForwardingTopology the “migration mode” is enabled on the EndpointorientedTopology. The migration mode only affects events. It does not change how events are published (that would break backwards compatibility) but changes how events are subscribed to. Subscribers running in migration mode will no longer fetch messages from subscriptions. Instead, they will perform the following:

  1. Make sure the forwarding topology specific parts are in place in case it isn’t yet by creating; a bundle topic bundle-1 for all events, a subscription for the subscriber endpoint and a filter rule per event type. The subscription will auto-forward the events to the endpoint’s input queue.
  2. Safeguard against duplicated items by creating a migration topic named migration if not present, that performs de-duplication and auto-forwards events to the bundle-1 topic.
  3. Auto-forward all events arriving at <subscriber_endpoint_name>.<event_type_name>subscriptions from the endpoint specific topic <publishing_endpoint_name>.events to the migration topic.

Below is an example of a publisher with two events, EventA and EventB, and a subscriber consuming those events. When an endpoint is running in migration mode, the topology is as follows:

Endpoint running in topology

Endpoint running in topology

Why is the migration topic required? When an instance of a publisher is been migrated to the forwarding topology, and another instance is still in migration, all events published by the endpoint still in migration need to be de-duplicated. The migration topic ensures events are de-duplicated regardless of which instance published them.

By introducing this migration mode our customers can now smoothly transition to the newly introduced topology without doing a big bang or even worse losing messages that would be already enqueued on the subscriptions (remember they behave like queues).

This wraps up this series on Azure ServiceBus topologies. As a final step, I would like to give a shout out to the ASB team. Their support continues to be outstanding. During the design and implementation phase, we discovered a bug on the broker side that would have made our migration approach not possible. The team triaged the issue and came up with a broker side fix that was rolled out globally so that our customers can benefit from our migration approach. Fantastic!

About the Author:

Daniel Marbach is a Solution Architect at Particular Software Ltd and the CEO of tracelight GmbH. His experience spreads from client and server development to building distributed systems. Daniel has an engineer degree from the University of Applied Science in Horw. He has strong skills in Extreme Programming Practices such as Continuous Integration, Acceptance Test Driven Development and Test Driven Development. Daniel has extensive knowledge in the .NET ecosystem including NoSQ L databases, asynchronous programming, open-source software development and much more. He is a Microsoft MVP for Integration,  a frequent speaker, coach and passionate blog writer. Daniel co-founded the .NET Usergroup Central Switzerland and continues his journey of software development with passion.

In his professional life, Daniel has worked for many companies, mainly focusing on software development for industrial, technological or medical applications. Examples include the implementation of firmware upgrade tool for field engineers of a company specialized in breastfeeding and health care, labor analysis software for gene expression and genetic variation with PCR-based light cyclers for a global player in the medical industry, a secure and hybrid elevator configuration utility for an elevator company, architecture and coaching from the introduction up to the marketability of an automated guidance system, coach for Agile Engineering Practices and co-architect of a highly distributed medical customer management system. In his current position, Daniel is improving and extending the Particular Platform and helping customers to develop thriving distributed systems with NServiceBus.

Reference:

Marbach, D. (2019). Azure Service Bus Topology migration with NServiceBus. Available at: https://www.planetgeek.ch/2019/01/30/azure-service-bus-topology-migration-with-nservicebus/ [Accessed: 5th February 2019]

Share this on...

Rate this Post:

Share: