Gartner’s Latest Advice on How to Choose an Event Broker

Back in July 2020, I summarized Gartner’s report “Choosing Event Brokers: The Foundation for Your Event-Driven Architecture” with this blog post.

Gartner updated that report in June 2022, but the advice was generally the same, and is best summed up in a single quote from that report:

“Not all event brokers are created equal. Even if you are already an expert user of one or more event brokers, you should still read, absorb and reference this information. You cannot assume that seemingly similar products will behave in line with your experience, and it is important to know where the differences exist.”
Source: Gartner “Choosing Event Brokers: The Foundation of Your Event-Driven Architecture, 6 June 2022, Gary Olliffe

Since then, Gartner has published a number of reports that further give clarity to choosing the right event broker for the use case at hand:

• Select the Right Event Broker Technologies for Your Event-Driven Architecture Use Cases
• Succeed With Event-Driven Implementations by Correcting Common Misconceptions
• Make EDA’s Events, Topics and Schemas Discoverable and Governable

Report Highlight: Select the Right Event Broker Technologies for your Event-Driven Architecture Use Cases

Gartner has found that “software engineers often try to use their preferred technology to solve all of their EDA use cases. This frequently stretches the technology beyond its functional design, leading to failed implementations.” ¹

Most often, their preferred technology might be the first one they used or had experience with, but as Gartner notes “Choosing the wrong technology for the use case can hamper implementation and lead to poor results.” ¹

The research goes on to identify the four primary/basic use cases that most requirements fall into:

• Application Messaging – This tends to be reliable message distribution within a single application domain. Gartner provides an example to illustrate this:
  “Example uses of application messaging are a component sending a query or update to a database and a web application sending a message to process a new order.” ¹
• Enterprise Messaging – This tends to be reliable message distribution between application domains. This may be within or between data centers, across branch locations, or across clouds and is generally delivered as a shared service. Gartner provides an example to illustrate this as well:
  “An example of the need for enterprise messaging would be the processing of a new order. This requires updates to the stock control, customer invoice and shipping systems. The order needs to be processed by each system to ensure customers receive their orders and get charged appropriately and to ensure another customer can’t buy the same item, however the systems are running in different domains and are managed by separate teams. Enterprise messaging provides a standardized way to pass data between the systems.” ¹
• Event Streaming – This is defined as providing reliable event ingestion and distribution or an ordered sequence of events. Again, Gartner gives an example:
  “Any change in status of a system can be used to form an event stream. Some examples are publishing changes in stock prices, a CRM system publishing that a customer address has changed and airports publishing flight departures and arrivals times.” ¹
• Edge Messaging – This aims to provide event distribution between devices and data centers. The example provided for Edge messaging reads:
  “Examples of edge messaging are appliances like refrigerators or washing machines notifying manufacturers of faults, oil pipelines sensor reporting on rate of flow of oil and car factory machines reporting current status like temperature.” ¹

To make selection straightforward, Gartner gives some very generalized advice for enterprise messaging and event streaming use cases, but notes that application and edge messaging may require more scrutiny to determine the best broker technology choice.

“Generally, you should select queue-orientated event brokers for enterprise messaging, and log-orientated for event streaming use cases. However, there is a degree of overlap between the four EDA use cases that make it harder to generalize about application messaging and edge messaging use cases where any type may be appropriate, depending on other requirements such as its scope, ownership model and maturity.” ¹

Overall, Gartner’s recommendation to organizations are to:

“Select the event broker technology that most directly meets the requirements of each of your initiatives. Avoid poor implementations by not using the same technology for all use cases and challenging if incumbent vendors and technologies are the best fit for your specific use case.” ¹

Source: Gartner Presentation “Trends and Use Cases for Event-Driven Architecture”, Andrew Humphreys, May 2023

Report Highlight:  Succeed With Event-Driven Implementations by Correcting Common Misconceptions

This report further reinforces the prior guidance by stating:

“Avoid the implementation challenges of a one-size-fits-all approach by identifying your organization’s core use cases and choosing the best-fit event-broker technology for each.” ²

The research then goes on to outline fundamental activities an organization should undertake in their quest for the best-fit event broker.  The high-level activities include:

• Identify Your Organization’s Core Use Cases and Choose the Best-Fit Event Broker Technology for Each
• Ensure Engineering Teams Understand the Event-Driven Use Case Before Choosing Technologies
• Encourage Team Members to Match the Solution to the Problem

Report Highlight: Make EDA’s Events, Topics and Schemas Discoverable and Governable

In this report, Gartner notes there has been a popularity surge in EDA, which according to the 2022 Gartner Software Engineering Survey is used by 92% of organizations polled. As a result of this, “event-driven architecture’s popularity has led to a surge or events, topics and schemas, and along with it greater discovery and governance challenges” ³.

To address this, Gartner advises that organizations connect relevant personas to the key discovery and governance capabilities needs in an event management toolset to meet their business outcomes.   They outline the personas and capabilities in the following figure ³:

From a persona perspective:

• Platform engineers tend to be concerned about making sure the platform runs reliably; therefore they are interested in most capabilities of event management.
• Data owners manage quality and usage or datasets; therefore they are most interested in a portal, lineage, and both schema and event registries.
• Architects are involved in creating the design standards, guidelines and policies so they tend to be interested in managing topic and schema lifecycle, as well as a topic registry (or catalog).
• Application developers and Data engineers have similar goals and therefore are generally interested in a searchable catalog for topics, events and schemas as well as a self-service way to register applications with event topics or create data pipelines.
• Quality Assurance want to know what happened and where so they most benefit from capabilities that provide lineage and inspection of what happened, where and when.
• Business Technologists tend to live outside the IT department and create technology or analytics for internal or external business where they are best enabled by low-code options and visual dashboards to create business value on top of event-driven implementations.

Gartner then makes the point that event management capabilities should be viewed as analogous to API management. Ie. they should have:

• A searchable catalog for events, topics and schemas;
• Lineage or topology capabilities to view end-to-end flows of data to understand upstream dependencies and reduce breaking changes;
• Inspection capabilities for troubleshooting and testing.

In Summary

Gartner’s advice on event brokers has remained consistent and can be summarized as follows:

• Understand your use case(s) and any unique requirements they might entail;
• Understand the capabilities of the various event broker technologies and match those to your use case requirements – do not just blindly leverage your existing (or preferred) technology without first matching them against your requirements because not all event brokers are created equally and picking the wrong one will cost you more in the end.
• Manage and govern your event driven assets (topics, event, schemas) as you would your REST-based API assets, by using an event management suite that is aligned to the personas in need and again match the capabilities to the requirements.

That’s some sound advice from the world’s leading authority in IT advisory.

Footnotes

¹ Source: Gartner “Select the Right Event Broker Technologies for Your Event-Driven Architecture Use Cases”, 19 June 2023, Andrew Humphreys, Keith Guttridge, Max van den Berk
² Source: Gartner “Succeed With Event-Driven Implementations by Correcting Common Misconceptions”, 30 May 2023, Max van den Berk, Keith Guttridge, Andrew Humphreys
³ Source: Gartner “Make EDA’s Event, Topics and Schemas Discoverable and Governable”, 25 Oct 2023, Max van den Berk , Andrew Humphreys, Keith Guttridge

The post Gartner’s Latest Advice on How to Choose an Event Broker appeared first on Solace.