Microservices: An ESB Anti-Pattern
I've been thinking a bit about microservices (μServices) recently. My immediate reaction is to think; "Isn't this just yet another new term for the same stuff: Web Services->SOA->APIs->Microservices?" Followed shortly by the thought, "Well, yes it is, but there are some important differences/distinguishing factors."
ESB vs. Microservices
What is an ESB?
ESB stands for Enterprise Service Bus and is an architecture. In ESB, a bus-like infrastructure hosts a set of principles and rules for the integration of multiple applications.
What's the Difference Between ESB and Microservices?
Microservices is an evolutionary paradigm born out of the need for simplicity (i.e. get away from the enterprise service bus) and alignment with agile (think DevOps) and scalable (think Containerization) development and deployment architectures. Martin Fowler and James Lewis defined nine core characteristics of microservices in their seminal post:
- Componentization via Services
- Organized around Business Capabilities
- Products not Projects
- Smart endpoints and dumb pipes
- Decentralized Governance
- Decentralized Data Management
- Infrastructure Automation
- Design for failure
- Evolutionary Design
In this post I'm going to focus solely on the concept of smart endpoints, dumb pipes. We'll get into others later on in what I expect will become a series of posts on microservices.
What is Smart Endpoints, Dumb Pipes?
The idea of “smart endpoints, dumb pipes” is that you should minimize the intelligence in your infrastructure, putting more of an onus on the services to implement their own functional, and non-functional capabilities; and on the service consumers to understand how to consume each of the microservices. This is essentially the enterprise service bus anti-pattern, and is a result of too much dependence on ESBs for non-functional requirements, and in some cases even for business logic. With the enterprise service bus, developers are encouraged to put more and more logic and intelligence into the central infrastructure (the pipes) and less and less into the services and the applications (the endpoints) that consume them. The microservices movement seeks to reverse this trend, in many ways getting back to the original intent of SOA. This would be a good time to point out that ESB and SOA are not synonymous. In fact, Akana, as a leading provider of SOA Governance solutions for many years, does not offer an ESB as part of our product portfolio.
The interesting question this growing backlash against ESBs raises for me is "if everyone hates ESBs so much, why did they become so ubiquitous?" The answer is that they deliver some very important, in some cases critical, capabilities, especially for non-functional requirements like security, monitoring, and mediation. The problem is that they have traditionally dragged a lot of baggage along with the value. Many of the ESBs emerged from integration platforms, and the market started pushing more and more capabilities into the enterprise service bus with things like business process management, and business activity monitoring.
Back on the subject of microservices: I do wonder if we're all getting caught up in the excitement, and are overlooking the concerns about non-functional requirements that were part of the reason why ESBs emerged in the first place. One thing that delayed the adoption of web services the first time around was that the original standards lacked effective security models, and when the standards did evolve to address this, they became over complex and unwieldy. This was coupled with the challenge of understanding what was happening in a distributed system, especially when it came to troubleshooting performance or functional issues. The enterprise service bus provided a solution to this by centralizing an architecture that was designed to be decentralized - in hindsight this seems like a poor compromise.
Microservices face a different security challenge, there are plenty of security standards out there, the questions will be which one to choose, and how to inform the service consumer of which standard it needs to comply with. This lines up with the challenge of letting the consumer know where to find the various endpoint(s) for the microservices, the latter being especially important where services use dynamic scaling models.
Another challenge that hasn't gone away, and really hasn't changed much over the years is how to effectively monitor what's happening in a distributed system. We’ll come back to that topic later along with looking at some of the benefits and trade-offs of microservices in this series of posts.