API Gateway as an Integration Server
In my last post, I gave a brief history of integration technologies and explained why you need an integration server. In this post, I explain how an API Gateway fills the integration role better than any other technology today, including the differences between an ESB and API Gateway, along with scenarios where you should (and should not) use one.
- APIs Inside the Enterprise
- Using an API Developer Portal as a Service Catalog
- Integration Technologies: From EAI to API Gateways
- API Gateway as an Integration Server
Not All Services Are Created Equal
Often the functions exposed as services by applications are too granular to be of much use to other applications. One of the roles often filled by an ESB or other integration solution is to orchestrate or aggregate multiple services together to create a higher-level business service that offers meaningful capabilities to consumers. Sure, you could continue to do this in an ESB, but again, the features in an API Gateway are often much better suited and much easier to use for this type of activity. This post on API Orchestration describes how the right API Gateway solution allows you to create APIs as orchestrations of other services.
All Your Data Is Not Always in One Place
An extremely valuable capability offered by integration solutions is aggregation of data from different sources. An obvious example is customer data. In many cases, data about customers is distributed across multiple applications. Think about your banking data, you likely have checking, savings, investment, mortgage, and maybe many other linked accounts. These are most likely not managed by the same backend applications, but your web or mobile application wants to be able to make a single API call to retrieve balance and status information for all these accounts. You could do this in an ESB, but these capabilities are often more easily and efficiently provided by your API Gateway solution. I describe an example of this type of use in this post about using the API Gateway as an Application Controller for a microservices application.
I just described three kinds of scenarios where integration still plays a valuable role and discussed how an API Gateway fills that role better than existing technology solutions. Now I’ll dive a bit deeper into the difference between an ESB and an API Gateway.
What’s the Difference Between an ESB and an API Gateway?
This is definitely a bit of a “how long is a piece of string” question. There are many different types of ESB, ranging from a simple rebrand of a legacy EAI platform through a newly built (in the mid 2000’s) product. Most of them share certain characteristics that render them unsuitable for API integration:
Not API native
Most ESBs come from a time where SOAP was dominant and they still use it as a primary communication protocol, often with RESTful capabilities loosely-bolted on. Don’t get me wrong, I think SOAP still has its place and is better suited than RESTful approaches for some scenarios, but an API integration platform needs a deep understanding of modern API protocols, content-types, security standards and approaches, and definition languages.
Prescriptive not declarative
ESBs tend to require developers to write code to manage even fairly simple mediation tasks, as they act in a prescriptive manner, doing exactly what they are instructed to do. A good API Gateway abstracts interfaces from implementations, and further abstracts policies allowing for a configuration-driven approach to integration.
Network edge deployments
A core function of an API Gateway is threat management and prevention. API Gateways provide extensive security capabilities that are typically missing from an ESB. This means that an ESB is not suitable for network edge (DMZ) deployment.
Why is this such an issue for internal integration?
Because the definition of internal has changed. In many cases, with enterprise adoption of cloud platforms, the system you’re integrating with is distributed across multiple datacenters, business partners, and cloud providers. You need an integration solution that provides the protection you need while still offering the rich integration capabilities you want.
Why Replace Existing Systems With an API Gateway?
I had a whole section planned out to talk about this, but I think I’ve made my case above. API Gateways are:
- Declarative – easier to use, less expensive to create integrations
- More efficient – higher performance requiring less infrastructure
- More secure – suitable for deployment in the DMZ
- API native – directly supporting modern applications
When Should You NOT Use an API Gateway?
I’ve presented a compelling (I think) argument for why you should start using an API Gateway instead of an integration server. There are still some enterprise integration patterns (good book by the way) where the API Gateway isn’t the right choice, although I would argue that perhaps you should be looking to modernize your approach to ditch old patterns.
API Gateways like to work in a stateless manner offering high-performance and seamless scaling. Long-running transactions consume resources and force a different model in the Gateway. It’s not that API Gateways can’t handle long-running transactions, it’s more that there are often better approaches.
Human interaction and workflow
Where transactions involve human interaction (a special form of long-running transaction) you should really be thinking about breaking the transaction up into multiple different API calls, rather than trying to implement the workflow in the API Gateway. This is one of the areas where use of ESBs got a bit out of hand in many organizations.
I’m trying to be a bit careful on this one. Most API Gateways do not include their own messaging system, but a good API Gateway can act as an intermediary between the messaging system, and a really good one can even maintain the guaranteed “once and once-only” heuristic of a messaging system. Where API Gateways come into their own is in creating APIs (REST/JSON for example) from an application that listens on and writes to a queue. Good Gateways can also do this the other way around, listening on queues themselves and acting appropriately when they see a new message. All this said, let me reiterate that an API Gateway will not typically include its own message broker, and this can be a good thing.
This is another fine line. In the past, we would always say that a Gateway should stick to syntactic mediation (think in terms of dealing with the envelope of a message, or the packaging of a parcel) and should not involve itself in semantic mediation (handling the meaning of the content). A classic example would be transformation of a purchase order form from one company’s format to another. Today this has changed somewhat. Many Gateways offer strong content mediation (even if it’s just XMLJSON) and some even offer sophisticated mechanisms for converting documents from one semantic form to another. Still, be careful about how sophisticated you want to make your document mappings in the Gateway as it can quickly become a management challenge.
I’ve covered a bit of integration technology history and discussed some scenarios where an API Gateway provides an excellent integration solution. I’ve talked about the differences between an ESB and an API Gateway and have shared a few thoughts on some areas where you need to be careful using an API Gateway as an integration solution.
So, now that you’re armed with all this information (or maybe I should call it opinion), what should you do next?
I suggest looking at your mainstream integration scenarios to see if they would be better served by an API Gateway. Gather a list of the ways you use integration today and prioritize it. Then work through the list, examining exactly what your current solutions are doing and compare them with the capabilities of a modern API management platform. At the very least ensure that when you look at new projects, you take advantage of new capabilities rather than continuing to pour money into an already aging and expensive infrastructure.