APIs Inside the Enterprise
It’s been a few years now since the software industry popularized the use of the term “API” to mean a business capability exposed as a technical service for developers to use. This use of API began as a backlash against the heavy-handed drive towards formalized SOA governance practices most often driven by enterprise architecture teams.
In 2009, Gartner Group’s Anne Thomas-Manes famously declared that SOA was dead, before going on to say that it was actually still very much alive, albeit in a different form. We can now look back on this and realize that we have come full circle, and that APIs are rapidly becoming the way in which organizations can realize the benefits of SOA.
A couple of years ago, a large portion of the industry would have branded me a heretic for claiming that API was just another name for web service, making all sorts of arguments about REST vs. SOAP, JSON vs. XML, etc. Today, however, this statement is likely to be more widely accepted, as developers are less focused on protocol specificity and more focused on how APIs can meet a wide range of business needs. The modern use of the term “API” originally implied a RESTful service, but over the last few years, pragmatism has won out over purism, and so while the vast majority of APIs use HTTP as a transport, pure RESTful services are in a small minority. Most people would now accept that the term “API” can refer to services over many protocols (HTTP, AMQP, JMS) with a wide range of exchange patterns (RESTful, RPC, synchronous, asynchronous), and an even wider array of content types, the most common still being JSON, XML, and specific XML variants like SOAP, Accord, HL7, and more.
Accepting that API is just another name for service is really a technical stance, though. I would argue that there are certain distinctions between types of service. I like to think of services as being in three buckets (I’m sure the future will bring more):
- SOA services (most often SOAP): provider-centric, often implemented with an “if I build it, they will come” mentality, the intent being to reduce IT spending by promoting reuse of assets
- APIs (often REST/JSON): consumer-centric, often driven by a product management approach, the goal being to drive new business opportunities by sharing business capabilities with partners and external developers
- Microservices: application-centric, built to deliver a specific feature for an application in a massively scalable and fully independent way. This is best explained in Martin Fowler and James Lewis’ seminal article
There is really no argument that, on a technical level, all three of these “definitions” refer to services of one kind or another, and while there are specific requirements that might lead to one technology being a more common approach for one or other of these types, any of the service types could be implemented in pretty much any of the available technologies.
This leads me back to the original theme of this article: APIs inside the enterprise. We are now starting to see companies adopting the kinds of concepts and technologies that have made APIs popular as a business construct inside their enterprise boundaries.
One way to look at this trend is to examine the things that went well and poorly with old-school SOA initiatives, and see how the application of consumer-centric technologies and ideas can improve things. I’ll get to that in a later blog. For now, let’s look at why enterprises want to use APIs inside the enterprise. In other words, let’s revisit some of the old excitement behind SOA, only this time with a more modern twist:
Cost and time efficiency
One of the primary reasons IT organizations embarked on SOA initiatives was to minimize duplication of effort by providing a library of shared assets (business services) that application teams could draw upon instead of constantly reinventing the wheel. SOA detractors often point to the failure of these initiatives to really drive significant reuse, especially compared to the heavyweight processes and infrastructure used to implement them. There are many reasons why SOA initiatives may not have delivered on their promises – some of them related to technology – but regardless of these failures, it’s hard to ignore the promise of significant cost and time savings achievable with a well-managed reuse program.
Perhaps by combining modern consumer and application-centric concepts with IT-driven provider-centric ideals, companies could start to realize some of this potential, making new application development faster and less expensive by using existing services rather than building everything from scratch every time.
Consistency of data
Every time a developer builds the same function in a different application, they increase the risk of getting different results from what is supposed to be the same thing. Take a simple but all too common example of a customer database: every company has at least one database of all their customers, but pretty much every time a new application comes along, the designers decide that the existing customer database doesn’t quite fit their needs, so they build a new one, sometimes trying to synchronize data, sometimes just starting from scratch. What if there were a single source of truth about customer data, and this source of truth could be readily extended to meet the requirements of new applications? This way, companies would only have to update changing data in one location, and customers would be able to much more easily access consolidated information.
The challenge here is twofold:
- Convincing new application developers that existing services are good enough to meet their needs
- Being agile enough to extend existing services to meet changing needs
This is an area where modern developer portals can help enormously. More on this later.
Application portfolio rationalization
Technologies may become obsolete, but they never die. The mainframe is the classic example. In the months leading up to Y2K, we were all convinced that this would finally see the end of the mainframe. But guess what? Pretty much every large enterprise continues to spend as much, if not more, on their mainframes today than they did in the ’90s. Why is this?
Partly it’s because of the difficulty of replacing old applications. Systems come to rely on them and use highly proprietary mechanisms to integrate with them, making it very difficult to replace core systems. What if you could abstract the backend implementation from the consuming application by putting a service layer between them? This was the original intent behind the ESB (Enterprise Service Bus). Although, as with many SOA constructs, ESBs became overly burdensome to operate and manage, and became legacy systems as much as the backend applications they were designed to abstract.
What we’re seeing today is that more of the backend systems themselves are surfacing standards-based interfaces (services) that can be readily consumed by modern applications, reducing direct dependencies and making it easier to contemplate replacing, or at least modernizing the original applications. The trick is making it easier for developers to consume these services than it is to directly integrate with the backend.
This is a bit of a tricky one. Many developers and architects physically recoil when they hear the word “governance,” but to an enterprise architect, and to a CIO, governance has a lot of value. Governance run amok is a terrible thing to behold and will quickly ensure the failure of any SOA initiative, but an effective governance program will often mean the difference between success and failure. At its core, an effective governance program helps ensure you build the right things (services), build them right, and build them to run right.
Governance applies just as much to API initiatives as it did to SOA programs. As API adoption moves into the enterprise, we have to ensure that we manage the proliferation of APIs properly. In an external API initiative, companies have product managers in charge of their APIs and a business justification for new APIs and changes to their APIs. This is governance. These same principles apply inside the enterprise boundary.
In future posts, we’ll take a look at how the evolution of the technologies used to drive API and microservices adoption can be applied to help ensure the success of internal API (SOA by another name) initiatives.