Tag Archives: APIs

Clouds Down Under

When I was young I was fascinated with the idea that the Coriolis effect—the concept in physics which explains why hurricanes rotate in opposing direction in the southern and northern hemispheres—could similarly be applied to common phenomenon like water disappearing down a bathtub drain. On my first trip to Cape Town many years ago I couldn’t wait to try this out, only to realize in my hotel bathroom that I had never actually got around to checking what direction water drains in the northern hemisphere before I left. So much for the considered rigor of science.

It turns out of course that the Coriolis effect, when applied on such a small scale, becomes negligible in the presence of more important factors such as the shape of your toilet bowl. And so, yet another one of popular culture’s most cherished myths is busted, and civilization advances ever so slightly.

Something that definitely does not run opposite south of the equator turns out to be cloud computing, though to my surprise conferences down under take a turn in the positive direction. I’ve just returned from a trip to Australia where I attended the 2nd Annual Future of Cloud Computing in the Financial Services, held last week, held in both Melbourne and Sydney. What impressed me is that most of the speakers were far beyond the blah-blah-blah-cloud rhetoric we still seem to hear so much, and focused instead on their real, day-to-day experiences with using cloud in the enterprise. It was as refreshing as a spring day in Sydney.

Greg Booker, CIO of ANZ Wealth, opened the conference with a provocative question. He simply asked who in the audience was in the finance or legal departments. Not a hand came up in the room. Now bear in mind this wasn’t Microsoft BUILD—most of the audience consisted of senior management types drawn from the banking and insurance community. But obviously cloud is still not front of mind for some very critical stakeholders that we need to engage.

Booker went on to illustrate why cross-department engagement is so vital to making the cloud a success in the enterprise. ANZ uses a commercial cloud provider to serve up most of its virtual desktops. Periodically, users would complain that their displays would appear rendered in foreign languages. Upon investigation they discovered that although the provider had deployed storage in-country, some desktop processing took place on a node in Japan, making this kind of a grey-area in terms of compliance with export restrictions on customer data. To complicate matters further, the provider would not be able to make any changes until the next maintenance window—an event which happened to be weeks away. IT cannot meet this kind of challenge alone. As Randy Fennel, General Manager, Engineering and Sustainability at Westpac put it succinctly, “(cloud) is a team sport.”

I was also struck by a number of insightful comments made by the participants concerning security. Rather than being shutdown by the challenges, they adopted a very pragmatic approach and got things done. Fennel remarked that Westpac’s two most popular APIs happen to be balance inquiry, followed by their ATM locator service. You would be hard pressed to think of a pair of services with more radically different security demands; this underscores the need for highly configurable API security and governance before these services go into production. He added that security must be a built-in attribute, one that must evolve with a constantly changing threat landscape or be left behind. This thought was echoed by Scott Watters, CIO of Zurich Financial Services, who added that we need to put more thought into moving security into applications. On all of these points I would agree, with the addition that security should be close to apps and loosely coupled in a configurable policy layer so that over time, you can easily address evolving risks and ever changing business requirements.

The entire day was probably best summed up by Fennel, who observed that “you can’t outsource responsibility and accountability.” Truer words have not been said in any conference, north or south.

Introducing Layer 7’s OAuth Toolkit

“If your tools don’t work for you, get rid of them,” is a simple creed I learned from my father in the workshop. Over the years, I have found it is just as relevant when applied to software, where virtual tools abound, but with often-dubious value.

OAuth is an emerging technology that has lately been in need of useful tools, and to fill this gap we are introducing an OAuth toolkit into Layer 7’s SecureSpan and CloudSpan Gateways.  OAuth isn’t exactly new to Layer 7; we have actually done a number of OAuth implementations with our customers over the last two years. But what we’ve discovered is that there is a lot of incompatibility between different OAuth implementations, and this is discouraging many organizations from making better use of this technology. Our goal with the toolkit was to provide a collection of intelligently parameterized components that developers can mix-and-match to reduce the friction between different implementations. And thanks to the generalization that characterize the emerging OAuth 2.0 specification, this toolkit helps to extend OAuth into interesting new use cases beyond the basic three-legged scenario of version one.

I have to admit that I was suspicious of OAuth when it first appeared a few years ago. So much effort had gone into the formal specification of SAML, from core definition to interop profiles, that I didn’t see the need for OAuth’s one use case solution and had little faith in the rigor of such a grass roots approach. But in time, OAuth won me over; it fits well with the browser-centric, simple-is-better approach of the modern Internet. The mapping to more generalized, token server-style interactions in the new version of the spec appeals to the architect in me, and the opening up of the security token payload indicates a desire to play well with existing infrastructure, which is a basic enterprise requirement.

However, adding extensibility to OAuth will also bring about this technology’s greatest challenge. The 1.0a specification benefitted enormously from laser focus on a use case so narrow that it was a wonder it gained the mindshare that it did. OAuth in 2011 has no such advantage—generalization being great for architects but hell for standards committees and vendors. It will be interesting to see how well the OAuth community satisfies the oftentimes-conflicting agendas of simple, standard, and interoperable.

Here at Layer 7 we predict a bright future for OAuth. We also think it’s very useful today, which is why we introduced a toolkit instead of a one-size-fits-one approach. We see our customers using OAuth in concert with their existing investments in Identity and Access Management (IAM) products, such as IBM’s Tivoli Access Manager  (TAM) or Microsoft’s Active Directory (AD). We see it being used to transport SAML tokens that require sophisticated interpretation to render entitlement decisions. Taking a cue from OAuth itself, the point of our toolkit is to simplify both implementation and integration. And the toolkit’s parameterization helps to insulate the application from specification change.

I’ll be at the Gartner/Burton Catalyst show this week in San Diego where we’ll be demonstrating the toolkit. I hope you can drop by and talk about how it might help you.

Why Cloud Brokers Are The Foundation For The Resilient API Network

Amazon Web Services crashed spectacularly, and with it the illusion that cloud is reliable-by-design and ready for mission-critical applications. Now everyone knows that cloud SLAs fade like the phosphor glow in a monitor when someone pulls the plug from the wall. Amazon’s failure is an unfortunate event, and the cloud will never be the same.

So what is the enterprise to do if it can’t trust its provider? The answer is to take a page from good web architecture and double up. Nobody would deploy an important web site without at least two identical web servers and a load balancer to spray traffic between them. If one server dies, its partner handles the full load until operators can restore the failed system. Sometimes the simplest patterns are the most effective.

Now take a step back and expand this model to the macro-level. Instead of pair of web servers, imagine two different cloud providers, ideally residing on separate power grids and different Internet backbones. Rather than a web server, imagine a replicated enterprise application hosting important APIs. Now replace the load balancer with a Cloud Broker—essentially an intelligent API switch that can distribute traffic between the providers based  both on provider performance and a deep understanding of the nature of each API.

It is this API-centricity that makes a Cloud Broker more than just a new deployment pattern for a conventional load balancer. Engineers design load balancers to direct traffic to Web sites, and their designs excel at this task. But while load balancers do provide rudimentary access to API parameters in a message stream, the rules languages used to articulate distribution policy are just not designed to make effective decisions about application protocols. In a pinch, you might be able to implement simple HTTP fail over between clouds, but this isn’t a very satisfactory solution.

In contrast, we design cloud brokers from the beginning to interpret application layer protocols and to use this insight to optimize API traffic management between clouds. A well-designed cloud broker abstracts existing APIs that may differ between hosts, offering a common view to clients decoupled from local dependencies. Furthermore, Cloud Brokers implement sophisticated orchestration capabilities so they can interact with cloud infrastructure through a provider’s APIs. This allows the broker to take command of applications the provider hosts. Leveraging these APIs, the broker can automatically spin up a new application instance on demand, or release under-utilized capacity. Automation of processes is one of the more important value propositions of cloud, and Cloud Brokers are means to realize this goal.

For more information about Cloud Brokers, have a look at the Cloud Broker product page at Layer 7 Technologies.

Blowing Holes in the Web of Trust

The Register today published an excellent summary of the latest issues with SSL. In the typically blunt and mordant style for which the publication is so famous, Dan Goodin illustrates how the gossamer-thin SSL web of trust is built on a superstructure of astonishingly dubious merit. It’s a wonder the whole thing works at all.

Have a careful read of How is SSL hopelessly broken? Let us count the ways and then re-examine the cartel certs that anchor your own web browsing experience. As you roll out your API strategy, make sure you deploy your SSL endpoints with certificates that were subject to organizational or (much better) extended validation. Encourage—or if you can, demand—that your API clients limit their trust stores to a small subset containing only the most legitimate CAs.

The opportunity is largely over in the browser world; affecting massive change there will only happen when individuals personally lose money on a grand scale. But APIs still have a chance to regain some level of trust through rigorous application of SSL best practices, and API providers and developers can take the initiative here.

The API Gateway As Endpoint

The US Customs and Immigration station at Point Roberts is a small building set on the edge of the coastal rainforest of Washington State. It is the last customs station in a vast western chain strung out along the 49 parallel. I enjoy crossing the border at Point Roberts. The abrupt change from sprawling Canadian suburbs to American rural countryside always appeals to me.

The customs station, despite it’s small size, offers a range of services from granting a visa to simply giving advice about where to find a good deal for gas in Point Roberts. It’s not simply a gateway controlling access into the US, but a provider of a broad range of services associated with crossing an international boundary.

There exists a similar duality with API gateways. Although the common deployment is as a border guard, protecting APIs hosted by an organization, an API gateway can also act as an endpoint providing valuable standalone services. Nearly all of my customers first purchase SecureSpan Gateways to fulfill the role the name implies: that is, an API or service gateway. The border guard deployment pattern is now so commonplace that I no longer need to evangelize it as I did in the early days of Layer 7, close to a decade ago. Architects recognize this pattern as an accepted best practice for securing and managing APIs. But like the Point Roberts border station, a SecureSpan Gateway can also provide services that have nothing to do with access control or transaction confidentiality, but provide value on their own, independent of any APIs they may be guarding.

Here’s a very simple example illustrating the endpoint pattern using a SecureSpan Gateway. Normally I might deploy my gateway in front of a web server, acting as an authenticating reverse proxy for my web pages. In this role, the gateway is responsible for access control, SSL management, audit, lightweight load balancing, etc. All classic gateway functions.

Figure 1: Gateway as boarder guard at the edge of a network. This is classic perimeter security.

But suppose I wanted to use the gateway itself as the web server? In other words, it’s not acting as a gateway, but the server endpoint in itself?

Figure 2: Gateway as service endpoint. Here the gateway is providing valuable APIs on its own without necessarily routing a request to an internal host.

It’s very simple to configure a SecureSpan Gateway as an endpoint. You simply create a policy that never routes a request onward to an internal host, but instead acts on the message content internally. I sometimes think of it as an internal loop back mechanism. I’ll build a policy implementing my web site, and add the assertion Return Template Response to Requester. I will bind this policy to the gateway URL http://localhost:8080/helloworld as shown in the screen snippet below from the SecureSpan management console:

With just one assertion, this is pretty much the simplest policy you can write, and so fitting for the time-honoured Hello World example. Let’s expand the assertion to examine at what the template actually contains:

Pretty plain-vanilla HTML. The only departure I’ve made from the deliberate austerity of the Hello World tradition is to add a time stamp to showcase the use of predefined context variables. There are dozens of context variables that are automatically set as the gateway processes a transaction (regardless of whether it’s actually being a gateway or an endpoint—remember we consider this just a policy implementation pattern, not a different capability). Context variables cover everything from accessing individual fields in the X509 certificate used for client-side SSL authentication to storing the IP address of a request sender. And of course, you can set your own context variables at any time within a policy.

Just to be complete, here’s what happens when I point my browser at the gateway-as-endpoint:

Now, before you think I’ve gone mad, I am not advocating that you deploy a web site like this; there are much better tools for that purpose. But the point I’m making is that there are legitimate endpoint services that do indeed lend themselves to implementation on a gateway. These are just a little more complicated than what I’ve shown here, but nevertheless simple to implement using the declarative policy language in SecureSpan. For example:

  • Signing Services: In the notary pattern, a gateway with an on-board Hardware Security Module (HSM) responsible for protecting key material can easily provide authoritative signing services for all kinds of data, from XML documents to arbitrary text.
  • Encryption Services: Same idea as signing, with the goal to avoid insecure proliferation of keys.
  • Validation Services: Ensure data conforms to a particular schema.
  • Threat Detection Services: Scan content for SQL injection, Cross Site Scripting (XSS), XML threats, or custom threat signatures.
  • Security Token Services (STS): Issue security tokens like SAML or OAuth tokens. Validate or exchange tokens.
  • Centralized Session or Caching Management: Cache frequently used data items according to a scheme that is tuned to your application.
  • Policy Decision Point (PDP) Services: Receive and act on SAMLP requests.
  • XACML Services: Act as a centralized XACML engine providing authorization services.

The list above is by no means exclusive; however, at Layer 7 we’ve implemented all of these at one time or another using SecureSpan Gateways. It’s an important design point for us that the only real difference between a gateway and an endpoint is how you define your policy.

The API Lab

Promotion is a problem faced by every API developer. Long nights of coding have given form to the stroke of genius you had six months ago in the cafe. You’ve just written the API that will serve as the front door into your application. But how do you document this so that your peers will use it—and hopefully make you rich in the process?

Java had Javadoc, an innovation that managed to strike a surprisingly effective balance between ease of use and systematization (three cheers for strong typing and static binding). Web services “solved” the interface definition problem with WSDL, a standard only its authors could love (and some of these won’t admit to participating in conception). To the registry crowd was left the task of devising clever ways to document a SOAP API around the torturous abstractions of WSDL so that humans could grok it as effectively as the machines. Few would argue that their solutions were, well, solutions.

RESTful APIs neither benefit nor suffer from formalized approach and automation. Every so often WADL pokes its head up, only to be knocked decisively back into its hole in a community game of Standards-Whac-A-Mole. REST is—and indeed, has always been—a style that resists any formalism that threatens its simplicity and accessibility. For this, above all, the community deserves praise.

But this does leave a hole around documentation. APIs tend to be described-by-wiki, which can be great or just awful, depending on a developer’s ability to write and discipline to update. Good ideas like ProgrammableWeb showcase both of these extremes; however, the site suffers in particular from out-of-date API descriptions which is too bad.

Outside of the eventual acceptance en masse of some kind of standardized wiki template for APIs (which I think is actually quite likely), we probably won’t see substantial change in API documentation soon. Minor changes, though, can still count for a lot, and it’s worth pointing these out.

Google, always an innovator to watch, create a lot of APIs. They recently published the Periodic Table of Google APIs, which as its name suggests is a clever way to visually categorize their API portfolio into groups like mobile, data, geo, etc. It’s a bit gimmicky, but it is also fun and actually inspired me to try some new Google APIs that I wasn’t aware of before.

So, isn’t it about time for more creative solutions around API documentation?

REST is Simple, But Simple is not REST

Last December, William Vambenepe posted a provocative blog entry about RESTful APIs in the cloud. In it, he states that Amazon proves that REST doesn’t matter for Cloud APIs. Writing over at InfoQ, Mark Little re-ignited the controversy in his recent post asking Is REST important for Cloud? And Vambenepe responded to the re-invigorated commentary in Cloud APIs are like Military Parades. All of these posts are very much worth reviewing, and be sure to read the comments left by members of the community.

Amazon’s query APIs are certainly not textbook examples of the RESTful architectural style, a point the Restafarians are quick to make. But they are simple, understandable, and very easy to implement, which is largely the set of characteristics that attracted developers to REST in the first place. REST does have a satisfying elegance and undeniable philosophical appeal, but accessibility probably counts for more in the popularity stakes. If there is one theme that repeats throughout the history of computing, surely it is the story of simple and good enough winning the day.

Amazon’s APIs are not the reason that AWS is such a resounding success; it is the service offering that was responsible for that. However, the APIs were good enough that they didn’t hinder Amazon’s success, and this is the relevant point. Dig a little deeper, though, and Amazon’s API story offers some interesting insights.

Amazon, it turns out, is a good case study illustrating the evolution of Internet APIs. When AWS first appeared, Amazon engineers faced what at the time was an unresolved question of style: should a cloud publish SOAP services, or RESTful APIs? I’ve heard that question led to some pretty heated internal debates, and I don’t doubt this is true. Amazon lies in Microsoft’s backyard, and .NET in those days was very firmly in the SOAP camp. REST, however, was rapidly gaining mind share, especially due to the growing popularity of AJAX in Web 2.0 apps.

The engineer’s solution was to hedge, producing a mix of SOAP, a “REST-like” query (HTTP with query params as NVPs), and REST APIs somewhat in parallel. Developers gravitated toward the later two options so much more than SOAP that AWS left the orbit of WS-* for good. Amazon gave the market choice, and the market spoke decisively. This was an interesting experiment, and in performing it Amazon actually led the way—as they have in so many ways—for future cloud providers. Each new provider had the luxury of an answer to the question of API style: REST won (even if this wasn’t really REST). Now all an emerging cloud provider needed to do was to design their API according to current best practices, and this is something for which experience counts. Roy Fielding’s thesis may have come out in 2000, but the appreciation of what constitutes good RESTful design among developers is a much more recent phenomena, one that has benefited greatly from the growing popularity of this style.

APIs are of course simply an interface between components of a distributed application. Like any interface, properly versioned, they can evolve. One thing I’ve noticed lately is that our customers are increasingly using Layer 7 technology to refine their APIs as they come to better understand their own needs.

For example, we have a customer in the publishing industry that is actually adapting existing SOAP-based APIs to RESTful design principles. They use our SecureSpan Gateways to present a true REST facade, and adapt this to existing SOAP messages on-the-fly. This transformation is fairly mechanical; what is more important is that the exercise also gave their developers the opportunity to refine their API suite to better meet their needs. Many of their new RESTful APIs leverage SecureSpan’s orchestration capabilities to consolidate multiple SOAP calls behind a single REST call. The resulting API 2.0 is a better match to their current business—and best of all, they were able to do all of this without writing a line of new code.

The lesson here is that while we like to think that APIs are perfectly static and never change, the truth is that we rarely get interfaces right the first time. Requirements change, business moves, and our understanding advances. What is important is that your APIs can respond with agility to changing needs, because change, we all know, is inevitable.

Defeating the Facebook Hack

This week, Facebook fell victim to hackers who managed to deface Mark Zuckerberg’s page, no doubt earning the perpetrators tremendous props within their own social community. Facebook quickly closed the door on that particular exploit, but by then of course the Internets were abuzz and the damage was done. The company quickly followed up with some unrelated security distractions: HTTPS, good for countering Firesheep (I love that name); social authentication instead of CAPTCHAs (this is actually interesting and plays to their strengths); and an announcement that this Friday is “Data Privacy Day” (Ouch).

Their aren’t many details available on the hack (the Guardian has a great investigation examining some of the clues that were left behind), but it appears that one particular API didn’t perform sufficient authorization on a POST. This is a common problem when you don’t make the configuration of basic security functions like AAA highly visible, auditable, and tunable. Leave security to the developers, and rigor is often overlooked because of a coder’s naturally intense focus on functionality. Decouple security out of the API, promoting security management to a first class citizen administered by internal experts—well, then you have a much greater chance to avoid embarrassing gaffs like this one. Consistency is the soul of good security, and the decoupling strategy makes consistency an achievable goal across all of an organization’s APIs. Specialize where necessary, but make everything highly visible to the experts so they can easily move from big picture to necessary minutia. Make security configuration highly declarative both to advance understanding and facilitate rapid change in response to evolving threats. This is how API management must be in 2011.

Facebook has its pick of the best and brightest these days—and they still face these problems. As other, less privileged organizations attempt to create APIs—and this is very much the trend of the day—we can expect to see many more such attacks. Having the CEO’s page defaced is certainly embarrasing. But in the long run, it’s a lot less expensive than a real privacy breach, or cleaning up from massive sustained fraud.

The time for do-it-yourself security and management of APIs has passed. This is the time to deploy professional solutions to the problem that have been proven out by the military, intelligence community, and health care—groups that have come to understand the best practices around API security out of absolute necessity.

To learn more, take a look at Layer 7 Technologies solutions for securing and managing APIs.

Upcoming Webinar: How To Implement Enterprise-scale API Management: The secret to making your business into a platform.

Jeffery Hammond, Principal Analyst with Forrester Research and I will be jointly delivering a webinar Tuesday, Sept 28th at 9am Pacific time. The topic we are discussing is API management and security. We’ll look at why APIs are important, and discuss the best practices for effectively leveraging these in your business.

Figure 1: The role of gateways in API management.

This promises to be a very good presentation, and I’d urge you to attend. We’re doing something a little different this time and delivering a much more interactive discussion than some of my past webinars. Since Jeffery and I are both traveling over the next few weeks, we’ve run through our rehearsals early. The material is top notch; Jeffery absolutely understands the issues organizations face as they attempt to expose core business applications using APIs. We are very much on the same page, and I have a strong feeling that this is going to be a very good show. I’m looking forward to it, and I hope you can join us.

You can register for this webinar here.