Tag Archives: REST

REST is Simple, But Simple is not REST

Last December, William Vambenepe posted a provocative blog entry about RESTful APIs in the cloud. In it, he states that Amazon proves that REST doesn’t matter for Cloud APIs. Writing over at InfoQ, Mark Little re-ignited the controversy in his recent post asking Is REST important for Cloud? And Vambenepe responded to the re-invigorated commentary in Cloud APIs are like Military Parades. All of these posts are very much worth reviewing, and be sure to read the comments left by members of the community.

Amazon’s query APIs are certainly not textbook examples of the RESTful architectural style, a point the Restafarians are quick to make. But they are simple, understandable, and very easy to implement, which is largely the set of characteristics that attracted developers to REST in the first place. REST does have a satisfying elegance and undeniable philosophical appeal, but accessibility probably counts for more in the popularity stakes. If there is one theme that repeats throughout the history of computing, surely it is the story of simple and good enough winning the day.

Amazon’s APIs are not the reason that AWS is such a resounding success; it is the service offering that was responsible for that. However, the APIs were good enough that they didn’t hinder Amazon’s success, and this is the relevant point. Dig a little deeper, though, and Amazon’s API story offers some interesting insights.

Amazon, it turns out, is a good case study illustrating the evolution of Internet APIs. When AWS first appeared, Amazon engineers faced what at the time was an unresolved question of style: should a cloud publish SOAP services, or RESTful APIs? I’ve heard that question led to some pretty heated internal debates, and I don’t doubt this is true. Amazon lies in Microsoft’s backyard, and .NET in those days was very firmly in the SOAP camp. REST, however, was rapidly gaining mind share, especially due to the growing popularity of AJAX in Web 2.0 apps.

The engineer’s solution was to hedge, producing a mix of SOAP, a “REST-like” query (HTTP with query params as NVPs), and REST APIs somewhat in parallel. Developers gravitated toward the later two options so much more than SOAP that AWS left the orbit of WS-* for good. Amazon gave the market choice, and the market spoke decisively. This was an interesting experiment, and in performing it Amazon actually led the way—as they have in so many ways—for future cloud providers. Each new provider had the luxury of an answer to the question of API style: REST won (even if this wasn’t really REST). Now all an emerging cloud provider needed to do was to design their API according to current best practices, and this is something for which experience counts. Roy Fielding’s thesis may have come out in 2000, but the appreciation of what constitutes good RESTful design among developers is a much more recent phenomena, one that has benefited greatly from the growing popularity of this style.

APIs are of course simply an interface between components of a distributed application. Like any interface, properly versioned, they can evolve. One thing I’ve noticed lately is that our customers are increasingly using Layer 7 technology to refine their APIs as they come to better understand their own needs.

For example, we have a customer in the publishing industry that is actually adapting existing SOAP-based APIs to RESTful design principles. They use our SecureSpan Gateways to present a true REST facade, and adapt this to existing SOAP messages on-the-fly. This transformation is fairly mechanical; what is more important is that the exercise also gave their developers the opportunity to refine their API suite to better meet their needs. Many of their new RESTful APIs leverage SecureSpan’s orchestration capabilities to consolidate multiple SOAP calls behind a single REST call. The resulting API 2.0 is a better match to their current business—and best of all, they were able to do all of this without writing a line of new code.

The lesson here is that while we like to think that APIs are perfectly static and never change, the truth is that we rarely get interfaces right the first time. Requirements change, business moves, and our understanding advances. What is important is that your APIs can respond with agility to changing needs, because change, we all know, is inevitable.

REST Security Does Exist—You Just Need To Apply It

On the eve of the RSA conference this year, Chris Comerford and Pete Soderling published a provocative article in Computerworld titled Why REST security doesn’t exist. It’s a prelude to a talk the author’s are delivering at the conference. Their premise is that while good REST security best practices do indeed exist, developers just don’t seem to follow them.

Comerford and Sodering attribute this state of affairs to a combination of two things. First, REST lacks a well-articulated security model. Few would argue with this—REST, by virtue of its grassroots origins, suffers from a security just-do-it-like-the-web nonchalance that’s certainly done it no favors.

The second issue concerns developers who tend to rush implementation without giving due consideration to security. Truthfully, this is the story of security across all of IT, but I might suggest that with REST, the problem is especially acute. The REST style owes much of its popularity to being simple and fast to implement, particularly when faced with the interest-crushing complexity and tooling demands of the WS-* stack. It’s reasonable to think that in the enthusiastic dash to cross the working application finish line, that security is conveniently de-emphasized or forgotten altogether.

REST, of course, can be secured, and the author’s offer sound advice to accomplish this deceptively simple task. They recommend that API developers:

  • “Do employ the same security mechanisms for your APIs as any web application your organization deploys. For example, if you are filtering for XSS on the web front-end, you must do it for your APIs, preferably with the same tools.
  • Don’t roll your own security. Use a framework or existing library that has been peer-reviewed and tested. Developers not familiar with designing secure systems often produce flawed security implementations when they try to do it themselves, and they leave their APIs vulnerable to attack.
  • Unless your API is a free, read-only public API, don’t use single key-based authentication. It’s not enough. Add a password requirement.
  • Don’t pass unencrypted static keys. If you’re using HTTP Basic and sending it across the wire, encrypt it.
  • Ideally, use hash-based message authentication code (HMAC) because it’s the most secure. (Use SHA-2 and up, avoid SHA & MD5 because of vulnerabilities.) “

I agree with this advice. And just to demonstrate how easy it is to implement, I’ve constructed a simple policy for the Layer 7 Technologies SecureSpan Gateway demonstrating their directives:

In this policy, I’m ensuring that the REST client is using SSL for three things: confidentiality, integrity, and server authentication. I could require client-side certificate authentication here, but instead I’m using HTTP digest, to emphasize the requirement to avoid using plain text HTTP basic or simple user keys. I’m authorizing access based on group membership here, restricting access to members of the sales group.

Finally, I’ve added a scan for cross site scripting attacks.

In the interest of deeper vigilance, I’m also searching for PHP and shell injection signatures. This is admittedly broad, but it covers me in case the developer of the service changes implementation without warning.

This last point—that there is an explicit separation made between developers and the security administrators writing and enforcing policy—is an important one. Developers will be developers: some will be rigorous about implementing security best practices; others won’t be. The only way to manage this is to assume a defensive posture in service policy, both from the perspective of incoming transactions, but also around the services themselves. The best practice here is to externalize policy enforcement and assign dedicated security professionals to administer policy.

This defensive approach to securing REST services fits well with the spirit of Comerford and Soderling’s directives. It addresses, in particular, their point about leveraging peer-reviewed frameworks. This is precisely what Layer 7’s SecureSpan Gateway is—a peer-reviewed security framework offering great depth of functionality. SecureSpan is undergoing Common Criteria Review of its implementation, as well as the entire development process for the product. We’re certifying to EAL4+, which is particularly rigorous. This provides assurance that the technology is sufficiently robust for deployment at the highest levels of military and the government. Common Criteria is an arduous process, and going through it demonstrates Layer 7’s deep commitment to security. You should not ever consider a security gateway—for REST, or for XML messaging—that isn’t undergoing the Common Criteria evaluation. Remember, Common Criteria is a necessary stamp of approval for governments around the world; it should also be a basic requirement for you.

Try SecureSpan yourself, and see how you can implement robust application security and monitoring without changing code. Download an evaluation of the SecureSpan virtual appliance right here.

How to Secure REST and JSON

Here at Layer 7 we get asked a lot about our support for REST. We actually have a lot to offer to secure, monitor and manage REST-style transactions. The truth is, although we really like SOAP and XML here at Layer 7, we also really like REST and alternative data encapsulations like JSON. We use both REST and JSON all the time in our own development.

Suppose you have a REST-based service that you would like to publish to the world, but you are concerned about access control, confidentiality, integrity, and the risk from incoming threats. We have an answer for this: SecureSpan Gateway clusters, deployed in the DMZ, give you the ability to implement run time governance across all of your services:

Pictures are nice, but this scenario is best understood using a concrete example. For the services, Yahoo’s REST-based search API offers us everything we need–it even returns results in JSON format, instead of XML. Yahoo has a great tutorial describing how to use this. The tutorial is a little dated, but it’s simple, to the point, and the REST service is still available. Let’s imagine that I’m deploying a SecureSpan Gateway in front of the servers hosting this API, as I’ve illustrated above. The first thing I will do is create a very simple policy that just implements a reverse proxy. No security yet–just a level of indirection (click on the picture for detail):

This is just about as simple as a policy can get. Notice that the policy validator is warning me about a few potential issues. It’s pointing out that the transaction will pass arbitrary content, not just XML. Because I’m expecting JSON formatted data in the response, this is the behavior I expect. The validation is also warning me that this policy has no authentication at all, leaving the service open to general, anonymous access. We’ll address this in the next step.

I’ve explicitly attached this policy to the gateway URL:

 http://scottssg/yahooImageSearch

If need be, I could easily add wild card characters here to cover a range of incoming URLs. For this demonstration, I’m just running a virtual SecureSpan Gateway here on my Macbook; it’s not actually residing in the Yahoo DMZ, as would be the case in a real deployment. But from the perspective of an administrator building policy, the process is exactly the same regardless of where the gateway lives.

I’ve also placed a restriction on the listener to only accept HTTP GET verbs:

Now I can point my web browser to the gateway URL shown above, and get back a JSON formatted response proxied from Yahoo. I’ll use the same example in the Yahoo tutorial, which lists pictures of Madonna indexed by Yahoo:

http://scottssg:8080/yahooImageSearch?appid=YahooDemo&query=Madonna&output=json

This returns a list looking something like this:

{"ResultSet":{"totalResultsAvailable":"1630990", "totalResultsReturned":10, "firstResultPosition":1, "Result":[{"Title":"madonna jpg", ...

which I’ve truncated a lot because the actual list spans thousands of characters. The Yahoo tutorial must be fairly old; when it was written, there were only 631,000 pictures of the Material Girl. Clearly, her popularity continues unabated.

Now let’s add some security. I’d prefer that nobody on the Internet learns that I’m searching for pictures of Madge, so we need to implement some privacy across the transaction. I can drag-and-drop an SSL/TLS assertion into the top of my policy to ensure that the gateway will only accept SSL connections for this RESTful service. Next, I’ll put in an assertion that checks for credentials using HTTP basic authentication. I’ll use the internal identity provider to validate the username/password combination. The internal identity provider is basically a directory hosted on the SecureSpan Gateway. I could just as easily connect to an external LDAP, or just about any commercial or open source IAM system. As for authentication, I will restrict use of the yahooImageSearch REST service to members of the development group:

HTTP basic authentication isn’t very sophisticated, so we could easily swap this out and implement pretty much anything else, including certificate authentication, Kerberos, SAML, or whatever satisfies our security requirements. My colleague here at Layer 7, Francois Lascelles, recently wrote an excellent blog post exploring some of the issues associated with REST authentication schemes.

Let’s review what we this simple policy has given us:

  1. Confidentiality, integrity, and server (gateway) authentication
  2. Authentication
  3. Authorization
  4. Virtualization of the internal service, and publication to authorized users

This is good, but I’d like to add some more REST-specific constraints, and to filter out potential REST attacks that may be launched against my service. I can do this with two simple assertions: one that validates form field in HTML, and another that scans the content for various code injection signatures:

The form data assertion allows me to impose a series of tight constraints on the content of query parameters. In effect, it let’s me put a structural schema on an HTTP query string (or POST parameters). I’m going to be very strict here, and explicitly name every parameter I will accept, to the exclusion of all others. For the Yahoo search API, this includes:

  • appid
  • query
  • output
  • callback

The latter does some wrapping of the return request to facilitate processing in JavaScript within a browser:

Depending on my security requirements, I could also be rigorous with parameter values using regular expressions as a filter. I’ll leave that as an exercise for the reader.

Naturally, I’m concerned about REST-born threats, so I will configure the code injection assertion to scan for all the usual suspects. This can be tuned so that it’s not doing unnecessary work that might affect performance in a very high volume situation:

That’s it–we’re done. A simple 6 assertion policy that handles confidentiality, integrity, authentication, authorization, schema validation, threat detection, and visualization of RESTful JSON services. To call this, I’ll again borrow directly from the Yahoo tutorial, using their HTML file and simply change to URL to point to my gateway instead of directly to Yahoo:

<html>
<head>
<title>How Many Pictures Of Madonna Do We Have?</title>
</head>
</body>
<script type="text/javascript">
function ws_results(obj) {
alert(obj.ResultSet.totalResultsAvailable);
}
</script>
<script type="text/javascript" src="https://scottssg:8443/yahooImageSearch?appid=YahooDemo&query=Madonna&output=json&callback=ws_results"></script>
<body></body>
</html>

Still can’t get over how many pictures of Madonna there are.

I ran it a few times and here’s what it looks like in the dashboard. I threw in some policy failures to liven up the display:

So where can we go from here? Well, I would think about optimization of the policy. Depending on predicted loads and available hardware, we might want to check for code injection and validate the schema before performing authentication, which in the real world would likely call out to an LDAP directory. After all, if we are being fed garbage, there’s no sense in propagating this load to the directory. We can add SLA constraints across the service to insulate back end hosts from traffic bursts. We could also provide basic load distribution across a farm of multiple service hosts. We might aggregate data from several back-end services using lightweight orchestration, effectively creating new meta-services from existing components. SecureSpan Gateways provide over 100 assertions that can do just about anything want to an HTTP transaction, regardless of whether it contains XML or JSON data. You can also develop custom assertions which plug into the system and implement new functionality that might be unique to your situation. Remember: when you are an intermediate, standing in the middle between a client and a service–as is the case with any SecureSpan Gateway–you have complete control over the transaction, and ultimately the use of the service itself. This has implications that go far beyond simple security, access control, and monitoring.

Using URI Templates on XML Security Gateways

Earlier this fall, Anil John put out the following Twitter challenge:

“@Vordel, @layer7, @IBM_DataPower If you support REST, implement support for URI templates in XML Security Gateways”

Somebody brought Anil’s tweet to our attention this week, and Jay Thorne, who leads our tactical group, put together a nice example of just how to do this using SecureSpan Gateways.

URI templates are a simple idea to formalize variable expansion inside URI prototypes. A receiving system can then trivially parse out substituted components of the URI and use these as input. There’s an IETF submission here that describes the approach. It turns out that it was co-authored by my old friend and ex-IBM colleague Dave Orchard. Another co-author is Mark Nottingham, who I worked with at the WS-I. I guess I should have looked into this earlier. Sorry guys.

Here’s a real example of how URI templates work. Let’s begin with the template:

http://somesite/Template/{Noun}/{Verb}/{Object}

The braces represent variables for run time substitution. Suppose these variables equaled:

Noun=Dog
Verb=bites
Object=man

The result of the substitution would then be:

http://somesite/Template/Dog/bites/man

Which is pretty easy to parse up and process by a web application residing at http://somesite/Template

URI templates are also very easy to process on a SecureSpan Gateway. Here’s a policy that demonstrates how to access the variables in the URI. Basically, all this policy does is parse up the URI, audit the results, and echo the content back to the sender:

I’ll bind this policy to a particular URI range on my SecureSpan gateway virtual instance. This happens to be running with an HTTP listener on port 8080 on my Macbook Pro:

Now let’s walk through the policy in detail. The first assertion uses a regular expression to parse the URI.:

It places the content of the expansion variables Noun, Verb, Object into Vars[1] , Vars[2] and Vars[3] respectively. The next two assertions in the policy simply audit the resulting variables into the SSG audit subsystem; this can sink to cluster storage, or go off box. Finally, the policy populates a template XML document with the contents of the variables and echos this back to the original sender:

When I fire up my browser and hit the REST service http://scottssg:8080/Template/Dog/bites/man I get:

Obviously in a real policy, you would do something more interesting than just echoing parameters, such as route the message to a particular internal host based on template content. And you can certainly compose outgoing URIs using existing variable substitution capability in SecureSpan. The take away is this is very simple to implement, and I think that it highlights that here at Layer 7, we think that supporting RESTful services is just as important as supporting SOAP.

Integrating SecureSpan Gateways and Sun’s OpenSSO

François Lascelles, who is Technical Director for Europe here at Layer 7, has just published an excellent article on Sun’s Developer Network site titled Delegating XML Gateway Runtime Authorization to OpenSSO. It goes into detail about how entitlements in Sun’s OpenSSO can be enforced for Web services, XML, and REST transactions using a SecureSpan Gateway and OpenSSO server.

This combination of Policy Decision Point (PDP)–in this case, OpenSSO–and Policy Enforcement Point (PEP)– the Layer 7 SecureSpan Gateway–is a common deployment pattern for us. Most organizations have already made PEP PDPan investment in Identity and Access Management (IAM) infrastructure; however, this is not sufficient on its own for SOA access control. That’s where Layer 7 can help. Deployed in combination with an IAM system like OpenSSO, SecureSpan does the heavy lifting of XML processing and enforcement, but delegates the access control decision process (and often identity token validation) to the existing, familiar IAM infrastructure. It’s a powerful combination, and one that extends existing investment in IAM into the SOA world.

Over the past seven years, we’ve built connectors into virtually all of the IAM systems out there. When we built SecureSpan, we were careful to build an effective framework for authentication and authorization so that it’s easy to build connectors into different systems. This is important, because unfortunately the IAM marketplace evolved rapidly and without a lot of standardization.

Have a look at François’ article. He’s been with the company since it’s beginning, and has as broad a perspective on this area as you will find.

eBizQ Forum Question: Are Web Services Protocols Such as SOAP and REST and AJAX Effective for Building SOA Off of Mainframe, Large Systems or Legacy Environments?

Joe McKendrick poses this interesting question on the eBizQ forum. The question of when to use MOM, SOAP, REST is one that I’m really interested in.