How to Secure REST and JSON

Here at Layer 7 we get asked a lot about our support for REST. We actually have a lot to offer to secure, monitor and manage REST-style transactions. The truth is, although we really like SOAP and XML here at Layer 7, we also really like REST and alternative data encapsulations like JSON. We use both REST and JSON all the time in our own development.

Suppose you have a REST-based service that you would like to publish to the world, but you are concerned about access control, confidentiality, integrity, and the risk from incoming threats. We have an answer for this: SecureSpan Gateway clusters, deployed in the DMZ, give you the ability to implement run time governance across all of your services:

Pictures are nice, but this scenario is best understood using a concrete example. For the services, Yahoo’s REST-based search API offers us everything we need–it even returns results in JSON format, instead of XML. Yahoo has a great tutorial describing how to use this. The tutorial is a little dated, but it’s simple, to the point, and the REST service is still available. Let’s imagine that I’m deploying a SecureSpan Gateway in front of the servers hosting this API, as I’ve illustrated above. The first thing I will do is create a very simple policy that just implements a reverse proxy. No security yet–just a level of indirection (click on the picture for detail):

This is just about as simple as a policy can get. Notice that the policy validator is warning me about a few potential issues. It’s pointing out that the transaction will pass arbitrary content, not just XML. Because I’m expecting JSON formatted data in the response, this is the behavior I expect. The validation is also warning me that this policy has no authentication at all, leaving the service open to general, anonymous access. We’ll address this in the next step.

I’ve explicitly attached this policy to the gateway URL:

 http://scottssg/yahooImageSearch

If need be, I could easily add wild card characters here to cover a range of incoming URLs. For this demonstration, I’m just running a virtual SecureSpan Gateway here on my Macbook; it’s not actually residing in the Yahoo DMZ, as would be the case in a real deployment. But from the perspective of an administrator building policy, the process is exactly the same regardless of where the gateway lives.

I’ve also placed a restriction on the listener to only accept HTTP GET verbs:

Now I can point my web browser to the gateway URL shown above, and get back a JSON formatted response proxied from Yahoo. I’ll use the same example in the Yahoo tutorial, which lists pictures of Madonna indexed by Yahoo:

http://scottssg:8080/yahooImageSearch?appid=YahooDemo&query=Madonna&output=json

This returns a list looking something like this:

{"ResultSet":{"totalResultsAvailable":"1630990", "totalResultsReturned":10, "firstResultPosition":1, "Result":[{"Title":"madonna jpg", ...

which I’ve truncated a lot because the actual list spans thousands of characters. The Yahoo tutorial must be fairly old; when it was written, there were only 631,000 pictures of the Material Girl. Clearly, her popularity continues unabated.

Now let’s add some security. I’d prefer that nobody on the Internet learns that I’m searching for pictures of Madge, so we need to implement some privacy across the transaction. I can drag-and-drop an SSL/TLS assertion into the top of my policy to ensure that the gateway will only accept SSL connections for this RESTful service. Next, I’ll put in an assertion that checks for credentials using HTTP basic authentication. I’ll use the internal identity provider to validate the username/password combination. The internal identity provider is basically a directory hosted on the SecureSpan Gateway. I could just as easily connect to an external LDAP, or just about any commercial or open source IAM system. As for authentication, I will restrict use of the yahooImageSearch REST service to members of the development group:

HTTP basic authentication isn’t very sophisticated, so we could easily swap this out and implement pretty much anything else, including certificate authentication, Kerberos, SAML, or whatever satisfies our security requirements. My colleague here at Layer 7, Francois Lascelles, recently wrote an excellent blog post exploring some of the issues associated with REST authentication schemes.

Let’s review what we this simple policy has given us:

  1. Confidentiality, integrity, and server (gateway) authentication
  2. Authentication
  3. Authorization
  4. Virtualization of the internal service, and publication to authorized users

This is good, but I’d like to add some more REST-specific constraints, and to filter out potential REST attacks that may be launched against my service. I can do this with two simple assertions: one that validates form field in HTML, and another that scans the content for various code injection signatures:

The form data assertion allows me to impose a series of tight constraints on the content of query parameters. In effect, it let’s me put a structural schema on an HTTP query string (or POST parameters). I’m going to be very strict here, and explicitly name every parameter I will accept, to the exclusion of all others. For the Yahoo search API, this includes:

  • appid
  • query
  • output
  • callback

The latter does some wrapping of the return request to facilitate processing in JavaScript within a browser:

Depending on my security requirements, I could also be rigorous with parameter values using regular expressions as a filter. I’ll leave that as an exercise for the reader.

Naturally, I’m concerned about REST-born threats, so I will configure the code injection assertion to scan for all the usual suspects. This can be tuned so that it’s not doing unnecessary work that might affect performance in a very high volume situation:

That’s it–we’re done. A simple 6 assertion policy that handles confidentiality, integrity, authentication, authorization, schema validation, threat detection, and visualization of RESTful JSON services. To call this, I’ll again borrow directly from the Yahoo tutorial, using their HTML file and simply change to URL to point to my gateway instead of directly to Yahoo:

<html>
<head>
<title>How Many Pictures Of Madonna Do We Have?</title>
</head>
</body>
<script type="text/javascript">
function ws_results(obj) {
alert(obj.ResultSet.totalResultsAvailable);
}
</script>
<script type="text/javascript" src="https://scottssg:8443/yahooImageSearch?appid=YahooDemo&query=Madonna&output=json&callback=ws_results"></script>
<body></body>
</html>

Still can’t get over how many pictures of Madonna there are.

I ran it a few times and here’s what it looks like in the dashboard. I threw in some policy failures to liven up the display:

So where can we go from here? Well, I would think about optimization of the policy. Depending on predicted loads and available hardware, we might want to check for code injection and validate the schema before performing authentication, which in the real world would likely call out to an LDAP directory. After all, if we are being fed garbage, there’s no sense in propagating this load to the directory. We can add SLA constraints across the service to insulate back end hosts from traffic bursts. We could also provide basic load distribution across a farm of multiple service hosts. We might aggregate data from several back-end services using lightweight orchestration, effectively creating new meta-services from existing components. SecureSpan Gateways provide over 100 assertions that can do just about anything want to an HTTP transaction, regardless of whether it contains XML or JSON data. You can also develop custom assertions which plug into the system and implement new functionality that might be unique to your situation. Remember: when you are an intermediate, standing in the middle between a client and a service–as is the case with any SecureSpan Gateway–you have complete control over the transaction, and ultimately the use of the service itself. This has implications that go far beyond simple security, access control, and monitoring.

On the Death of Design-Time Service Governance

Practically on the anniversary of Anne Thomas Manes now-famous SOA-is-Dead pronouncement, David Linthicum suggests we convene the vigil for design-time service governance. Dave maintains that cloud technology is going to kill this canonical aspect of governance because runtime service governance simply provides much more immediate value. Needless to say, rather than a somber occasion, Dave’s started more of a donnybrook. I guess it’s about time to get off of the bench and join in the fun.

The incendiary nature of is-dead statements often conceal  the subtle but important ideas behind them. Dave’s declaration is no different. What he’s really suggesting is that cloud will rapidly shift the traditional priorities with respect to services governance. In the big SOA era (before Jan 1, 2009), design-time governance was king. It fit nicely into the plan-manage-control discipline underpinning a large enterprise-scale SOA effort. And to support this, tool vendors rushed in and offered up applications to facilitate the process. Run time services governance, in contrast, was often perceived as something you could ignore until later–after all, on-premise IT has reasonably good security and monitoring in place. The technology might be old and the process a little byzantine, but it could tide you over. And if any internal staff were caught accessing services they shouldn’t be you could just fire them.

The cloud inverts this priority. If you deploy even one service into the public cloud, you must have runtime service governance in place. That one service needs to be hardened and protected as you would a DMZ-based application. You must have continuous visibility into its operation from the moment you turn it on. Run time governance simply cannot be put off.

The cloud is also about agility. We are attracted to the cloud because it offers rapid deployment, elastic computing, and that hard to refuse pay-for-what-you-need model. This same sense of agility is a leitmotiv throughout the entire development process. When you talk to teams that have been successful with the cloud, they are usually small, they rail against enterprise bureaucracy, and agile development process is in their DNA. They are comfortable and extremely efficient with the tools and process they have, and they don’t see a lot of reason to change this. Do they still need discipline, organization, process, communication, sharing? Absolutely–but they have this; it’s ingrained into their existing process. I see this in my teams developing products every day. It’s the mix of great management, excellent people, process where it’s been shown to be needed, and a basic set of tools we all rely on every day.

Will design-time services governance tools disappear completely? Of course not. But their value becomes apparent with scale, when you begin to approach “a lot of services” (and no,  nobody really knows what this means–probably more than a scoville, less than a parsec). The point is, in the cloud nobody starts there, but they do need to focus immediately on the threats.

In the end, governance priorities in the cloud come down to a pretty simple precept: Duplicating a piece of functionality probably won’t get you fired; leaving an open door that was used to compromise corporate data probably will.

Cloud Security Alliance Guidance v2 Released

Last week, the Cloud Security Alliance (CSA) released its Security Guidance for Critical Areas of Focus in Cloud Computing V2.1. This is a follow-on to first guidance document released only last April, which, gives you a sense of the speed at which cloud technology and techniques are moving. I was one of the contributors to this project.

The guidance explores the issues in cloud security from the perspective of 13 different domains:

Cloud Architecture

  • Domain 1: Cloud Computing Architectural Framework

Governing in the Cloud

  • Domain 2: Governance and Enterprise Risk Management
  • Domain 3: Legal and Electronic Discovery
  • Domain 4: Compliance and Audit
  • Domain 5: Information Lifecycle Management
  • Domain 6: Portability and Interoperability

Operating in the Cloud

  • Domain 7: Traditional Security, Business Continuity, and Disaster Recovery
  • Domain 8: Data Center Operations
  • Domain 9: Incident Response, Notification, and Remediation
  • Domain 10: Application Security
  • Domain 11: Encryption and Key Management
  • Domain 12: Identity and Access Management
  • Domain 13: Virtualization

I thought the domain classification was quite good because it serves to remind people that technology is only a small part of a cloud security strategy. I know that’s become a terrible security cliche, but there’s a difference between saying this and understanding what it really means. The CSA domain structure–even without the benefits of the guidance–at least serves as a concrete reminder of what’s behind the slogan.

Have a close look at the guidance.  Read it; think about it; disagree with it; change it–but in the end, make it your own. Then share your experiences with the community. The guidance is an evolving document that is a product of a collective, volunteer effort. It’s less political than a conventional standards effort (look though the contributors and you will find individuals, not companies). The group can move fast, and it doesn’t need to be proscriptive like a standard–it’s more a distillation of considerations and best practices. This one is worth tracking.

eBizQ: SOA in This Year and the Next

It’s that time when we look back on one year and forward to the next. Over at the eBizQ forum Peter Schooff asked about SOA’s past and future:

What Developments in SOA Are You Most Thankful For This Year?

What Do You Think Will be the Biggest Trend or Development for SOA in 2010?

Identity Buzz Podcast with Now Available: End-to-End Web Services Security

I recently had a great, freewheeling discussion with Daniel Raskin, Sun’s Chief Identity Strategist. Daniel runs the Identity Buzz podcasts. We talked about issues in identity and entitlement enforcement in SOA, compliance, and the problems you run into as you move into new environments like the cloud.

Daniel’s post about our podcast is on his blog.

You can download the podcast directly right here.

Upcoming Webinar: Controlling Your SOA Across The Global Enterprise and Cloud

I’ll be delivering a Webinar next week about Layer 7’s Enterprise Service Manager (ESM) product. ESM offers the global view of clusters of SecureSpan Gateways and the services under their management. It’s functions fall into three main areas:

Enterprise-scale Management

  • Centrally manage and monitor all Gateways and associated services across the extended enterprise and into the cloud

Automated Policy Migration

  • Centrally approve and then push policy to any Gateway across the enterprise, automatically resolving environmental discrepancies

Disaster Recovery

  • Remotely manage, troubleshoot, backup and restore all Gateways, supporting full disaster recovery

ESM is an important tool for managing SOA in the Enterprise, but it also plays a critical role when SOA moves to the cloud. In addition to extending an organization’s visibility and control, ESM provides critical tools for automating the migration of policy to new environments:

I’ll go into this process in detail in the webinar. Hope to see you there. Here is the official abstract:

Organizations have begun extending their SOA initiatives beyond traditional enterprise boundaries to encompass third-party, geographically remote, and even cloud-based resources. As a result, the complexity associated with migrating applications across these environments (for example, from development in India to test in the cloud to production in a hosted data center) has increased exponentially. In this webinar from Layer 7, you will learn how topology and identity issues between environments, geographies and settings (i.e., enterprise vs. cloud) can be easily resolved and even automated, dramatically reducing migration risk.

You can sign up for this webinar at http://www.layer7tech.com/main/media/webinars.html.

Using URI Templates on XML Security Gateways

Earlier this fall, Anil John put out the following Twitter challenge:

“@Vordel, @layer7, @IBM_DataPower If you support REST, implement support for URI templates in XML Security Gateways”

Somebody brought Anil’s tweet to our attention this week, and Jay Thorne, who leads our tactical group, put together a nice example of just how to do this using SecureSpan Gateways.

URI templates are a simple idea to formalize variable expansion inside URI prototypes. A receiving system can then trivially parse out substituted components of the URI and use these as input. There’s an IETF submission here that describes the approach. It turns out that it was co-authored by my old friend and ex-IBM colleague Dave Orchard. Another co-author is Mark Nottingham, who I worked with at the WS-I. I guess I should have looked into this earlier. Sorry guys.

Here’s a real example of how URI templates work. Let’s begin with the template:

http://somesite/Template/{Noun}/{Verb}/{Object}

The braces represent variables for run time substitution. Suppose these variables equaled:

Noun=Dog
Verb=bites
Object=man

The result of the substitution would then be:

http://somesite/Template/Dog/bites/man

Which is pretty easy to parse up and process by a web application residing at http://somesite/Template

URI templates are also very easy to process on a SecureSpan Gateway. Here’s a policy that demonstrates how to access the variables in the URI. Basically, all this policy does is parse up the URI, audit the results, and echo the content back to the sender:

I’ll bind this policy to a particular URI range on my SecureSpan gateway virtual instance. This happens to be running with an HTTP listener on port 8080 on my Macbook Pro:

Now let’s walk through the policy in detail. The first assertion uses a regular expression to parse the URI.:

It places the content of the expansion variables Noun, Verb, Object into Vars[1] , Vars[2] and Vars[3] respectively. The next two assertions in the policy simply audit the resulting variables into the SSG audit subsystem; this can sink to cluster storage, or go off box. Finally, the policy populates a template XML document with the contents of the variables and echos this back to the original sender:

When I fire up my browser and hit the REST service http://scottssg:8080/Template/Dog/bites/man I get:

Obviously in a real policy, you would do something more interesting than just echoing parameters, such as route the message to a particular internal host based on template content. And you can certainly compose outgoing URIs using existing variable substitution capability in SecureSpan. The take away is this is very simple to implement, and I think that it highlights that here at Layer 7, we think that supporting RESTful services is just as important as supporting SOAP.

Visualizing the Boundaries of Control in the Cloud

Two weeks ago, I delivered a webinar about new security models in the cloud with Anne Thomas Manes from Burton Group. Anne had one slide in particular, borrowed from her colleague Dan Blum, which I liked so much I actually re-structured my own material around it. Let me share it with you:

This graphic does the finest job I have seen of clearly articulating where the boundaries of control lie under the different models of cloud computing. Cloud, after all, is really about surrendering control: we delegate management of infrastructure, applications, and data to realize the benefits of commoditization. But successful transfer of control implies trust–and trust isn’t something we bestow easily onto external providers. We will only build this trust if we change our approach to managing cloud security.

Cloud’s biggest problem isn’t security; it’s the continuous noise around security that distracts us from the real issues and the possible solutions. It’s not hard to create a jumbled list of things to worry about in the cloud. It is considerably harder to come up with a cohesive model that highlights a fundamental truth and offers a new perspective from which to consider solutions. This is the value of Dan’s stack.

The issues in the cloud that scare us the most all fall predicatably out of the change in control this environment demands. Enterprise IT has carefully constructed an edifice of trust based on its existing on-premise security models. Cloud challenges these models. Cloud rips pieces from the foundation of this trust, leaving a structure that feels unstable and untrustworthy.

We cannot simply maintain existing security models in the cloud; instead, we need to embrace a new approach to security that understands the give-and-take of control that is inherent to the cloud. This demands we recognize where we are willing to surrender control, acknowledge that this conflicts with our traditional model, and change our approach to assert control elsewhere. Over time we will gain confidence in the new boundaries, in our new scope of control, and in our providers–and out of this will emerge a new formal model of trust.

Let’s consider Infrastructure-as-a-Service (IaaS) as a concrete example. Physical security is gone; low-level network control is gone; firewall control is highly abstracted. If your security model–and the trust that derives from this–is dependent on controlling these elements, then you had better stay home or build a private cloud. The public cloud providers recognize this and will attempt to overlay solutions that resemble traditional security infrastructure; however, it is important to recognize that behind this façade, the control boundaries remain and the same stack elements fall under their jurisdiction. Trust can’t be invested in ornament.

If you are open to building a new basis for trust, then the public cloud may be a real option. “Secure services, not networks” must become your guiding philosophy. Build your services with the resiliency you would normally reserve for a DMZ-resident application. Harden your OS images with a similar mindset. Secure all transmissions in or out of your services by re-asserting control at the application protocol level. This approach to secure loosely coupled services was proven in SOA, and it is feasible and pragmatic in an IaaS virtualized environment. It is, however, a model for trust that departs from traditional network-oriented security thinking, and this is where the real challenge resides.

How Secure is Cloud Computing?

Technology Review has published an interview with cryptography pioneer Whitfield Diffie that is worth reading. I had the great pleasure of presenting to Whit down at the Sun campus. He is a great scientist and a gentleman.

In this interview, Diffie–who is now a visiting professor at Royal Holloway, University of London–draws an interesting analogy between cloud computing and air travel:

“Whitfield Diffie: The effect of the growing dependence on cloud computing is similar to that of our dependence on public transportation, particularly air transportation, which forces us to trust organizations over which we have no control, limits what we can transport, and subjects us to rules and schedules that wouldn’t apply if we were flying our own planes. On the other hand, it is so much more economical that we don’t realistically have any alternative.”

Diffie makes a good point: taken as a whole, the benefits of commodity air travel are so high that it allows us to ignore the not insignificant negatives (I gripe as much as anyone when I travel, but this doesn’t stop me from using the service). In the long term, will the convenience of cloud simply overwhelm the security issues?

The history of computing, of course, is a history full of such compromise. Right now we are in the early days of cloud computing, where all of us in the security community are sniping at the shortcomings of the technology, the process, the legal and regulatory issues, and anything else that appears suspect. But truthfully, this is the ultimate low hanging fruit. Identifying problems with the cloud is effortless; offering real solutions is considerably harder.

Not surprising, Diffie offers a real solution, which is to look hard at trusted platforms. In the end, convenience will sweep over us all, so it is important to quickly establish the best secure baseline we can. The secure base for cloud computing needs to become like aircraft maintenance schedules–something that is a given part of the process and an important component that allows us to reasonably invest trust in the system as a whole.

I Went for Coffee and RDS was Waiting for Me When I Returned

Here at Layer 7, we’ve been really excited about Amazon’s Relational Data Service (RDS) ever since they announced it last month. RDS is basically a managed mySQL v5.1 instance running in the Amazon infrastructure. The point of RDS to provide another basic service that we all need all of the time, managed within the AWS ecosystem. It offers some great scaling options (in terms of instance sizing), but best of all, it provides automatic snapshoting of  database instances. This revolutionizes EC2 because it solves the nagging persistence problem that we all face when we terminate instances. We’ve all come up with clever ways of dealing with this using S3 and EBS,  but now it’s gotten much easier.

Since RDS is really mySQL under the covers, I had been hearing that it’s pretty easy to port to. We’ve been itching to play with it here, using Layer 7’s SecureSpan Gateway AMI that’s runs in EC2. Unfortunately, this Fall has been really busy, so none of us have had an opportunity to play with it until now.

The inimitable Jay Thorne, who is a musician first but holds down a day job here as Director of Development for the Tactical group, finally cleared an afternoon to put RDS through it’s paces. I had to step out for coffee with another of our execs, which turned into a longer-than-expected discussion. But by the time I got back, Jay was done: SecureSpan using persistent Amazon RDS storage. Hello, cloud registry/repository…

Here’s Jay’s summary, which I think speaks for itself:

Total elapsed time: 1.25 hours
Number of pdf documents read: 1
Number of web pages read: 3
Number of command copy/pastes from doc: 6
Number of dbs created by mistake until I got the zoning right: 2
Number of mistyped credentials until I learned to use a creds file: 7
Number of dumpfiles created source side: 1
Number of times I had to import to get it right: 1
Number of characters in the hostname of the db: 50
Number of hosts I put in the db firewall allow list: 1
Number of sets of user credentials I created: 1
Number of lines in our internal wiki article I wrote about this: 35
Number of bangs on the keyboard in frustration: 0