A lot of things are changing with the way we buy, sell, write and use software at the moment. We do live in very interesting times. The big umbrella movement called cloud is one of the most interesting things; for example there is a huge shift to on demand software and hardware, ideal for the agility it enables, as well as disintermediating expensive sales processes.

Hardware started commoditising a long time back, with the huge economies of scale in chipmaking and computers, but wheat has really hit is the 2005 shock when clock speeds hit the ceiling and Moore’s Law just went into more cores. Scale out was the only real option then for performance, although JIT compiler technology has been a hot topic, and if you program in some dynamic languages like Javascript you might be forgiven for thinking computers were still getting significantly faster. Simultaneously simpler lower power devices have been shifting up the performance curve, reaching a point where some people are starting to seriously build ARM servers, and mobile devices are reaching useful performance for wide ranging tasks.

Many thought that at this point we would be programming SMP architectures with shared memory models, but while you can now get systems with very large number of cores in the SMP/NUMA world, but these are very expensive systems, and little software outside specialist areas. The majority of multisocket systems now are probably virtualized to appear as multiple single socket ones. Scale out, message passing, commodity hardware are mostly where things are, with a sideshow of commoditized GPU technology giving us another parallelism to work with, or another competitor to the ageing x86 architecture. China is building its own MIPS64 based CPU too, designed to run Linux, more competition to lower prices. Computation, networking, storage are all getting very cheap.

I replaced my last dual socket 2GHz powerpc machine with a new dual core machine thats a fair bit faster but about 10-20% of the power consumption, and now almost all the ten or so computers around the house are small, low power, and dedicated to one function: netbook, 2 mobile phones, ipad, VOIP phone, router, storage, laptop, a couple of small computers. The highest power consumption is probably the old laptop, a last reminder of the old Intel that used to make chips up to 100W. It makes no sense for me to invest in large amounts of computer power in the home, because the bandwidth is not there: it takes too long to get larger datasets here, so I try to do computation where the data is and where there is bandwidth, so I outsource the computing.

When Douglas Parkhill wrote The Challenge of the Computer Utility in 1966, while working at the sinisterly capitalized MITRE, the reason for people to be interested in computers as utilities was the expense of actually owning a computer, which cost around $2m at the time, but could be rented at $450 an hour. But the economics of treating computers as a utility, improved utilisation, availability on demand are just as valid now, as well as the ability to shorten development feedback times and share common source code which he also cites. Utilities are largely centralised for reasons of economy of scale, ease of balancing changing demand, and ability to provide large quantities. Obviously some people have their own generators or solar cells and water supplies, and some people and companies sell power into the grid, but the majority comes from large sources, using varying technologies.

Other than PCs, which are increasingly mobile, most businesses don’t keep a lot of computing power on premise, most of it is in datacentres. And those datacentres are increasingly getting larger and further away, as the cost of power and real estate pushes them to cheaper locations. And they get bigger for cooling efficiency reasons, and start to turn into billion dollar projects, vastly more expensive in real terms than the computers of 1966. On demand computing suddenly starts to make sense for almost all businesses who do not wish to get into this type of expenditure. It also suddenly makes computation costs transparent, in terms of how much computation or accuracy is cost effective for a particular task, even if killing fixed cost budgeting is going to massively disrupt the financial planning of enterprises (another change that will need more computing power to manage).

What does a computer utility look like?

The early days of utilities are dominated by several issues, one being standardization, whether it be AC frequency and voltage, another being technological change, as from hollowed out trees, to clay pipes to plastic in water transmission, or financial boom and bust of the railway industry as the level of demand and supply were balanced out. Competition between new utilities and old ones was also significant, railways vs canals, roads vs railways, gas lighting vs electricity, hydraulic power vs electric power, and regulatory and legal issues from compulsory purchase, nationalisation and especially price regulation.

In the end though, the centralized delivery model means you end up with standardized services being delivered, you cannot choose the voltage and frequency you want your electricity delivered (other than wholesale bulk offerings, like three phase), the temperature or pressure of the tap water, the frequency of the trains, or the level of congestion on the roads. Utilities often have slightly odd externalities and natural distribution monopolies that vary from one to another.

Computer software architectures are very non standardized right now, as we are in the craft period of software. There is a lot more standardization than in 1966, when there was less mass production, and LSI was only just starting. Common abstractions have been slowly appearing, virtual memory being an early one in the B5000 in 1961, hiding the memory hierarchy in a way that simplifies a lot of code, even if other code needs to understand the abstraction for performance; that was around the same time that high level languages were being developed with mutable variables that matched the semantics of virtual memory. The next big abstraction, from the late 1960s, was the Unix (almost) everything is a file model, a universal stream processing model for many types of data and connections with a single software interface, which also allowed compositional software. In 1970 Codd created the relational data model, an abstraction for databases. It was a productive time, in which Paul Baran also invented the packet switched network made of unreliable components, another key building block of the cloud architecture. In the 1970s we had the language virtual machine, starting with the UCSD p-System which heavily influenced Java, the beginning of virtualisation of hardware and operating system characteristics that has had some, although still arguably limited, success and standardisation.

The characteristics of a utility architecture are those embodied in the internet: distributed, failure tolerant (up to a limit of course, we still get water and electricity outages, with very weak SLAs), standardised interfaces (plug sockets, HTML), usage agnostic, scalable. In particular the six constraints of the REST architectural style, client-server, stateless, cacheable, layered, optional code on demand, uniform interfaces currently give us our best architectural model of a scaleable, on-demand system, and so our best model of what the abstractions of a truly utility infrastructure will look like. Another characteristic is of course the use of open source software, which represents both the commoditisation of infrastructure software, and the right price for utility and on demand scaleable software.

Current environments

If we look at Amazon web services, the closest thing so far to a utility computing service (although with some transitional facilities bolted on), the storage abstraction, S3, looks pretty much like what we would expect from the REST style, resource based, supporting the HTTP concurrency model (etags, conditional updates); note it looks nothing like a POSIX file system, the previous abstraction. In the Amazon model, this is the only really persistent, reliable, scaleable, global storage. Other services like EBS are not fully reliable but can be snapshotted to S3, while SimpleDB is not really a database in the persistence sense, as it has severe limits on size and is not globally available; it is better seen as an abstract index, fulfilling that part of the database requirements without the persistence. Other storage such as local disks is even more ephemeral. I have another blog post in the works on how best to use these persistence models for real applications.

The network architectures in Amazon are similarly what you would expect for building REST architectures, with load balancers and an HTTP cache layer for example. In contrast, Amazon’s EC2 provides a Xen virtualised PC as the base abstraction which is pretty software architecture agnostic, compared to say Google App Engine, which provides an abstracted language runtime. The more general platform allows more experimentation, and crucially more legacy transition. We have a lot of historic usage of operating systems, security models, coding habits and productivity issues that make using “opinionated platforms” difficult.

In many ways though, whether the API provided on a utility compute resource is the Java servlet standard as in the GAE Java environment or virtual network devices as in EC2, thats really a middleware question, and a question of the amount of environment you have to wrap around your code. The Amazon approach encourages people to repackage services for different language communities on top of their offering. The important parts of the architecture are the parts that support failure, scale, reliability. The most important one of these, often neglected, is explicit management of state. The server itself must be stateless if it is to be failure proof. All state must be managed in the persistent storage abstraction, and software upgrades by starting up new machines, and upgradeable interface specifications, another REST idea. This is a big cultural leap for people, apart from not being used to actually accounting for every data change the program makes, sysadmins are used to patching systems at runtime, rather than starting up new, tested, instances when code needs to be updated. Unless state is explicit it is not testable or reliable, or known, or recoverable. These things were discovered of course in the Erlang community where backups are a sign of poorly designed systems that have not covered all the failure cases properly. More on this side of things in other posts soon too.

Once you have realised that there is only scale-out, all state must be explicitly managed in state abstractions, and failure must be designed for all over, the container shape makes much less difference, as your software architecture will be very similar. Sure there are points of difference, packets versus streams say, sync vs async, different security architectures, different programming languages, protocols, serializations and libraries. The network is really the only important IO point for the utility computation; the server environment can be much simpler in principle than the complex server environments we run predominantly now, and thus able to scale better on more and more commodity hardware, highly parallel and low power that we were talking about before.

Beyond Compute and Store

I said before that Amazon SimpleDB was best viewed as an index not a database, and lots of the software we use in a web environment is like that, key-value stores, memcache, and some of the NoSQL solutions are providing this type of function, not concentrating mainly on persistence; obvious product areas here are things like full text search which are not the ultimate data source. Others are trying to provide “small item persistence”, although most of these systems are not yet being aimed at utility computing provision, so they have not tended to cleanly separate store and index for example, and persistence is mainly aimed at hard drives. Given the historical popularity of databases, and the current flourishing of the NoSQL movement, I can see an opportunity for something like Cassandra with an S3-type backend as a utility computing key-value store. Why not SQL? The main problem is the lack of scalability of JOINs; making it difficult to scale as a true elastic utility service. In addition, I can see a place for a low latency reliable transaction log service built on replicated SSD that shifts over automatically to permanent long term storage (please Amazon), to cut the persistence time, especially for small data items.

Other facilities for utility computing include authorization infrastructure, message queues, payment infrastucture and so on, blending into more specialist services sold as SAAS on the infrastructure.

Beyond Now

Our current code model looks a bit like running code inside a web server; indeed that is one way to do organize the code, or you can reverse this and run a web library inside the language library, like Node.js does for example. Architectures could change though, perhaps (sometimes) moving the code to the storage for lower latency; the advantage of strongly decoupled code with explicit state management is that it is much easier to move it around; explicit state management with all lasting state over the network has the same properties as functional programming, indeed as we said before is very much like the Erlang functional process and message passing model. This flexibility in where code runs allows another level of infrastructure virtualisation and efficiency, moving code towards requesters or data based on latency and throughput requirements.

I don’t think this is going to change how programming works a lot, well at least it should be pretty familiar to web programmers, might be some adjustment for the enterprise lot of course. What is going to change is sysadmin. One of the issues now with cloud solutions is they require too much sysadmin to really allow us to scale up our usage. Jevon’s Paradox cannot come into effect and let us expand capacity as price falls if there are additional costs such as admin overhead. The infrastructure providers have managed to cut staff to less than 1 per 4000 machines, but the users of services like Amazon’s still have a lot of software environment management overhead. I think we can cut that too, substantially, using stateless servers and other changes, but that is another blog post.

This is perhaps a bit of a run through history, trying to make sense of the journey towards utility computing that has been going on for over 45 years but is really just starting to become mainstream, and where we still have the opportunity to build a new utility, which is something that does not happen very often. Some more concrete explorations coming soon.

This is a big change as one of the parts of the HTTP model is the small “uniform interface”, where there are very few things you can do to web resources. GET is the most common, to retrieve a resource representation. Then there is PUT to update a resource, and DELETE to delete it. Then there is POST, which tends to cover everything else you might want to do. The problem with that is that discovering the interface for POST is difficult, as is knowing exactly what it will do. (There are a few other verbs too).

PATCH is much more straightforward. PUT updates an entire resource with a new version, while PATCH just makes an amendment to a resource. For some types of resource, the entire resource may be large, so that just sending differences will save bandwidth. Also, sending the full resource may unnecessarily make the changes sequential, for example append operations where the order of the operations is not significant. One example given is a log file, where many processes may be adding entries, and if they had to retrieve the whole log, append a new entry and write it back there would be a lot of extra traffic, and a chance of either lost updates or processes having to retry if the resource was modified during this process. Clearly a PATCH operation here that does an append would make sense. I am not sure that is actually a very good example though, as you would almost certainly create a resource for each log entry, rather than one for the whole lot, but clearly other similar patterns exist.

HTTP is not a filesystem

When it was new, people tended to treat HTTP like a filesystem. After all that was the common model for storage, and web servers generally stored web pages as files, so they tended to e treated much like that, with filename extensions annd index files, and WEBDAV was created to try to make the web usable as a filesystem protocol. This model does not really work very well however, as it does not model the things you can and can’t do with HTTP. The methods are one example; updating entire resources at once means they tend to be small units, rather than, for example, log files. File systems generally struggle to store millions of tiny files without wasting a lot of space, and without becoming slower. The web resource does not have to support full Unix filesystem semantics (a topic that oddly Wikipedia seems to be missing an entry on! May have to rectify that), and supports a much simpler updte model.

Maybe the easist way of thinking about HTTP is to see every URL is a small Model View Controller (MVC) system. The model is an abstract resource, which we can see through GET, which retieves views. There can be multiple views, as a request can ask for different media types and encodings for a single resource. The controllers are the media types supported by PUT, which are usually the same as those for GET, but need not be; because both the view and the controller need to represent the whole resource state, they do tend to be quite complex representations, such as XML documents. PATCH however is also a controller, but a more interesting one in many ways, as it can send just state changes to the model, which tends to be how many MVC systems work.

Another thing that PATCH enables is resources that hide some of their state. A resource could only support PATCH and not PUT so that state modifications were only changes. If the state returned by GET is not the complete state, the resource could hide parts of the model. An example could be a voting method that records but does not reveal who has voted, only returning totals, which accepts votes as PATCH requests.

Server side scripting

One PATCH format that makes a lot of sense is actually to use executable code, rather than say diff files. There is no reason why you should not send the server a PATCH request that is some Javascript to modify the DOM of an HTML resource which can be executed serverside, or an XSL transform to modify and XML object. Sending code is an efficient way of making changes to a resource, and can be executed in a sandbox like the browser sandbox. This will be another driving factor for Javascript on the server side, as it is well suited for embedding like this, and already has a DOM model for transforms.

All these changes take us further away from the filesystem model. Web resources will more and more combine some storage with some computation, including ability to execute code in a contolled way. Smart resources will become more common, over dumb storage only resources.

In other news

Next in line for HTTP, hopefully, is the Link header which adds a new header for legacy document formats that do not include a native hyperlinking capability. This will allow relationships between these documents to be included in the retrieved resource, such as a link to metadata or other related resources. The HTTP replacement for the file system model is getting serious.

SPARQL DESCRIBE is a somewhat controversial feature of SPARQL that returns an RDF graph “around” the query, including all the referenced URIs and some sort of domain specific context. It is less well defined than CONSTRUCT, that also extracts a graph.

But there are not many guidelines for how to actually write useful interfaces for search type functionality for REST web service APIs.

A lot of this applies to other collection type interfaces, collection in the Atom sense, which is essentially a predefined filter rather than one you can submit a query to. Resource collections are a fundamental feature of any real world system, being able to take a set of resources and manipulate it as one. Query languages are generally an efficient way of defining interesting collections.

I do not specifically refer to SPARQL here. Like many of the W3C recommendations, it is XML document oriented in its specification, not HTTP oriented, and so does not try to address the types of issues here. I have some issues with the W3C approach, sidestepping implementation issues; indeed the whole RDF as Prolog wrapped in XML does not seem very productive, and I think a more pragmatic approach from practise will work better, as with some of the other W3C initiatives. Items like DESCRIBE imply much more domain knowledge, and it is unclear what interoperability would mean.

Whats the problem with query languages and REST?

The main issues are that a SQL style SELECT should map to an HTTP GET, but instead we are passing a query with another verb in it. One method of avoiding this would be to work with predefined refinement “queries” where you could walk through paths of filters from a resource that represents everything. Imagine a faceted navigation system drilling down to construct what becomes a filter query through a selection of refine links (where you might end up with a hierarchical URL in the end not a query document). This might work better in a system with a finite vocabulary, although it could still be navigated via forms potentially. However this starts to work less well with complex nested queries with sorting and limits, and subterms, as the number of opotions is large, and it would be complex to make a comprehensive set of link types to make them understandable (you might just end up tokenizing SQL in links which does not seem elegant).

REST search and GET queries

First off, for standard read only GET queries, there is one model that is really useful and is too rarely used. For some reason most search queries in “REST” frameworks involve submitting a query as a POST request because the query string would be too long for a GET request. This however generally does not create a resource, it just returns a response, or a temporary response resource. CMIS for example works along these lines. However the sane model along these lines is to submit a query document with POST (or PUT) and have it persistent until DELETEd, and be able to requery as needed. Also, it should be possible to create query resources with unbound variables, which can be used by a system like prepared queries in SQL databases; in particular the creation of these should allow a system to work out perhaps if it needs additional indexes based on the queries that have been created. These can then be evaluated by using query strings to bind the parameters. For some reason you rarely if ever see a query specification for a REST API that has unbound variables in the query, thus missing out on the ability to use this pattern.

Note that the query itself is a resource, so we can for example DELETE the prepared query eg http://example.com/query. There are also resources corresponding to the result sets eg http://example/com/query?q=red.

As well as unbound variables for constants, another missing feature from most query languages (probably because they were not designed for HTTP transport) is the lack of ability to specify a URL that corresponds to a subexpression. This can be considered as another extension of unbound variables; you can also see this as the extension of the labguage to support SQL style views that can themselves be queried. Like a view, you could either use the definition or the current materialized version for performing further queries.

REST and update queries

Although it rarely seems to be done, there is no particular reason that once you have a query resource like the ones defined above, you should not be able to DELETE http://example/com/query?q=red to delete all red objects, or PUT a new set of red objects to replace the existing ones. Most collection oriented protocols, such as Atom, do not allow PUT and DELETE methods on collections, only on the member resources. POST is allowed in Atom, to define collection membership; this only makes sense where collection membership cannot be defined by visible properties of a resource; the instruction create this item with whatever properties are needed to put this item in this query result is not necessarily something that works for any domain. Certainly one can think of models in which allowing PUT and DELETE on search results resources would be useful, although there might be issues with paging and large document sizes on some types of resource. PATCH would be a useful addition for doing the equivalent of SQL UPDATE, in order to reduce the document sizes, rather than having to PUT back everything.

There may be other domain issues, and in some systems consistency issues with mass deletion and insertion that mean that actions on individual items not collections are more efficient of course, but extending the uniform HTTP interface to collections should certainly not be ruled out for many domains.

Only solutions that produce partial results when partially implemented can succeed. The network is littered with ideas that would have worked had everybody adopted them. Evolvable systems begin partially working right away and then grow, rather than needing to be perfected and frozen. Clay Shirky

I came across that quote from Clay Shirky as I was writing this which seemed very relevant.

Jan Algermissen added the two following very good points:

the use of REST brings the decentralization aspects of the Web into the enterprise world and allows designers and developers of networked systems to create and evolve their components without the resource and time consuming effort of bringing them into a single room to discuss APIs. In geographically distributed organizations this might sometimes even be impossible.

the use of REST facilitates fragmented change. It deliberately provides room for evolution (e.g. format extensions) that can bypass the main organizational control. Evolutions that turn out to be valuable can flow back into the main line while others silently die. No harm to ongoing communication between systems is done by either of those.

These are really very important points. You cannot really undersell the success of the web, and the web took off with its own architecture despite attempts to force other architectures on it. The decentralization and the ability to reuse and mashup services to make new things is part of that architecture. The idea of introducing REST to the enterprise is to help the enterprise leverage the agility of the web architecture, and to make building new enterprise applications cheaper and easier.

Benjamin Carlyle made the good point that you can view REST as an evolution of SOA, as better easier to consume services. This does not quite fit with my view of history, but I do agree that it is a good way to sell it now as “SOA on steroids” as he says. He linked to useful sounding book coming out later this year on SOA and REST. He examines seven SOA objectives from Thomas Erl’s book, Principles of Service Design:

1. Increased Intrinsic Interoperability

   Made better with a uniform interface – Interoperability stops being an “easy to plug together at design time” thing to “easy to plug together at run-time”

2. Increased Federation

   This is not an explicit target of REST, to get everyone using the same service. However, it is compatible with REST. It also occurs in practice on the Web through market forces.

   While SOA emphasises reuse of services, REST emphasises reuse of components (including services), media types, and connectors.

3. Increased Vendor Diversity Options

   More or less the same for SOA and REST, however REST is probably still less mature in the enterprise space. Time will change this.

4. Increased Business and Technology Alignment

   ie, we build services that the business wants, and can change in pace with business objectives

   Again REST doesn’t particularly target specific valuable services, but is compatible with the notion

   REST is more capable of incremental upgrade than a classical SOA, especially “dynamic evolution”.

5. Increased ROI

   Classically realised in SOA by reuse of services, which again is compatible with REST

   Again, a uniform interface produces wins on this front when compared to classical SOA

6. Increased Organizational Agility

   Classically realised by being able to build new applications quickly by reusing existing services

   Again, compatible with REST and again a uniform interface produces improved performance

7. Reduced IT Burden

   Few legacy systems doing more means less maintenance and related support activities – again compatible with REST.

   REST makes maintenance easier with its focus on dynamic evolvability

Darrel Miller has obviously been through some maintenance pain. The web architecture was always designed to not have centralization, and that obviously mean that things are incrementally upgradeable.

For me the business case for REST is around client/server coupling and maintenance effort. My experiences with SOAP API’s on the Microsoft platform were that if you sneezed on the server API that defined the contract you had to recompile and redeploy the clients. And then there is versioning. In the WCF world versioning is a potential nightmare, with changing URLs, XML namespaces on service contracts, message contracts and data contracts.

I believe that REST’s use of hypermedia and conneg make maintenance, deployment and versioning much easier, which translates to lower costs to the business in the maintenance phase of an application.

Bill Burke responded to Darrel saying how he has been using it.

Also, I think interoperability is much easier to achieve with REST because the ubiquity of HTTP. This has a huge positive side-effect for integration as well. My gut tells me that SOA Governance could really reap the benefits of a RESTful architecture as well, but I don’t have any details yet on how.

All this plus Darrel’s comment on the fragility of SOAP stacks sold me on the idea of REST which is why I’ve been really pushing it hard at Red Hat and all our middleware projects.

A very useful set of responses, thanks to everyone. Will be writing more on this soon.

I went to a meeting a while back with a company starting to move to a web service based, internal API based architecture, and there was a minute where the CTO said (more or less) “does anyone know of any let or hindrance to this being a SOAP API?”. Like those moments at weddings, no one spoke out. But I do object. SOAP is not the backbone of a happy architecture, but we do not have the strength to say no right now.

So here is the beginning of the shout out. The enterprise architecture needs to be a resource oriented architecture (ROA) not a service oriented architecture (SOA). The enterprise needs to move from SOAP to REST.

Web developers know this. When given the choice, 9 out of 20 cats prefer to use REST. It is more productive. It does not involve programs to generate huge chunks of useless code. It involves hypertext (HATEOS) not opaque documents that are mappings of database schemas.

What are the resources in the enterprise? Start with customers. A CRM system is a clear example of something that needs to be modeled as resources. You need API calls that can find products a customer has bought, support tickets, all the core data that you would need to build customer service portals, internal support applications. What is your customer API?

Building this framework can be incremental. You need either tools that already provide REST APIs, which is becoming easier, or a web application development framework. This is the convergence point between application frameworks and web content management. The big issues are the domination of legacy applications with bolt on APIs that may have moved from SQL to CORBA to SOAP over the years. You can however build a REST API over at least parts of these applications to open them up into the enterprise.

OK thats the beginning. Apart from cheaper application development where is the real value?

Resources are the first part of the REST architecture, and resources are much easier to work with than services, as they share uniform semantics, and they are addressable, two keys to making application design simple. The big bit though is the (harder to understand) HATEOS, Hypertext as the Engine of Application State. What this involves is moving the vague and very expensive field of business logic from code (often code that is not even owned or understood by the enterprise, as it has been embodied into code written into systems from suppliers) into hypertext documents.

A concrete example might help here, based on another recent consulting example. A professional body has a set of different membership levels, exams, rules, CPD requirements and so on. These are resources, described states, and hypertext links of state transitions. This is the core of the business logic of the organization, embodied in documents. It can be used to build the membership application, as it describes for example the actions open to a member at the present time, as well as providing potentially a browsable structure, as well as a formal structure that can be used to ask questions about membership (what are the routes to becoming X, for example).

You can view this as a move of business logic to a declarative rather than imperative form. Things do not have to be stored in documents in the underlying storage, although the interface is document and hypertext based. Hypertext makes things human browsable, and declarative makes them computer browsable, and HATEOS makes them discoverable. Business logic becomes content rather than data, rather than there being tables of parameters that the business logic black box feeds off, the states themselves become resources that can be discovered, addressed, and reasoned with.

As states become resources in a REST model, because of the statelessness of web applications, both states and state transitions become first class objects. Constructing ad-hoc queries about state changes should become easy (what percentage of customers renew their contracts, say). The first class objects need to be the ones that are meaningful for the business.

There is more to this, it needs expanding. Perhaps we need a manifesto. The software architecture of the web is going to make huge changes to the architecture of other realms, more than most people realize it seems. It is inevitable as things get re-architected around the web that more areas will be affected; also there are the benefits of scalability and reliability that are being created for the web. If anyone has any other business case references for REST please post in comments.

(Image from this software architecture overview).