Now this has been something have been thinking about recently, and there are people who are moving big parts of their systems to just be built on search, such as the Guardian API which is served from Apache Solr. In this case though, the search engine is still not the system of record for the core data, which is still the Oracle based CMS which did not scale up enough to serve the API. There was some discussion at the talk about search engines that do support persistence (the D in ACID databases), something Lucene used to have a bad reputation for. My view here though is that, while actually making fsync work properly is a good thing, and you should not buy software that cannot recover from crashes, persistence involves a lot more than this now, such as replication, audit, versioning and access control. Building this directly into search products is a mistake. Another issue is that search engines are denormalized, and data stores of record should really be normalized to a large extent, to minimise the amount of data to be replicated.

There are two approaches that should work instead, however.

The first is more or less the current approach, to use the search engine as an index to a persistent store. I really like this approach if we follow it to its logical conclusion, which is that the persistent store in this type of application architecture should not be a relational database, but it should be a document store, that is a file, an HTTP resource, a document in a NoSQL document database, or an object in a replicated cloud storage system like S3. Modularize the database application, and split the persistence function from the index function. The persistence function provides the durability, versioning and audit and access control, with replication, backup. This can update the search index, and potentially any other types of index, such as a graph database for querying relationships, potentially even a relational database if that is the best way of querying some aspects of the data.

Obviously there is a potential consistency issue, if updates from the document store happen slowly, so potentially there is an eventual consistency model. Historically search was a bad offender here, as dynamic updates were not the norm and everything was batched into nightly updates, but that is going away and dynamic updates are more normal for search indexes. In principle you can have more consistency, especially in an architecture where there are fixed releases that can be consistently indexed, rather than distributed rolling updates, you choose your architecture and take your choice. Small consistency lags rarely matter in a lot of applications.

So you end up with an architecture with a well defined persistence layer that is not a relational database, and a set of indexes appropriate to the application, almost certainly including a full text search engine, but perhaps a graph engine too. Maybe you run consistency checks on your indexes for peace of mind.

The second approach is to see that search engines were some of the original NoSQL data stores, building custom storage and indexing engines, because they had such difficult problems. Indeed Google’s BigTable, and so the ancestry of a lot of NoSQL products came from search. However the search engines around now have not yet refactored themselves on top of the NoSQL engines that have emerged from this work, although this is starting with Lucandra which is Lucene persisted in Cassandra, which looks promising, offering seamless replication and distribution, and HBasene, an HBase Lucene backend. These make a huge amount of sense to me, as if you are developing sophisticated search algorithms, not having to build the whole index and persistence layer as well is a big advantage, as well as the scale out potential. Of course this approach does not conflict with the first one, in fact you could choose a NoSQL backend that is aimed more at read performance than persistence, and at storing small index values fast. The hard bits with this are that the search engines have specifically customised their data storage for the particular use cases, and reworking this onto a more general backend has few apparent advantages; as you can see from the examples above, most of these changes have come from people already using the backends in question and who want a single database to manage all their data requirements, particularly once they are working with high availability and replication. Software modularity really is not at the right level yet is it, I blame object oriented programming for this lack of reusability.

Anyway, back to the main point. For applications like content management, an architecture based on a content store that deals with persistence, versioning, access control, replication, with a set of indexes based on search engine techniques, graph databases, and anything else your applications needs. Ideally the indexes are all based on a common set of low level primitives so the backend can be swapped out or shared between the search store and other application specific indexing requirements, so there is a single low level indexing infrastructure that can be available as a common scalable service, with different implementations available. This type of architecture is quite buildable now, and is certainly used in quite a few applications, and I think it will become much more widespread, particularly in the cloud where it seems more natural, certainly for many types of application that fit into a document type model, such as content based applications.

Drupal (unofficially) competing in the BostonPHP Framework Bakeoff from New Leaf Digital on Vimeo.

The challenge

So first here comes the challenge to other CMS vendors and system integrators: spend three hours (or less) on repeating this using your system. You get an hour or so for planning, based on the wireframes. Note there are some incompletenesses in these (typical client!), like what the tag pages look like, so improvise, and some explicit open decision points (“require some sort of password”), so use whatever is easiest. You get up to an hour on the build (points given for less!) The aim is not a polished final shippable product, but a first alpha or beta. Then you are allowed another hour for annotating the screencast , putting the site online for us to look at etc, if you need it. You can do the annotation as you build if you are pushed for time and cut the planning and you should be able to get the whole thing done in an hour, so no excuses for anyone not to have a go.

Basically the site is a simple job board, in a circa 2005 Ruby on Rails style, so there is nothing that a decent web content management system should not be able to do out of the box. If there are bits your system technically finds hard its probably worth fixing them (the Drupal video reveals some small things that are apparently fixed in version 7 for example). But get it close enough, as really the aim is to give people an idea of how usable your system is for this sort of build task, the approach you have to take, and how quick it is! If your data model requires something a little different, by all means explain and make something close. There is no attempt for example to impose a REST architectural style and support an API that supports PUT and DELETE on jobs, but if that comes out of the box or trivially with the CMS you use, by all means show it off and put a bit more 2010 into the solution!

And no trying to get out of this, or I will write up your CMS as “pretty useless compared to Drupal” or your web CMS integrator as “rubbish, use New Leaf Digital, they have balls”. If you want to do it live as a webcast then feel free to invite me along (but record it too).

The bit about frameworks and CMSs

It is an interesting question though, whats the real difference between web development frameworks and CMS systems. Indeed, if you think that web frameworks are nicer in every way, feel free to enter the challenge above using your favourite framework.

My current thinking is that a CMS is best defined as a web development framework plus an IDE that persists configuration to a content repository or database. By all means disgree with me!

By web development framework, I mean something that gives you a full web server, front end and backend integration tools, authentication and access control, templating and so on. A CMS has an IDE in addition, a web frontend that lets you configure it, define datatypes, forms, layouts etc. But generally, unlike say visual code editors which persist to editable files, the results of these changes in the backend are persisted to the same content repository as the actual content, either as database entries or schemas, or as tree nodes (as in JCR based systems). Hence for example these changes do not generally end up in a source code control system, although they may be versioned in the way content is, or they may not as for example Drupal CCK schema changes change the underlying database, while with a standalone framework you might have to script schema changes for upgrades in the traditional way.

Also the issues in deployment with CMS products due to the use of a single repository for content and config are clear, even if many systems do try to make them separable by various means; the Drupal example shows this to some extent as all the frameworks have their code available on github but to set up your own instance of the Drupal example you would need some code, and a database dump which you would then have to filter the content in the example instance from.

This also makes it clear why CMS systems are usually very tied into one framework, as the IDE has to embody a lot of implicit knowledge of the data model and the way the framework wants to do things, as well as how to persist changes. Designing a more decoupled approach is much more difficult, although perhaps possible (thats another blog post to come, the decoupled CMS). But the streamlined coupling makes solving problems that fit the data model easy and quick, as the challenge above demonstrates. Of course if the type of problem does not match up to the the model that the CMS applies, then it comes down to trying to force a teapot into round hole.

In the beginning the web was editable in the browser. Tim Berners Lee made it so. But this did not last for long. Eventually Microsoft restored this, to some extent, with the contentEditable properties and related Javascript interfaces. Now these have a lot of foibles, and the current HTML5 standardization has not done a huge amount of work here yet, certainly not adding new features or changing the basic interface substantially, so a lot of behaviour is currently underdefined, which makes cross browser compatibility more difficult. While the main desktop browsers all implement the interface, mobile ones do not in general yet, even on form fators such as the iPad, which could very well be an effective content editing device.

Should you use one at all?

Almost every CMS supports wysywig HTML editing, but it is not entirely clear how much they are used. To be quite blunt, they have always been among the worst editing environments ever devised, overlaid with a poor HTML model and a tendency towards presentational markup. Most users do their editing elsewhere, at least for originating content. I think though that we are at the point where this can be improved.

The first thing is to kill the presentational markup, and instead support customised semantic use of classes. On the basic level this is easy to implement; there is probably more work to easily support more advanced HTML templates, for example to mark up recipes or other specialist content in consistent semantic ways through this type of interface. Pluggable schema guidance might be needed for this.

The second is to make the editors themselves sane. First step is of course local offline HTML5 storage, so that drafts are still there if you wander off, your computer runs out if battery or whatever. Then just make them nicer to use, keyboard friendly and so on.

Then there is the concurrency question, you may want to show concurrent editing, and the versioning model that applies. This does not have to be the Google Wave style simultaneous editng, although that is one option. People may be editing content for release in different versions.

Import from other editors is of course still important, and making it as painless as possible for the common tools is of course important.

Also, remember it is an editor, people like a choice of editors, so loosely couple it, make it standalone and easily changed.

What not to use

Old school implementations, with nothing now to recommend them that I can see; often they seem to need server side components too, which should not be required now when pure Javascript solutions should be sufficient.

1. Open Wysiwyg

   Not off to a good start here as the demo does not work in Chrome, works in Opera and Firefox. I guess it is not really in development then. Does not create paragraphs by default, just uses <br>. Still believes in web safe colours and fonts. A classic example of the old school, but not up to date. Open source.

2. CKEditor

   Much more promising demo, . Paragraphs are real (uses a &nbsp for blank ones, which is I suppose ok), there is a paragraph style dropdown that manages paragraphs, headings and divs. The style dropdown is a bit disappointing as it sets styles not classes, and you can still set fonts and so on. Works in Chrome, Firefox and Opera. The old school done right. Open source.

3. TinyMCE

   Ah yes, the classic one. Feature set similar to CKEditor, in taht it treats paragraphs the same, has a dropdown with predefined styles, that are style tags not classes. Seems a bit clunky and flaky in comparison, and has odd terminology, since when has a div been a layer? Both have a smiley insertion feature that suggests that people have been wasting time on pointless stuff rather than reconsidering how these programs should work. Open source.

4. XStandard

   Good blurb about supporting CSS properly and the importance of markup and presentation separation, but in a bout of massive fail there is no demo on the website. Your reviewer has no intention of installing it to test it, especially as it does not support Linux as a platform in a second fail. Commercial.

5. InnovaStudio

   No demo either, apart from animated screenshots. Why are these commercial companies so full of fail? Web 2.0 products with no web trial? Seems obsessively to be about dimensions and fonts from the features list, not about separation of concerns.

6. ActiveEdit

   ActiveX DHTML with Java fallback for other platforms. Need I say more? Commercial.

7. contentEditable

   Well at least here the demo is the homepage. All inline with a simple-ish mechanism to mark blocks as editable. Stupidly though, while it appears to be non modal, all the menus are modal popups, which completely destroys the usability.

8. widgEditor

   Very simple editor. Adds divs not paragraphs around everything. Open source.

Good implementations

Largely based on contentEditable, fixing up the inconsistencies, although Dijit is based on the less flexible designMode which means it has to run in an iFrame.

1. YUI editor

   Note there is also a YUI 2 version. In many ways the demo is not much use, as of course this is part of the YUI framework and needs some Javscript infrastructure to customise it effectively. There is also a YUI2 version demo, and linked demos of plugins. Much better than anything above. Open source. Iframe based for compatibility with the grade A browsers.

2. Dijit

   This is the editor that comes with the Dojo widget set. Still iframe based, not a full contentEditable implementation.

3. Aloha

   Some nice demos available, indeed probably the best demo site of any editor. Aims to fix up the issues and cross browser quirks contentEditable, and looks nice too. Open Source.

Other ways 1: Canvas

A survey of what is up is not complete without mentioning Skywriter, which is a (code) editor written entirely in canvas. As far as I can see this is an utterly pointless approach long run. It is especially a bad idea for HTML, where fitting in with the CSS of the page matters, so lets ignore this.

Other ways 2: pure Javascript

You can in principle implement the whole of a contentEditable-like editor in pure Javascript, using a little div as a cursor and everything, no help at all from the browser. That is a lot of work, of course. I know of two implementations, one being the 2010 release of Google Docs, which explains why the did that, in order to be able to provide the best cross browser support. The other implementation I know of is the one for Squiz CMS, no online demo, not open source and tied into one product. This is pretty functional although it had some browser compatibility issues last time I used it, and ther website still says it does not support Webkit.

Other ways 3: Markdown and Wiki markup

I have to say I use Markdown a lot, this blog is generally written in it. There are two issues I see. One is that it is limited, and while you can add native HTML, this often means everything has to be redone in HTML (trying to add anchors to a list for example). There are extensions with more features, but then you get into compatibility issues. ALso there is no sanely reversible transformation that will generate the sme output as there are notational choices, so it is generally best to keep the content in Markdown all the way through. There are much the same issues with Wiki markup, although it is extensible too, which causes more issues of trying to remember constructs, and compatibility is weakened. The difficulty is that HTML is not a very good markup for humans to write, but once you get past the very restricted domain that say Markdown handles, producing something that works well is hard, maybe impossible.

Other ways 4: Don’t use HTML

A number of people I have worked with, and some content management systems go with a very minimalist use of HTML, pretty much text is paragraphs, everything else is structured fields. A slightly enhanced version of this, effectively a very minimalist tiny markup, corresponding to even less than Markdown is the COPE system.

Personally I believe we need to expand the use of rich text, for many use cases “just the words” is not enough, and we need equations, images, notes, asides, sidebars, tables, captions, and all the richness of rich text, so I don’t think this is viable for much serious writing, and we need to expand from the dumbed down web that this enforces.

Concluding

While the wysiwyg editor as it has been in content management systems is pretty awful, I think we need to expand the principle and try to do better, go back to the really editable web, and support well structured semantic HTML in rich ways, not forms and simple text boxes. There is a lot to do to get this working though still.

The basic model is that we have resources, which may have some content and metadata attached. We are mainly interested here in the properties that can be attached to a resource, and the fact that some of those properties are relations to other resources. We are less concerned about what goes inside the structured body of a resource, although there are some issues about how many “bodies” a resource can have. So we have a model that has resources, each of which has key-value pairs, some of the values may be links to other resources, which presumably have referential integrity support

Properties of properties

• STRING. Resources can have string valued properties.

  This is a basic starting point; I don’t think I know of a CMS that does not support this. You can store any other type in a string if necessary, though binary values are more efficient in some cases.

• VALID. Property values can be validated.

  Many core repositories do not validate property values at all, it is just a validation proxy layer that does, so as a repository principle this is fairly rare, although a small number of types (numbers, dates) might have native validated representations.

• TYPE. Property names can be validated.

  Many systems have a typing facility that restricts the set of property names a resource can have. Others are unstructured, and any property can be added to any resource. There may be type composition mechanisms, such as mixins or type inheritance. Unlike value VALID this is more often tied to the core repository model, rather than a proxy layer, if the repository is typed for internal performance or indexing reasons, that is tied to dense rather than sparse storage.

• BINARY. Resources can have at most one binary property.

  I have split this property as some content management systems can only have one binary property (such as an image file) on a particular resource, and multiple ones have to be constructed from multiple linked resources. This is not generally a huge limitation in an otherwise flexible system, but in a weaker system could be annoying.

• N-BINARY. Resources can have any number of binary properties.

  This is the fully flexible version; one may still be a distinguished value in some way, but you can store all the sizes of an image (say) as properties of one resource, which makes managing them easier, although it may actually make things more difficult if it is not easy to iterate over properties (STRUCTPROP), and using multiple resources could be easier.

• STRUCTPROP. Properties can be structured.

  Some systems have structured properties, for example some systems have a JSON representation for properties, rather than the flat key-value namespace of other systems. JSON supports arrays that can be iterated over, and structures that can be repeated. To make this sort of structure with only key-value properties you may need to use more resources. Structured properties though add a lot more complexity, and perfectly useful, but different, systems can be made with or without this model type. Structured properties often have partial update interfaces, which adds complexity, so that one subproperty can be modified at a time. Note while technically JCR does not have structured properties, you can use the distinguished tree below any resource as a tree of properties, so it is rather similar to this model. Note also that property naming can informally add structure, such as in the way slashes denote URI hierarchy, they can denote property hierarchy in a technically flat namespace.

• MULPROP. Properties can have multiple values.

  Structured properties can usually have multiple values (JSON array for example), but not all systems with key-value type properties allow the same key to be set multiple times with different values. This is the model with say HTML metadata, where each property (key) can be set multiple times; however some key value systems only allow a key to hold a single value, and so the user would have to make a structured value to hold the multiple data items instead, by some encoding scheme without the system providing support directly. Having multiple properties complicates the simple set and get interfaces that single valued properties have.

• TREE. Resources can be in exactly one tree structure.

  Another split one. Many systems have one distinguished tree structure that content items must be in, and that tree has special operations, like fast access to parents and children; other trees might be constructed by other means, like using a general relation, but the operations on them might be difficult. Children in a tree are almost always ordered and can be reordered, although some systems might not have this property.

• N-TREE. Resources can be in any number of trees.

  The distinguished tree is very common (although Lily for example does not have one); but I do not think I know of any system with multiple named trees that share a common tree interface (like a parent function). You can make a tree with general relations, but you will not get help in making it acyclic for example. So while this is a possible design, it is complex to implement, although arguably useful as a modeling tool. Generally you will have to manage general relations yourself to do this.

• CLONE. A resource can be cloned.

  This means that the same item can appear as more than one resource at the same time, each of which will update in the same way. This is similar to say a Unix hard link. This is the usual way of turning a TREE into a dag, which adds some flexibility. Different tree locations of the cloned resources may affect properties such as permissions in some systems, or inheritance so this property can add a fair amount of modeling flexibility; conversely without these it is of less use.

• RENAME. A resource can be renamed or moved.

  Many content management systems provide this operation in their model, but it is not a native operation in others at the repository level, as the end user visible name just be a property for example. HTTP does not provide a rename operation, but WebDAV does.

• REL. General relations between resources can be created.

  This is the basic relation (named by the key name) between two items. It corresponds to the HTML metadata element, or an RDF triple. It turns the resources into a directed graph with named edges. It is certainly essential for any content management system; I will talk more about how you want to be able to use and query it later.

• RELNS. Relations have a different namespace from properties.

This is a distinguishing feature between XML, which has attributes and child elements (relations) syntactically distinguished, versus JSON that does not. Non child relations in XML are still attributes though. Generally seems a pointless distinction, and using a single namespace is simpler.

• RELPROP. Relations can have properties.

  This is an interesting one. Adding a value to the relationship triple to make it a quad, means that a number (or other value) can be assigned to a relation, making each relation a weighted directed graph (or you can view the system as a matrix). The general model of the property graph has properties for edges, but for example the RDF model does not, although they are often what blank nodes are used to model, although of course blank nodes can have relations as well as properties. You can make up for a lack of properties on edges/relations by adding extra nodes like this, but they may proliferate and need managing, so allowing properties may help. It is also worth noting that a system without MULPROP can use naming of properties to implement RELPROP, as a relation could have a naming convention for its properties; similarly STRUCTPROP generally allows storing the extra information in the property structure.

• REFINT. Referential integrity is preserved for relations.

  Preserving referential integrity at the repository layer is a fair amount of work, relational databases can do this, but not all content repositories do, for example over delete operations.

• ORDREL. Properties are ordered.

  True key-value models do not tend to have an ordering for properties. As with many of these things, ordering adds interface complexity. Structured properties may however be ordered, and if the model supports a distinguished tree (TREE) this almost certainly has ordered children. If you have to build an ordered tree simple from basic relations it is quite complex. A sort order on a relation is another relation property that seems to be rarely supported for general relations, like weights.

• REIFY. Properties can have properties.

  RDF in principle lets properties themselves be resources (reification), so that they can in turn have properties. This allows me to add information about the properties, such as where they came from. This rarely seems to be useful in common models. Giving different properties different permissions might be a more useful side effect.

• EXTREL. Relations can be defined externally to their subject.

  HTML originally had a rev relation, which defined a relation backwards from object to subject, and RDF triples can be stored in any document, divorced from subject and object referants. This causes all sorts of issues with updates and managing (even finding) relations, while adding no descriptive ability except potentially REIFY.

• INHERIT. Relations can inherit properties.

  The inheritance tree might be set from other properties, or from a distinguished tree, but one model is that properties not explicitly set can be inherited from another resource, or a prototype. This often makes models simpler, as rather than explicitly walking a tree, you can implicitly do it though inheritance. Seems surprisingly uncommon in content repositories.

• MINHERIT. Multiple inheritance.

  Allow inheritance from multiple resources, not just for example based on the primary distinguished tree. More complex.

• NAMESPACE. Namespacing on properties.

  Some systems have a type of namespacing on properties, often used for multiple language variants for example, so that a property may differ across these namespaces. This can also be implemented with multiple resources, structured properties or inheritance. Usually not all properties are namespaced at once; some may not vary, which makes the set and get interface more complex.

• ATOMIC. All the properties of a resource must be updated together.

  A resource and all its properties are all updated at once to a complete new state (this will also be the versioning state). Other than for versioning, this also affects how concurrent access works. The alternative is that each property can be updated independently. Atomic updates are the HTTP resource update model.

• RVERSION. Resources may be versioned.

  An entire resource may be versioned; this is a similar namespacing operation, where a namespace is available to retrieve the old values of a resource.

• PVERSION. Properties may be individually versioned.

  Some systems (such as Lily) allow versioning to be turned on or off on a per property basis. This property generally implies that ATOMIC is not true.

• DELVERSION. Versions are deleted when a resource is deleted.

  This is surprisingly common, if versions are namespaced properties, then they are often deleted along with the resource when it is deleted. The better solution is not to delete versioned resources, just give them a tombstone (whiteout) marker.

• SNAPSHOT. The versioning namespace is whole system state not resource state based.

  Although versioning of the total state of a system is now common in source code control systems, many content management systems only let individual ressources be versioned (hence creating issues such as DELVERSION). The main issue here is that you cannot apply or undo a set of changes together, only individually. Apart from the difficulty in making easy user interfaces, whole system versioning is superior in every way to versioning of individual resources or properties and no one should be designing a system that does not behave like this.

• TREEVERSION. Versioning supports branching and merging.

  A full versioning model like git or subversion, rather than just a linear series of checkpoints is another model. It generally simplifies the concurrent updates (ie can avoid both CAS and LOCK in theory). Although these provide the richest model of versioning, it is the hardest to present to the non technical user. Note also that this is one clear area where the content model for delivery can differ from the one for authoring; for authoring there are much more complex operations that are useful, while for delivery performance is key, and versioning may not be required at all, depending on how updates are applied.

• CAS. A CAS-type operation is supported on resource updates.

  Some type of atomic update-if-unchanged since this version operation is supported, for lockless updates. HTTP Etags are the canonical example. This is the simplest choice for API access, and simple for users too. The unit of atomicity is usually the whole resource here, making it the unit of transactions; atomic update only of individual properties does not let two properties be updated in a single transaction so is not so useful.

• LOCK. A locking operation is supported.

  The traditional alternative to CAS is a locking operation, that disables write operations while the operation is locked. Some administrative or time based unlock operations are required as well. Less suited than other methods to automated APIs, due to issues like deadlock. As multiple locks can potentially be obtained, cross resource transactions are possible, although this could impact concurrency.

• TRANSACTION. Transactions across multiple resources are supported.

  Generally individual resources are the unit of transaction, or possibly individual properties. Some systems however allow a transaction in which multiple resources are updated together. JCR is probably the main example of these. A system with snapshots may also have this property if moving between versions is atomic. HTTP deliberately does not have this sort of transaction, as it does not work well if the resources are distributed, and system design for HTTP should ensure that resources model the right things so that transactions across resources are not needed.

Queries

With the property model above, you can retrieve resources, and read and modify their properties. There may also be some additional maybe slightly different properties (the ones that TREE might expose for example, parent and child relations). We can traverse between resources by following their links. However we do generally want to make more complex queries, either about global questions, or more complex traversals based on properties. There are a lot of query models we should really explore, particularly we need to focus on how properties are indexed. I suspect that the analysis below is just a starting point.

• PINDEX. Property values are indexed.

  This is not necessarily essential, as for most interesting properties one would create a node rather than a value, and use relations, though you need reverse relation indexes anyway.

• REV. Relations have a reverse index.

  This lets me find the opposite direction of a relation. This is a key property, as relations are directional, and important properties are in the other direction, such as finding all the resources tagged with a particular tag value.

Interfaces to properties

I mentioned above that some of the property models have differing interface complexities, and I think it probably helps to show what some of the interfaces look like.

The canonical interface in web content management is that one exposed by HTTP. Resources and all their properties have to be updated atomically (ATOMIC) – PATCH is just an optimization. CAS is available (Etags or last update). No versioning is specially supported (although the system could create resources for old versions and add properties to access them). Other property behaviours depend on the document types, so HTML for example supports a meta and link flat property namespace, but other schemes are possible. A resource TREE is very loosely defined by ‘/’ in URLs, but does not provide any properties, so it is barely a tree.

WebDAV changed the HTTP data model to push it much more close to one traditional content model, supporting LOCK, RENAME, and TREE on top of HTTP, and an explicit property model independent of the resources in question. The property model is flat, with no STRUCTPROP although extending it is mentioned in the RFC, and no MULPROP. CLONE is allowed, as resources can have more than one URI. Updates to properties are not ATOMIC, as the PROPPATCH method can update some properties without others. The main HTTP resource is the body, which allows storage of one BINARY property, as the other properties are XML strings. This is pretty much the standard document management style set of properties.

As I was looking at Lily CMS recently, it is pretty different. There is no TREE, you have to construct it yourself. There is TYPE, no STRUCTPROP, there is NAMESPACE and PVERSION. There is no CAS or LOCK, conflicts are resolved by time of modification alone. It is an interestingly different model which I will look at in more detail in another post, as it chooses complexity in some areas and simplicity in others.

One of the common themes of the NoSQL movement is saying things along the lines of if you only give up this one feature we can make the storage layer that much simpler and faster, push some more work up to higher layers to resolve, issues like conflict resolution say, or referential integrity, or tree structures. This is the path of a simpler core repository which does not implement all common usage patterns for a CMS application, with some layering and conventions on top to build the next level. This is not so different from the relational model, with a low level relational algebra and a set of database management tools, then the application. What is still unclear is exactly where to make that split, but certainly large monolithic repository models that try to do everything listed above end up very large and complex to use.

There is definitely a case for moving some of these properties out of the repository into the authoring tools. Referential integrity at the repository level is actually quite hard to work with, as you cannot refer to something you are about to create, for example, but the authoring layer can provide tooling to help the user here.

I will post some follow-ups about some of the other issues arising from this, and what I think the best set of constraints to work in is, and more on the CMIS and JCR models.

Standards

• repository
• feeds
• CMIS
• JCR
• terminology, ways of thinking, industry model

Standardization has really started to affect the content management industry. The industry was very immature, a bit of a landgrab, and not very customer focussed. This has changed rapidly, with the wide adoption of the JCR standards, but particularly with process around CMIS. What is being set now is the model of the industry for the next five years, what the customers expect and what the products will deliver. Setting the agenda matters, and now is the opportunity to participate.

CMS as a platform

• build applications on a content platform
• API driven development
• SOA
• embed code everywhere in domain level scripting languages

A content management system is at last becoming less of a product that lets you do some stuff and more of a platform for working with content and building content centered applications and a service oriented world. Pervasive invasion of scripting languages such as Javascript into this is coming. The web programming model of pervasive agile scripting and rich REST APIs is going to be the norm, not large scale Java programming or application specific templating languages.

Co-opetition and community

• collaboration on standards, infrastructure
• open source as community
• twitter, blogs, enterprise 2.0
• end of NIH
• customers are community too

In the last year especially the landscape of content management as a community has changed. First through the standards processes, particularly CMIS and JCR, and then through social media, particularly twitter, as well as via events and blogs, there is now a growing cross vendor technical content management community, particularly with the open source players, and joint projects, for example with CMIS. This is in addition to the developer communities that are strongest around the open source products, although the .net products are trying hard to build around the Microsoft developer relations model. And of course the community of customers, who are becoming more vocal.

Rich content

• richer xhtml and xml
• enhanced metadata; richer metadata in other formats
• constraints not just validation
• RDF and semantic web, linked data
• relations and IA expressed in metadata
• enhancement via deeply integrated search
• document management, DAM and WCM converge
• richer presentation layers, richer APIs
• Flash is dead, plugins are dead, HTML5 is winning faster than anyone thought

As we have moved from document management, where the focus was on whole documents, to web content management, which is more component and assembly based, there has been a gradual push to do more with the documents. Standardized rich document semantics are after all one of the main advantages of web documents. It is taking a while but making use of the potential here is beginning to happen, now we have Google indexing rich snippets and even Tesco using RDFa. There is a lot more standardization work to do here.

In the front end the aim of this backend information enhancement is to build richer interfaces more easily, and to enhance findability, search and navigation, as well as to enable repurposing, richer APIs, and linked data. Authoring is the biggest challenge, as the majority of users need to be given interfaces that are independent of the IA, simple to use, but support generation and modification of complex data structures.

SAAS and the service business

• cloud
• internal delivery in a SAAS way
• devops
• APIs and standardization forced by SAAS
• changes to customer service model

Software as a service models are winning because no one wants to buy software as a product any more. I will cover more of this in another article I have been working on for a bit, but the main point is that enterprise software is a paid big ticket product is dead. The replacements are open source software and SAAS. These are not alternatives though, as people want the open source software delivered as a service, albeit maybe a more commoditized one if there are multiple providers, and many of the SAAS products delivered will be largely built of open source components by companies that run a mixed model. Microsoft is going headlong into cloud in a way that redefines what the operating system is. Even purchased software will be delivered in internal clouds.

This changes both how code as written and administered, with the web operations joining up into rolling delivery and creating the emerging field of devops. Developers need to understand operations and how to build code for this environment.

The service business as a business is different from the product business. Open source companies have got that better than product based vendors, but the less there is lockin the more key these changes become. The success of the customer using the services becomes the key business driver.

Performance and scaling, real time

• cloud has pushed scale up out of picture
• scale out transparently
• new technologies beyond RDBMS that fit CMS
• dynamic generation becoming the norm; Google pushing the performance thing; the industry norm of 100ms will fall
• real time becomes more important – dynamic updates, forget crawling, Google is going push
• backend: queuing (0MQ, AMQP)
• frontend: websockets, XMPP, long polling

Just buying big hardware for scale up is really becoming difficult; the web vibe has always been to scale horizontally on commodity hardware. There is a lot of development around scale out technologies such as NoSQL which fit into the WCM data models, which are those of the web after all.

As well as scaling for volume, latency and real time are becoming key. Google’s time to crawl has been falling rapidly, to a few or less, but is moving to real time with push updates. Twitter has really pushed the boundaries of expectation for real time. Behind the scenes there are a lot of technologies for efficiently pushing around notifications and events both at the backend and on the frontend. Real time is going to become increasingly pervasive.

Page generation times will need to fall; the standard industry benchmark of 100ms per component will probably need to be halved; overall total times under 1s will become the norm.

Security

• web increasingly hostile
• every bug is a potential security issue
• security focussed on fewer areas, push into the OS not out to applications

I read the excellent Linux Weekly News every week, and every week there are security exploits for many pieces of software; one that really struck me recently was the major exploit against the CMS e107. What happened here was the a group of crackers found a serious security flaw in the CMS, which they began attacking systematically. When the patch was released however, they already had control of the developer’s website via the flaw, so they replaced the patched version of the code with a version with a backdoor. Hacked websites are a vital part of the underground online crime scene, and a content management system is a high value target. Expect much more of this, and be prepared.

Narrowing the security into fewer points of vulnerability, sandboxing, using every available facet of the operating system’s security layers; make the most of processes, permissions, everything that you get there; I wrote more about this in an earlier post on emerging trends. File format parsing is another area of vulnerability that is common.

It is war out there on the internet, and many people underestimate or ignore the issues, and too many programmers do not code defensively by habit.

Summary

It is an exciting time in web content management right now; the industry is growing up beyond its beginnings as a way of getting web sites up, towards being the core of the broader content management industry. The choices made now will shape the industry; the next generation of products will be a big step forward forr the industry.

The data model for content management

I have another draft post on this subject in more detail, which I am working on as parrt of my REST modelling in content management work, but I will outline some of the types of data relations that are important. I will be quite abstract here, if you want more concrete examples you will have to wait for the other post: database models like the ones we are talking about here are more easily understood in the abstract I think.

First we our unit of modeling. This in itself is the first issue. Content management tends to deal with, at the conceptual level, something that looks like a document. It may be a fragment, in the sense that it is say a page component (asset if you use that terminology) rather than a whole item, but the unit for the user to edit and which is usually versioned is a structured object itself. The processing model tends to treat it as almost of binary blob, except that certain properties can be extracted, such as metadata, links in HTML and so forth, but it is stored as an item rather than decomposed further.

OK, so we have a piece of content and some attributes extracted from it as one basic model. This corresponds pretty much to the JCR data model for example. There are variations; sometimes people do not store metadata in the file formats, as historically many file formats had poor support for arbitrary structured metadata, although that is largely obsolete now, and the advantages of actually storing metadata and relations substantially within documents are high. External storage does not change the model much, just complicates processing and storage. Another variant, often seen in document management systems is to be able to have multiple ‘streams’ ie several document variants rolled into one, for example a video and a still from it. You can however from the modelling point of view regard these as anotehr compound document format kept together because conceptually they are a bundle of content; you might distribute them as a zip file if you havent got any other suitable container format.

So now we have a storage model where we have a blob, with rich media operations on it, and extracted structural and metadata information. There is also versioning to consider, but let us ignore that and treat it either as part of the blob, or as a new document with some relation to the old ones, those being the two core versioning models, this does not really affect anything else.

There are two kinds of metadata, although they are more similar than they appear, properties and relations. Properties are the standard attributes (this picture depicts sheep), while relations join two items in the repository (this is a cropped version of this other picture). Although this distinction seems clear, in the end richer information architectures demand that everything becomes a relation, so I can browse a sheep node and find all the sheep items, turning every attribute value of any significance into a node with relations instead. Pure attribute values are only left for the less interesting properties (this PDF file is 176k in size).

They are also less interesting from a relational versus non relational storage point of view, although there is one important point, which is the dense versus sparse question, so let us take a look at this. Most real world attributes are sparse, that is most attributes aare not set on most items. In the relational model we have a row for our item, and columns for all the attributes, so we are saying most are NULL. (I was brought up on matrix algorithms and still think in terms of sparse versus dense matrices as this is exactly the same problem, and matrices represent graphs anyway). Storing huge mainly null tables is not very efficient, so there are two common practices in relational mapping of attributes in content management systems. First is to define a type based system, where a particular type of content item is defined to have certain attributes (or at least fewer NULLs!), and each set of that type therefore can have its own table which is assumed to have fewer NULL values. Mixins, sets of properties that live across types can potentially be added to this model, as can inheritance schemes, but the basic idea is one table per type. This gives a nice simple direct database programming model, and causes a complete nightmare if you ever want to change the schema, for example add an attribute, as for any large database most DBMSs will effectively shutdown the system while a schema change takes place, as schema changes require pretty much all locks. Silverstripe is one example of a content management system built like this; there are many others.

The alternative is the entity attribute value (EAV) model (terrible Wikipedia article, please fix), where rather than a direct mapping of the attributes to relations, you indirectly map, creating a table that joins entites, attributes and values; this table of course looks just like RDF triples. Doing this though loses everything that makes a relational database useful: constraints, typing, query optimization. It adds an extra layer of logical schema above the physical schema which the database layer does not understand. This is a pretty common relational mapping for content management systems, as it allows full flexibility in defining and redefining attributes. To implement well it needs a large mid layer to manage the constraints, provide an API layer, generate efficient queries, effectively to manage the logical layer to physical layer map. The Drupal CCK is an example of this model.

Of course this is not to say that neither of the two relational models do not work. The direct mapping works well with simple, unchanging content types in small websites, for example, or in models where attributes are not very sparse, or the sparseness is worth the overhead, and changing the schema is rare. EAV works well too, if managed carefully; it helps if the type of queries required on the model are not too complex.

Once you add relations as well as attributes, the already difficult mapping layer gets harder; you add another set of operations (recursion to handle tree structures) that the relational model does not handle well, so you may need to add more into the mapping layer. The promise of NoSQL is that you can bypass this for these types of applications, and program directly to a database model that handles sparse attributes and relations natively. But how much do the NoSQL databases get you? You can argue that if you are already looking at EAV, then you are already not getting much from a relational database, and you are building a modeling layer on top of it, so dropping that and going for something that maps the logical data layer directly does make sense from a development point of view. Whether that really helps performance is less clear; much of the original work for NoSQL has come out of huge scaling, big problems, not actually providing efficient solutions to the types of data mapping problem we are seeing here on a medium scale; of course for huge sites there may be benefits.

The types of NoSQL database vary in their level of support for attributes and relations as they are used in content management. Document oriented databases do not give you much more than retrieval of content items; associative ones give key value type attribute lookups; graph databases should let you query relations directly, expressing the types of queries that are needed for information architecture problems directly, in principle. Examples I am thinking of are things like tag clouds, which is simple to express as a graph problem as it is simple a count of the number of edges from a set of nodes. Indeed most information architecture problems look like graph problems, and also like OLAP processing operations which also do not work well on relational databases. And of course one of the things that NoSQL has shared with OLAP is the use of denormalization; you can use simpler models if you denormalize data to match the queries you will be using, rather than assuming that the types of query you will use can necessarily be optimized and made efficient by a general purpose system.

Denormalization is not without its difficulties, although arguably it could become a tool embedded in databases like indexes are now. One of the issues with NoSQL is most of the database systems leave denormalization to the user: you need to use it because joins are not available, but you have to manage that yourself. Building an infrastructure to explicitly manage denormalization as a first class database item akin to an index might be interesting. So that gives us a first issue, as in any NoSQL system except a graph database we will either need to denormalize or compose queries to get the results we want.

So I think there are four realistic models for content management backends going forward:

1. The direct relational model for small systems with simple data models, rare attribute changes, little or no use of relations.
2. EAV models wrapped in a content modeling layer; JCR is an example of this, hiding the underlying SQL layer very well, and indeed allowing it to be replaced with another underlying storage model potentially; I am sure someone is testing a Neo4J backend somewhere. This is where most production solutions are at now.
3. Direct, nondenormalized graph database backends, with the raw content stored in a document store. Cuts out a special purpose middle level by mapping the domain more directly. As Emil Neo says, it may not scale right up as far as the othe NoSQL technologies, but it cuts complexity of implementation; there are also issues about whether all the kinds of queries required are available efficiently. I think this will be the sweet spot in a few years once the products mature and we see more open source activity in the field. Of course RDF based solutions, for example using SPARQL fall into this category too, and the maturity of products around these technologies will help drive this category as well as the NoSQL models.
4. Big, denormalized systems, probably with software support for managing the denormalization, and using underlying simple but scalable technologies like key-value stores. These already exist in large scale web applications, but may remain niche if the development effort remains high. If frameworks for modelling more easily on these turn up they may trickle down for performance reasons even on smaller datasets; a key value store runs fine on a relational database backend, although the types of processing required probably means a specialized backend is useful.

Note that the Lily CMS which there was a talk about fits very much into the fourth option above; this is where the NoSQL technologies have perhaps seen most use, but I think there will be a lot of work in order to build a CMS like this now, in particular in terms of tools to support denormalization strategies that are needed. The outlined approach sounded much like the outlines I have been thinking about for this type of model, although I would focus more on tooling for denormalized queries and less on scaling other parts like full text search right now. It will be interesting to follow the progress of this project.

We are at an interesting juncture, where it looks like there are some options that will let us do domain modelling in a way that corresponds more directly to the domain, but there are a lot of interesting challenges on the way.

Just press the play button to move around.

@McBoof I left London dazed and confused when it comes to open source. Somebody pls. help me explain what open source really means #idiot

Now I only spoke to him very briefly before he had to rush to the airport, but hopefully the following will be helpful: first an overview of the important things about open source in general, and then how they are and will affect content management in particular.

Open Source

I think it is easiest for software developers to understand open source. It came from that community, and it addresses our needs. For a long time no one outside that community was really concerned with it. I think the first time I noticed someone who was not a developer showing an interest was when I was stopped at a tram stop in Vienna by an American as I was wearing a Redhat T-shirt just after their float and was asked who the next open source IPO was going to be, that was 1999 ten years ago now. I suppose that IPO was a big event in the spreading awareness of open source, although it did not perhaps spread much information about what it was really about.

I think the best place to start with trying to understand open source is with three things. I tend to have a bit of a historical approach to things…

The first is Richard Stallman. I recommend him in person rather than in writing actually. Actually that reminds me the first time I ever saw him I was sitting in the Foundry in Old Street and he walked in and proceeded to autograph a woman’s breasts. Anyone who wants to understand open source should hear him explain the roots of the open source movement. I will not really try to explain all that here, but openness is what created the scientific method, and the idea that software got to the point where it was no longer possible to make it do what you wanted because you did not have access to source code, the point where you could not build on stuff any more or fix it, where control of your tools is taken away is a key part of it. Some people have tried to sanitize Richard out of things (the open source vs free software mess) but that is a mistake.

Second is Eric Raymond’s essay The Cathedral and the Bazaar which wass very influential at the birth of commercial open source. It is strictly about software development methodologies, and much of the discussion about the cathedral methods is applicable to open source software too. It is about the huge changes that the internet brought in open source development, the birth of a development method that no longer copied the methods of closed source development but utilised the openness to create true large scale community development in a way that was not possible before, and which closed source cannot replicate. Linux is of course the classic early example of this.

Which brings us to the third thing, community. Open source is first of all participatory, not just for consumption, perhaps a bit against the grain of late twentieth century culture. Actually I am an optimist, with Clay Shirky and against the sitcom, and think culture is swinging this way but we shall see. So for open source, start by using it, then participate. No you do not have to code, although you can learn, there are other ways, bugs, documentation, all sorts. If you just want too see what the community looks like, I can’t recommend anything better than going to a good conference, like FOSDEM next month in Brussels.

Open source in content management

Open source has not affected content management much yet. Almost all content management by volume takes place on open source products (by volume WordPress, Joomla! and Drupal far outweigh anything else). By value it is less clear, open source always has an issue with by value calculations as the revenue models are different, Linux is not the leading server operating system by value, but is by installed base, but is also probably by the value of the services running on it.

But arguably open source content management software has not affected the industry yet, looking now at the larger installations, and the areas that Janus is interested in, indeed that I am. The industry has grown up in a mess as far as standards, ideas, infrastructure are concerned, but the reality checkpoint has been reached. Two standards have so far started to change the technology landscape of content management, JCR and CMIS, and almost all the implementations of these are open source, and most are cross-vendor projects. This change will grow as more standardization and commoditization sweeps the industry, as the industry adopts a web infrastructure rather than the pre-web legacies inherited from the document management history of the business. Everything that this business deals with will be served through the web; almost all web infrastructure is open source software; content management will be no different.

In this field this is all just beginning. Like open source as I said above, it started with developers, about more efficient ways of building, architecting and delivering software; in terms of influence on the end users it is still small. But things are turning as people become aware of open source in the industry, but they clearly still need some help understanding it. I hope this has helped.

Other links are here: Jon Marks, Irina Guseva, Ian Truscott; other participants Adriaan Bloem, Andrew Liles, Philippe Parker (first use of Wave over GotoMeeting?)

Well it was fun. Technical difficulties, lost sync and crashed a few browsers, some people lost whole machines though. Safari coped better than Firefox. It took a while to realize what was happening here, hey but this is in beta!

As a brainstorming tool at worked pretty well. I thought it scaled pretty well. The named cursors indicate who is in the bit you are in, but for brainstorming you can look, write another point, move, continue, not edit much. After half an hour of getting to bulletted lists, a bit of moving around the heavy writing started (after a discussion at the top in our proxy process section; we should have split the thing up a bit).

There is a great tendancy to write temporary notes about the discussion and then just delete them. Which feels odd, data and metadata together of course. The editing process was odd, you would find orphaned bits, move things, try to join stuff up to make it flow, while it was all changing around you. Pretty chaotic. Bits that no one expanded into prose got junked (quite a good edit method, as they couldnt stand up themselves).

Here is the “finished” article… which cannot be attributed to anyone individually of course… the subject was chosen about 10 minutes in, just as something people would have something they could easily contribute into this situation, there are some good points in there though!

Things We Hate About Content Management

- By The Motley Crew

It was a lovely Friday morning/afternoon, and we were Waving. The experiment initiated by McBoof (yes, that one) brought together 6 CMS folks from around the world. The event gathered together analysts, journalists, vendors, system integrators to Wave on a topic that was decided at that very moment. We had one hour (in between conference calls and other job thingys) to pick a topic and Wave it.

A little collab on what exactly to Wave about later, we decided to do “a mindmap of things we find annoying in CMSs.” To up the ante, we also decided to take the original bullet points (deemed “too easy”) and convert the whole thing to prose. Was the tool given really up to the task? Were our minds flexible enough to wrap around this kind of realtime collaboration?

In the beginning — we blame the tool — we were Drowning, not Waving. We (almost) didn’t fight about edits. We almost didn’t step on each other’s toes. All in all, it turned out to be a fun and productive collaborative exercise. Read on to see for yourself.

Cosmetic Issues

There really should be a CMS UI fashion police. As there should be a Magic Quadrant for shoes and handbags. Why? Well, there’s a couple of issues.

For instance, sloppy, non-designed design. You know the kind of thing that has not been thought about and reworked and made to feel right. The sort of thing coders do if you don’t force them. But at the same time, over-designed interfaces can be just as bad: the designers and developers really need to be on speaking terms.

When building a system that works, you can’t have the development team in the basement on a sustenance of Jolt coding away into the night, and the designers in the penthouse in turtleneck sweaters sipping espressos. Too many CMS designs end up being programmer vs. end-user friendly. And this is not the best way to charm away those marketing and web content folks.

Developers and designers need to talk to each other and essentially, both should talk to users – not just eat your own dogfood – but listen to what dogs like to eat. A developer or UI designer are not content editors, marketers or knowledge and information workers.

Some vendors say that the agonizingly and depressingly black UI backgrounds are hip and modern. Well, they are not, really. Who told you that? Especially if you add a Star Trek theme to it and sprinkle in some stars and cosmic swirls, because if Apple does it, it must be cool right? Not pointing any fingers, but I would quit if I were a content manager having to spend my 9-5 staring into the “black hole” of some of the CMS UIs that are out there on the market.

Even pop-ups seem less annoying when compared to dark UIs. Which brings us onto…

Interface Issues

Interfaces need a comfortable lived in feel. Content management is something people work with every day, it is their interface to their job. You meet people who hate the interface, and that makes their work a heap of pain. I have seen people who describe the 44 clicks it takes to insert an image. You have a responsibility to these people, to make them love the content and make the tool disappear.

We all hate it when the interface does something on its own that ruins your context. E.g. a page refresh, or in Wave the jumping around of the scrolled window in some cases Or the lack of an easy way to bookmark, so you can reference someone to the content. Remember people will be collaborating and need to send links around. Make sure the UI is a proper web application with URLs. And why do tasks that are easy to describe and often repeated in exactly the same way still take more than a few clicks? (Or maybe even dozens of clicks.) With bonus points for forcing users to use dialogs or tabs to enter mandatory information. Remember people do not have all the information in the right order.

Also, we need sane conflict merges. Check in and check out is too extreme for most uses. But people want to edit offline still. Of course Wave doesn’t have an offline: Google thinks this problem is going away, it’s real time so there are never conflicts (that’s defined in the XML protocol; it’s quite interesting if you are that way geeky). Does Google have the right answer here? Well, the Motley Crew is struggling here, and some browsers lost sync during this experiment.

“Power users” (those who use it all day long) of CMSs needed to have a “Desktop” experience. What does Desktop Experience mean? Well, it doesn’t really have to be on the desktop — these days it is perfectly possible to get very close to a hitherto Desktop experience in a browser or similar. these are qualities: very low latency from action to response, no page refreshes, modal and modal-less dialog boxes as appropriate, “push” notification.

Architectural Issues

Architectural issues of the wave overtook any architectural issues of Content Management Systems. The fact that we authored this entire article in a single blip didn’t help, and slowed everything down enormously. McBoof learned the hard way that he really need a new laptop and spent most of the session giving his machine CPR. Next time we’ll do each paragraph in its own blip to stop FireFox going down like a Led Zeppelin.

Monolithic systems. Build it out of pieces that the client can not use all of. Obviously your pieces may work together better, but there should be components. Do not try to reinvent all kinds of wheel. “Best of breed,” though, is just another weasel marketing idea, as if systems are pinnacles not about meeting requirements.

Marketeers are adroit at using the term Best Practice to position Their Way as the only way that a particular matter can be solved. (Many of us live in that netherland of having to pedal that point of view, but it is a falsehood that the careful buyer should try to see through.) I think this devalues genuine best practice, vendors should cite references

Most often a marketeer’s Best Practice view is the only one they subscribe to as their product development has paddled up the wrong stream and cannot or won’t reverse their architectural design (probably because of the cost of doing so). This intransigence most often causes a product to doom itself. (Think of IBM and The Mainframe Is The Only Way To Do Serious Business).

Who really still believes that there is a place in this world for Flash or Java Applet based Rich Text Editors? TinyMCE, FCKeditor and others are filling the gap left by Ektron when they bit the hand that feeds and entered the CMS market. Ephox is trying to spread, but I find it difficult to come up with an excuse to use an Applet over HTML with javascript these days. Stick with the standard.

Business Issues

Where you are buying into something that you may very well need to change or integrate with there is strong benefit in considering Open Source. Open Source used to frighten commercial software companies but we have come along way on that road to understand that commercial organisation can operate in an Open Source world and benefit. This does not necessarily mean that their prized system needs to be fully opened up, but taking the spirit of it to mean that you are completely open to people seeing and learning from your code how it operates.

Exactly what you need to see opened up varies. In a CMS there may be a subsystem that stores the content or one that allows a Rich Text Editor. These arguably don’t need to be opened up, but when a CMS ships with modules for, for example, an RSS feed widget, calendaring tool, prebuilt webforms, users who then want a variation on this module can benefit from seeing how the “pros” did it, they can then use it as a starting point for their own different implementation.

We really don’t need vendors that pay lip service to the buzzwords. When they think the new CMS buzzword “engagement” is just a screenshot of Google Analytics. Or when they add an image picker and call it DAM. And a cross-over between WCM and ECM? Don’t think WCM is like ECM and it’s about organizing content, not about effectively communicating with the audience. And don’t think that if you organize the content, you can automatically communicate effectively.

Completely different, but equally frustrating, is procurement (and the procedures that go with it.) Procurement folk don’t recognise the importance of user adoption to the success of the project — of the black background and all the UI issues pointed out previously. If a CMS is procured according to procedure, the selection is a success to them. But those same rules are often a recipe for ignoring what the users really need.

At the same time, budgets that aren’t transparent are an issue – customer and vendor should be able to have a sensible grown up conversation. As a customer, of course you want good value, but how cheap are you? But to vendors: many licensing models don’t make any sense, and force you to do stupid things. People are scared to have that conversation – the best architectural fit first I say, lets figure out an appropriate license around that.

Conclusion

So much hatred rolled up into a tight little ball of anti-CMS rage. Who would have expected it from such a respected bunch of CMS folk. We hate the designs, the interfaces, the architectures and the business. Time for a beer/wine? Wave good bye!

So what are these, why do you want them, and what do they mean for your content strategy?

Toby Bell at Gartner says

Smart companies have begun linking more of their content to industry-specific, human-centric processes, such as insurance claims handling, or supporting research on new drug development. This approach usually means building or modifying the content-enabled vertical applications (CEVAs) on top of ECM environments. CEVAs typically help to automate complex processes that previously required workers to manually sort through paper documents and other forms of content (in effect, a way to manage down costs of exception handling) and optimize the remainder of the work.

This seems to be not a great summary though. Look at it from another view. Your enterprise content was disorganized, living on network shares, random websites, legacy systems, all over the place. Your ECM strategy was to first to consolidate to reduce costs of multiple systems, and to improve findability, get to a base enterprise content management position. But what next? Where is the next value?

Content strategy starts here. There are many parts to this, covering creation, lifecycle and reuse, audit, consolidation quality and so on. The CEVA part is about the delivery and interaction with content within other non content focussed areas of the business. Content management often seems to be about specific content focussed parts of a business, such as, historically, technical documentation and, more recently, online marketing material. Plus a load of unstuctured stuff like emails and generic office documents. The areas such as technical documentation have had high value often for legal and regulatory reasons, so structured processes were created early; these effectively created the content management industry initially. Web content historically had different solutions because it turned up and became important when the general purpose tools (Word!) could not usefully author it.

But all the stuff classified as “other” does have underlying processes, content processes. Some are formalized in systems, the original paperless office systems, the roots of the document management industry in forms and scanned paperwork processing. This stuff generally sits more on the “data” not “content” side of business processes. Long term this distinction is not such a useful one, and data and content resources will merge together into a single enterprise resource architecture. The majority of processes with content though take place through informal channels, particularly email with Microsoft Office documents. These are the document types you tried to take control of through ECM.

So ECM took the documents that were behind many processes and made them findable and organized them. But at the base level a content repository is just that, a repository. It deals with basic issues such as versioning and permissions, search and findability, and some organization, but it does not really deal with process and processing.

Process and processing are the valuable parts in the lifetime of most content. Imagine the lifetime of an insurance contract say, with payments and claims and disputes, or an employees personnel file, or a technical manual over the lifetime of a product. A CEVA or CCA is an application to support these lifetime processes.

It is also an application to support the relation of a document’s lifetimes processes to other systems. Your CRM system may need to know about insurance claims, your sales department may need to know about expiry, your website may need to know about new documentation releases, content changes do not happen in a vacuum.

One class of CCA that is common but is rarely perceived as that is a software application with embedded content. Once that was just embedded “help screens” with content tools to manage them, then came internationalization, with a different set of tools. But these desktop applications are being rapidly replaced by web applications. Web applications are much more content driven, they may live in an SEO facing world, they may live in a customer facing world that may consider usability, they may be multilingual, and they are not driven by the developer-centric ideas of help screens and manuals. Content and application can live together, but this requires new ways of using, reusing and versioning content, and pulling content out of application release cycles so it can reflect non application changes, such as the marketing environment, usability improvements, corrections and enhancements. These applications were historically development led but as they mature the content aspects become key business drivers, needing content management integration.

So what do you need to build this type of application, and what should your decision criteria on platforms be?

Stéphane Croisier says in a good survey “So rapid raw composite assembly, fast integration and ease of use are the three new pillars of next generation content solutions.”

The first thing to bear in mind is that you need more than a repository, you are looking for an application platform too. The ease of use issue is important. Long term you need to be looking for something that staff can build simple tools from, even if you are hiring specialists for the complex projects. Ease of use is a two way thing, as you need an easy to use platform that lets you build easy to use applications. And ease of modification and maintenance is equally important as these applications may need to be fluid. You are likely to need external support to build more complex applications on the same platform, so availability of this is important too. Ignore the jargon of portlets, widgets and mashups: none of these so called standardisations have much traction; we are talking application development, use what you have or can hire developers to do. Ask the vendors what their platform strategy is.

Stéphane identifies a trend towards solutions, ready to go solutions for common problems; these may be useful but I would not choose a development platform on the basis of the availability of particular solutions or you may end up buying a platform for every solution. A longer term view of the viability of a platform for other solutions is necessary too.

Long term, remember that content application strategy is part of content strategy, and comes after that. You need to know what your content applications are and will be, and have built an underlying respository, authoring and reuse strategy first. Applications are where developers need to interact with this to achieve the long term goals.