Standards

• repository
• feeds
• CMIS
• JCR
• terminology, ways of thinking, industry model

Standardization has really started to affect the content management industry. The industry was very immature, a bit of a landgrab, and not very customer focussed. This has changed rapidly, with the wide adoption of the JCR standards, but particularly with process around CMIS. What is being set now is the model of the industry for the next five years, what the customers expect and what the products will deliver. Setting the agenda matters, and now is the opportunity to participate.

CMS as a platform

• build applications on a content platform
• API driven development
• SOA
• embed code everywhere in domain level scripting languages

A content management system is at last becoming less of a product that lets you do some stuff and more of a platform for working with content and building content centered applications and a service oriented world. Pervasive invasion of scripting languages such as Javascript into this is coming. The web programming model of pervasive agile scripting and rich REST APIs is going to be the norm, not large scale Java programming or application specific templating languages.

Co-opetition and community

• collaboration on standards, infrastructure
• open source as community
• twitter, blogs, enterprise 2.0
• end of NIH
• customers are community too

In the last year especially the landscape of content management as a community has changed. First through the standards processes, particularly CMIS and JCR, and then through social media, particularly twitter, as well as via events and blogs, there is now a growing cross vendor technical content management community, particularly with the open source players, and joint projects, for example with CMIS. This is in addition to the developer communities that are strongest around the open source products, although the .net products are trying hard to build around the Microsoft developer relations model. And of course the community of customers, who are becoming more vocal.

Rich content

• richer xhtml and xml
• enhanced metadata; richer metadata in other formats
• constraints not just validation
• RDF and semantic web, linked data
• relations and IA expressed in metadata
• enhancement via deeply integrated search
• document management, DAM and WCM converge
• richer presentation layers, richer APIs
• Flash is dead, plugins are dead, HTML5 is winning faster than anyone thought

As we have moved from document management, where the focus was on whole documents, to web content management, which is more component and assembly based, there has been a gradual push to do more with the documents. Standardized rich document semantics are after all one of the main advantages of web documents. It is taking a while but making use of the potential here is beginning to happen, now we have Google indexing rich snippets and even Tesco using RDFa. There is a lot more standardization work to do here.

In the front end the aim of this backend information enhancement is to build richer interfaces more easily, and to enhance findability, search and navigation, as well as to enable repurposing, richer APIs, and linked data. Authoring is the biggest challenge, as the majority of users need to be given interfaces that are independent of the IA, simple to use, but support generation and modification of complex data structures.

SAAS and the service business

• cloud
• internal delivery in a SAAS way
• devops
• APIs and standardization forced by SAAS
• changes to customer service model

Software as a service models are winning because no one wants to buy software as a product any more. I will cover more of this in another article I have been working on for a bit, but the main point is that enterprise software is a paid big ticket product is dead. The replacements are open source software and SAAS. These are not alternatives though, as people want the open source software delivered as a service, albeit maybe a more commoditized one if there are multiple providers, and many of the SAAS products delivered will be largely built of open source components by companies that run a mixed model. Microsoft is going headlong into cloud in a way that redefines what the operating system is. Even purchased software will be delivered in internal clouds.

This changes both how code as written and administered, with the web operations joining up into rolling delivery and creating the emerging field of devops. Developers need to understand operations and how to build code for this environment.

The service business as a business is different from the product business. Open source companies have got that better than product based vendors, but the less there is lockin the more key these changes become. The success of the customer using the services becomes the key business driver.

Performance and scaling, real time

• cloud has pushed scale up out of picture
• scale out transparently
• new technologies beyond RDBMS that fit CMS
• dynamic generation becoming the norm; Google pushing the performance thing; the industry norm of 100ms will fall
• real time becomes more important – dynamic updates, forget crawling, Google is going push
• backend: queuing (0MQ, AMQP)
• frontend: websockets, XMPP, long polling

Just buying big hardware for scale up is really becoming difficult; the web vibe has always been to scale horizontally on commodity hardware. There is a lot of development around scale out technologies such as NoSQL which fit into the WCM data models, which are those of the web after all.

As well as scaling for volume, latency and real time are becoming key. Google’s time to crawl has been falling rapidly, to a few or less, but is moving to real time with push updates. Twitter has really pushed the boundaries of expectation for real time. Behind the scenes there are a lot of technologies for efficiently pushing around notifications and events both at the backend and on the frontend. Real time is going to become increasingly pervasive.

Page generation times will need to fall; the standard industry benchmark of 100ms per component will probably need to be halved; overall total times under 1s will become the norm.

Security

• web increasingly hostile
• every bug is a potential security issue
• security focussed on fewer areas, push into the OS not out to applications

I read the excellent Linux Weekly News every week, and every week there are security exploits for many pieces of software; one that really struck me recently was the major exploit against the CMS e107. What happened here was the a group of crackers found a serious security flaw in the CMS, which they began attacking systematically. When the patch was released however, they already had control of the developer’s website via the flaw, so they replaced the patched version of the code with a version with a backdoor. Hacked websites are a vital part of the underground online crime scene, and a content management system is a high value target. Expect much more of this, and be prepared.

Narrowing the security into fewer points of vulnerability, sandboxing, using every available facet of the operating system’s security layers; make the most of processes, permissions, everything that you get there; I wrote more about this in an earlier post on emerging trends. File format parsing is another area of vulnerability that is common.

It is war out there on the internet, and many people underestimate or ignore the issues, and too many programmers do not code defensively by habit.

Summary

It is an exciting time in web content management right now; the industry is growing up beyond its beginnings as a way of getting web sites up, towards being the core of the broader content management industry. The choices made now will shape the industry; the next generation of products will be a big step forward forr the industry.

The data model for content management

I have another draft post on this subject in more detail, which I am working on as parrt of my REST modelling in content management work, but I will outline some of the types of data relations that are important. I will be quite abstract here, if you want more concrete examples you will have to wait for the other post: database models like the ones we are talking about here are more easily understood in the abstract I think.

First we our unit of modeling. This in itself is the first issue. Content management tends to deal with, at the conceptual level, something that looks like a document. It may be a fragment, in the sense that it is say a page component (asset if you use that terminology) rather than a whole item, but the unit for the user to edit and which is usually versioned is a structured object itself. The processing model tends to treat it as almost of binary blob, except that certain properties can be extracted, such as metadata, links in HTML and so forth, but it is stored as an item rather than decomposed further.

OK, so we have a piece of content and some attributes extracted from it as one basic model. This corresponds pretty much to the JCR data model for example. There are variations; sometimes people do not store metadata in the file formats, as historically many file formats had poor support for arbitrary structured metadata, although that is largely obsolete now, and the advantages of actually storing metadata and relations substantially within documents are high. External storage does not change the model much, just complicates processing and storage. Another variant, often seen in document management systems is to be able to have multiple ’streams’ ie several document variants rolled into one, for example a video and a still from it. You can however from the modelling point of view regard these as anotehr compound document format kept together because conceptually they are a bundle of content; you might distribute them as a zip file if you havent got any other suitable container format.

So now we have a storage model where we have a blob, with rich media operations on it, and extracted structural and metadata information. There is also versioning to consider, but let us ignore that and treat it either as part of the blob, or as a new document with some relation to the old ones, those being the two core versioning models, this does not really affect anything else.

There are two kinds of metadata, although they are more similar than they appear, properties and relations. Properties are the standard attributes (this picture depicts sheep), while relations join two items in the repository (this is a cropped version of this other picture). Although this distinction seems clear, in the end richer information architectures demand that everything becomes a relation, so I can browse a sheep node and find all the sheep items, turning every attribute value of any significance into a node with relations instead. Pure attribute values are only left for the less interesting properties (this PDF file is 176k in size).

They are also less interesting from a relational versus non relational storage point of view, although there is one important point, which is the dense versus sparse question, so let us take a look at this. Most real world attributes are sparse, that is most attributes aare not set on most items. In the relational model we have a row for our item, and columns for all the attributes, so we are saying most are NULL. (I was brought up on matrix algorithms and still think in terms of sparse versus dense matrices as this is exactly the same problem, and matrices represent graphs anyway). Storing huge mainly null tables is not very efficient, so there are two common practices in relational mapping of attributes in content management systems. First is to define a type based system, where a particular type of content item is defined to have certain attributes (or at least fewer NULLs!), and each set of that type therefore can have its own table which is assumed to have fewer NULL values. Mixins, sets of properties that live across types can potentially be added to this model, as can inheritance schemes, but the basic idea is one table per type. This gives a nice simple direct database programming model, and causes a complete nightmare if you ever want to change the schema, for example add an attribute, as for any large database most DBMSs will effectively shutdown the system while a schema change takes place, as schema changes require pretty much all locks. Silverstripe is one example of a content management system built like this; there are many others.

The alternative is the entity attribute value (EAV) model (terrible Wikipedia article, please fix), where rather than a direct mapping of the attributes to relations, you indirectly map, creating a table that joins entites, attributes and values; this table of course looks just like RDF triples. Doing this though loses everything that makes a relational database useful: constraints, typing, query optimization. It adds an extra layer of logical schema above the physical schema which the database layer does not understand. This is a pretty common relational mapping for content management systems, as it allows full flexibility in defining and redefining attributes. To implement well it needs a large mid layer to manage the constraints, provide an API layer, generate efficient queries, effectively to manage the logical layer to physical layer map. The Drupal CCK is an example of this model.

Of course this is not to say that neither of the two relational models do not work. The direct mapping works well with simple, unchanging content types in small websites, for example, or in models where attributes are not very sparse, or the sparseness is worth the overhead, and changing the schema is rare. EAV works well too, if managed carefully; it helps if the type of queries required on the model are not too complex.

Once you add relations as well as attributes, the already difficult mapping layer gets harder; you add another set of operations (recursion to handle tree structures) that the relational model does not handle well, so you may need to add more into the mapping layer. The promise of NoSQL is that you can bypass this for these types of applications, and program directly to a database model that handles sparse attributes and relations natively. But how much do the NoSQL databases get you? You can argue that if you are already looking at EAV, then you are already not getting much from a relational database, and you are building a modeling layer on top of it, so dropping that and going for something that maps the logical data layer directly does make sense from a development point of view. Whether that really helps performance is less clear; much of the original work for NoSQL has come out of huge scaling, big problems, not actually providing efficient solutions to the types of data mapping problem we are seeing here on a medium scale; of course for huge sites there may be benefits.

The types of NoSQL database vary in their level of support for attributes and relations as they are used in content management. Document oriented databases do not give you much more than retrieval of content items; associative ones give key value type attribute lookups; graph databases should let you query relations directly, expressing the types of queries that are needed for information architecture problems directly, in principle. Examples I am thinking of are things like tag clouds, which is simple to express as a graph problem as it is simple a count of the number of edges from a set of nodes. Indeed most information architecture problems look like graph problems, and also like OLAP processing operations which also do not work well on relational databases. And of course one of the things that NoSQL has shared with OLAP is the use of denormalization; you can use simpler models if you denormalize data to match the queries you will be using, rather than assuming that the types of query you will use can necessarily be optimized and made efficient by a general purpose system.

Denormalization is not without its difficulties, although arguably it could become a tool embedded in databases like indexes are now. One of the issues with NoSQL is most of the database systems leave denormalization to the user: you need to use it because joins are not available, but you have to manage that yourself. Building an infrastructure to explicitly manage denormalization as a first class database item akin to an index might be interesting. So that gives us a first issue, as in any NoSQL system except a graph database we will either need to denormalize or compose queries to get the results we want.

So I think there are four realistic models for content management backends going forward:

1. The direct relational model for small systems with simple data models, rare attribute changes, little or no use of relations.
2. EAV models wrapped in a content modeling layer; JCR is an example of this, hiding the underlying SQL layer very well, and indeed allowing it to be replaced with another underlying storage model potentially; I am sure someone is testing a Neo4J backend somewhere. This is where most production solutions are at now.
3. Direct, nondenormalized graph database backends, with the raw content stored in a document store. Cuts out a special purpose middle level by mapping the domain more directly. As Emil Neo says, it may not scale right up as far as the othe NoSQL technologies, but it cuts complexity of implementation; there are also issues about whether all the kinds of queries required are available efficiently. I think this will be the sweet spot in a few years once the products mature and we see more open source activity in the field. Of course RDF based solutions, for example using SPARQL fall into this category too, and the maturity of products around these technologies will help drive this category as well as the NoSQL models.
4. Big, denormalized systems, probably with software support for managing the denormalization, and using underlying simple but scalable technologies like key-value stores. These already exist in large scale web applications, but may remain niche if the development effort remains high. If frameworks for modelling more easily on these turn up they may trickle down for performance reasons even on smaller datasets; a key value store runs fine on a relational database backend, although the types of processing required probably means a specialized backend is useful.

Note that the Lily CMS which there was a talk about fits very much into the fourth option above; this is where the NoSQL technologies have perhaps seen most use, but I think there will be a lot of work in order to build a CMS like this now, in particular in terms of tools to support denormalization strategies that are needed. The outlined approach sounded much like the outlines I have been thinking about for this type of model, although I would focus more on tooling for denormalized queries and less on scaling other parts like full text search right now. It will be interesting to follow the progress of this project.

We are at an interesting juncture, where it looks like there are some options that will let us do domain modelling in a way that corresponds more directly to the domain, but there are a lot of interesting challenges on the way.

Just press the play button to move around.

@McBoof I left London dazed and confused when it comes to open source. Somebody pls. help me explain what open source really means #idiot

Now I only spoke to him very briefly before he had to rush to the airport, but hopefully the following will be helpful: first an overview of the important things about open source in general, and then how they are and will affect content management in particular.

Open Source

I think it is easiest for software developers to understand open source. It came from that community, and it addresses our needs. For a long time no one outside that community was really concerned with it. I think the first time I noticed someone who was not a developer showing an interest was when I was stopped at a tram stop in Vienna by an American as I was wearing a Redhat T-shirt just after their float and was asked who the next open source IPO was going to be, that was 1999 ten years ago now. I suppose that IPO was a big event in the spreading awareness of open source, although it did not perhaps spread much information about what it was really about.

I think the best place to start with trying to understand open source is with three things. I tend to have a bit of a historical approach to things…

The first is Richard Stallman. I recommend him in person rather than in writing actually. Actually that reminds me the first time I ever saw him I was sitting in the Foundry in Old Street and he walked in and proceeded to autograph a woman’s breasts. Anyone who wants to understand open source should hear him explain the roots of the open source movement. I will not really try to explain all that here, but openness is what created the scientific method, and the idea that software got to the point where it was no longer possible to make it do what you wanted because you did not have access to source code, the point where you could not build on stuff any more or fix it, where control of your tools is taken away is a key part of it. Some people have tried to sanitize Richard out of things (the open source vs free software mess) but that is a mistake.

Second is Eric Raymond’s essay The Cathedral and the Bazaar which wass very influential at the birth of commercial open source. It is strictly about software development methodologies, and much of the discussion about the cathedral methods is applicable to open source software too. It is about the huge changes that the internet brought in open source development, the birth of a development method that no longer copied the methods of closed source development but utilised the openness to create true large scale community development in a way that was not possible before, and which closed source cannot replicate. Linux is of course the classic early example of this.

Which brings us to the third thing, community. Open source is first of all participatory, not just for consumption, perhaps a bit against the grain of late twentieth century culture. Actually I am an optimist, with Clay Shirky and against the sitcom, and think culture is swinging this way but we shall see. So for open source, start by using it, then participate. No you do not have to code, although you can learn, there are other ways, bugs, documentation, all sorts. If you just want too see what the community looks like, I can’t recommend anything better than going to a good conference, like FOSDEM next month in Brussels.

Open source in content management

Open source has not affected content management much yet. Almost all content management by volume takes place on open source products (by volume Wordpress, Joomla! and Drupal far outweigh anything else). By value it is less clear, open source always has an issue with by value calculations as the revenue models are different, Linux is not the leading server operating system by value, but is by installed base, but is also probably by the value of the services running on it.

But arguably open source content management software has not affected the industry yet, looking now at the larger installations, and the areas that Janus is interested in, indeed that I am. The industry has grown up in a mess as far as standards, ideas, infrastructure are concerned, but the reality checkpoint has been reached. Two standards have so far started to change the technology landscape of content management, JCR and CMIS, and almost all the implementations of these are open source, and most are cross-vendor projects. This change will grow as more standardization and commoditization sweeps the industry, as the industry adopts a web infrastructure rather than the pre-web legacies inherited from the document management history of the business. Everything that this business deals with will be served through the web; almost all web infrastructure is open source software; content management will be no different.

In this field this is all just beginning. Like open source as I said above, it started with developers, about more efficient ways of building, architecting and delivering software; in terms of influence on the end users it is still small. But things are turning as people become aware of open source in the industry, but they clearly still need some help understanding it. I hope this has helped.

Other links are here: Jon Marks, Irina Guseva, Ian Truscott; other participants Adriaan Bloem, Andrew Liles, Philippe Parker (first use of Wave over GotoMeeting?)

Well it was fun. Technical difficulties, lost sync and crashed a few browsers, some people lost whole machines though. Safari coped better than Firefox. It took a while to realize what was happening here, hey but this is in beta!

As a brainstorming tool at worked pretty well. I thought it scaled pretty well. The named cursors indicate who is in the bit you are in, but for brainstorming you can look, write another point, move, continue, not edit much. After half an hour of getting to bulletted lists, a bit of moving around the heavy writing started (after a discussion at the top in our proxy process section; we should have split the thing up a bit).

There is a great tendancy to write temporary notes about the discussion and then just delete them. Which feels odd, data and metadata together of course. The editing process was odd, you would find orphaned bits, move things, try to join stuff up to make it flow, while it was all changing around you. Pretty chaotic. Bits that no one expanded into prose got junked (quite a good edit method, as they couldnt stand up themselves).

Here is the “finished” article… which cannot be attributed to anyone individually of course… the subject was chosen about 10 minutes in, just as something people would have something they could easily contribute into this situation, there are some good points in there though!

Things We Hate About Content Management

- By The Motley Crew

It was a lovely Friday morning/afternoon, and we were Waving. The experiment initiated by McBoof (yes, that one) brought together 6 CMS folks from around the world. The event gathered together analysts, journalists, vendors, system integrators to Wave on a topic that was decided at that very moment. We had one hour (in between conference calls and other job thingys) to pick a topic and Wave it.

A little collab on what exactly to Wave about later, we decided to do “a mindmap of things we find annoying in CMSs.” To up the ante, we also decided to take the original bullet points (deemed “too easy”) and convert the whole thing to prose. Was the tool given really up to the task? Were our minds flexible enough to wrap around this kind of realtime collaboration?

In the beginning — we blame the tool — we were Drowning, not Waving. We (almost) didn’t fight about edits. We almost didn’t step on each other’s toes. All in all, it turned out to be a fun and productive collaborative exercise. Read on to see for yourself.

Cosmetic Issues

There really should be a CMS UI fashion police. As there should be a Magic Quadrant for shoes and handbags. Why? Well, there’s a couple of issues.

For instance, sloppy, non-designed design. You know the kind of thing that has not been thought about and reworked and made to feel right. The sort of thing coders do if you don’t force them. But at the same time, over-designed interfaces can be just as bad: the designers and developers really need to be on speaking terms.

When building a system that works, you can’t have the development team in the basement on a sustenance of Jolt coding away into the night, and the designers in the penthouse in turtleneck sweaters sipping espressos. Too many CMS designs end up being programmer vs. end-user friendly. And this is not the best way to charm away those marketing and web content folks.

Developers and designers need to talk to each other and essentially, both should talk to users – not just eat your own dogfood – but listen to what dogs like to eat. A developer or UI designer are not content editors, marketers or knowledge and information workers.

Some vendors say that the agonizingly and depressingly black UI backgrounds are hip and modern. Well, they are not, really. Who told you that? Especially if you add a Star Trek theme to it and sprinkle in some stars and cosmic swirls, because if Apple does it, it must be cool right? Not pointing any fingers, but I would quit if I were a content manager having to spend my 9-5 staring into the “black hole” of some of the CMS UIs that are out there on the market.

Even pop-ups seem less annoying when compared to dark UIs. Which brings us onto…

Interface Issues

Interfaces need a comfortable lived in feel. Content management is something people work with every day, it is their interface to their job. You meet people who hate the interface, and that makes their work a heap of pain. I have seen people who describe the 44 clicks it takes to insert an image. You have a responsibility to these people, to make them love the content and make the tool disappear.

We all hate it when the interface does something on its own that ruins your context. E.g. a page refresh, or in Wave the jumping around of the scrolled window in some cases Or the lack of an easy way to bookmark, so you can reference someone to the content. Remember people will be collaborating and need to send links around. Make sure the UI is a proper web application with URLs. And why do tasks that are easy to describe and often repeated in exactly the same way still take more than a few clicks? (Or maybe even dozens of clicks.) With bonus points for forcing users to use dialogs or tabs to enter mandatory information. Remember people do not have all the information in the right order.

Also, we need sane conflict merges. Check in and check out is too extreme for most uses. But people want to edit offline still. Of course Wave doesn’t have an offline: Google thinks this problem is going away, it’s real time so there are never conflicts (that’s defined in the XML protocol; it’s quite interesting if you are that way geeky). Does Google have the right answer here? Well, the Motley Crew is struggling here, and some browsers lost sync during this experiment.

“Power users” (those who use it all day long) of CMSs needed to have a “Desktop” experience. What does Desktop Experience mean? Well, it doesn’t really have to be on the desktop — these days it is perfectly possible to get very close to a hitherto Desktop experience in a browser or similar. these are qualities: very low latency from action to response, no page refreshes, modal and modal-less dialog boxes as appropriate, “push” notification.

Architectural Issues

Architectural issues of the wave overtook any architectural issues of Content Management Systems. The fact that we authored this entire article in a single blip didn’t help, and slowed everything down enormously. McBoof learned the hard way that he really need a new laptop and spent most of the session giving his machine CPR. Next time we’ll do each paragraph in its own blip to stop FireFox going down like a Led Zeppelin.

Monolithic systems. Build it out of pieces that the client can not use all of. Obviously your pieces may work together better, but there should be components. Do not try to reinvent all kinds of wheel. “Best of breed,” though, is just another weasel marketing idea, as if systems are pinnacles not about meeting requirements.

Marketeers are adroit at using the term Best Practice to position Their Way as the only way that a particular matter can be solved. (Many of us live in that netherland of having to pedal that point of view, but it is a falsehood that the careful buyer should try to see through.) I think this devalues genuine best practice, vendors should cite references

Most often a marketeer’s Best Practice view is the only one they subscribe to as their product development has paddled up the wrong stream and cannot or won’t reverse their architectural design (probably because of the cost of doing so). This intransigence most often causes a product to doom itself. (Think of IBM and The Mainframe Is The Only Way To Do Serious Business).

Who really still believes that there is a place in this world for Flash or Java Applet based Rich Text Editors? TinyMCE, FCKeditor and others are filling the gap left by Ektron when they bit the hand that feeds and entered the CMS market. Ephox is trying to spread, but I find it difficult to come up with an excuse to use an Applet over HTML with javascript these days. Stick with the standard.

Business Issues

Where you are buying into something that you may very well need to change or integrate with there is strong benefit in considering Open Source. Open Source used to frighten commercial software companies but we have come along way on that road to understand that commercial organisation can operate in an Open Source world and benefit. This does not necessarily mean that their prized system needs to be fully opened up, but taking the spirit of it to mean that you are completely open to people seeing and learning from your code how it operates.

Exactly what you need to see opened up varies. In a CMS there may be a subsystem that stores the content or one that allows a Rich Text Editor. These arguably don’t need to be opened up, but when a CMS ships with modules for, for example, an RSS feed widget, calendaring tool, prebuilt webforms, users who then want a variation on this module can benefit from seeing how the “pros” did it, they can then use it as a starting point for their own different implementation.

We really don’t need vendors that pay lip service to the buzzwords. When they think the new CMS buzzword “engagement” is just a screenshot of Google Analytics. Or when they add an image picker and call it DAM. And a cross-over between WCM and ECM? Don’t think WCM is like ECM and it’s about organizing content, not about effectively communicating with the audience. And don’t think that if you organize the content, you can automatically communicate effectively.

Completely different, but equally frustrating, is procurement (and the procedures that go with it.) Procurement folk don’t recognise the importance of user adoption to the success of the project — of the black background and all the UI issues pointed out previously. If a CMS is procured according to procedure, the selection is a success to them. But those same rules are often a recipe for ignoring what the users really need.

At the same time, budgets that aren’t transparent are an issue – customer and vendor should be able to have a sensible grown up conversation. As a customer, of course you want good value, but how cheap are you? But to vendors: many licensing models don’t make any sense, and force you to do stupid things. People are scared to have that conversation – the best architectural fit first I say, lets figure out an appropriate license around that.

Conclusion

So much hatred rolled up into a tight little ball of anti-CMS rage. Who would have expected it from such a respected bunch of CMS folk. We hate the designs, the interfaces, the architectures and the business. Time for a beer/wine? Wave good bye!

So what are these, why do you want them, and what do they mean for your content strategy?

Toby Bell at Gartner says

Smart companies have begun linking more of their content to industry-specific, human-centric processes, such as insurance claims handling, or supporting research on new drug development. This approach usually means building or modifying the content-enabled vertical applications (CEVAs) on top of ECM environments. CEVAs typically help to automate complex processes that previously required workers to manually sort through paper documents and other forms of content (in effect, a way to manage down costs of exception handling) and optimize the remainder of the work.

This seems to be not a great summary though. Look at it from another view. Your enterprise content was disorganized, living on network shares, random websites, legacy systems, all over the place. Your ECM strategy was to first to consolidate to reduce costs of multiple systems, and to improve findability, get to a base enterprise content management position. But what next? Where is the next value?

Content strategy starts here. There are many parts to this, covering creation, lifecycle and reuse, audit, consolidation quality and so on. The CEVA part is about the delivery and interaction with content within other non content focussed areas of the business. Content management often seems to be about specific content focussed parts of a business, such as, historically, technical documentation and, more recently, online marketing material. Plus a load of unstuctured stuff like emails and generic office documents. The areas such as technical documentation have had high value often for legal and regulatory reasons, so structured processes were created early; these effectively created the content management industry initially. Web content historically had different solutions because it turned up and became important when the general purpose tools (Word!) could not usefully author it.

But all the stuff classified as “other” does have underlying processes, content processes. Some are formalized in systems, the original paperless office systems, the roots of the document management industry in forms and scanned paperwork processing. This stuff generally sits more on the “data” not “content” side of business processes. Long term this distinction is not such a useful one, and data and content resources will merge together into a single enterprise resource architecture. The majority of processes with content though take place through informal channels, particularly email with Microsoft Office documents. These are the document types you tried to take control of through ECM.

So ECM took the documents that were behind many processes and made them findable and organized them. But at the base level a content repository is just that, a repository. It deals with basic issues such as versioning and permissions, search and findability, and some organization, but it does not really deal with process and processing.

Process and processing are the valuable parts in the lifetime of most content. Imagine the lifetime of an insurance contract say, with payments and claims and disputes, or an employees personnel file, or a technical manual over the lifetime of a product. A CEVA or CCA is an application to support these lifetime processes.

It is also an application to support the relation of a document’s lifetimes processes to other systems. Your CRM system may need to know about insurance claims, your sales department may need to know about expiry, your website may need to know about new documentation releases, content changes do not happen in a vacuum.

One class of CCA that is common but is rarely perceived as that is a software application with embedded content. Once that was just embedded “help screens” with content tools to manage them, then came internationalization, with a different set of tools. But these desktop applications are being rapidly replaced by web applications. Web applications are much more content driven, they may live in an SEO facing world, they may live in a customer facing world that may consider usability, they may be multilingual, and they are not driven by the developer-centric ideas of help screens and manuals. Content and application can live together, but this requires new ways of using, reusing and versioning content, and pulling content out of application release cycles so it can reflect non application changes, such as the marketing environment, usability improvements, corrections and enhancements. These applications were historically development led but as they mature the content aspects become key business drivers, needing content management integration.

So what do you need to build this type of application, and what should your decision criteria on platforms be?

Stéphane Croisier says in a good survey “So rapid raw composite assembly, fast integration and ease of use are the three new pillars of next generation content solutions.”

The first thing to bear in mind is that you need more than a repository, you are looking for an application platform too. The ease of use issue is important. Long term you need to be looking for something that staff can build simple tools from, even if you are hiring specialists for the complex projects. Ease of use is a two way thing, as you need an easy to use platform that lets you build easy to use applications. And ease of modification and maintenance is equally important as these applications may need to be fluid. You are likely to need external support to build more complex applications on the same platform, so availability of this is important too. Ignore the jargon of portlets, widgets and mashups: none of these so called standardisations have much traction; we are talking application development, use what you have or can hire developers to do. Ask the vendors what their platform strategy is.

Stéphane identifies a trend towards solutions, ready to go solutions for common problems; these may be useful but I would not choose a development platform on the basis of the availability of particular solutions or you may end up buying a platform for every solution. A longer term view of the viability of a platform for other solutions is necessary too.

Long term, remember that content application strategy is part of content strategy, and comes after that. You need to know what your content applications are and will be, and have built an underlying respository, authoring and reuse strategy first. Applications are where developers need to interact with this to achieve the long term goals.

It started with an idea I had, in an innocent sort of way. I thought if I looked at the JCR specs for a bit I might find some kind of way of building a non Java interface with them. You know, maybe there might be a nice REST architecture waiting to get out. But of course there is no such thing. It is an application definition. There are not even that many ways of implementing it, other than choosing your object persistence method to be a database, file, or something else.

The REST architecture is notionally provided by another layer, such as Apache Sling, but Sling is in no way a REST layer, it is a URL dispatcher and scripting and application layer with which some REST style applications can be developed. With that you end up with a pretty heavyweight development framework, indeed together you have much of Day’s CMS offering in effect, rather than a lightweight REST repository solution.

I had a look at CMIS again. Fielding once laid into CMIS for not being REST and you can see why, although some improvements have been made since that. Although resources are discoverable through hypertext, there is a fair amount of semantics that needs to be known to understand what a type or a checkout means, and the search queries are obviously just RPC wrappers. It is not too bad though, but unfortunately the data model does not map well onto web content management right now for obvious historical document management reasons. Fixable? I think it serves a part‎icular purpose well and should probably not be forced into anything else, as we need it to succeed in its field.

Day claims that JCR is not a Java standard in an odd way, that you can implement the API in another language. Thats a strange argument to make, especially as the types are defined as Java types, and standards without interoperability are pretty vague. Without some sort of wire format or ABI this is meaningless outside the JVM world. People are making JCR like repositories in PHP but outside any standards process, so in the end this just becomes a PHP repository project; Typo3 seems to be building another, also closely aligned to JCR.

The problem with these efforts is that it is not helping the balkanization of web CMS, which is already fragmented by language and API, which is ridiculous in an industry that is about the web. The web has an architecture (REST) and an API (HTTP). Building web content management on Java APIs or PHP APIs or .NET is a legacy way of thinking; it is acceptable for document management given its role in existing enterprise architectures, but it is not going to work if we want to get widespread acceptance in web development; in the short term it is the easy path, it is what people are used to, but a forward thinking industry needs to look at defragmenting the landscape and building future proof tools.

The odd thing is that a web content repository alone surely lends itself to a simple REST architecture. Content is after all lots of small resources with relations. Hypertext. It is pretty much in presentation a fairly dumb web application, although with a fair amount going on behind the scenes. It takes content, relates it to other content, and serves it back, with authentication and versioning. Everything else is in other system layers, transforming it and so on. Not simple, but well defined; lower level than JCR + Sling say

So we need to work on a web content repository model, as a community. Process wise, it makes sense for this to sit in an organization like AIIM, as a content management based industry body. It may well be that what ends up coming out of this is more standardized architectures and semantics and open source implementations rather than the tighter prescriptions of JCR and CMIS; I have some ideas along these lines that I need to code up. I have had some discussions and there is a degree of interest in some sort of solution; who is interested? Or is infrastructure dead, everything ust wants interfaces?

“Spent a week in a dusty library waiting for some words to jump at me” Camera Obscura.

Kas Thomas in his contribution to Julian Wraith’s popular thread on the future of content management really managed to make me disagree, the first of the posts that has!

The theme of this blog (yes it has got one) is that content management is changing as the web way of working starts to infiltrate the enterprise. And the web way of metadata is not what the old document oriented way was.

Kas says “Keeping knowledge about a file separate from the file itself is a hugely important concept.”

Looking at that point first. Look at any newish document format, from PDF/A to EXIF via the HTML meta tag, even a Word document and you will find embedded metadata. Remember that documents are emailed around, generally get lost. Metadata is the dog collar with a name and phone number on, saying version me and send me home. This trend is not going away, documents are becoming self contained.

Kas then continues “A file’s metadata becomes its interface to the outside world. It’s like a service descriptor.” That is simply not the way the web works. Resources use self describing formats like HTML for core data, and then all the other important metadata is linking information. It used to be that you had a blob file type and a program that could understand it, that was the basis for the desktop architecture, but that is not the case any more, even on the desktop you have a choice of applications that can understand a given file type, and your choice depends on what you can do with them. The web architecture goes much further, and resources become fully self-describing, a browser can understand all the web as every resource carries its own description, and bundled code to help you interpret it. A web page is its own service descriptor, and defines application state through hyperlinks. The web architecture has never had service descriptions.

There is one vital part of metadata that is not kept with a file, that is the link. Kas says “content, on the whole, is becoming richer, less structured”, missing out completely on the big picture that content is being structured by the imposition of links onto it, by its transformation into a hypertext, that creates a much richer structure than the individual documents have in themselves. Documents contain metadata about other documents in the form of links. The semantic web project is an attempt to add further richness to this structure. “What does the trend toward richer, less structured content mean for management of content?” well that means that content management is going to be about managing those links and relations between items, a lot more than it is now, when it came from a background of just managing documents, each an isolated item.

In a way that does come back to “Keeping knowledge about a file separate from the file itself” but not at all in the way Kas was trying to argue. Now time to link this to his argument to create a structured discussion…

6f82f1d2683dc522545efe863e5d2b73

Stéphane Croisier in his post in the thread says

There is currently an unclear separation between applications frameworks and content infrastructure. But at the end of the day everything is content and every application has first to deal with content items rather than with processes, states, UI components or other application oriented paradigms.

In my general work in content management I think this is one of the things that has become very clear, and that “unclear separation” is very apparent. First content; it is a very good start, and every project needs to be grounded in content, and in the structure of content, the architecture of content and the user IA. However, the processes, states, UI and other parts of the web application are beginning to dominate projects. In the Gartner terminology we are building CEVAs (Content Enabled Vertical Applications that is), as integration, process, e-commerce, CRM parts of the project start to dominate the requirements over the purely content based parts.

It seems that most of the contributors to Julian Wraith’s future of content management thread who mention it see content management moving to a clear split between repositories (Common Content Information Infrastructure as Stéphane calls them) and applications and content management systems and CEVAs implemented on top of these.

I don’t think we can yet see what the successful content infrastructure stack will be; as I said in my earlier post there are technical decisions that have to be made that there is not yet agreement on (except between me and Peter Monks!) and the existing putative standards (CMIS and JCR) do not extend far enough to take a position on. But we can see that this is the way things are going. Quite clearly the standards for the infrastructure will be open, and most implmentations will be open source. There will be some vendors who do not embrace standards, but they will need to be the few large ones or they will lose out. Infrastructure environments remember (think Linux, Apache) are mainly open source, although there is scope for proprietary layers at the very high end (think Amazon, Google).

At the application layer, as Stéphane says, everything is a mashup, content from different systems, content from other APIs, this is the we application layer. It needs to be content aware, very much so, but it needs to be an application development environment. This is where most people will see the value added in the content management business, although in fact the value here is in implementation, design and integration services, not the technology itself. Application development environments no longer make a lot of money, and again they are dominated by open source (think Java, Eclipse, JBoss, Django).

Once you take out content infrastructure and application development, and the other tools like search, workflow, there is a core of tools for working with content, to support reuse, refactoring, cleaning, import and export, that one might call a Content Workbench. There is a lot of potential value if these types of tools are the value added end of the business, as they can differentiate vendors and add value. Interfaces for merging changes and so on would be part of this type of toolkit. This is the stuff where good UX means timesaving for content workers, but it is difficult to build on a customized per-project basis, so this still offers value from a particular vendor.

Overall then we see a picture where the monolithic CMS starts to break apart into infrastructure, application and toolkit layers, that can perhaps gradually be mixed and matched together to build content applications. We are just seeing the beginnings of this now.

6f82f1d2683dc522545efe863e5d2b73

Although the vast majority (but not all) of open source content management systems are continually trying to reinvent the blog, we are talking about internet infrastructure here, and the future of content is going to be open source, like the rest of internet development. I also believe that long term the web project will overwhelm the legacy areas of document management, although it may take some time. Hypertext, the web architecture, XML, HTML, and all those standards are here to stay and to dominate long term. Content management will also become pervasive long term, as the blogging projects show, as the right tools make content management a natural part of workflow. Content management succeeds when it replaces the file and folder paradigm with a content-led paradigm.

In my conversation the other day with Jon I was arguing that although we agree on many of the technical issues there are real decisions that need to be made about what needs to be built to get the the content management future. Below are some of my lists of differences. Generally, I think the future of content management is going for the left hand one of the pairs, although some are not clear yet. I have probably missed a lot of the things to determine, but it is a start.

Architecture – API differences

These may cause API and other more significant differences, though some may not matter (eg git can read svn repos, but not vice versa).

• REST vs SOAP
• REST vs Java native interfaces
• distributed version control (git) vs file based (SVN)
• compositional vs monolithic
• structured content vs files
• relations vs metadata
• web (hypertext) content vs documents
• URIs vs referential integrity
• web applications with content management vs content management systems

Architecture – performance differences

These could have different implementations with different performance characteristics potentially. These are basically IA differences to a large extent, so they do depend on the type of problem being modelled and the modelling process. Models and performance are linked though, and the best we can do is to make parts of this pluggable so that a range of performance characteristics can be used.

• unstructured vs structured
• sparse vs dense
• untyped vs typed
• NoSQL vs RDBMS
• permission hierarchy vs permission graph
• scaleable vs local

Development process

This is key to getting the product to where you want it to be.

• open source vs proprietary
• API driven vs UX driven
• ubiquitous content management vs isolated systems
• agile vs monolithic

Architecture – usage differences

These could potentially just come down to the ways or tools with which components are joined together, maybe they do not affect architecture per se.

• social media vs controlled content
• programming languages (Javascript, XSLT) vs templating systems

6f82f1d2683dc522545efe863e5d2b73