The basic model is that we have resources, which may have some content and metadata attached. We are mainly interested here in the properties that can be attached to a resource, and the fact that some of those properties are relations to other resources. We are less concerned about what goes inside the structured body of a resource, although there are some issues about how many “bodies” a resource can have. So we have a model that has resources, each of which has key-value pairs, some of the values may be links to other resources, which presumably have referential integrity support

Properties of properties

• STRING. Resources can have string valued properties.

  This is a basic starting point; I don’t think I know of a CMS that does not support this. You can store any other type in a string if necessary, though binary values are more efficient in some cases.

• VALID. Property values can be validated.

  Many core repositories do not validate property values at all, it is just a validation proxy layer that does, so as a repository principle this is fairly rare, although a small number of types (numbers, dates) might have native validated representations.

• TYPE. Property names can be validated.

  Many systems have a typing facility that restricts the set of property names a resource can have. Others are unstructured, and any property can be added to any resource. There may be type composition mechanisms, such as mixins or type inheritance. Unlike value VALID this is more often tied to the core repository model, rather than a proxy layer, if the repository is typed for internal performance or indexing reasons, that is tied to dense rather than sparse storage.

• BINARY. Resources can have at most one binary property.

  I have split this property as some content management systems can only have one binary property (such as an image file) on a particular resource, and multiple ones have to be constructed from multiple linked resources. This is not generally a huge limitation in an otherwise flexible system, but in a weaker system could be annoying.

• N-BINARY. Resources can have any number of binary properties.

  This is the fully flexible version; one may still be a distinguished value in some way, but you can store all the sizes of an image (say) as properties of one resource, which makes managing them easier, although it may actually make things more difficult if it is not easy to iterate over properties (STRUCTPROP), and using multiple resources could be easier.

• STRUCTPROP. Properties can be structured.

  Some systems have structured properties, for example some systems have a JSON representation for properties, rather than the flat key-value namespace of other systems. JSON supports arrays that can be iterated over, and structures that can be repeated. To make this sort of structure with only key-value properties you may need to use more resources. Structured properties though add a lot more complexity, and perfectly useful, but different, systems can be made with or without this model type. Structured properties often have partial update interfaces, which adds complexity, so that one subproperty can be modified at a time. Note while technically JCR does not have structured properties, you can use the distinguished tree below any resource as a tree of properties, so it is rather similar to this model. Note also that property naming can informally add structure, such as in the way slashes denote URI hierarchy, they can denote property hierarchy in a technically flat namespace.

• MULPROP. Properties can have multiple values.

  Structured properties can usually have multiple values (JSON array for example), but not all systems with key-value type properties allow the same key to be set multiple times with different values. This is the model with say HTML metadata, where each property (key) can be set multiple times; however some key value systems only allow a key to hold a single value, and so the user would have to make a structured value to hold the multiple data items instead, by some encoding scheme without the system providing support directly. Having multiple properties complicates the simple set and get interfaces that single valued properties have.

• TREE. Resources can be in exactly one tree structure.

  Another split one. Many systems have one distinguished tree structure that content items must be in, and that tree has special operations, like fast access to parents and children; other trees might be constructed by other means, like using a general relation, but the operations on them might be difficult. Children in a tree are almost always ordered and can be reordered, although some systems might not have this property.

• N-TREE. Resources can be in any number of trees.

  The distinguished tree is very common (although Lily for example does not have one); but I do not think I know of any system with multiple named trees that share a common tree interface (like a parent function). You can make a tree with general relations, but you will not get help in making it acyclic for example. So while this is a possible design, it is complex to implement, although arguably useful as a modeling tool. Generally you will have to manage general relations yourself to do this.

• CLONE. A resource can be cloned.

  This means that the same item can appear as more than one resource at the same time, each of which will update in the same way. This is similar to say a Unix hard link. This is the usual way of turning a TREE into a dag, which adds some flexibility. Different tree locations of the cloned resources may affect properties such as permissions in some systems, or inheritance so this property can add a fair amount of modeling flexibility; conversely without these it is of less use.

• RENAME. A resource can be renamed or moved.

  Many content management systems provide this operation in their model, but it is not a native operation in others at the repository level, as the end user visible name just be a property for example. HTTP does not provide a rename operation, but WebDAV does.

• REL. General relations between resources can be created.

  This is the basic relation (named by the key name) between two items. It corresponds to the HTML metadata element, or an RDF triple. It turns the resources into a directed graph with named edges. It is certainly essential for any content management system; I will talk more about how you want to be able to use and query it later.

• RELNS. Relations have a different namespace from properties.

This is a distinguishing feature between XML, which has attributes and child elements (relations) syntactically distinguished, versus JSON that does not. Non child relations in XML are still attributes though. Generally seems a pointless distinction, and using a single namespace is simpler.

• RELPROP. Relations can have properties.

  This is an interesting one. Adding a value to the relationship triple to make it a quad, means that a number (or other value) can be assigned to a relation, making each relation a weighted directed graph (or you can view the system as a matrix). The general model of the property graph has properties for edges, but for example the RDF model does not, although they are often what blank nodes are used to model, although of course blank nodes can have relations as well as properties. You can make up for a lack of properties on edges/relations by adding extra nodes like this, but they may proliferate and need managing, so allowing properties may help. It is also worth noting that a system without MULPROP can use naming of properties to implement RELPROP, as a relation could have a naming convention for its properties; similarly STRUCTPROP generally allows storing the extra information in the property structure.

• REFINT. Referential integrity is preserved for relations.

  Preserving referential integrity at the repository layer is a fair amount of work, relational databases can do this, but not all content repositories do, for example over delete operations.

• ORDREL. Properties are ordered.

  True key-value models do not tend to have an ordering for properties. As with many of these things, ordering adds interface complexity. Structured properties may however be ordered, and if the model supports a distinguished tree (TREE) this almost certainly has ordered children. If you have to build an ordered tree simple from basic relations it is quite complex. A sort order on a relation is another relation property that seems to be rarely supported for general relations, like weights.

• REIFY. Properties can have properties.

  RDF in principle lets properties themselves be resources (reification), so that they can in turn have properties. This allows me to add information about the properties, such as where they came from. This rarely seems to be useful in common models. Giving different properties different permissions might be a more useful side effect.

• EXTREL. Relations can be defined externally to their subject.

  HTML originally had a rev relation, which defined a relation backwards from object to subject, and RDF triples can be stored in any document, divorced from subject and object referants. This causes all sorts of issues with updates and managing (even finding) relations, while adding no descriptive ability except potentially REIFY.

• INHERIT. Relations can inherit properties.

  The inheritance tree might be set from other properties, or from a distinguished tree, but one model is that properties not explicitly set can be inherited from another resource, or a prototype. This often makes models simpler, as rather than explicitly walking a tree, you can implicitly do it though inheritance. Seems surprisingly uncommon in content repositories.

• MINHERIT. Multiple inheritance.

  Allow inheritance from multiple resources, not just for example based on the primary distinguished tree. More complex.

• NAMESPACE. Namespacing on properties.

  Some systems have a type of namespacing on properties, often used for multiple language variants for example, so that a property may differ across these namespaces. This can also be implemented with multiple resources, structured properties or inheritance. Usually not all properties are namespaced at once; some may not vary, which makes the set and get interface more complex.

• ATOMIC. All the properties of a resource must be updated together.

  A resource and all its properties are all updated at once to a complete new state (this will also be the versioning state). Other than for versioning, this also affects how concurrent access works. The alternative is that each property can be updated independently. Atomic updates are the HTTP resource update model.

• RVERSION. Resources may be versioned.

  An entire resource may be versioned; this is a similar namespacing operation, where a namespace is available to retrieve the old values of a resource.

• PVERSION. Properties may be individually versioned.

  Some systems (such as Lily) allow versioning to be turned on or off on a per property basis. This property generally implies that ATOMIC is not true.

• DELVERSION. Versions are deleted when a resource is deleted.

  This is surprisingly common, if versions are namespaced properties, then they are often deleted along with the resource when it is deleted. The better solution is not to delete versioned resources, just give them a tombstone (whiteout) marker.

• SNAPSHOT. The versioning namespace is whole system state not resource state based.

  Although versioning of the total state of a system is now common in source code control systems, many content management systems only let individual ressources be versioned (hence creating issues such as DELVERSION). The main issue here is that you cannot apply or undo a set of changes together, only individually. Apart from the difficulty in making easy user interfaces, whole system versioning is superior in every way to versioning of individual resources or properties and no one should be designing a system that does not behave like this.

• TREEVERSION. Versioning supports branching and merging.

  A full versioning model like git or subversion, rather than just a linear series of checkpoints is another model. It generally simplifies the concurrent updates (ie can avoid both CAS and LOCK in theory). Although these provide the richest model of versioning, it is the hardest to present to the non technical user. Note also that this is one clear area where the content model for delivery can differ from the one for authoring; for authoring there are much more complex operations that are useful, while for delivery performance is key, and versioning may not be required at all, depending on how updates are applied.

• CAS. A CAS-type operation is supported on resource updates.

  Some type of atomic update-if-unchanged since this version operation is supported, for lockless updates. HTTP Etags are the canonical example. This is the simplest choice for API access, and simple for users too. The unit of atomicity is usually the whole resource here, making it the unit of transactions; atomic update only of individual properties does not let two properties be updated in a single transaction so is not so useful.

• LOCK. A locking operation is supported.

  The traditional alternative to CAS is a locking operation, that disables write operations while the operation is locked. Some administrative or time based unlock operations are required as well. Less suited than other methods to automated APIs, due to issues like deadlock. As multiple locks can potentially be obtained, cross resource transactions are possible, although this could impact concurrency.

• TRANSACTION. Transactions across multiple resources are supported.

  Generally individual resources are the unit of transaction, or possibly individual properties. Some systems however allow a transaction in which multiple resources are updated together. JCR is probably the main example of these. A system with snapshots may also have this property if moving between versions is atomic. HTTP deliberately does not have this sort of transaction, as it does not work well if the resources are distributed, and system design for HTTP should ensure that resources model the right things so that transactions across resources are not needed.

Queries

With the property model above, you can retrieve resources, and read and modify their properties. There may also be some additional maybe slightly different properties (the ones that TREE might expose for example, parent and child relations). We can traverse between resources by following their links. However we do generally want to make more complex queries, either about global questions, or more complex traversals based on properties. There are a lot of query models we should really explore, particularly we need to focus on how properties are indexed. I suspect that the analysis below is just a starting point.

• PINDEX. Property values are indexed.

  This is not necessarily essential, as for most interesting properties one would create a node rather than a value, and use relations, though you need reverse relation indexes anyway.

• REV. Relations have a reverse index.

  This lets me find the opposite direction of a relation. This is a key property, as relations are directional, and important properties are in the other direction, such as finding all the resources tagged with a particular tag value.

Interfaces to properties

I mentioned above that some of the property models have differing interface complexities, and I think it probably helps to show what some of the interfaces look like.

The canonical interface in web content management is that one exposed by HTTP. Resources and all their properties have to be updated atomically (ATOMIC) – PATCH is just an optimization. CAS is available (Etags or last update). No versioning is specially supported (although the system could create resources for old versions and add properties to access them). Other property behaviours depend on the document types, so HTML for example supports a meta and link flat property namespace, but other schemes are possible. A resource TREE is very loosely defined by ‘/’ in URLs, but does not provide any properties, so it is barely a tree.

WebDAV changed the HTTP data model to push it much more close to one traditional content model, supporting LOCK, RENAME, and TREE on top of HTTP, and an explicit property model independent of the resources in question. The property model is flat, with no STRUCTPROP although extending it is mentioned in the RFC, and no MULPROP. CLONE is allowed, as resources can have more than one URI. Updates to properties are not ATOMIC, as the PROPPATCH method can update some properties without others. The main HTTP resource is the body, which allows storage of one BINARY property, as the other properties are XML strings. This is pretty much the standard document management style set of properties.

As I was looking at Lily CMS recently, it is pretty different. There is no TREE, you have to construct it yourself. There is TYPE, no STRUCTPROP, there is NAMESPACE and PVERSION. There is no CAS or LOCK, conflicts are resolved by time of modification alone. It is an interestingly different model which I will look at in more detail in another post, as it chooses complexity in some areas and simplicity in others.

One of the common themes of the NoSQL movement is saying things along the lines of if you only give up this one feature we can make the storage layer that much simpler and faster, push some more work up to higher layers to resolve, issues like conflict resolution say, or referential integrity, or tree structures. This is the path of a simpler core repository which does not implement all common usage patterns for a CMS application, with some layering and conventions on top to build the next level. This is not so different from the relational model, with a low level relational algebra and a set of database management tools, then the application. What is still unclear is exactly where to make that split, but certainly large monolithic repository models that try to do everything listed above end up very large and complex to use.

There is definitely a case for moving some of these properties out of the repository into the authoring tools. Referential integrity at the repository level is actually quite hard to work with, as you cannot refer to something you are about to create, for example, but the authoring layer can provide tooling to help the user here.

I will post some follow-ups about some of the other issues arising from this, and what I think the best set of constraints to work in is, and more on the CMIS and JCR models.

It started with an idea I had, in an innocent sort of way. I thought if I looked at the JCR specs for a bit I might find some kind of way of building a non Java interface with them. You know, maybe there might be a nice REST architecture waiting to get out. But of course there is no such thing. It is an application definition. There are not even that many ways of implementing it, other than choosing your object persistence method to be a database, file, or something else.

The REST architecture is notionally provided by another layer, such as Apache Sling, but Sling is in no way a REST layer, it is a URL dispatcher and scripting and application layer with which some REST style applications can be developed. With that you end up with a pretty heavyweight development framework, indeed together you have much of Day’s CMS offering in effect, rather than a lightweight REST repository solution.

I had a look at CMIS again. Fielding once laid into CMIS for not being REST and you can see why, although some improvements have been made since that. Although resources are discoverable through hypertext, there is a fair amount of semantics that needs to be known to understand what a type or a checkout means, and the search queries are obviously just RPC wrappers. It is not too bad though, but unfortunately the data model does not map well onto web content management right now for obvious historical document management reasons. Fixable? I think it serves a part‎icular purpose well and should probably not be forced into anything else, as we need it to succeed in its field.

Day claims that JCR is not a Java standard in an odd way, that you can implement the API in another language. Thats a strange argument to make, especially as the types are defined as Java types, and standards without interoperability are pretty vague. Without some sort of wire format or ABI this is meaningless outside the JVM world. People are making JCR like repositories in PHP but outside any standards process, so in the end this just becomes a PHP repository project; Typo3 seems to be building another, also closely aligned to JCR.

The problem with these efforts is that it is not helping the balkanization of web CMS, which is already fragmented by language and API, which is ridiculous in an industry that is about the web. The web has an architecture (REST) and an API (HTTP). Building web content management on Java APIs or PHP APIs or .NET is a legacy way of thinking; it is acceptable for document management given its role in existing enterprise architectures, but it is not going to work if we want to get widespread acceptance in web development; in the short term it is the easy path, it is what people are used to, but a forward thinking industry needs to look at defragmenting the landscape and building future proof tools.

The odd thing is that a web content repository alone surely lends itself to a simple REST architecture. Content is after all lots of small resources with relations. Hypertext. It is pretty much in presentation a fairly dumb web application, although with a fair amount going on behind the scenes. It takes content, relates it to other content, and serves it back, with authentication and versioning. Everything else is in other system layers, transforming it and so on. Not simple, but well defined; lower level than JCR + Sling say

So we need to work on a web content repository model, as a community. Process wise, it makes sense for this to sit in an organization like AIIM, as a content management based industry body. It may well be that what ends up coming out of this is more standardized architectures and semantics and open source implementations rather than the tighter prescriptions of JCR and CMIS; I have some ideas along these lines that I need to code up. I have had some discussions and there is a degree of interest in some sort of solution; who is interested? Or is infrastructure dead, everything ust wants interfaces?