In "The new dynamics of strategy: Sense-making in a complex and complicated world"  Cynthia Kurtz and Dave Snowden introduce the term "unorder". As "undead" describes something neither truly dead nor alive, "unorder" describes things which are neither systematically ordered, nor completely disordered. In chaotic or complex real-world systems patterns of behaviour or structure can often be found. A system showing these "emergent" patterns exhibits unorder.

Systems can be both ordered and unordered at the same time. For example an organisation can have a formal, ordered management hierarchy, but its decisions are driven, at least in part, by an informal influencing network.

The architect's traditional domain is ordered. He or she receives a coherent set of requirements from identified stakeholders, and uses his technical skill to derive a solution structure which optimally balances the various design forces. Lucky architects occasionally get to work in this deterministic fashion.

But in the real world, it doesn't often work like that. The stakeholders may be difficult to identify, or keep changing. The requirements change frequently, and conflict with one another or the overall objectives. Architecturally sound solutions are derailed by politics, or because the technology doesn't work as it's supposed to. These are characteristic of an unordered domain.

However, the architect's skills may be of great use in such circumstances, proposing structure to help stabilise the chaotic, or using analysis and abstraction to look for patterns in complexity. Having identified emerging patterns, he can help suggest or take action to reinforce and stabilise desirable patterns while discouraging undesirable ones.

In their paper, Kurtz and Snowden introduce the Cynefin Framework, which divides the space in which we make decisions or solve problems into four "domains":

In "known" space, things are nice and predictable. We know exactly how to handle things, with established, deterministic rules and procedures, and predict the outcome. For each new problem, we simply have to categorise the data, and apply the appropriate rule. This is the domain in which "best practice" has most meaning.

In "knowable" space, things require a bit more work. We may not know the answer immediately, but know that someone does, or that an answer can be found through formal analysis. The solution may be complicated, but it won't be "complex" in the mathematical sense - we can establish clear cause and effect relationships and act on them.

"Complex" situations are not so amenable to analysis. While cause and effect relationships exist, they are neither simple nor direct, and patterns of behaviour may vary over time, so you can't guarantee repeatable outcomes. Formal analytical processes will run into several problems (see the next section for details). We can probe the situation, look for emerging patterns, and work to stabilise and exploit them, but don't expect everything to boil down to simple, stable rules.

Finally, we have the domain of chaos, where behaviour seems random, unrelated to causes. The only thing we can do here is try and help bring some stability. Of course, no real situation is truly random, but a "very complex" state may be the same in effect.

The known and knowable are the domains of order, while the complex and chaotic domains are those of unorder.

The Kurtz and Snowden paper also considers transitions between the domains. Situations change over time. As our knowledge increases, there's a natural progression clockwise from Chaos to the Known, while forces of unorder tend to push systems the opposite way. Transitions can be gradual or sudden, and can be decisively one-way, or an oscillation between two domains. The following picture identifies the typical transitions with some explanatory labels:

The Cynefin framework helps you to understand why different solutions are appropriate in different circumstances. If you can characterise your problem space using the model, it may help direct you both in how you should act, and the type of solution you should be designing.

Structured, analytical methods suit the known and knowable domains. Both packages and bespoke solutions may be suitable in either domain, but it is more likely that the known will be supported by standard solutions, possibly with minimal analysis, while the knowable requires careful analysis and may require unique solutions.

In complex or chaotic situations probing, agile, techniques are more appropriate. Attempting to impose formal, "big analysis up front" methods will fail, as the analysis will encounter a variety of problems:

• Things may be changing so fast the analysts simply can't keep up,
• Formal analytical techniques may be overwhelmed by detail, or too many variables,
• Repeated attempts at analysis may get a different answer every time.

If any of these happens then "analysis paralysis" will set in, and any formal "waterfall" process is set to fail.

In chaotic situations change is the dominant characteristic, and very lightweight, reactive developments the most appropriate solution. The introduction of some solution, even a package or an agile prototype, may itself help to bring stability, but sometimes the most stabilising thing is to discourage any change, and a good architect should recognise when that's the right approach.

Solutions in the known space are often relatively inexpensive and painless, so management tends to seek them out. The difficulty is that they won't fit problems in the other domains. Part of an architect's value is in recognising this mismatch, and explaining to management why "buy a package" is the wrong answer.

Similarly a lot of IT procurement assumes a known domain: "We're building an X, so we want a ready-made expert in X". But if the problem is in the knowable or complex domains, you really want an architect who has skills in those domains. And that may mean someone who doesn't know it all already, but is willing to learn...

If you're designing a solution, you need to understand the "dimensions of change", so your solution has appropriate flexibility. Thinking about what is genuinely known, knowable or unknown about the problem may help. Look not only at the known requirements, but at any historic pattern of change or instability in them. (See my paper "Strategies for Flexibility" for more ideas.)

If your problem is inherently complex then you may need a system which seeks the best answer itself. Evolutionary, "genetic", algorithms or a "Monte Carlo"-type simulation may be better than an analytical solution. You may also need to guide against a simplistic imposition of order, by making sure your models are of sufficient complexity.

In unordered space traditional roles and responsibilities may be blurred. An architect may need to cross the boundaries with management, analysis and development roles, acting as needed to help find the right answer. Your own agility of approach may be an important contribution.

The architect must understand the forces of unorder. For example, rational analysis and design is founded on three questionable assumptions:

• People act deliberately. While usually true, we have all done things by accident, on a whim, or on a hunch. So does everyone else.
• People act consistently. This is untrue. A human's decisions and actions are guided by context. An individual may act differently for himself, for his family, for his company and so on.
• People act to maximise reward. Again untrue. Altruism aside, humans are good at considering the big picture as well as local forces, intangible as well as tangible benefits. Simplistic models which don't consider these will produce the wrong result.

The architect must question requirements, constraints and technical assessments which don't seem to fit an appropriate pattern. Failure of one or more of these assumptions is usually the cause, and if you understand the reason for someone's position, then it's much easier to either work with or work to change it.

We use patterns and models to describe behaviour, problems and solutions. In ordered space it should be possible to apply existing analysis and solution patterns. In unordered space the patterns may be new, and they may emerge from apparent chaos, but finding them could be a major contribution to progress.

Architects and analysts will naturally look for "technical" patterns in data, business rules or solution components. However, they must also be aware of patterns of a commercial or political nature. In unordered space these will strongly influence rates of change, the emergence of other patterns, and the acceptability of possible solutions, more so than in ordered situations. In the example of a company with both formal and informal influencing networks, smokers may have undue influence on a company's decision making, because they have the ear of the CEO on his cigarette breaks.

Emergent patterns may be positive, for example helping to understand the behaviour of a complex system. But they can just as easily be negative. For example gradually decreasing system performance may indicate an incipient failure. The architect must be alert to patterns of all forms, seeking a thorough, clear explanation whether or not it brings good news.

Abstraction is a vital tool in all of this. Separating the general from the specific is key to finding emergent patterns, and also to designing solutions which manage complexity and rapid change. Focus not only on what's different, but also on what's the same, but always be on your guard against imposing simple solutions where they don't belong.

As patterns emerge or models develop, they must be explained to appropriate stakeholders. It is always one of the architect's duties to help do this, but it becomes essential in complex situations. Stories which use metaphors or historical anecdotes can be a very powerful way of communicating complicated or technical issues using everyday language. Story techniques are becoming a popular tool for this reason. The Cynefin website contains a number of interesting papers about them.

Alternatively, clear, simple pictorial models may help. These may be the best way to explain the degree of complexity, or forces acting to push a system from an ordered state into an unordered one. Clear, concise visions can help reinforce the development of desirable patterns, and support moves from complexity to order. Provided they are not abused, tools like Strategic System Visions can promote a "virtuous circle" of both stability and increased understanding.

IT Architects or their clients who work only in ordered space are either extremely lucky, or deluding themselves. Most of us work in environments with significant elements of unorder. A wise architect works to understand the nature of his or her domain, exploits appropriate architectural skills and tries to develop well-matched solutions. I hope that the framework and ideas presented in this paper will help you to do just that.

The Cynefin Framework is described in "The new dynamics of strategy: Sense-making in a complex and complicated world" by Cynthia Kurtz and Dave Snowden, originally published in the IBM Systems Journal Vol 42 No 3, 2003. (Also available in pdf).

"Cynefin" is a Welsh word analogous to "heritage" or "background" but with a wider scope, essentially "everything which makes us what we are". The Cynefin web site is at www.cynefin.net.

Organisations need to protect and maximise the value of their IT assets. To protect against threats from business and technological change systems need to be flexible: able to change to support new functions, new workloads and new working environments. Flexibility does not happen by accident - it is usually the result of planning, forward thinking and adopting strategies known to enhance and encourage it. My paper "Strategies for Flexibility"   (PDF format) presents some of those strategies.

A number of books discuss the issue of situations too complex or too chaotic for normal analysis, but one book focuses on these issues specifically. It's Wicked Problems, Righteous Solutions, by Peter DeGrace and Leslie Hulet Stahl, (Prentice Hall 1990). You can buy it from Amazon UK or Amazon.com.

While not focused exclusively on these problems, AntiPatterns, by William J. Brown, Raphael C. Malveau, Hays W. “Skip” McCormick III and Thomas J. Mowbray (Wiley 1998) is an amusing book about common ways in which things can go wrong, and how to recognise when they do. If you think you’re failing, and you’re not sure why, this is a good place to start. You can buy it from Amazon UK or Amazon.com.

Are there specific situations in your architecture practice that call for a top-left approach? I think it would really progress the debate and the understanding if we had some solid examples of how the Cynefin framework applies to specific architectural projects in specific businesses. Cynefin is currently being picked up by the Agile Methods crowd, and it would be good to have some practical examples of how Cynefin makes a real difference to what architects can achieve

Yes, I do have some real, current examples where complexity is forcing me to say to the client “you can’t analyse this”. Watch out for a follow-on “examples” piece sometime soon.

My paper is a straightforward application and extension of Dave Snowden and Cynthia Kurtz’s 2004 work, and properly credits that work. Dave has indicated that he is happy with this.

Tom Graves has recently referred to this paper, I believe mainly as a source for the Cynefin diagrams without having to seek permission directly from Dave. Tom has not contacted me in any way, or sought my permission to re-use the diagrams in his article. I do not in any way endorse his views, or have any relationship to this derivative work.

Andrew: re: “Tom Graves has recently referred to this paper, I believe mainly as a source for the Cynefin diagrams without having to seek permission directly from Dave.”

I referred to this paper because I thought it was good work. The assertion that I referred to this paper “mainly as a source for the Cynefin diagrams without having to seek permission directly from Dave” is both insulting and absurd – not least because the Cynefin diagram is explicitly in the public domain anyway (see Snowden’s licensing notice on the Wikipedia page on Cynefin).

In the past I have done very extensive on ‘the Cynefin categorisation’, in particular on attempting to integrate the Chaotic domain, which is barely addressed in Snowden’s work (though it is addressed in some depth in Kurtz’s more recent work). The methods and approaches I used in that work are most certainly not ‘derivative’ – a fact which seems to be the main source of Snowden’s very public ire (including an extraordinary out-of-context misuse of two of my diagrams in his ‘History of Cynefin’, apparently for the sole purpose of mockery, and certainly without any apparent understanding of their proper context or use). It is certainly true that most of my work around ‘the Cynefin categorisation’ has a different practical and theoretical base – for example, Snowden concentrates on complexity-science, whereas my work leverages iterative/recursive techniques from the futures disciplines (such as Causal layered analysis) and enterprise-architectures (such as TOGAF ADM, as also extended beyond IT). At Snowden’s request, I have explicitly and publicly separated my work from his, although you might note that Kurtz does explicitly incorporate some of my ideas and material in her current work on ‘Confluence’.

Richard Veryard above asks “it would be good to have some practical examples of how Cynefin makes a real difference to what architects can achieve”, to which you replied “Yes, I do have some real, current examples where complexity is forcing me to say to the client ‘you can’t analyse this’: watch out for a follow-on ‘examples’ piece sometime soon”. However, it is now three months later: would you give us a timeline as to when you publish these examples? (In the meantime, if anyone is interested, there are many examples of real-life usages of a ‘Cynefin-like categorisation’ linked to proven enterprise-architecture methodologies available in my books – see TetradianBooks – and on my weblog.)

I do acknowledge that Snowden and I have disagreed strongly in the past over our significantly different approaches to theory and practice in the ‘unorder’ space, and I appreciate that people may sometimes choose to ‘take sides’ in such cases of ‘conflict of ideas’. However, ‘taking sides’ does not actually further the progress in the field. You might also note that Snowden’s work is not designed to work directly with and in enterprise-architectures; whereas mine is. In that sense, might I request that you at least consider my work properly in its proper context, rather than dismissing it outright on the say-so of someone from a largely unrelated field of enquiry?

Tom,

Apologies I have not contacted you sooner. I have been on holiday until last week, followed by a busy catch-up with my clients, and I was trying to digest your comments before taking action.

The reference to my paper is in your post “Tackling uniqueness in enterprise-architectures” at http://weblog.tomgraves.org/index.php/2010/06/03/uniqueness-in-ea/#more-986. I read this when my attention was brought to it by a Google blog scan back in June, not by any other mechanism.

Your post contains one direct reference to my paper, and no comment that I could discern on my own assessment/use of Cynefin. The only other references are where you reprint the diagrams, where you promptly make notes that these are not the “up to date” versions against which you would like to comment. In the comments to the piece you then used the phrase (addressed to Dave Snowden) “I would also point out that since you’ve publicly accused me of plagiarism if I do reference the original sources…”. You never contacted me to post any sort of comment on my article itself.

Taking these points together I think my inference about the use of my paper was unavoidable. If that was incorrect I apologise, but I hope you’ll accept it was a reasonable assessment based on the evidence I had.

I’m not a “Dave Snowden follower”, and I don’t agree with everything he says. However, I have found his/Cynthia’s work on Cynefin extremely useful, and I think we are both comfortable about the relative relationship between his work and my use of it. I was keen to “set the record straight” to ensure that your use of my paper as a source was not seen to challenge this relationship, hence the comment on my blog, and an equivalent comment on Dave’s blog.

I did read your paper, but didn’t find anything in it which I could latch onto as a working Solution/Enterprise Architect. That’s not to denigrate your work in this field, just an assessment based on my need for very simple, clear concepts which I can use with developers and business people with little or no knowledge or interest in the formal knowledge management space.

A quick point on Richard Veryard’s comment. That was originally made when the paper was published in 2005, and at the time I had the best intentions of publishing a follow-up piece with real world examples. However, as my clients are major organisations and my relationships with them mostly on public record, it has proven very difficult to find ways of documenting examples of “unorder” in a way which will not challenge those ongoing relationships. I may resolve this at a future date, but may not do so very quickly.

I hope this sets the record straight with you. I am happy to publish your comment alongside this explanation, if you will do the same.