Very early on in any course in marketing or economics you will encounter the concept of the "Unique Selling Proposition", the USP, that factor which differentiates a given product or service from its competitors. It’s "what you have that competitors don’t", a key reason to buy this one rather than an alternative.

With the current trend away from development specialisms such as architect towards relatively homogenous development teams, it is perhaps instructive to ask "What is the architect’s USP?" Why should I employ someone who claims that specialism, and give him or her design responsibility, rather than just expecting my developers to cover it?

I have written elsewhere about why I don’t buy into the ultra-agile concept of "architecture emerging from the code", any more than I would bet money on the script for Hamlet "emerging" from a finite group of randomly typing monkeys. (Of course, if you have an infinite number of monkeys then it’s more achievable, but that’s infinity for you…) However that argument is about process, and I believe that almost irrespective of process a good architect’s skills and perspectives can have a significant beneficial effect on the result. That’s what I want to explore here.

The Architect’s Perspective

One key distinction between the manager, the architect and the developer is that of perspective. As an architect I spend a lot of time understanding and analysing the different forces on a problem. These design forces may be technical, or human: financial, commercial or political. The challenge is to find a solution which best balances all the design forces, which if possible satisfies the requirements of all stakeholders. It is usually wrong and ultimately counter-productive to simply ignore some of the stakeholders or requirements as "less important" – any stakeholder (and by stakeholders I mean all those involved, not just senior managers) can derail a project if not happy.

Where design forces are either aligned or orthogonal, there is usually a "sweet spot" which strikes an acceptable balance. The problem effectively becomes one of performing a multi-dimensional linear analysis, and then articulating the solution.

However, sometimes the forces act in direct opposition. A good example is system security, where requirements for broad, easy access directly conflict with those for high security. In these cases the architect has to invest heavily in diplomacy skills – to invest a lot of time understanding and addressing different stakeholder positions. One common problem is "requirements" expressed as solutions, which usually hide an underlying concern that can be met many ways, once understood and articulated.

In cases of diametrically opposed requirements, there are usually three options:

• Compromise – find an intermediate position acceptable to both. This may work, but it may be unacceptable to both, or it may fatally compromise the architecture.
• Allow one requirement to dominate. This has to be a senior level business decision, but the architect must be sensitive to whether the outcome is genuinely accepted and viable, or whether suppressing the other requirements will cause the solution to fail.
• Reformulate the problem to remove or reduce the conflict. In the security example the architect might come up with a cunning partitioning of the system which allows access to different elements under different security rules.

Of course, you can’t resolve all the problems at once – that way lies madness. An architect uses techniques like layered or modular structures, and multiple views of the architecture to "separate concerns". These are powerful tools to manage the problem’s complexity.

The architect must look at the big picture, balance the needs of multiple stakeholders, and bring to bear an understanding of the business, of strategy, of technology and of development project work at the same time. If these responsibilities are split among too many heads and isolated within separate organisational confines then you lose the ability to see how it all fits together, and increase the danger of things "falling through the cracks".

The Architect’s Responsibilities

The architecture, and its resolution of the various design forces (i.e. how it meets various stakeholder needs) have to be communicated to many who are not technical experts. The architect acting as technical leader must take much of this responsibility. The messages may have to be reformulated separately for different audiences: I have had great success with single-topic briefing papers, which describe aspects like security in business terms, and which are short and focused enough to encourage the readers to also consider their concerns separately.

The architect must listen to the voice inside, and carry decisions through with integrity. For an architect, the question is whether the architecture is elegant, and will deliver an adequately efficient, reliable and flexible solution. If the internal answer to this is not an honest "yes", it is important to understand why not, and decide whether all the various stakeholders can live with the compromises.

The architect must protect the integrity of the solution against the slings and arrows of outrageous projects. (Hamlet again?) Monitor in particular those design aspects which reflect compromises between design forces, because they will inevitably come under renewed pressure over time. The architect must not only do the right thing, but ensure it is done right.

While every person on the project should be doing these things, there is a natural tendency for most to allow delivery priorities to take precedence. A developer’s documentation, for example, must be adequate to communicate the solution to other developers and maintainers, but does not have to be comprehensible to other stakeholders. However for the architect integrity, fit and communication of the solution are primary responsibilities, not optional. In addition the architect should have sufficient independence to call out and challenge conflicts of interest when they do occur.

The Architect’s Skills

The architect should be equipped with a distinct set of skills in support of these responsibilities. These will include:

• Design patterns and knowledge of how to apply them
• Tools and techniques to formally document both detail designs and wider portfolios

• Methods to ensure that requirements, especially non-functional ones, are documented unambiguously

• Methods to review a solution design, model its behaviour and confirm the solution’s ability to meet requirements

• The ability to clearly communicate solutions, issues and potential resolutions to a wide variety of stakeholders

• The ability to support the project and programme managers in handling the impact of issues and related decisions

Now it’s perfectly possible (and highly desirable) that others on the project will have many of these skills between them. However their combination in the architect is key to the delivery of the architect’s value, and a solution with a good chance of meeting its various objectives.

The Architect’s Position

A good architect should be able to operate in various organisational positions or roles and still deliver the above. Irrespective of the official organisation chart I often end up working between two or more groups, and I suspect this is a common position for many architects. It may actually be a natural result of adopting the architect’s unique perspectives.

The architect’s role may to some extent overlap with that of developers, analysts or product owners, and in smaller organisations or projects the architect may also take on one of these roles. In that case the architect must be able to "wear the appropriate hat" when focusing on a specific project issue or taking a wider view. The architect must then ensure that his or her ability to look at the wider picture is not compromised by the project relationship.

Conversely, a central architecture group may become accused of being in an ivory tower, separate from the realities of the business and the developers at the coal face. An architect in such a position must actively display an interest in and willingness to help with practical project issues.

A good architect will reconcile the need for a broad perspective and the specific responsibilities of a given position, thereby delivering distinct value compared with someone who has a more specific scope. I may on occasion be challenged for taking a wider interpretation of scope than others, but the insights which accrue from that perspective are almost always seen as valuable.

Conclusions

These are generalisations, and in practice there are as many variants on the architect’s role, skills and delivery as there are individuals who take the title. However it is generally true that an architect’s involvement increases the chance that a solution’s behaviour will be predictable, understood, and a good fit to its objectives. That’s the fundamental USP of the architect.

The Challenge

I often hear a statement which worries me, especially but not exclusively in agile projects, along the lines of “we’ll make sure it works when we test it later”.

Now you may think this is an odd view coming from a man who has written testing courses, presented conference papers on testing and developed testing tools, but let me explain myself.

First up, there’s the old chestnut that the objective of testing is not to prove something works, but to find errors. All you can actually do by testing is locate problems to be fixed, although obviously if problems are hard to find, that increases confidence in your product. However the much deeper issue is that testing is commonly viewed as an alternative to properly understanding and documenting the expected behaviour of a system, and reviewing in advance whether a proposed design will deliver that behaviour. That can be a recipe for failure.

Obviously in some areas this is an acknowledged and viable trade-off. If we are exploring functional alternatives, or working in a problem space where extracting documented requirements is tricky, then agile development and testing is a powerful solution, and we accept the rework that may result where we get it wrong. Having said that, even in something like UI development it may be better to develop cheap models such as wireframes, and at least attempt to explore solution fit before we commit too much to code.

The problem is that when we come to the more fundamental architectural elements and non-functional behaviour, the dynamics change dramatically. The best way to show this is a variant of the testing “V Model”:

For functional details, the gap between development and testing is small, and they can quickly be reworked and retested. However some of the key architectural and non-functional aspects can only be fully tested late in the delivery process (and frequently only late in the overall programme), if at all. The “testing gap” becomes huge, the impact of any change substantial, and the rework path lengthy.

One challenge is that many non-functional tests require an environment representative of the technology and scale of the production system. If this is provided at all, it is typically late in the project, or testing has to be shoe-horned into a short window on the production system before operations commence. If that uncovers a major issue, it is simply too late.

That’s assuming that the issue is detectable. In an agile development, it may be difficult to understand “what acceptable looks like”, if there is no adequate agreed, documented definition of the expected non-functional behaviour.

The other challenge is that good non-functional testing is hard, and limited in what it can achieve. Simulating a peak load is difficult, especially with the variety of data in a real production peak. You can simulate planned and unplanned equipment failures and restarts, but by definition only predictable events. If a problem only emerges from lengthy running or a “perfect storm” event, then testing is unlikely to uncover it. Basically resilience is testable, performance may be testable, reliability isn’t. Similar considerations also apply to other non-functional aspects like security.

The Solution

The solution is to adopt an analytical and predictive approach: trying to understand, articulate and document the expected behaviour of the solution, before you build it. Importantly this is not just thinking about the solution (although thinking is vital), but thinking with models.

Models in this context take many forms. They can be diagrams, possibly based on UML, but not necessarily: for example reliability block diagrams or fault tree analyses are powerful tools to understand resilience and reliability. They can be spreadsheets, for example profiling expected transaction mixes and their relative resource requirements. They can also be active software, whether simulations of some expected behaviour, or point implementations to quantify some aspect of the solution, but the point is that their purpose is to understand the solution before a major technical commitment, not to deliver functionality. Irrespective of form all models should lend themselves to a quantitative understanding of the solution, not just “what?”, but “how much?” and “how well?”.

For example, here’s a simple redundancy scheme modelled using RelQuest, my own Visio-based fault tree analysis tool, from which we can not only understand the various combinations of failures which lead to loss of service, but the relative probability and impact (e.g. Mean Time to Repair) for each combination.

Models and simulations provide you with an early understanding of the system behaviour, so you can understand whether something should work, or not, and if not where to focus your efforts. They can be detailed, like the example fault tree above, or doing an early first pass on a platform provider’s sizing tool, but a more approximate approach may also provide value.

Numbers are your friends. I am a great fan of Fermi estimates (see the sidebar) – quick “order of magnitude” approximations to see if you have understood the key elements in a problem, and whether the answer looks viable or not.

You can easily get viable estimates of this type for performance, capacity or reliability. If the answer is “no problem”, like we can easily accommodate millions of transactions per hour on a single server and we expect thousands, then you’re probably fine. If the answer is the other way round, like the developer who proudly presented me a solution which would take 1s CPU time to do a calculation, but we needed to do a thousand a second, then the design needs to change (I got it down to 2ms, which was acceptable). If it’s marginal, then you probably need to do a more accurate model and calculation, or build a greater degree of flexibility into the solution.

Simulations or low-volume experiments may be a valid way to understand CPU, storage and memory usage, network bandwidth requirements, threading, virtualisation, and even failover behaviour. Anything which scales linearly can be measured at low volume and extrapolated, but you need to be wary of areas such as network latency or storage throughput where that may not be valid.

Ultimately anything which builds your understanding and proves that you have thought about the problems in advance is good, even if some detail may only be confirmed at later stages. The key point is that the problems become targets for analytical thinking rather than hope and prayers, and that makes them solvable.

The Conclusion

Testing on its own is absolutely necessary, but very much not sufficient. For tests to be meaningful you have to describe the predicted behaviour in advance, and for the system to have any chance of passing those tests it has to be engineered accordingly. We increasingly seek to drive functional development from written user stories and behaviour specifications. In the same way professional development must be driven by quantitative models which forecast non-functional behaviour for testing to confirm, not discover in surprise.

A former colleague, Neil Schiller, recently wrote an excellent article, https://www.linkedin.com/pulse/agile-data-programmes-neil-schiller/, on the challenge of using agile approaches in data-centric programmes. In it, he referenced and reviewed a classic cartoon by Henrik Kniberg which is often used to promote the advantages of agile delivery:

Now it’s wholly possible that I am reading more into a limited analogy than appropriate, but I think this same diagram can also be used to explain some of the fundamental issues with agile approaches.

Think about what the bottom line is claiming: that by a set of small incremental deliveries we can somehow achieve the equivalent of transforming a scooter into a bicycle, into a motorbike and then into a car, each a fully working vehicle meeting the user’s requirements. In the real physical world this is laughable: each has a wholly different architecture with no commonality whatsoever between equivalent subsystems at any of the stages. Key properties arise from the fundamental structure – a simple tubular chassis for bike, a more complex frame including complex stressed moving parts like the engine and transmission for the motorbike, typically a monocoque chassis/exoskeleton for the car. These underlying elements form the basis, and you have to get them right as you can’t modify them later: you can’t “add strength” to a car by adding more tubes after the event (unless you are going banger racing!).

In the real physical world you create a complex engineered artefact by understanding its required properties, creating a layered structure which is designed to meet them, and then building up those layers to progressively deliver the required result. This requires that the most fundamental, least readily changed layers have to be right, and stable, early on, and only then can you add the upper more flexible elements. The first version of the process in the diagram is actually wholly correct, the second a joke.

Is it so very different for software? If we’re talking about major systems with real-world complexity and non-functional demands, I’m not convinced. The “ultra-agile” argument that it is always possible to “refactor” code to make changes. This is true up to a point, but it can be difficult and costly to change the underlying structure. If that does not meet requirements for security, or reliability, or performance, then no amount of fiddling will fix it, and if changes amount to a fundamental rewrite then it’s difficult to see where any advantage has been gained.

Obviously there are differences. The vehicle designer seeks to both create and use solutions which once right will be re-used many times (from hundreds to millions of instances), but will be difficult to change once in production. Software development is still largely about one-offs. Software requirements are typically less well-defined than for established hardware products. In vehicle manufacture, the roles of engineer/designer and constructor are distinct, whereas in software the designers often have an ongoing role in construction, and may at least subconsciously seek to extend that role (guilty as charged 🙂 ). On the other hand, the car designer knows that an approved design will be built largely as documented, whereas the software designer has no such assurance.

Fundamentally, however, I believe that software development can benefit from engineering disciplines just as much as the design of physical products. For example, it is much better to attempt to understand and predict up front how a given design will respond against non-functional requirements. Testing is a very good way to confirm that your solution basically works and to refine refinements. It is a very bad way to uncover fundamental deficiencies, especially if this occurs late in the development process.

This doesn’t mean that I don’t believe in agile development. Far from it, I am a great believer in iterative and incremental development, and structures such as Scrum sprints to manage them. However, I really don’t believe in architecture “emerging from the code”, just the same as I would not expect to see a great car design “emerge” from the work of a group of independent fabricators working on small parts of the problem without any overarching design. Cars “designed” in such a way tend to be more Austin Allegro (or AMC Pacer) than Bugatti Veyron.

Instead, Architecture has to be understood as providing the structure within which the code is developed, with that overall structure developed using engineering disciplines: assess the various forces on the design, articulate how these forces will be resolved (including what compromises are required), then document and model the solution to predict its properties.

If the requirement is for a sports car, design a sports car, don’t try and “refactor” a pushbike…

Creation of such designs, documents and models is a distinct discipline from coding. Some of this may be the domain of specialists, some may be performed by those who also have other development roles, but as a separate activity requiring appropriate skills and experience. Ironically I think Tom Gilb got it about right in his 1988 book “Principles of Software Engineering Management”, when he defined “Software Engineer” as someone who “can translate cost and quality requirements into a set of solutions to reach the planned levels”, and who has the skill to change any given quality dimension of a system by a factor of ten if required. The latter challenge would uncover a lot of people who call themselves “architects”.

In addition complex designs need some form of centralised, overall ownership and design control – this again requires specialist skills and cannot just be allocated randomly, but will sit with an Architect and/or a Product Owner.

Within such a framework concepts such as continuous integration and testing still make sense. Development, both functional and non-functional can still be managed via the backlog and sprint plans, epics and stories. However the “minimum viable product” may require completion of much of the underlying architecture as well as major functional capabilities. Major capabilities, both functional and non-functional, have to be analysed and designed up front, not left to stories somewhere in the backlog. The intermediate delivery is a car, albeit incomplete, not a complete bicycle.

Agile development and architecture are not incompatible, but complementary. Successful development of a complex real-world system will inevitably follow the first model in Kniberg’s cartoon, no matter how much the agilists would like it to be the second. At scale, and in the face of more challenging requirements software development needs to be treated as an engineering discipline, with agile structures in service of that discipline, not avoiding it.