I recently attended a day of the Butler Group "Application Development Strategies" Symposium. The following is a short report on some of the more interesting discussions and presentations.
The keynote dealt with the need for IT to be driven by the business, not the reverse. This has a number of implications:
We don't actually spend much money on delivering new value. Application development expenditure is typically about 30% of IT costs, but over half of this goes on maintenance and modernisation, with less than half for new initiatives. Assuming a typical figure that the IT budget is about 4% of the business's costs, only 0.5% of costs are actually being spent on delivering new value via IT. We need to get better value from this, but we also need to direct a greater proportion of IT budget at new capabilities.
The other major problem is growing complexity, down to technological change and growing business integration, but also significantly to short-termist decision making. Distributed and outsourced development increase the challenge, especially in respect of control and measurement.
I don't know whether the organisers had prompted them, but almost all of the presentations started with a reminder that we still have a "software crisis" - the vast majority of software projects fail to deliver to their original targets and estimates. The presentations suggested three independent, but not exclusive, approaches to try and resolve the problem:
There were a couple of interesting case studies. One is detailed below. Another, less detailed, suggested that a combination of process and architecture improvements can deliver a 45% improvement in development productivity, and 77% improvement in reliability.
There was also surprising agreement on things which won't solve the problem:
The Microsoft presentation was a bit disappointing, but introduced a useful "software factory" concept. Basically mass production techniques exploit "economies of scale", but are not relevant to software development because every software product is different. Instead we need to create stable "mass customisation" processes which deliver multiple, evolving products by exploiting "economies of scope or design" : common components, patterns and architecture. A "software factory" essentially consists of support for a given set of models, templates, patterns and components.
The latest generation of Microsoft tools deliver much better support for team working, patterns, and a higher level of abstraction than previously. In particularly they have introduced the concept of "Domain-specific languages" supported by graphical designers which can sketch out and then generate much more of the skeleton and "plumbing" of a complex system design than has previously been possible.
Probably the best presentation of the day was from Borland, about their work at BT. BT face a familiar set of challenges, but on a large scale. The business has to both "defend the core" (improve efficiency for established business) and "grow the new" (support innovation). They have a highly fragmented IT portfolio, a range of project sizes (from less than 20 to more than 500 people), and a highly distributed mix of internal, contract, SI and ODC effort.
A new CIO has decided to focus on agile development approaches, mandating that both hardware and software products should be delivered in cycles of a maximum of 90 days. Every cycle starts with a "Hothouse" which delivers some form of "prototype" and guarantees customer involvement. The short cycles are deliberately designed to feedback, and if necessary fail, quickly.
A couple of strategies ensure quality, maintainability and ongoing value, rather than the "stovepipe" systems which can result from agile methods. Testing is built in early, with good support for automation, and all developments have to fit within an overall architecture. This strongly echoes my vies on the importance of architecture in agile development.
They are aiming for substantial re-use of all sorts of artefacts from requirements to testware. To achieve this they are using much more scalable and integrated tools for requirements gathering and modelling, not just Word and Excel. Processes and collaboration are enforced by the workflow features integrated into the Borland tools.
A great deal of effort has gone into making the transition painless and productive. Central servers and support capability allow very quick project start-up using standard templates and tools etc. Recognising that the programme includes changing basic attitudes and ways of thinking it has been treated as a business change in its own right with a lot of focus on roll-out and communication techniques. Initial document-based roll-out materials were being ignored, so now there's much more use of mind maps, demos, mentoring, CBT, FAQ repositories and so on. However, the presenter emphasised that the greatest single aid was the visible high-level championship from the enthusiastic, knowledgeable CIO.
While there's not much numeric proof for the success so far, the programme is proceeding well, with a lot of anecdotal evidence of rapid delivery of value in areas where progress had previously been difficult (for example where an expensive requirements-gathering process had been completely ignored). The financial case is interesting - the first year of work has been entirely paid for by software licensing rationalisation. It will be interesting to see how this proceeds.
Supporting the Borland/BT presentation another speaker also talked about "Architecture-first iterative development" in an unusual context - development of complex systems for government, specifically defence.
Lengthening development cycles (from 9 months for the Harrier software to 5 years for Tornado) have caused increasing problems with requirements and technology churn, late discovery of flaws etc. Steria have been introducing agile techniques to try and address these issues:
In addition to this profile for technical risk, the presenter asserted that up-front estimates for IT projects are typically based on about 3% of the required information, so a 300% error should not be surprising! Therefore agile techniques, where each cycle is estimated separately building on what has been learned previously, should also reduce cost risk.
The presentation highlighted a number of key factors for success using agile processes in such risk-sensitive environments:
The focus has been on reducing risk rather than costs, but 20% productivity increases have been measured.
Several vendors were demonstrating their latest tools. Several things distinguish the best of the latest generation:
Some of the claimed productivity requirements can be explained simply by the tighter integration of things for which we currently have multiple tools and quite separate processes, but a realist would have to acknowledge that getting the full benefit would require a substantial IS process change programme like that being followed by BT.
Compuware's presentation was basically a sales presentation for "model driven architecture" (MDA), the use of tools to generate some or all of an application from models, but they did make two important general points:
The Object Management Group (who are basically a standards body) were also pushing MDA, but in the process of doing so made a couple of interesting observations:
In sharp contrast to equivalent presentations a couple of years ago, the only presentation focused on "service orientation" was quite downbeat. The key messages were:
There is some good news. Tools are moving closer to the target of integrated support for the whole application life-cycle, and various combinations of agile methods and sophisticated model-based techniques are being successfully adopted by large, complex organisations. However, taking advantage is likely to mean significant investment, being prepared to guide customers and suppliers in adopting more sophisticated approaches, and a change of direction from the waterfall-based methods which many organisations still mistakenly try to follow.
If you'd like to comment on this article, with ideas, examples, or just to praise it to the skies then I'd love to hear from you.