Conference Report - Application Development Strategies

I recently attended a day of the Butler Group "Application Development Strategies" Symposium. The following is a short report on some of the more interesting discussions and presentations.

Keynote

The keynote dealt with the need for IT to be driven by the business, not the reverse. This has a number of implications:

• Bespoke development is still crucial, to provide innovation and competitive advantage. Although commoditised applications are growing in scope, they will always be supported by both vertical market specialties and custom applications. Also recent studies suggest that a typical SAP implementation is 40% custom code, so that even core "package" ERP systems are substantially bespoke.
• Management expectations, regulatory compliance and legislation are driving a need for greater transparency in governance and the development process.
• A focus on costs instead of value means "IT driving the business" as surely as technology-led initiatives

We don't actually spend much money on delivering new value. Application development expenditure is typically about 30% of IT costs, but over half of this goes on maintenance and modernisation, with less than half for new initiatives. Assuming a typical figure that the IT budget is about 4% of the business's costs, only 0.5% of costs are actually being spent on delivering new value via IT. We need to get better value from this, but we also need to direct a greater proportion of IT budget at new capabilities.

The other major problem is growing complexity, down to technological change and growing business integration, but also significantly to short-termist decision making. Distributed and outsourced development increase the challenge, especially in respect of control and measurement.

Common Themes

I don't know whether the organisers had prompted them, but almost all of the presentations started with a reminder that we still have a "software crisis" - the vast majority of software projects fail to deliver to their original targets and estimates. The presentations suggested three independent, but not exclusive, approaches to try and resolve the problem:

• Adopting better, more agile processes to address fundamental weaknesses in "waterfall" processes,
• Adopting better tools and techniques to improve development productivity and the integration of the application life-cycle,
• Enforcing a stronger "enterprise architecture" framework for development.

There were a couple of interesting case studies. One is detailed below. Another, less detailed, suggested that a combination of process and architecture improvements can deliver a 45% improvement in development productivity, and 77% improvement in reliability.

There was also surprising agreement on things which won't solve the problem:

• No-one was promising a technical or product "silver bullet". The only presentation on service orientation was very pragmatic and surprisingly downbeat.
• No-one was suggesting that we should just "try harder" with old-fashioned tools and processes.
• There's no "one size fits all" solution. For example it's a mistake to force a formal, high-ceremony process onto small business systems developments.
• Excessive technical standardisation is also not the answer. The drawbacks include "lowest common denominator" technical solutions and inflated costs where the standard solution is "overkill".

Microsoft

The Microsoft presentation was a bit disappointing, but introduced a useful "software factory" concept. Basically mass production techniques exploit "economies of scale", but are not relevant to software development because every software product is different. Instead we need to create stable "mass customisation" processes which deliver multiple, evolving products by exploiting "economies of scope or design" : common components, patterns and architecture. A "software factory" essentially consists of support for a given set of models, templates, patterns and components.

The latest generation of Microsoft tools deliver much better support for team working, patterns, and a higher level of abstraction than previously. In particularly they have introduced the concept of "Domain-specific languages" supported by graphical designers which can sketch out and then generate much more of the skeleton and "plumbing" of a complex system design than has previously been possible.

BT / Borland

Probably the best presentation of the day was from Borland, about their work at BT. BT face a familiar set of challenges, but on a large scale. The business has to both "defend the core" (improve efficiency for established business) and "grow the new" (support innovation). They have a highly fragmented IT portfolio, a range of project sizes (from less than 20 to more than 500 people), and a highly distributed mix of internal, contract, SI and ODC effort.

A new CIO has decided to focus on agile development approaches, mandating that both hardware and software products should be delivered in cycles of a maximum of 90 days. Every cycle starts with a "Hothouse" which delivers some form of "prototype" and guarantees customer involvement. The short cycles are deliberately designed to feedback, and if necessary fail, quickly.

A couple of strategies ensure quality, maintainability and ongoing value, rather than the "stovepipe" systems which can result from agile methods. Testing is built in early, with good support for automation, and all developments have to fit within an overall architecture. This strongly echoes my vies on the importance of architecture in agile development.

They are aiming for substantial re-use of all sorts of artefacts from requirements to testware. To achieve this they are using much more scalable and integrated tools for requirements gathering and modelling, not just Word and Excel. Processes and collaboration are enforced by the workflow features integrated into the Borland tools.

A great deal of effort has gone into making the transition painless and productive. Central servers and support capability allow very quick project start-up using standard templates and tools etc. Recognising that the programme includes changing basic attitudes and ways of thinking it has been treated as a business change in its own right with a lot of focus on roll-out and communication techniques. Initial document-based roll-out materials were being ignored, so now there's much more use of mind maps, demos, mentoring, CBT, FAQ repositories and so on. However, the presenter emphasised that the greatest single aid was the visible high-level championship from the enthusiastic, knowledgeable CIO.

While there's not much numeric proof for the success so far, the programme is proceeding well, with a lot of anecdotal evidence of rapid delivery of value in areas where progress had previously been difficult (for example where an expensive requirements-gathering process had been completely ignored). The financial case is interesting - the first year of work has been entirely paid for by software licensing rationalisation. It will be interesting to see how this proceeds.

FMI Solutions / Steria

Supporting the Borland/BT presentation another speaker also talked about "Architecture-first iterative development" in an unusual context - development of complex systems for government, specifically defence.

Lengthening development cycles (from 9 months for the Harrier software to 5 years for Tornado) have caused increasing problems with requirements and technology churn, late discovery of flaws etc. Steria have been introducing agile techniques to try and address these issues:

In addition to this profile for technical risk, the presenter asserted that up-front estimates for IT projects are typically based on about 3% of the required information, so a 300% error should not be surprising! Therefore agile techniques, where each cycle is estimated separately building on what has been learned previously, should also reduce cost risk.

The presentation highlighted a number of key factors for success using agile processes in such risk-sensitive environments:

• Continuous quality verification processes
• Multiple pilots
• Not adopting radical new processes on major, business-critical project
• A high-ranking process champion

The focus has been on reducing risk rather than costs, but 20% productivity increases have been measured.

Tool "Labs"

Several vendors were demonstrating their latest tools. Several things distinguish the best of the latest generation:

• Full integration of requirements, change, problem, test and source management with modelling and/or development. The aim, which some are starting to approach, is "application lifecycle management" rather than "development" tools,
• Built-in workflow and discussions to enforce and ease the process - this is vastly superior to the "island" solutions of older generations,
• Good integration with one or both of the two leading IDEs: Microsoft Visual Studio and Eclipse. Most also have a web interface for external access, but this is not the primary interface.
• Standards support, for example UML for modelling, XMI for model exchange.

Some of the claimed productivity requirements can be explained simply by the tighter integration of things for which we currently have multiple tools and quite separate processes, but a realist would have to acknowledge that getting the full benefit would require a substantial IS process change programme like that being followed by BT.

Compuware

Compuware's presentation was basically a sales presentation for "model driven architecture" (MDA), the use of tools to generate some or all of an application from models, but they did make two important general points:

• Business needs to decide on the extent and focus of testing. Otherwise regression tests in particular tend to become focused on minor / visible defects, and not on the real functional issues.
• Businesses must not outsource their key intellectual property. This means keeping business, architecture and application expertise in house. To do this and still exploit cheap outsouced labour for bulk test and development activities you need to use better modelling and requirements management tools which allow a controlled, standards-based handoff at appropriate points in the life-cycle.

OMG

The Object Management Group (who are basically a standards body) were also pushing MDA, but in the process of doing so made a couple of interesting observations:

• Integration is the major challenge, with too many computers, languages and operating systems, and an increasing churn.
• We must stop pretending that today's delivery platform is the last - each only lasts around 7 years (if you're lucky). Given this it's probably a mistake to invest intellectual capital mainly in code, which is another justification for better modelling.

Services (Tom Welsh)

In sharp contrast to equivalent presentations a couple of years ago, the only presentation focused on "service orientation" was quite downbeat. The key messages were:

• Web services are about interfacing application logic, not an end in themselves. There's probably no justification for their use unless you need to link heterogeneous platforms, as other mechanisms are much easier between like platforms within one enterprise.
• If you do need to use them, stick to "Middleweight" use, based on the core standards. This is well supported by current tools and can be effective, cheap and good value.
• Avoid "Heavyweight" use based on the full set of standards. Tools and standards are still evolving, and this is very challenging.
• Just like web sites a few years ago, Web Services have the potential to start small but suddenly become heavily used services critical to multiple organizations. It's therefore important to design carefully, and not commit too early to SLAs.
• Security is also a potential issue. There are solutions, but these are more complex than originally thought.

Conclusions

There is some good news. Tools are moving closer to the target of integrated support for the whole application life-cycle, and various combinations of agile methods and sophisticated model-based techniques are being successfully adopted by large, complex organisations. However, taking advantage is likely to mean significant investment, being prepared to guide customers and suppliers in adopting more sophisticated approaches, and a change of direction from the waterfall-based methods which many organisations still mistakenly try to follow.

Comments

If you'd like to comment on this article, with ideas, examples, or just to praise it to the skies then I'd love to hear from you.

Comment on this article

Please share: