Category Archives: Agile & Architecture

Agile Development & Software Architecture

Enterprise Architecture Conference

Halfway through, and this is shaping up to be the best EAC I have attended for a while.

I was umming and aahing about whether to attend yesterday’s seminar sessions, and couldn’t make up my mind which to join. In the end I made up my mind about the morning session while having a cup of coffee on the way, when I recognised one of the speakers, Lawrence Helm, as having given an excellent presentation a couple of years ago on NASA’s knowledge management problems. This time he and his colleague Robert Stauffer were talking about NASA’s adoption of Capability Modelling, and how they have put it to use supporting some very high level decisions about NASA’s future shape.

This was another stimulating session, and really benefitted from the extra space from making it a half-day session. Lawrence and Robert actually ran out of time, which was probably a testament to the depth of the material and the discussions it engendered.

The principle of relating capabilities to strategic objectives was not new to me, although the NASA examples certainly were. What did surprise me was the level of detail required for capability definitions in that environment. For example, the launch capabilities relate specifically to certain target longitudes and temperature ranges, and could not be moved to a location outside those ranges (for example Korou or Baikonur) without re-engineering the rocket platforms.

The afternoon session was also a bit random, as I got confused between Mike Rosen’s half-day seminar and his separate one hour talk for which I had the slides. Not a problem, the half day session on case study methods was very educational. The example, of how Wells Fargo created a federated model to integrate their various systems under a common customer model was interesting, and plays nicely into my EAI talk tomorrow. Like a good sermon, I didn’t learn much new, but I felt thoroughly validated that Wells Fargo did what I would have recommended, and succeeded with it. We had a very robust discussion on the importance of stable service interfaces, so hopefully that will drum up some support for my talk.

You get a very good class of attendee at these sessions. Alec Sharp joined the NASA session, and John Zachman joined the afternoon session, although he didn’t participate much.

Thursday’s highlights have probably been the two keynotes: this morning on how different companies have developed different strategies to come through and out of the recession, and this afternoon on “how to think like a CEO” and get your messages across to senior managers. However, there was also an excellent talk this morning by David Tollow on how EA feeds management and planning of long term outsourcing deals, from the supplier’s viewpoint. Very relevant to many of us in the current day and age.

Just to make things interesting, Sally has asked me to swap slots with someone else tomorrow, so my talk which was carefully trimmed to the constraints of the last slot on Friday will now be at 10 am. This may or may not be a good thing.

Wish me luck!

– Posted using BlogPress from my iPad

Location:Portman Towers,Paddington,United Kingdom

Posted in Agile & Architecture, Thoughts on the World | Leave a comment

Practical Enterprise Integration

I’m speaking at the IRM Enterprise Architecture Conference 2011, in London next week. My topic is “Practical Enterprise Integration: Realising the Benefits of a Strong Canonical Architecture”. In the paper I discuss the evolution of an EAI environment at National Grid, and how over time some key decisions on the underlying architecture have delivered significant benefits.

I’ve just uploaded the slides to my website. You can download them here.

Unfortunately, I’ve been put into a real “graveyard slot”, right at the end of business on the Friday afternoon. And I thought Sally, the conference chair, was my friend! If you are attending, I would very much appreciate your support. If not, I’d appreciate your thoughts on this topic, as always.

See https://www.andrewj.com/publications/EAI%20Evolution%20Slides.pdf
Posted in Agile & Architecture, My Publications | Leave a comment

The Wrong Orientation?

In an odd confluence, multiple streams of activity have come together to convince me that current IS thinking may be suffering from a bad dose of “the wrong orientation”. My work on data modelling at National Grid, an excellent course on business process modelling, my frustrations with the iPad software architecture and finally some exposure to Oracle’s latest SOA tools have all convinced me that we need to get back to designing systems with an “object-action orientation”, rather than the regressive move back to the alternative.

 

Read the full article
Posted in Agile & Architecture, Thoughts on the World | 2 Comments

A$$hole Driven Development and Other Anti-Patterns

During a project management meeting today, I was driven to look for a reference to “Document Driven Development”, a great anti-pattern developed a few years ago by the Agile crowd, in order to emphasise the importance of working solutions, not documents, as the goal of IT projects. I was in for a few surprises…

Oddly, although the wonderful “Waterfall 2006” web site still exists, I couldn’t find DDD on it. So I checked with Google and found a couple of references to non-ironic (as far as I can tell) papers on the subject. Yes, some people seem to think that document-driven development is a good idea! Now I might be prepared to concede this for applications where documents are themselves the key business objects (some legal processes, for example), but as far as I can see this isn’t what those papers were referring to. If that’s the case, they really haven’t understood…

What I did find, however, was a wonderful blog post from a few years ago with the excellent title “Asshole Driven Development“, in which Scott Berkun has collected a wide variety of development and project management anti-patterns. It takes a while to read through all the comments, but doing so is quite rewarding, if mainly as a form of therapy. At least you know you’re not alone.

The list is pretty comprehensive, but despite over 300 contributions, I couldn’t see my own bte noir. A lot of large corporate organisations now seem to follow a governance methodology I term IAKOM (the “It’s A Knock Out Method”), known on the continent as la Methode Jeux Sans Frontieres (MJSF). Those of a certain age will remember a series of hilarious television games in which relatively simple tasks (such as carrying a bucket of water) were rendered impossible by the imposition of progressive handicaps and obstacles (such as carrying the bucket up a greased slope against a rubber bungy while wearing clown shoes and being pelted with wet sponges).

Some IT governance is like that. Just when you think you might have a fair run at doing something, a new governance hurdle or document check is inserted into the process. It wouldn’t be so bad if it all made sense, but sometimes it feels almost capricious. Some organisations are more enlightened than others, but as a general industry trend it’s inescapable.

I don’t know what the answer is. If you do, let me know!

See http://www.scottberkun.com/blog/2007/asshole-driven-development/
Posted in Agile & Architecture, Thoughts on the World | Leave a comment

Red Roof Reflections

St. Nicholas Abbey, Barbados
Camera: Canon EOS 40D | Lens: EF-S17-85mm f/4-5.6 IS USM | Date: 20-04-2010 17:16 | ISO: 200 | Exp. bias: -2/3 EV | Exp. Time: 1/125s | Aperture: 11.0 | Focal Length: 17.0mm (~27.6mm) | Lens: Canon EF-S 17-85mm f4-5.6 IS USM

As my “photographic eye” develops, I find I’m noticing much more readily the colour of light, and how it can be modified by things both inside and outside the scene. This shot of St. Nicholas Abbey on Barbados is an interesting case in point. The porch has a flat roof, and that flat roof is obviously painted red to match the railings and guttering. We can’t see it directly, but its effects are very dramatic. I’ve boosted the colour saturation slightly to make this work on the web, but only slightly – the pink glow was immediately obvious as we looked back to take this shot.

St. Nicholas Abbey, despite its name, was never an Abbey, but a plantation house. It’s recently been revived, and sits at the hub of a busy farming and rum distilling business. It’s also an interesting example of the challenges of architectural re-use. It was built from a set of plans developed and used for a similar manor house in England. These were faithfully followed, including all the fireplaces and chimneys. In nearly 400 years Barbados has never had a day cold enough for any of the fires to be lit!

In Agile development, the mantra is that you don’t build features you don’t need, but I’ve rarely seen a discussion on what to do if those features come “free with the design”. Re-using an established design has significant benefits, particularly if the architectural effort comes at a significant cost (which was obviously the driver for the decision about St. Nicholas Abbey). Tailoring that design to omit features you don’t need will have a cost, and a risk that by doing so you break some other capability. For example, chimneys tend to be built very strongly, and often have an important structural role in a building. On the other hand, building features which won’t be used is also costly. If you can’t find exactly the right design pattern, you will have an interesting decision – whether to change it, or whether to follow it regardless.

View featured image in Album
Posted in Agile & Architecture, Barbados, Photography | Leave a comment

Beauty is Only Skin Deep

I’m currently reading a book called “Beautiful Architecture“. This has at its core the concept that some software structures are inherently elegant, things of beauty as well as great function, like many of our greatest buildings.

The trouble is that for every St. Paul’s there must be a Bletchley Park – an architectural mish-mash which while possibly important, successful or even revered is inherently inelegant, or even downright ugly.
My analysis is that behind the glossy facade, the iPad software architecture has to be the best current example of “Ugly Architecture”.

In many ways it’s strongly reminiscent of PCs in the days of DOS, or maybe Windows 3.0, before the emergence of strong component-based architectures and unifying design standards in Windows 95 and NT.
The fundamental problem is the application-centric model, in which each application is a stand-alone combination of code and data, with very few shared services or components. This naturally leads each application developer to “do their own thing”, implementing separate, widely varying solutions for communications, document storage, printing support and so on. Apart from a token “open in another app…” supported by some applications, there’s effectively no cross-application linking, leading to massive duplication of functionality and data, and some significant functional limitations, for example the inability to directly open a URL embedded in a document.

Each application has its own data area, which may or may not interact with iTunes, web sites or a PC via FTP, websites via WebDAV or various different cloud storage services. Data which should arguably be general visible just isn’t – you can upload video files to the photos area, but they won’t be visible in the videos list. To test a variety of editors with a document you need to deliver a different copy of the document to each app.

Each application supports different models for document exchange, and different cloud stores, so a user potentially has to have multiple separate cloud accounts. While “public” cloud storage may be fine for individuals’ personal data (although individuals may still have valid security and privacy concerns), it is a real concern if used for corporate information. In corporate contexts, connectivity, security, copyright, access rights, service levels, data protection and privacy obligations, regulatory and legal constraints may all be compromised or complicated by cloud use, and become significant issues.

There’s also an interesting security implication to this which you don’t often see discussed. Because there’s no accessible file system, and no extensibility model for the application filing model, there’s nowhere for anti-virus solutions to run, and as of today iPhones and iPads are effectively unprotected devices. There are probably numerous iPads in the wild acting as festering reservoirs of infected documents. Those who are security conscious can’t be happy about this, and I know that many corporate security departments are making moves to ban connectivity to corporate services for that reason.

Even if an application interacts with the host PC more directly, you get multiple copies of documents, typically the original, a copy in iTunes and one on the device, with no mechanism to synchronise them or compare version information. Apple’s own applications such as Pages are even worse, with a completely separate iTunes space from their own “My Documents” spaces, and an additional copy step in each direction. This is a version control and management nightmare!

Why could the iPad not support a simple shared filing area with proper two-way synchronisation to the host PC, as the Pocket PC has had from day 1?

The communications architecture is a similar mess. The only application which can communicate with the host PC over USB is iTunes, but iTunes can’t use WiFi. All other apps have to use WiFi, but there’s no real shared comms application infrastructure, so the result is another diverse and fragmented “roll your own” free for all. The most obvious way for a companion device to talk to its host PC, BlueTooth, isn’t supported at all!

The WiFi only design works fine in the confines of, say, a small home office. Elsewhere it’s problematic at best. Paid WiFi (e.g. in a hotel) is typically limited to a single device, so you’ll end up paying twice if you want to connect both devices. Corporate WiFi systems are typically similar, and you may not be allowed to connect the iPad directly. Even if you do get connectivity, these networks are often set up to prevent routing between devices, as a security measure, so that’s that, then.

The alternative is to set up either the PC or iPad as a hot-spot itself. On the iPad, this is only possible on jailbroken devices. On the PC, it can be complicated and opens up potential security issues. Neither is ideal.

Apple’s policies effectively put software development back in the Stone Age, in the particular sense that “monolithic” means “single lump of rock”. Each application has to be “stand alone”, implementing many things which should arguably be shared. For example, each file management application implements its own storage management dialogs, its own comms model, its own browser, its own PDF and Word file viewers, each with their own subset of functionality, dialogs and gesture support, and so forth. There simply doesn’t seem to be any real concept of shared components or companion applications. Let’s be clear: I’m not criticising the application developers for trying their best to provide a comprehensive solution – my criticism is directed squarely at the crass architecture through which Apple force such an approach.

Even those applications which implement the “open in another app…” capability to open documents in other viewers suffer two common problems: you frequently have to open the document natively before you can send it elsewhere, and the act of doing so usually creates yet another copy of the document to manage separately! 🙁

Ironically, where there are shared components they impose significant constraints and limitations. The most obvious is the keyboard. Essentially there’s only one way to get text directly into any application, and that’s to use the on-screen keyboard configured exactly as the application developer decides. It’s “my way or the highway”. This is a dramatic contrast with the Microsoft world, where even a humble 2003-era Pocket PC supports not only a variety of built-in and third-party on-screen keyboards, but also handwriting recognition, character recognition (like the Palm Pilot), Swype, and even limited voice recognition. Importantly, these are all user-selectable at any time text input is required. On the iPad you can buy an app with a different keyboard layout, or dictation capability, but you have to cut and paste the raw text into your target application and typically reformat it to suit. This is simply primitive.

What makes all this worse is that the iPad application approval/delivery model makes it unlikely that anyone will innovate a better solution. No approved application can have legal access to another app’s or central iTunes data. Without approval, you won’t appear in the App Store or run on non-jailbroken devices, so Apple simply impose their will, whether good or bad.

OK. I am starting to love my iPad, but the software architect within me is incredibly frustrated. This great hardware is hamstrung by a clumsy, unimaginative, software architecture and oppressive centralist control by those who worship according to The Book of Jobs. It could be so much better.

</rant>

Posted in Agile & Architecture, iPad, Reviews, Thoughts on the World | Leave a comment

Some Good News

I’ve just had a bit of excellent news – my submission for the 2011 Enterprise Architecture Conference in London has been accepted. The provisional title is “Practical Enterprise Integration – Realising the Benefits of a Strong Canonical Architecture” and I’m going to tell the story of the evolution and benefits of a strong Enterprise Integration Architecture at National Grid with which I’ve been closely involved over several years.

Interestingly, a very similar submission last year didn’t make the cut. Whether the change is due to an increase in the quality of my submission, or a decrease in that of the competition, only time will tell… 🙂

Posted in Agile & Architecture, My Publications, Thoughts on the World | Leave a comment

The Half Arsed Agile Manifesto

If you’re wondering why agile methods don’t work in your organisation, look no further than this. Very funny, but scarily accurate.

In the words of the Tao of Lao-Tsu, “If you want to control something, you must first let it go free”. Big organisations just don’t get this, because too many people want to hold onto their favourite control mechanism. Agility is a culture, and trying to do it inside a culture of formal control is very difficult. Occasionally you can succeed in spite of the controllers, and that’s a real achievement.

See http://www.halfarsedagilemanifesto.org/
Posted in Agile & Architecture | Leave a comment

21st Century Schizoid Man

My good friend and sometime manager, Mike Rawlins, has just started a new blog ruminating on leadership. In his first post, he discusses the question of how to decide what to do, to “do the right thing”.

Now I’m not sure whether his guidance on decision making process is generic, or whether that process depends on your organisational position and role in determining “the right thing”.  I don’t know whether the key difference in our perspectives is between leadership as a manager versus leadership as an influencer, or the difference between managerial and technical leadership, or the difference between synthesising solutions and deciding which to adopt, but Mike’s article portrays a very different perspective to mine.

Mike portrays as key the ability to focus on key issues, and exclude those which are “not relevant”.

In my experience as an architect and technical leader, I spend a lot of time understanding and analysing the different forces on a problem. These design forces may be technical, or human: financial, commercial or political. The challenge is to find a solution which best balances all the design forces, which if possible satisfies the requirements of all stakeholders. It is usually wrong and ultimately counter-productive to simply ignore some of the stakeholders or requirements as “less important” – any stakeholder (and by stakeholders I mean all those involved, not just senior managers) can derail a project if not happy.

Where design forces are either aligned or orthogonal, there is usually a “sweet spot” which strikes an acceptable balance. The problem effectively becomes one of performing a multi-dimensional linear analysis, and then articulating the solution.

However, sometimes the forces act in direct opposition. A good example, currently personally relevant, is system security, where requirements for broad, easy access directly conflict with those for high security. In these cases the architect has to invest heavily in his skills in diplomacy – to invest a lot of time understanding stakeholder positions. One common problem is “requirements” expressed as solutions, which usually hide an underlying concern which can be met many ways, once understood.

In cases of diametrically opposed requirements, there are usually three options:

  1. Compromise – find an intermediate position acceptable to both. This may work, but it may be unacceptable to both, or it may fatally compromise the architecture.
  2. Allow one requirement to dominate. This has to be a senior level business decision. As an architect, you then have to be sensitive to whether the outcome is genuinely accepted and viable, or whether suppressing the other requirements will cause the solution to fail.
  3. Reformulate the problem to remove or reduce the conflict. In the security example the architect may come up with a cunning partitioning of the system which allows access to different elements under different security rules.

Of course, you can’t resolve all the problems at once – that way lies madness. An architect uses techniques like layered or modular structures, and multiple views of the architecture to “separate concerns”. These are powerful tools to manage the problem’s complexity.

It’s also important to remember that the architecture, and its resolution of the various design forces (i.e. how it meets various stakeholder needs) have to be communicated to many who are not technical experts. The technical leader must take much of this responsibility. I have had great success with single-topic briefing papers, which describe aspects like security in business terms, and which are short and focused enough to encourage the readers to also consider their concerns separately.

One area where I do agree with Mike is the need to listen to the voice inside, and carry decisions through with integrity. For an architect, the question is whether the architecture is elegant, and will deliver an adequately efficient, reliable and flexible solution. If your internal answer to this is not an honest “yes”, you need to understand why not, and decide whether you and your users can live with the compromises.

And finally, the architect must protect the integrity of the solution against the slings and arrows of outrageous projects. Monitor in particular those design aspects which reflect compromises between design forces, because they will inevitably come under renewed pressure over time. You have to not only do the right thing, but ensure it is done right.

Non-Sequiteur

About the weird title: Mike is attempting to create his blog based largely on 1970s Prog Rock references. As a tribute to such an excellent idea, I feel compelled to join in (at least on this occasion)!

Posted in Agile & Architecture, Thoughts on the World | 1 Comment

A Parable

In 2008, Bibble 4, supported by its community of plug-ins, was probably the best RAW convertor available – it was certainly my favourite. Then in late 2008 Bibble abandoned support for it, although Bibble 5 was not yet available (for another year, as it turned out). I wrote a short parable to express my frustration at this inexplicable decision.

Although mainly about photography, I think there are lessons for anyone contemplating a disruptive, rewrite from scratch change to a software product – beware!

Read the full article
Posted in Agile & Architecture, Photography, Thoughts on the World | Leave a comment

A Shortage of Analysts?

I’ve just spent two days at the 2008 Enterprise Architecture Conference in London. It was a very high quality event, with a range of speakers covering topics from pragmatic analysis techniques to how to manage knowledge through the life of NASA’s Mars programme, more than any single working lifetime.

Overall there was much less focus on technology (read SOA and modelling tools) this year, and a vigorous and renewed focus on business alignment and business architecture, which, if we can deliver, potentially places architecture where it should be, as the business’s agent.

But there’s a problem. Good business analysis is fundamental to this, yet several delegates bemoaned the current lack of good business analysts. User organisations often struggle to articulate and abstract their needs, and this feeds into all downstream processes. Modelled requirements are an increasing rarity, poorly substituted by imprecise verbal statements in Word or PowerPoint.

The problem is, of course, not unique to analysts, and may have common cause with the equal lack of architects. Senior architects and analysts both tend to have several big birthdays under the belt, and many learned their trade as developers, gaining both practical method skills and the experience of turning ideas into working code. (The majority of exceptions have other “making it work” experience, such as building networks or running data centres.)

But in the current world of ERP packages and large-scale outsourcing, many organisations no longer build anything themselves. The live classroom has been thrown away.

I have worked with a number of good, keen young analysts, but most work for large supplier companies who still have both well-funded training programmes and the breadth of work to build experience and a broad skill set. These guys and girls can do a good job, but at the risk of higher costs and potential conflicts of interest.

We already know that this may reduce organisations’ ability to ensure the right solution to their needs, or assure its quality. Recent observations suggest that organisations who forgoe getting their hands dirty in IT will also suffer an increasing difficulty in creating a clear, concise and structured statement of those needs themselves.

Posted in Agile & Architecture, Thoughts on the World | Leave a comment

Paradigm Shift – Clear Memory Now!

I’ve been musing lately on why we in IT insist on forgetting so much valuable knowledge. I don’t know whether it’s because of our youth-obsessed culture and our focus on the newest and best, because of our tendency to prioritise on-the-job over traditional learning, or whether there’s simply too much in the “architect’s book of knowledge” (ABOK), and we all have to focus on the new to keep up.

I explore two very different examples: the value of understanding RS232 in this 3G+ world, and some recent discussions on service reliability, both of which can be resolved using some quite old knowledge…. (Read More…)

Read the full article
Posted in Agile & Architecture, Thoughts on the World | 2 Comments