Now I’m not sure whether his guidance on decision making process is generic, or whether that process depends on your organisational position and role in determining “the right thing”.  I don’t know whether the key difference in our perspectives is between leadership as a manager versus leadership as an influencer, or the difference between managerial and technical leadership, or the difference between synthesising solutions and deciding which to adopt, but Mike’s article portrays a very different perspective to mine.

Mike portrays as key the ability to focus on key issues, and exclude those which are “not relevant”.

In my experience as an architect and technical leader, I spend a lot of time understanding and analysing the different forces on a problem. These design forces may be technical, or human: financial, commercial or political. The challenge is to find a solution which best balances all the design forces, which if possible satisfies the requirements of all stakeholders. It is usually wrong and ultimately counter-productive to simply ignore some of the stakeholders or requirements as “less important” – any stakeholder (and by stakeholders I mean all those involved, not just senior managers) can derail a project if not happy.

Where design forces are either aligned or orthogonal, there is usually a “sweet spot” which strikes an acceptable balance. The problem effectively becomes one of performing a multi-dimensional linear analysis, and then articulating the solution.

However, sometimes the forces act in direct opposition. A good example, currently personally relevant, is system security, where requirements for broad, easy access directly conflict with those for high security. In these cases the architect has to invest heavily in his skills in diplomacy – to invest a lot of time understanding stakeholder positions. One common problem is “requirements” expressed as solutions, which usually hide an underlying concern which can be met many ways, once understood.

In cases of diametrically opposed requirements, there are usually three options:

1. Compromise – find an intermediate position acceptable to both. This may work, but it may be unacceptable to both, or it may fatally compromise the architecture.
2. Allow one requirement to dominate. This has to be a senior level business decision. As an architect, you then have to be sensitive to whether the outcome is genuinely accepted and viable, or whether suppressing the other requirements will cause the solution to fail.
3. Reformulate the problem to remove or reduce the conflict. In the security example the architect may come up with a cunning partitioning of the system which allows access to different elements under different security rules.

Of course, you can’t resolve all the problems at once – that way lies madness. An architect uses techniques like layered or modular structures, and multiple views of the architecture to “separate concerns”. These are powerful tools to manage the problem’s complexity.

It’s also important to remember that the architecture, and its resolution of the various design forces (i.e. how it meets various stakeholder needs) have to be communicated to many who are not technical experts. The technical leader must take much of this responsibility. I have had great success with single-topic briefing papers, which describe aspects like security in business terms, and which are short and focused enough to encourage the readers to also consider their concerns separately.

One area where I do agree with Mike is the need to listen to the voice inside, and carry decisions through with integrity. For an architect, the question is whether the architecture is elegant, and will deliver an adequately efficient, reliable and flexible solution. If your internal answer to this is not an honest “yes”, you need to understand why not, and decide whether you and your users can live with the compromises.

And finally, the architect must protect the integrity of the solution against the slings and arrows of outrageous projects. Monitor in particular those design aspects which reflect compromises between design forces, because they will inevitably come under renewed pressure over time. You have to not only do the right thing, but ensure it is done right.

Non-Sequiteur

About the weird title: Mike is attempting to create his blog based largely on 1970s Prog Rock references. As a tribute to such an excellent idea, I feel compelled to join in (at least on this occasion)!

I recently converted my blog from an old bespoke (= “custom”, for my American friends) solution to one based on WordPress. However, this created a problem, in that the WordPress model is to hold all content in the database, and that wasn’t the right model for me.

Firstly, I have a number of articles which are very long for a blog post, and I had no interest in restructuring them. I also didn’t want to break external links to the existing articles.

Next, I decided that I wanted the freedom to continue to write in that style. Some of my writing takes several weeks, and it works for me to draft it as separate HTML pages. I also sometimes want to include active content or multiple images, and I don’t want to create a large and unwieldy WordPress database full of such stuff.

Finally, my online photo galleries are managed and generated using Jalbum, and I wanted to find a way of neatly integrating single images into my blog, complete with the watermarks and metadata extraction which Jalbum manages so well, without duplicating that functionality in WordPress.

This is probably typical of many older web sites, but WordPress doesn’t really embrace the integration of external content. This article describes how I solved this problem, and a WordPress plugin I have developed to make my solution reusable.

Now, I think there’s a danger they are swinging the opposite way. Yesterday, in the manual for my new Canon 7D, I found a word I never expected to find in a camera user manual: “hereinafter”. Correctly used as well, but is this quite the language we expect of a user manual for the masses?

I have therefore now created a number of new “views” of my blog, and a number of specialist feeds, as follows:

• Thoughts on the World. This is the original and complete blog and feed, and will give you access to all my content, in chronological order. Expect a mix of articles including my observations of the world of software development and architecture, humorous observations, photography and personal announcements.
• The blog address is www.andrewj.com/blog
• The feed is http://feeds.feedburner.com/ThoughtsFromAndrewJohnston
• The Professional Blog. This will focus on professional articles based around my observations of the world of software development and architecture. Articles for AgileArchitect.org will appear here, as will updates on my products and publications, and I may cross-post relevant reviews. You’ll also see any practical announcements relating to my websites and feeds.
• The blog address is www.andrewj.com/blog/pro
• The feed is http://feeds.feedburner.com/StructureOfTheWorld
• The Photo Blog. This will focus on my photos, and photography-related articles and reviews. You’ll also see any practical announcements relating to my websites and feeds. There’s also a separate feed for updates to my gallery.
• The blog address is www.andrewj.com/blog/photo
• The feed is http://feeds.feedburner.com/ImagesOfTheWorld
• The feed for my gallery is http://feeds.feedburner.com/GalleryOfTheWorld
• The Review Pages. This presents my reviews of books, films and music. You’ll also see any practical announcements relating to my websites and feeds.
• The blog address is www.andrewj.com/blog/reviews
• The feed is http://feeds.feedburner.com/ReviewsOfTheWorld

For more details, visit my Blog Views and Feeds page.

Camera manufacturers have made enormous strides in very difficult technology areas, but current DSLRs fall down in so many simple usability areas. Why?

I remember the grand old days of Windows 3.0. This was Microsoft’s third attempt to deliver a window-based environment on the PC, and had a load of technical innovations which showed that this could at last be a reality. In practice, it was a bit flaky, with some enormous frustrations (does anyone else remember the old File manager?!!)

Then came Windows 3.1. This was solid, fast, and worked so well that some people are still using it.

Windows 95 introduced a radically overhauled architecture, with the object-oriented user interface we all know and love, and a much cleverer structure for common components like drivers and communication components. In practice, it was a bit flaky, with the odd enormous frustration.

Then came Windows 98. This was solid, fast, and worked so well that some people are still using it.

Is anyone else spotting a pattern here?

Windows 2000 introduced a load of technical innovations, merging the “NT” and “9x” code bases into a single workstation line and a separate server stream based on the same core. Interestingly, although this worked pretty well, I even caught Microsoft salesmen saying to corporate clients “there’ will be an update out next year – wait for that”.

That was Windows XP. This was solid, fast, and worked so well that some people are still using it. I still run it on my laptops, although the big beast now runs Windows 7 (and Frances’ laptop manages on Vista).

If you look at the history of other Microsoft products (Word, for example), you see the same pattern: an “architectural innovation” release, followed by two or three consolidation releases which build on the new architecture and make it stable. Any the reality is that the same is equally true for many other software suppliers – see my recent postings on Bibble for another example.

So here’s my threefold confident prediction:

1. Windows 8 will introduce a load of new technology, which will move the world of computing on. It will also be full of frustrations and most people will hate it. The critics will pan it and explain that it’s the end of Microsoft and computing as we know it. There will generally be a great wailing and gnashing of teeth.
2. As a result, some people will still be using Windows 7 in 2020. It wouldn’t surprise me if a few are still also using XP, 98 and 3.1!
3. Windows 8.1/9 will be solid, fast and people will love it.

Don’t say I didn’t tell you!

My new blog is now live. This allows a lot more flexibility, including commenting etc.

Please can you make sure you are taking your RSS feed from one of the following:

http://feeds.feedburner.com/ThoughtsFromAndrewJohnston (preferred)

http://www.andrewj.com/blog/index.php/feed/ (straight from my site)

I still need to update some of the links to longer old articles, so please bear with me if these are a bit odd over the next few days. Also you may see some duplicate items in the Feedburner feed – if so, please ignore these, and they should clear shortly.

Thanks, and enjoy the new blog.

Andrew

Migration of our web sites and email to the new hosting server is almost
complete. See previous article/rant for some of my findings. After some initial
frustration with my hosting provider (WebFusion) when I discovered the
limitations of their new Linux shared hosting services compared with the old
one, I have to say a big “Thank You” for their efficiency in finding a better
solution for me, and providing me with effective technical support to get it up
and running.

If you do have any problems with our sites or email, let me know…

I’m currently developing a new blog, based on WordPress. This will make it
easier to post “on the fly” than with the current solution, hand-carved from XML
and ASP (now PHP).

When the new blog is running, I’m going to have a regular post for fans of my
photography, so you can see what I’ve been working on. In the meantime, I’ve
updated my gallery pages so they are a bit easier to navigate, bookmark and
search.

And talking about my photography, I’ve recently been accepted by the Alamy
stock agency. So please all rush at once and spend lots of money licensing my
pictures for all those uses you’ve dreamed of but were too polite to mention

See you soon,

Andrew

I’ve just spent a week-end migrating my website to a new hosting server. As
part of that process, I had to rewrite all my old ASP code using PHP. Here’s
what I learned:

1. The Apache/Linux community have misleadingly changed the meaning of “ASP”.
   If you bought a Linux-based hosting service 5+ years ago with “ASP”, it meant
   a *nix port of Active Server Pages. That worked for me, as I could develop it
   on Windows. Now, if you buy a Linux hosting service with “ASP” it means
   “Apache Server Pages”, and the embedded language is Perl. Useless!
2. PHP has positively the worst combination of features for a language:
   • A c-based language’s sensitivity to case, ending semicolons and curly
     bracket counts,
   • None of the protections against errors in the latter that a C++/Java (or
     VB) language gives you, like strong typing and forced variable
     declaration,
   • No single-step debugging. Now I accept that this may not be 100% true,
     so don’t all write in with the names of all the debuggers I didn’t find in a
     quick search for tools on Sunday morning, but certainly I don’t have one at
     the moment,
   • It runs differently on Windows and Linux, and in a way I haven’t yet
     understood 100%, so I can only test by uploading to my live
     website.

That said, I’ve still got it! I’ve managed to convert my blog and my book
reviews, and I’ve actually improved on my old code for the latter. Just please
let me have VB.NET back for my next major project.

OK. </rant>

The modern DSLR is a great image making machine, and I don’t want to change it’s core paradigm. But as I use my cameras in different ways, I recognise a number of lost opportunities and frustrations which have yet to be addressed. Now is the time to divert a small fraction of that enormous engineering effort from the hunt for ever more megapixels to the development of software and mechanical features which will make the DSLR a more flexible and better-integrated photographer’s tool.

In this article, I suggest a round dozen ways in which DSLR manufacturers could easily improve the product, mostly through simple software or mechanical refinements, to ease the process of taking and processing photos.

Even better, “I and I own comments here man” is also deemed fine!

I may explore this further – be afraid, be very afraid…

A few years ago I toyed with replacing my trusty PDA, phone and digital camera with a combined unit. That was not a success, and I ended up with a second hand SmartPhone, an iPaq 4700 PDA, and the T-Mobile MDA as my car satnav (a job it does acceptably). You can read an analysis of my trials and tribulations, entitled “Annoyance-Based Technology Selection”.

Last year, with my old mobile phone wearing out, I tried again, with depressingly similar results. Someone once said “Those who do not learn from History are condemned to repeat it.” Why didn’t I follow this excellent advice?

Although mainly about photography, I think there are lessons for anyone contemplating a disruptive, rewrite from scratch change to a software product – beware!

Mike’s solution to this is to suggest that each photographer’s website should start with “A photographer’s ten best, or ten favourite, or ten most characteristic pictures, up front.” He gives this concept the excellent name of the tenset.

Suitably chastened and inspired, I’ve taken this suggestion to heart, and my website now contains  my very own “Tenset”, as the first grouping on my gallery page. If you’re not sure what makes me tick as a photographer, the subjects which interest me or the style I am trying to develop, look here. If you like what you see, continue browsing. If you don’t, then go no further.

I’ve also written a short article providing a bit more background on the tenset concept, and how I managed to choose my own selecton.

In July 2008 my wife and I attended the Northampton Balloon Festival. An enjoyable time was had by all, notwithstanding a complete absence of balloons, occasioned by the British weather.

One of the highlights was a successful attempt on the world record for the greatest number of performers singing “Y.M.C.A”! This shot captures the scene:

OK, it’s not “Afghan Girl” or “Migrant Mother”, but I liked it, capturing a British Bobby policing his community exactly as required.

This photograph is legal. Had I taken it after February 16th 2009 I would have committed a crime. The stupidity of that law hands victory to the terrorists.

Andrew

My photography zone has been updated with images from my trip to Maine, Vermont and New York last autumn, and from the dramatic Winter weather in the UK. Photos from my trip to the Venice carnivale should start to follow shortly.

This isn’t a Bond film, it’s like a bad entry in the Bourne series. Where is the elegance, the charm we should expect of Bond? Bond films have always traditionally leavened the action with humour and beauty. Both were spectacularly missing from this episode. Read the rest of my review…

The problem is that there’s little reliable advice on how to correct this. There are a few articles on the web, but I haven’t found any to provide a reliable and repeatable process which can be applied to “casual” underwater photography.

After a fair amount of research and experimentation, here’s my analysis of the options available, and a suggested technique which I have developed and which I haven’t seen documented anywhere else.

Overall there was much less focus on technology (read SOA and modelling tools) this year, and a vigorous and renewed focus on business alignment and business architecture, which, if we can deliver, potentially places architecture where it should be, as the business’s agent.

But there’s a problem. Good business analysis is fundamental to this, yet several delegates bemoaned the current lack of good business analysts. User organisations often struggle to articulate and abstract their needs, and this feeds into all downstream processes. Modelled requirements are an increasing rarity, poorly substituted by imprecise verbal statements in Word or PowerPoint.

The problem is, of course, not unique to analysts, and may have common cause with the equal lack of architects. Senior architects and analysts both tend to have several big birthdays under the belt, and many learned their trade as developers, gaining both practical method skills and the experience of turning ideas into working code. (The majority of exceptions have other “making it work” experience, such as building networks or running data centres.)

But in the current world of ERP packages and large-scale outsourcing, many organisations no longer build anything themselves. The live classroom has been thrown away.

I have worked with a number of good, keen young analysts, but most work for large supplier companies who still have both well-funded training programmes and the breadth of work to build experience and a broad skill set. These guys and girls can do a good job, but at the risk of higher costs and potential conflicts of interest.

We already know that this may reduce organisations’ ability to ensure the right solution to their needs, or assure its quality. Recent observations suggest that organisations who forgoe getting their hands dirty in IT will also suffer an increasing difficulty in creating a clear, concise and structured statement of those needs themselves.

I’ve also always had great admiration for those photographers whose work can really move the emotions, particularly the great landscape photographers like Ansel Adams. Taking good representational photographs is hard enough, but establishing a composition which can both emotionally inspire the viewer and satisfy the critics is even harder.

Over the last year or so I’ve been rediscovering photography with a Digital SLR, and on my new photography pages you’ll find some of my efforts, plus (in coming weeks) some hints, tips and tools for the D-SLR photographer.

I explore two very different examples: the value of understanding RS232 in this 3G+ world, and some recent discussions on service
reliability, both of which can be resolved using some quite old
knowledge…. (Read More…)

• “Freakonomics” by Steven D Levitt & Stephen J Dubner
• “Examples: The Making of Forty Photographs” by Ansel Adams
• “Dark Matter” by Philip Kerr
• “Sevent Ancient Wonders” by Matthew Reilly

I presented a paper on Agile Architecture. If you regularly read my work you’ll recognise many of the ideas, but I’ve managed to bring them all together for the first time. You can download my slides and script here.

What was very interesting was how the thrust of the material has changed from a few years ago. No-one was claiming that a given framework, process or toolset can solve EA problems. At the risk of being uncharitable I thought John Zachman’s ideas sounded very tired, and there was almost no mention of alternative frameworks such as TOGAF. I may have self-selected by not attending any vendor sessions, but there was also no promotion of tools or technology. A common view was that EA, SOA and many supporting concepts are currently entering the trough of the hype cycle.

Instead the focus was largely on people-related problems and approaches. The labels varied, but several speakers introduced ideas familiar to agile architects. Maybe we’re doing something right after all.

Why you need architects, in song and dance.

Enjoy!

As a first step, I am trying to develop a comprehensive list of test automation “best practices”, grouped roughly by life-cycle (or iteration) stage. I’m trying to find practices which are broadly independent of specific methods and technologies, although obviously tool support may vary depending on the chosen technology.

This article is my first draft of such a list.

I’d welcome suggestions from my readers if you think there are any omissions (or if you substantially disagree with anything I’ve included).

Thanks

Andrew

I’m presenting a paper entitled “The Agile Architect”. This focuses on both how agile projects can have a strong architecture, and how architects can learn and benefit from agile approaches. I take a rather different approach to some recent papers with a similar title (e.g. at this week’s otherwise excellent Microsoft Architecture Insight conference), which suggest that agile projects can “do away with the architect”.

I look forwards to seeing you there.

Andrew

But I’m not sure that’s the key question. I think the right question is “Who are the architects?”

Somehow, a number of tasks must be discharged, but how varies from project to project. In the last year I’ve had a modest building project which tells an interesting story about how different people contribute to “the architecture”. Read more here…

Sorry, there’s no Coppertrees Calendar for 2006 – another casualty of our very busy 2005. Hopefully it will make a return in 2007.

I’m currently working on a new paper, looking at the different ways of modelling dependencies in software. I’m hoping it’s possible to merge some of the matrix-based techniques with pattern-based approaches – watch this space for more details.

A very Happy New Year to all my readers. Here’s hoping that we’ll all have a slightly easier 2006.

This is a story of a project I worked upon which followed an informal, agile process, but delivered a successful architecture. Hopefully it serves to support my assertion that agile can have an architecture, but needs an agile architect.

However, UML doesn’t help when it comes to the transformations themselves, and typically the detail has to be captured either in code, or a proprietary format. Most good integration tools provide some sort of “visual mapping tool”, where the developer drags and drops to create links between representations of the two structures, usually imported directly from their physical schemas. Here’s an example using SeeBeyond. Altova provide a good stand-alone data mapping tool called Mapforce – here’s an example showing it in use. The problem is that these tools work directly with the physical structure, and don’t export the mapping information in a reusable format, so that information is completely disconnected from the UML analysis or design models.

I have experimented with trying to represent mapping information in a UML model, but so far without much success. The best solution I’ve found so far is to use some sort of “pseudo code” (it could be OCL, pseudo-Java, pseudo-VB or anything similar). For example, we could easily annotate the model with code fragments such as:

Database.order_table.order_no = Message.Header.OrderNo

(where each element refers to a UML Package.Class.Attribute combination).

The problems are that it’s not clear where to put this annotation, most UML modelling tools won’t help generate it, and there’s no graphical representation. Ultimately, writing pseudo-code like this is probably not much better than abandoning the model and moving straight to using your integration tool.

My question is: does anybody know a better way? Has anybody found a good way of representing mapping information in UML? And if so, is there any good tool support?

If you know, please send me a message.

While the metaphor might still be valid, some people are beginning to question how far it should be taken. Helland’s article, like others before it, implies that “good” EA looks rather like a medium-sized modern American town, complete with relatively standard services, civic buildings and commercial venues. In an answer to the original “Metropolis” article Richard Veryard and Philip Boxer have published “Metropolis and SOA Governance” which challenges several of Helland’s assumptions.

I think that maybe we should extend the metaphor by thinking about cities, or Enterprise Architectures, as very diverse entities. What sort of “city” do you live in? To what extent is it planned? What is the vision, and do the citizens share in it? Does the EA resemble a nice neat midwest town, a dark, brooding Gotham City, a glass and steel Utopia, a federation of small towns with lots of empty space between them, a medieaval walled town, or a wartime mid-european ghetto?

And the metaphor can be taken further. Do you want to promote “infill development”, closing up functional gaps, or do you want to keep clear separation between the various zones? Do you want the shared services to be clearly visible, as they are in modern, purpose-built towns or hidden beneath a facade which looks much older or simpler? Do you expect to eventually knock down and rebuild older “legacy” zones, or do you want to preserve them with the minimum of change (a common requirement for our valuable historic buildings)? Do you want to accomodate the small hardware shop (read small the bespoke system) as well as the new DIY superstore (the ERP package)?

Finally, remember that it is extremely rare for a city to be truly planned and designed from scratch. You usually start with something established. Even if the city has been flattened by a bomb, you’ll have to observe land rights (this is what stopped Christopher Wren and Charles II realising their grand design after the Fire of London). This is equally true of Enterprise Architectures.

The city planning metaphor is a powerful one, but its true power may come if we use it to explore problems as well as utopian ideals.

Keep reading!

Andrew

Regular readers will know that I’ve highlighted several articles published in Microsoft’s Architecture Journal. A week ago I went to post a note on another article, to find that all my previous hyperlinks were broken. Thanks to my regular correspondents Richard Veryard and Arnon Rotem-Gal-Oz I discovered that the cause is an internal reorganisation within Microsoft, and there is a new web location for the journal, although it wouldn’t surprise me if that changes again. (To add insult to injury, the new URLs are very cryptic, and don’t paste easily into my blog!)

Now if you follow Microsoft’s advice when building systems, interfaces should be immutable. Otherwise you just don’t know what will break. The Microsoft advice is to never change an existing interface – if you need a different one, create a new interface, or at least a different version. And maintain the old one as long as dependent systems need it.

Microsoft are actually very good (not perfect, but quite good) at following this rule in their software systems. But they don’t seem to understand that the same rule should apply to that big public interface called the website. There are, of course, perfectly good strategies which would avoid this problem.

First, don’t try to reflect internal structural changes in the MSDN website. Doing so is like changing a system’s interface just because the implementation has been updated – the opposite of good practice. The public interface should be independent of implementation details.

Second, if you must create a new interface, keep the old one working. In systems, you can usually wrap the new interface to mimic the old. The same is true for a website. A set of auto-redirect pages at the old addresses, and I would never have even noticed the change.

Unfortunately Microsoft have done neither of these. And they seem to have a corporate blindness to the fact that documents are interfaces too. MS SharePoint is based on a web idiom, in which documents are identified by their position in a hierarchy. Re-arrange the hierarchy, and any external references or cross-references suddenly break.

Professional-quality document management systems don’t do this. They identify and control documents via a unique, immutable key into the underlying repository, and the primary document access via this key is guaranteed. Of course, you also want to show the document in a hierarchical structure, but any such entry is just a pointer to the underlying document. And if you want to change the hierarchy, or expose the same underlying document at multiple places in multiple hierarchies, it’s easy to do. The world of blogging has a similar concept, with “permalinks” which (should) survive a reorganisation.

Memo to Microsoft: links are interfaces too!

Three short quotes from Helland’s conclusions bear repeating directly. On heterogeneity he says:   

Remember that heterogeneity happens. Unless you have a very simple application portfolio, shared services will not be achieved by trying to put all
of your applications on one version of one platform. Even if you could, the next
merger would change that! Rather, you have to design for interoperability and
integration across platforms. This is the force that is driving the industry
wide work in service-oriented architectures.

He extends the popular “city planning” metaphor to IT investment:   

IT investment is a balance of funding the sacred, protecting historic monuments, and allocating spending between infrastructure and business opportunity. Striking this balance is a key facet in effective governance, and in realizing the potential of IT in your organization.

And finally, those who seek to maintain control of their enterprise
architecture through heavy governance would be well advised to note: 

You have to maintain a light hand. It is counterproductive to try to dictate
what happens in every structure in town, what color shirts are made, and how much is charged for soap. You have to embrace the semi-autonomous approach to governance that is characteristic of our cities, and allow the process owners to optimize and achieve efficiencies with as few constraints as
possible.

Update

Thanks to my regular corrspondents Richard Veryard and Arnon Rotem-Gal-Oz I’ve managed to track down what’s happened. The journal is now at http://www.microsoft.com/architecture/default.aspx?pid=journal

This “broken interface” breaks so many architecture rules it deserves a separate post, so I’ve written one!

However, Martin’s essay goes beyond common programming and data techniques to look at the development of specialist tools which he calls “Language Workbenches”. I’m not completely convinced that we need these in the world of XML and XSD. If you have a defined schema for you XML-based DSL (and aren’t all the many *ML langauges just different DLSs?) then any schema-sensitive editor will provide you with good design and editing support. The leading IDEs (e.g. Visual Studio) all have such a tool built into their core capabilities. Surely we now have a sufficiently sophisticated set of XML-based tools and standards that we have an opportunity to exploit synergies rather than re-inventing the wheel?

In a paper entitled “The New Methodology” Martin Fowler, the guru of object-oriented development, suggests a couple of reasons why this might be.

My article answers Martin’s, suggesting a couple of other
considerations, and whether we have to completely abandon the physical construction analogy as a result.

Almost all of the presentations shared a reminder that we still have a “software crisis” – the vast majority of software projects fail to deliver to their original targets and estimates. The presentations suggested three independent, but not exclusive, approaches to try and resolve the problem:

• Adopting better, more agile processes to address fundamental weaknesses in “waterfall” processes,
• Adopting better tools and techniques to improve development productivity and the integration of the application life-cycle,
• Enforcing a stronger “enterprise architecture” framework for development.

This last one was surprising, with several papers echoing my view that a strong architecture is essential if agile development is to succeed on a large scale or in complex and critical applications.

There was also surprising agreement on things which won’t solve the problem:

• No-one was promising a technical or product “silver bullet”. This includes web services!
• No-one was suggesting that we should just “try harder” with old-fashioned tools and processes.
• There’s no “one size fits all” solution. For example it’s a mistake to force a formal, high-ceremony process onto small business systems developments.
• Excessive technical standardisation is also not the answer. The drawbacks include “lowest common denominator” technical solutions and inflated costs where the standard solution is “overkill”.

Read my report for more details.

The paper is also available from the Identity Blog.

The paper’s thrust is that we need to develop a unifying set of identity-related technologies, but that these must observe certain key “laws”, and must accomodate varying technologies and requirements, much as unifying APIs provide access to a variety of hardware technologies.

I started thinking about the most common form of digital identify at the moment, the email address. It can be used in accordance with many of the laws. I can (usually) control when I release it. I can have different identities in different contexts, and choose which one to disclose. The identity is verifiable (to a limited extent) – someone can send me mail to check my address is valid. A variety of service providers and technologies are supported.

The big problem with email, of course, is that I can’t usually verify that email is from the claimed sender. For example, my spam whitelist admits email apparently from microsoft.com, but some of these emails are offers of dodgy mortgages and promises of increased manhood, obviusly not from the claimed source!

As a result, I wonder whether there is a missing “law of identity”. I need to be able to verify a claimed identity by methods I trust. I’d express the law something like “A party must be able to validate any identity claim, particularly its ownership, by reference (directly or indirectly) to resources he or she trusts.” This is implied in the current laws, but might be important enough to promote to a law in its own right.

Steele’s argument is highlighted by the paper’s unique style – he uses a small but growing language himself. He writes using only words of one syllable, gradually adding other words he has defined in these terms.

The paper works at many levels. As well as the fascinating intellectual exercise in style, it makes a strong case for:

• simple but extensible programming languages,
• improving the extensibility of Java, rather than its core vocabulary,
• an agile community process for developing languages, rather than up-front design of great monoliths,
• the communication power of simple language and words.

Steele exhorts us to think about extensibility mechanisms – if we get these right then the core functionality can always grow. And by example, he encourages us to use simple, direct language and benefit from its discipline. On both accounts I agree wholeheartedly.

However, reading the book has prompted me to do some more research in this field. I’m currently looking at whether combining the core valuation equation with probability decision trees might be useful. I’ll post more when I have some results.

The messages of these books are that mathematics is important to everyday life, and that it’s fun. Have you ever wondered why flowers often have five petals, how bookies’ odds work, how you always end up in the slowest queue, or, indeed, why buses come in threes? If so, then this is the book for you.

Whether as an introduction if you’ve never enjoyed maths before, or a reminder if you have, I thoroughly recommend both books.

In my recent work I’ve discovered a few patterns of my own, and I’ve started a page to document them.

First up is entitled “Change Indicator” . You may have a legacy system with an EAI adaptor, or a similar source of messages, which reports on the current state of key business objects. The messages will typically tell you when something has changed, but not necessarily what has changed, but the latter may be important to downstream systems. This pattern shows how to use EAI elements to add this information in a way which is totally transparent to the legacy system and existing users of the EAI scheme.

But Gergor’s article doesn’t really touch on understanding the dynamics of business change, and how these should dictate the form of architectural extensibility. Neither does he discuss the problem of being an architect who understands what flexibility is required, but being unable to get either the customers, or the suppliers, to agree. I seem to spend a lot of my time saying “I told you so” when a problem I predicted occurs – this is frustrating, even if I’m being proven right.

If you want to understand more about the dynamics of change, you might look at my previous postings on the subject:

Architects
- Masters of Order and Unorder

Business
Flexibility

Strategies
for Flexibility

One recent experience suggests to me that we may well be at risk of this. Read about my first experience of Web Service development and judge for yourself: have we thrown the baby out with the bath-water?

The first part of “DasBlog:
Notes from Building a Distributed .NET Collaboration System” is an accessible review of the phenomenon of blogging, and its implications for collaboration and knowledge management. Blogs cross several traditional boundaries: diaries, content management, collaboration, and news publishing. As a result both authors and readers are changing the way they get news and opinion from the web, and the new patterns are beginning to be reflected in corporate and project communications.

The second part of the article discusses some of the underlying technologies, like RSS, and how they can lead to increasingly rich linkage between the work of many authors.

Finally, the third section discusses the challenges of building a full-function open-source blogging engine using .NET technology. It’s very interesting, but gets quite technical. However you don’t have to follow, or even read, this to get benefit from the excellent first part.

Of particular interest, there’s an article in the first edition by Cap Gemini on “Enterprise Architecture Design and the Integrated Architecture Framework”. Obviously every consultancy has their own architecture framework, and they all share some
features, but what makes this one a bit different is the strong focus on the contextual (business context), conceptual (vision) and logical (solution independent capability) aspects, with technical details relegated to a bottom physical layer. It also has one of the best discussions I’ve seen on the relationship and differences between enterprise and project architectures.

For the technically inclined, I’ve built my blog using a combination of VB.NET, Active Server Pages, and a bit of XSLT. This allows me to develop my site using FrontPage and IIS, and run it under Apache. If anyone’s interested I’ll write a technical note on it.

This rather tongue in cheek piece was inspired by a “group therapy” session at the IRM/Meta group Enterprise Architecture Conference 2004.

This article, co-written by Richard Wiggins and originally published in the Rational Edge in February 2003 describes a new approach, based on UML, which meets the real requirements of modelling the Enterprise Data Architecture.

However, they rely entirely on Monte-Carlo simulation. I believe that where the curve is defined by, or can be approximated by, a few discrete points, a relatively simple analytical solution can then be used in place of simulation.

This paper (in Adobe Acrobat Format), originally published by the CBDi Forum, presents some of those strategies.