Thoughts on the World

Enterprise Integration Patterns

Integration, like other design activities, can benefit from sharing ideas and proven strategies in the form of patterns. I highly recommend Gregor Hohpe and Bobby Woolf's Enterprise Integration Patterns website and book.

In my recent work I've discovered a few potential patterns of my own, which are documented here. The pictures use the pattern language developed in the Hohpe/Woolf book. If you want to be informed when I add new material, please subscribe to my RSS feed.

Change Indicator

Suppose you have a legacy system with an EAI adaptor, or a similar source of messages, which reports on the current state of key business objects. The messages will typically tell you when something has changed, but not necessarily what has changed. It's common for such a message source just to report on the latest state of the objects. However, many changes may not be relevant to downstream systems - to properly filter the message stream they need to understand the nature of the change.

How can we understand the nature of each change, when the source system can't provide this information?

If the source system can't provide the information, then can we derive it? Yes. If we've seen a previous message describing the state of the same object, we can compare the state now with the state reported in that message.

Therefore the obvious solution is to add this change information using the EAI system itself. The solution will need a way of storing and retrieving messages, a way of comparing them with one another, and a way of adding the "change" information to the message. Fortunately these are all things we know how to do.

Use a combination of a message store, a message comparator and a content enricher to track changes between the latest message and the previous one, and add this change information to the outgoing message before it is published to other systems.

The message retrieval step needs to identify the contents of the message and retrieve the last  message for that business object from the message store. For example, if it is a "customer's details changed" message, this will need to extract the customer ID from the message, and find the previous message indexed by that customer ID. Ideally the required indexing information will be held in the message title or envelope, but otherwise this function will have to unpack and parse some of the message contents.

The form of message storage will depend on the infrastructure set up for the messaging system, and the number of business objects which must be tracked. If that number is small, it might be enough to store the messages in files named using the object ID. If larger, then a proper database may be appropriate. In either case we only need to store the latest message for each business object, so the required storage does not grow indefinitely.

The "changed items" information is added to the message by a "content enrichment" step. Assuming that the message is formatted in XML, this can take the form of a new element attached to the message header, which should be entirely transparent to message consumers not looking for it. If the messaging system uses a Common Message Model then the changed items node can be added to the common part of the structure as per the following model:

The change information can be as simple as a list of changed elements, or can be enhanced to include before and after values, and to identify moves, additions and deletions separately from simple value changes. If certain changes have specific business significance, they can be detected and highlighted at this step.

The message comparison step may be a bit of a challenge. Depending on the message format it may be as simple as a line by line comparison, or it may be more complex. For example, if two XML messages have the same content in the same element, but the position of the element has changed then this may not be significant. Fortunately the common platforms provide tools which make this task easier. For example in the Microsoft environment you can use the XMLDiff library which supports intelligent, fast, XML comparison where the programmer can easily choose which differences are significant and which should be ignored.

The message store can either be populated progressively by each change as it happens. This will result in one "everything's changed" message for each business object. Alternatively it can be primed by running some process in the legacy system to "touch" each appropriate business object in turn.

This design has several strengths:

  • Transparency - the design has no impact on the legacy system and its adapter, or any existing message consumers,
  • It can detect any required granularity of change down to individual elements and attributes,
  • Incorporation of the changed items information into the message makes it easy to add appropriate filtering code into recipient message processors,
  • It's easily extended to as many business objects as you want, without significant change to the core code, reducing the ongoing maintenance cost,
  • It doesn't require you to know in advance what changes may be significant.

Comments

If you'd like to comment on this article, with ideas, examples, or just to praise it to the skies then I'd love to hear from you.

Comment on this article

Please share: All Addthis servicesTweet thisFacebook thisLink thisYam thisShare on Google

© Questa Computing Ltd. 2005
Page last updated 14 May, 2015 12:39

Questa Computing Ltd. is registered in England and Wales number:2889117.
Registered office: Ember House, 35-37 Creek Road, East Molesey, KT8 9BE UK
For credits, copyright details and cookie policy, see here.

Search my Sites:

Share this page:

Share |

Indexes

Favourite/Major Articles
All Articles by Category
All Articles by Title
Review Index

Site Map:

Welcome / Home Page
Contact Me
Search the Site
Thoughts on the World
Consultancy Services
An introduction
Key Skills
Career Summary
Case Studies
IT Knowledge/Education
Customer Profiles
Training Courses
Previous Experience
Download Full CV
Customers
National Grid Group Plc
British Energy Power and Energy Trading
Legal Marketing Services Ltd.
Addison-Wesley
Marks and Spencer Plc
Faith Footwear Ltd.
Barclays Sales Financing Ltd.
Oracle Corporation UK Ltd.
Livingston Rental Group Ltd.
National Power Plc.
Blog
Agile Development
Coding Projects & Thoughts
General Thoughts
Humour
Personal
Photography
Products and Projects
Publications
Reviews: Books, Music & Films
Website Announcements
Full Index
Publications and Papers
A Hacker's Guide to Project Management
Conference Papers
Agile Architecture
Practical Enterprise Integration
Modelling an Enterprise Data Architecture
Strategies for Flexibility
Getting Sizing Right
Muzzling the Alligators
Evolution of a Test Method
Photography
Photography Articles & Discussion
Photo Gallery
Projects and Products
RelQuest - Reliability Modelling Tool
ConQuest - Container Yard Management System
Reviews
Full Index
Architecture
Biography and Personal Endeavour
Construction & Civil Engineering
Cryptography
Fiction
Human-Computer Interaction
Mathematics & Statistics
Military & 20th Century History
Modelling & Analysis
Photography & Photographers
Physics & Cosmology
Project & People Management
Psychology & Human Behaviour
Science, General
Links

Contact Me

Email me
@TweetAndrewJ
Others

Feeds and Tweets

Follow me on Twitter

@TweetAndrewJ

Thoughts on the World
(main feed)

Feedburner XML
RSS Version XML (direct)

How many subscribers?

Other Feeds

Professional Blog
Photo Blog
Photo Album
Review Pages

About my feeds

Google Blog Search

References to www.andrewj.com
References to www.agilearchitect.org

Sister Sites:

www.agilearchitect.org