Integration Message Comparison for Development for National Grid Plc.
Identify the nature of changes to key assets from messages providing just the latest information for each asset
National Grid developed an enterprise integration scheme in which changes to details of the network components in the enterprise asset management system (Ellipse/MIMS) were published, using a standard common message format, to a variety of downstream systems.
One early problem was that significant changes to asset details could be “swamped” by changes not relevant to downstream systems, such as a global change to accounts system codes. This caused performance and reliability issues for key downstream consumers such as the field force system.
The suppliers of Ellipse could not offer any way to filter relevant messages at source, or even to mark the messages indicating the nature of the change to the asset records. Essentially the message just said “this asset has changed” and dumped the current data.
I came up with a solution to hold the last message related to any given asset in an intermediate message store. When a new message indicated a change to that asset, the two messages would be compared, and the new message enriched with an additional node listing the changes. The next stage in the integration flow could then read this information, and use it to appropriately route or suppress further flow of that message.
Key to the development was creation of a node-by-node comparison tool for XML: straight text comparison of XML is meaningless as node order or whitespace can change significantly for the same content. I developed the initial version of this code, which was then adopted and productionised by the suppliers of the integration solution.
Problems and Challenges
No commercial library or published solution for the XML comparison problem. I had to initially develop a simple .NET version, and then convert to Java
Working solution adopted into the production system
Maintained as a key component of the NG integration architecture for many years
Tools and Technologies
Technologies: Java, JAXB
I developed a component for National Grid’s EAI architecture which compares and enriches related XML messages as they flow through the system, enabling better downstream routing depending on what has changed.