I talk to a lot of people about enterprise integration, and I find that there are several widely held beliefs which are preventing people from really understanding the benefits integration can bring their organisations. Below are some that I believe are most important and the truth behind each one.
1) All integration products are the same
There are two main types of integration tool: on one side you have data synchronisation / upload tools; and on the other is process-based integration. This isn’t just a choice between integration patterns but behaviours, in particular whether a given tool works as an A to B link, or a full data orchestration.
This isn’t to say that one type of tool is necessarily better for you, as which behaviour you need is dependent on your requirements: if you’re after an ETL or data synchronisation, then a data synchronisation tool is ideal; and if you want to integrate workflows or processes then you need a process-based integration tool.
Not all tools are the same: making sure that you fully understand what you need today and what you might need tomorrow will help you choose the correct one.
2) We use cloud-based solutions, so we need a cloud-based integration product
This is a widely-held belief but modern integration tools can connect to any data source, regardless of whether it is cloud-based or on-premise. Ultimately, this is a decision that should come down to a total cost of ownership (TCO) decision factoring in the performance of a hosted solution, as well as the control and elasticity offered by the system regardless of where it lives.
More importantly than whether a tool is cloud-based, consider how scalable and fault-tolerant the system would make your IT systems: technologies like in-memory data grids (IMDGs) are able to guarantee message delivery and recover from errors with no data loss, as well as massively increasing the speed of batch jobs. You need to decide for yourself what importance you would give this resiliency and peace of mind in your TCO considerations, but the bottom line is to focus on functionality, not location.
3) I don’t need a third party integration tool because my system vendor provided one
Many system vendors provide integration tools and these vary in terms of how they work and how well. While it is entirely possible that this tool will be adequate for your needs, there is always the possibility that a vendor tool will be optimised toward their technology stack only.
Third-party tools are also optimised to deal with vendor’s technology stacks (especially those which have vendor-certified connectors and capabilities) but they are also optimised to integrate between stacks, which is naturally less of a concern for vendors. So if you are thinking of integrating technologies from multiple vendors or want to keep your options open for the future, it’s worth looking into vendor-agnostic tools as these will provide best-of-breed capabilities out of the box.
4) Everything is going mobile, and that allows my users to work without integration
Integration isn’t just about linking to a data source, but creating a workflow that can move seamlessly throughout your IT environment, with full governance and a single gateway to secure. In fact, the move to mobile is making integration more important, not less, because on mobile users have a shorter attention span and are less willing to work with a complex environment of multiple screens and systems.
A well-integrated workflow can take the user through, for example, the entire sales process from lead generation to contract signature in a series of clear, simple screens that minimise data entry.
5) Many integration projects fail, it’s not worth the risk
It’s true that many integration projects fail, but this is most often a management challenge rather than a technical issue. I’m not saying that it’s bad management; on the contrary the problem is that an integration project represents major change, and that always leads to push-back. The good news is that it’s possible to dramatically increase the chances of your project succeeding and get value out even before all the steps have been completed. Here’s how:
Firstly, top management buy-in is vital, to provide a strategic view and ensure the right resources are dedicated to the project: even though many projects are tactical, they affect so many areas that the change should be derived from an overall strategy,
Having the power to derail the project, IT must be seen as more than just executors: management needs their input and they must be brought into the project from the beginning, and given the big picture view as a significant partner,
The use of champions inside affected business units helps to sell the project to users, being able to articulate the benefits to their colleagues and also (through management sponsorship) acting as a channel back to management for concerns,
Avoid blame culture by fostering a strategic view and attitudes of openness, cooperation and problem-solving across all management and departments on the project,
Focus on processes to understand what users are actually trying to achieve when accessing data,
Adopt a philosophy of continuous change, where the overall strategy is broken down into small steps, each of which will bring demonstrable value,
Bring on board the vendors and their subcontractors, to resolve any potential issues before they become roadblocks.
6) Integration projects are never-ending
Integration projects are typically wide-ranging and complex, following a grand strategic vision of how the company should look; this view means that the reality often looks incomplete and with endless work ahead. Adopting a philosophy of continuous change helps here, breaking the vision down into projects that can be implemented independently of each other.
This allows you to carry out, develop, test and make live the first component, then wait and see what happens. However, this does mean that you need to choose an integration tool which allows this kind of flexible, process-driven integration where future changes are not just a possibility but the norm. You need a tool which doesn’t require all your connections to be rewritten every time a system is changed.
Continuous improvement is one way to deal with the tendency of projects to run on; another is to keep integration projects small and closely controlled, especially if the objective is simply to synchronise between a data source and a warehouse for example. However in this case you need to be aware of the dangers of increasing complexity when choosing your tool: many integration tools are able to connect systems in a few hours, but sometimes every change requires existing work to be modified leading to “spaghetti code” and complex, risky projects. Always remember that integration projects can become lengthy even if you try to avoid it, and factor this into your TCO calculations.
7) Monitoring and performance management isn’t needed
Along with fault-tolerance, resilience and elasticity, monitoring and performance management capabilities are the key additions to integration solutions. The increased requirements for guaranteed message delivery means monitoring is vital, as if a system fails during transmission the integration tool needs to recognise when it can resend the message. Further, monitoring capabilities allow systems to automatically cache transmissions that cannot be sent, and grant extra resources to deal with sudden peaks in demand.
8) Security isn’t necessary in integration
On the contrary, when moving from a siloed system to a workflow-based environment it is critical to secure the processes, especially on mobile. Essentially, what you are doing is taking data sources in the enterprise back-end which only a handful of people used to have access to, and exposing that data to other systems which are available via a smartphone. This requires a shift in thinking about security, from securing individual systems to securing a process.
On mobile alone, there are a range of ways to achieve this, from securing the device with passwords, MDM solutions, and geofences, to securing access to the applications, often through containers, and finally securing the data itself, with encryption and ensuring it is not stored on the device.
9) Our C-suite already gets all the data they need
Top management is used to basing their decisions on a certain type, age and granularity of data, but this doesn’t mean that that’s what they should continue to have available in the future, especially as real-time data analytics become more widespread among the competition. Equally, simply having a Big Data analytics tool connected to a data warehouse isn’t that valuable; the real value comes from integrating day-to-day processes into the analytics tool and running real-time analytics on this.
10) Integration is too expensive
Integration tools can be expensive, but the process shouldn’t be considered as a cost, because the effect it has on the business is more complex and there is an opportunity cost associated with not being able to recover the benefits. For example, the ability to bring real-time data and Big Data into day-to-day business processes means management are able to take decisions based on the real situation, rather than old and abstracted data, especially when using a process-based integration platform.
I worked with a large shipping insurer which used integration to track the location and integrity of its containers; and by using real-time data was able to dramatically reduce its rate of theft and losses which led directly to a significant reduction in its insurance costs. Integration can have a “knock-on” effect in the business, and the true value may not lie where you expect; so rather than not being able to afford to integrate, can you afford not to?
By David Akka, Managing Director at Magic Software Enterprises UK.
