This is the age of hyper-connectivity. With all of our smartphones, laptops, and tablets we are instantly available anytime, anywhere. In 1624 when John Donne wrote, “No man is an island, entire of itself”, he had no idea how far technology would take us.
Now not only are people hyper-connected, the same is true for devices. All the lingo about digital transformation, Internet of Things (IoT), Industry 4.0 and big data, the cloud can be summed up in just one word – “connectivity”. With intelligence being collected and then shared on individual devices, and then merging together in the cloud, there is an enormous opportunity to gain huge insights on machine and human behaviour that can be used to tweak operations to improve productivity and the customer experience. All of this connectivity should make us wiser, but it doesn’t always work that way.
For many companies, it’s basically impossible to get a comprehensive view of their data assets since information is scattered across the organisation in different ERP, CRM, e-commerce, and customised solutions for logistics and manufacturing. In many cases this fragmentation is also the result of what’s called “shadow IT,” a practice where individual employees, or even whole departments or remote locations (ROBOs) take the responsibility of meeting their IT needs into their own hands. Gartner estimated that shadow IT management would account for more than a third of total IT expenditures in 2016.
So here is the challenge – how do you create a bridge between all these systems to enable the free exchange of data to enable companies to benefit from all the insights it brings?
Many organisations have in-house staff who are experienced and readily available, so the temptation is to tackle the integration challenge in-house and to implement point-to-point integrations. When infrastructures include only a few components, for example integrating an enterprise resource planning (ERP) system with a manufacturing execution system (MES), costs are low at the project level. Since, historically, this is how IT has evaluated success, the majority of IT solutions are still based on the P2P approach.
However, the end result can be a complicated web of connections that makes adapting to every new requirement very cumbersome, slowing down development, and requiring IT departments to focus resources on integration instead of what really counts—creating applications for their business users.
The P2P integration, termed by Gartner “pervasive integration”, also results in application developers, line of business (LOB) IT services and business users having to be increasingly involved with the process of integrating data.
Integration platforms can provide a better alternative because they are optimised to deal with different vendors’ technology stacks (especially stacks that include certified connectors and functionality) and to integrate between stacks. They can also include various capabilities, such as resilience, fault-tolerance, monitoring and performance management. Monitoring is essential to guarantee message delivery. If during transmission a system fails, the integration tool needs to be able to recognise when it can resend the message. In addition, monitoring capabilities provide systems with the ability to automatically cache transmissions that cannot be sent and extra resources to handle sudden peaks in demand.
In addition, low-code middleware solutions enable employees who aren’t trained programmers to get involved with the integration process, which is even more essential as more and more systems from different departments and disciplines are responsible for collecting, consuming, and analysing data.
Connectivity is essential to reap the benefits of big data, IoT, and digital transformation. With all the different data siloes and specialised systems, having a plan for long term data mediation can keep a business on track with becoming more and more data driven. Without the flexibility of middleware, the insights from connectivity can stay beyond a business’s reach.
