4 Real-Life Examples of Data Integration

Data integration has come a long way in the past few decades. In the past, integration was based on ETL processes that used a few skilled developers to fulfill data integration requirements. Fast forward to today and things look quite different. A new dynamic and fluid model of data integration has taken its place – bringing data from across a business to users when and how they need it. Much of the changed approach is driven by a broader diversity of cloud data consumption models, as well as a surge in the number and types of applications demanding real-time data delivery.

For organizations running legacy systems such as mainframe and IBM i, the challenge becomes a practical path forward that brings these systems into the modern data flow. Additionally, one must find the right data integration tools that can bridge the gap between the investments of the past and the cloud platforms of the future.

The data integration imperative

Most IT managers working within data integration must help the business do two things. First, provide data analysts, lines of business and others with access to the best enterprise data in which to make decisions on. Second, there is the need to accelerate the adoption of powerful modern technologies while still maintaining existing applications and storage.

A lot more data integration takes place simply due to the fact a customer’s data can come in from multiple places and sources. For example, when a customer places an order online, the order information needs to flow from the e-commerce system into ERP for order fulfillment, cash journal entry, and perhaps other purposes. It may also need to flow to the CRM system so that the customer’s record can be updated. Additionally, this customer may also be interacting with the same company via social media or customer service.

Streaming data for analytics has grown significantly. This growth is triggered by data scientists and analysts seeking to aggregate ever-larger volumes of information and leveraging vast increases in processing power to draw valuable insights. Given the robust capabilities of data analytics platforms and the scalability of cloud-based systems, leading-edge companies are turning more and more to streaming data.

A few Precisely customer examples

With that background in mind, let’s consider a few real-life scenarios in which data integration plays a critical role:

Insurance

For a large insurer running its core applications on IBM/z, native mainframe data formats presented a particular challenge. The company wanted to move IBM/z mainframe data in real-time into Cloudera for use in an enterprise datahub. Because Cloudera offers no native connectivity or processing capabilities for EBCDIC formatted mainframe files, the company’s existing processes for moving that data was very labor-intensive, and required personnel whose skillsets included both IBM/z and distributed cloud technologies.

Working with Precisely Connect, the customer was able to capture and load IBM/z data into Kafka in real-time, and to batch that IBM/z data in Kafka for delivery to Cloudera.

Anti-Money Laundering (AML)

A global bank needed to meet AML transaction monitoring and Financial Conduct Authority (FCA) compliance. When looking to act on these requirements, the bank found they had two critical problems. First, it had disparate systems in their organization, including mainframe, RDBMS, cloud, and more. Second, it lacked the internal resources required to integrate these disparate systems into their data lake, which would be a critical source of data for its AML process.

Using Precisely Connect, the bank could integrate the disparate data sources without the need for specialized skills into the data lake. Additionally, Precisely Connect enabled the bank to have end-to-end data lineage as data moved from source to target.

Global Shipping

When a global maritime services firm needs to provide real-time shipment tracking and information to its multitude of partners, ports, and local carriers worldwide, integration from multiple legacy platforms into a cloud-based data hub for direct access by various operations and analytics systems can be challenging.

Using Precisely Connect CDC, data can be streamed from Oracle SQL and IBM mainframe systems to Kafka.  From there, data can be batched and delivered to distributed HDFS/Hive database clusters by Precisely Connect ETL.

Retail

A luxury retailer who sells their goods in stores and through direct-mail and corporate merchandising had an inventory data problem.  With transaction values that exceeded six-figures, the retailer could not afford to mistakenly sell an out of stock item to a customer online. The company needed to know all transaction information in real-time.

Struggling to accomplish this with existing infrastructure, the retailer evaluated Precisely Connect. Today the company uses Precisely Connect to replicate Microsoft SQL Server data globally in support of their business. As they expand to cloud deployments, the retailer is now looking to use Precisely Connect usage to provide real-time replication to Microsoft Azure.

Making legacy system data integration a reality

Integrating legacy systems like mainframe and IBM i into cloud platforms is challenging because of a lack of skills, complexity of data and time. For example, many of the anomalies of mainframe data formats, such as COBOL copybooks and variable-length records, must be accounted for. Capturing changes to data on legacy systems on the fly and streaming updates to external systems securely and reliably requires enterprise-grade tools built with these challenges in mind.

Precisely Connect is designed to help businesses stream real-time data from legacy systems into next-gen cloud and data platforms with one solution. Precisely Connect keeps your data synchronized in real-time without overloading networks, affecting the source database performance, or the need for specialized skills.

IT executives are constantly challenged by the need to accelerate the adoption of the most modern and powerful technologies while still having to rely upon their existing installed base of storage and compute infrastructure. The only practical path forward is to enable backward compatibility.

For more information, get our checklist of the 10 key features when hiring a data integration vendor. Click to read The Data Integration Top 10.