Data Trends for 2023

Advanced analytics and AI/ML continue to be hot data trends in 2023. According to a recent IDC study, “executives openly articulate the need for their organizations to be more data-driven, to be ‘data companies,’ and to increase their enterprise intelligence.”

At the same time, IDC’s analysts note that a wide gap remains between expectations and reality. Nearly two-thirds of data practitioners believe they are expected to make data-driven decisions, yet only 30% believe that their actions are genuinely supported by data analysis. In its most recent Data Trust Survey, IDC found that just over a quarter (27%) of data practitioners fully trust the data they are working with on a routine basis.

As the drive toward data-driven business decisions continues, most executives are keenly aware of this trust gap. Consequently, they are investing heavily in data integrity in an effort to increase trust, accelerate data-driven transformation, and produce better business results.

Here are the top data trends our experts see for 2023 and beyond.

1. DataOps Delivers Continuous Improvement and Value

In IDC’s spotlight report, Improving Data Integrity and Trust through Transparency and Enrichment, Research Director Stewart Bond highlights the advent of DataOps as a distinct discipline. DataOps is a combination of technologies and methodologies for ensuring the quality and consistency of data to deliver continuous improvement and business value.

Like the similarly named discipline of DevOps, DataOps applies lean and agile principles such as continuous development and testing to improve communication, integration, and automation of data flows – and ultimately ensure more predictable data delivery. According to the IDC report, “organizations that have implemented DataOps have seen a 40% reduction in the number of data and application exceptions or errors and a 49% improvement in the ability to deliver data projects on time.”

2. Cloud Adoption Will Continue Steadily

Cloud computing and its inherent scalability and elasticity offer distinct advantages, especially with respect to AI/ML and advanced analytics. As cloud data platforms and powerful analytics tools gain in popularity, the march toward the cloud continues at a rapid pace.

According to IDC’s analysts, cloud-based data platforms contribute to increased data democratization, offering broader distribution of data, improved data diversity, and a more dynamic environment. But to achieve these benefits, organizations must invest in data integrity to set the stage for success. The ability to integrate data into the cloud, cleanse and validate it, enrich it with third-party data and location-insights, and ensure it is cataloged and governed ensure that the data in your cloud is ready to support the business. As privacy and security regulations and data sovereignty restrictions gain momentum, and as data democratization expands, data integrity becomes a must-have initiative for companies of all sizes.

3. AI and Machine Learning Come of Age

Artificial intelligence and machine learning (AI/ML) find practical applications across all industries. From insurance claims management to supply chain optimization and fraud detection, AI:

• Discovers correlations
• Assesses potential outcomes
• Automates routine decisions

Despite the advances such technologies make possible, data practitioners are keenly aware that the problem of poor data integrity may be magnified by large-scale automation. AI/ML algorithms can only deliver reliable insights to the extent that they are trained using high volumes of accurate, consistent, and contextualized data. Systems trained on poor data may make bad recommendations or bad decisions, with potentially disastrous results.

AI/ML are not only the downstream beneficiaries of data integrity programs. They are also instrumental in managing data integrity at scale. Precisely leverages AI to automate the discovery of data issues in real time, recommend data quality rules, and suggest data enrichment opportunities. By automating the collection of intelligence about data, inferring relationships among various data entities, and detecting anomalies, AI automates many of the key elements of data integrity – including data observability, data quality, and data enrichment.

4. Data Observability Fixes Problems Earlier

Data observability involves the continuous monitoring and testing of data pipelines to alert decision-makers in the organization to anomalies that could indicate potential problems or highlight unexpected changes that indicate opportunities or threats.

Anomalous data can occur for a variety of different reasons. Significantly higher or lower than normal data volumes could indicate a breakdown in data collection systems or unexpected behavior that calls for a response. If values for a particular data field are unexpectedly falling outside of the normal range, it might suggest any number of problems, depending on the nature of the data set. In any case, data observability provides early notice to data practitioners, prompting rapid root cause analysis and resolution.

Just as early detection of software bugs leads to faster, easier, and less costly fixes, early detection of anomalous data offers an opportunity for early intervention that usually leads to fixing a problem earlier and at less expense than if it is detected when a business application is impacted.

For organizations to benefit from data observability, investing in technologies and processes that support good data quality and governance, as well as accurate data cataloging and profiling, is imperative.

5. ESG is on the Ascent

Environmental, social, and governance (ESG) standards are garnering increased attention around the world, with a stated goal of increasing transparency into corporate activities. For many enterprises, ESG reporting presents unique, new challenges with respect to corporate governance and sustainability standards.

According to a November 2021 IDC survey on governance, risk, and compliance (GRC), a majority of enterprises are already using their data reliability and accreditation capabilities to monitor and report on ESG compliance today. Most respondents in that study stated that improved monitoring of ESG-related metrics is a top priority for the coming three years.

Some leading-edge adopters of ESG have been challenged as to the accuracy of their data. To avoid accusations of “greenwashing,” it’s essential to develop reporting mechanisms that deliver trustworthy results. Precisely helps organizations to meet their ESG reporting requirements by ensuring the integrity of the data they are using for reporting.

Data trends for 2023 point to the need for enterprises to manage data integrity at scale. To learn more about these and other data trends, download your free copy of the IDC spotlight report, Improving Data Integrity and Trust through Transparency and Enrichment.