Data Quality in the Age of AI: Addressing the Challenge of "Dirty Data"

One of the most common questions I ask clients about their AI implementations is: How clean does your dirty data need to be to actually invest in AI? This question strikes at the heart of a critical consideration for any AI project.

Understanding Data Quality Dimensions

Data quality is a complex domain with multiple dimensions that affect AI performance. When evaluating your data's readiness for AI applications, several key factors come into play:

Completeness vs. Accuracy

There's an important distinction between these two concepts:

• Completeness: Does your dataset contain all the necessary elements? For example, if your complete dataset should include numbers 1-6, but you only have 1-3, that's an incomplete dataset.
• Accuracy: Are the values in your dataset correct? If your dataset contains values 1, 2, 3, 9, and 10 when it should only contain 1-6, the data isn't accurate.

These distinctions matter significantly when training AI models, as both incomplete and inaccurate data can lead to biased or incorrect outputs.

Data Collection Considerations

The quality of your data is also affected by:

• Age of the dataset: How current is the information?
• Capture method: Was the data captured at the source or further downstream where it might have been modified?
• Capture frequency: Was data collected once a month, every week, or in real-time?
• Data point timing: Was the data captured at the right time to be relevant and reliable?

The Five V's of Data

You can consider the quality of data and analyze it from different dimensions. When collecting the data and assessing its quality for AI, I often refer to what I call the "five V's":

• Veracity: How truthful is the data when it was captured for the intended purpose?
• Variety: What different types and formats of data are included?
• Velocity: How quickly is the data being captured and processed?
• Volume: How much data is available?
• Value: What business value can be derived from the data?

If you need more granularity to assess the quality of the data over time, you can include three more “V’s”:

• Validity: How correct is the data at the time it was captured and since that time?
• Visualization: How will different user audiences view and consume the data over time?
• Volatility: How quickly does the data lose business value and become obsolete over time?

For your AI initiative, you can consider all or some of the data quality dimensions — the “V’s” — depending on how data quality impacts your business processes, users, objectives, and outcomes. For example, what is the quality of your customer data? Does poor data quality limit cross-selling and up-selling opportunities, while also reducing the accuracy of marketing data used for trend analysis? If your answers raise concerns, it may be time to reevaluate your data quality before proceeding with your AI implementation.

The Impact of Poor Data Quality

The consequences of using poor quality data in AI systems can be significant. As mentioned, it can include biased outputs, inaccurate analysis / predictions, legal vulnerabilities, wasted investment, etc.

Strategies for Addressing Data Quality

When preparing data for an AI implementation, consider these approaches:

1. Understand your data gaps: Know where your data is incomplete or potentially inaccurate
2. Identify areas of high-quality data: Focus your AI implementation where data quality is strongest
3. Consider synthetic data: In some cases, synthetic data can help fill gaps, but be cautious — if your model for creating synthetic data is flawed, you're just compounding the risk associated with poor data quality.
4. Implement data quality monitoring: Continuously assess and improve data quality as part of your AI operations

The Data Quality Threshold

Determining exactly how clean your data needs to be isn't a one-size-fits-all proposition. The threshold varies based on your specific application. For example:

• What decisions will be made based on the AI outputs?
• What level of risk is acceptable for those decisions?
• What regulatory requirements apply to your use case?

For instance, an AI solution used in mortgage application decisions would probably require stricter data quality. In contrast, a solution that monitors inventory levels and recommends a supplier may operate effectively with lower data quality.

Data quality will always be critical for AI success. If you can't trust your data due to quality issues, you can't trust the outputs, which may lead to incorrect decisions and potential legal issues.

Concluding Thoughts: Data Quality for AI Success

Before diving into your AI implementation, take the time to assess your data quality across these various dimensions. Understanding where your data “strengths and weaknesses” are, will help you develop a more effective, reliable AI strategy — ultimately delivering better business outcomes.

This blog post is based on an original AIIM OnAir podcast. When recording podcasts, AIIM uses AI-enabled transcription in Zoom. We then use that transcription as part of a prompt with Claude Pro, Anthropic’s AI assistant. AIIM staff (aka humans) then edit the output from Claude for accuracy, completeness, and tone. In this way, we use AI to increase the accessibility of our podcast and extend the value of great content.