Evaluation Criteria

View the 5 evaluation criteria.

Enterprises should consider both current and likely future requirements

Data observability products have a tall order to fill. They must scale to support high volumes of varied data across heterogeneous environments while still satisfying real-time business demands. They must alert stakeholders to critical issues by extracting clear signals from lots of confusing noise. Data observability tools also must help enterprises demonstrate compliance with internal policies and external regulations, in particular those related to the handling of personally identifiable information (PII.)

 To overcome these challenges, Eckerson Group recommends that enterprises evaluate data observability tools according to five criteria: breadth of functionality, performance and scale, ease of use, open architecture, and governance. In each case enterprises should consider both current and likely future requirements.

Close

The 5 criteria are:

1

Breadth of functionality.

Data observability tools should include functions such as ML-driven anomaly detection, monitoring, and alerting, as well as the ability to enforce both custom and default rules.

2

Performance and scale.

Data quality observability tools must monitor data without impeding data consumption. Data pipeline tools, meanwhile, must help data teams meet SLAs for latency, throughout, and concurrency while closely managing cost.

3

Ease of use.

Data observability tools should make data engineers, data analysts, data scientists, and other key stakeholders more productive as they manage and consume data. A graphical interface, default configurations, user prompts, and fine-grained controls for alerting all help ensure this ease of use.

4

Open architecture.

Data observability lives in a dynamic ecosystem, so must exchange data and interact with many related elements. Evaluate data observability tools by the level of effort required to integrate them with data sources, targets, pipeline tools, data catalogs, data fabrics, and governance platforms.

5

Governance

Enterprises must demonstrate compliance with internal policies and external regulations, especially those related to the handling of PII. Data teams should evaluate how data observability tools ensure data access and quality, and assist auditors with documentation.