Evaluation Criteria

Eckerson Group uses a worksheet to evaluate data catalogs that consists of 180 criteria in 45 categories. As a shortcut to product evaluations, data teams can evaluate five items: 

  • Company heritage. What was the problem or pain point the vendor initially tried to solve when it introduced its product? That represents the strength and focus of the vendor, no matter how much they have since enhanced the product
  • Target customers. Similarly, a vendor targets customers that exhibit the pain points its product is best designed to address. Vendors whose product have a business-oriented graphical user interace target customers whose business users find data catalog tools too complex to use.
  • Architecture. Architectural issues can eliminate products from the start. Does it only run in a cloud SaaS environment, but you need it to run on premises? Does it only support cloud applications but you need it connect to legacy systems? Understand your target environment before spending time evaluating tools. 
  • Pricing. A vendor’s pricing determines whether the product is geared to enteprrise or mid-market companies. It’s not just the cost of the product, but the complexity of the pricing. Vendors targeting mid-market firms or departments of larger firms will provide simplified pricing that businesspeople without technical knowledge can understand.
  • Customer References. As with any product, it’s valuable to speak directly with customers using the product under evaluation. It’s even better to speak with consultants or systems integrators who may have used multiple tools in production and understand the strengths/weaknesses of each.

“Data governance platforms are typically anchored by a data catalog….”

Figure 9. Data Catalog Evaluation Worksheet


Evaluation Criteria