{"id":16,"date":"2022-06-09T12:49:33","date_gmt":"2022-06-09T12:49:31","guid":{"rendered":"https:\/\/eck.uidev.site\/?page_id=16"},"modified":"2023-06-01T17:20:05","modified_gmt":"2023-06-01T17:20:05","slug":"maturity-2","status":"publish","type":"page","link":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/maturity-2\/","title":{"rendered":"Success Factors"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\"><\/h2>\n\n\n<div class=\"entry-content\">\n\t<div style=\"\" class=\"simplesub wow zoomIn bgdblue b100w notoppover nofootpoverpad ceven withtitle singleno\">\n\t\t\t\t<h1>Success Factors<\/h1>\n\t\t\n\t\t<div class=\"textblockwrap\">\n\n\t\t\t\t\t\t<div class=\"txt\">\n\t\t\t<p><span style=\"font-weight: 400\">The success of technology always depends on more than just cool tools. To understand which factors drive the success of data pipelines, consider the phases of pipeline design, operation, and adaptation.<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><b>Design<\/b><span style=\"font-weight: 400\">. To design effective data pipelines, data engineers and architects must first align with their company\u2019s business priorities by defining and stack ranking use cases according to their expected business value. As they design pipelines to support those use cases, they must scope likely future requirements and how to meet them. They also must embrace graphical tools that support a no code\/low code approach, increasing productivity.<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Operate<\/b><span style=\"font-weight: 400\">. The more data and analytics engineers can automate the design, execution, and monitoring of pipeline processes, the more they improve efficiency and reduce risk. Engineers can further boost productivity by creating standard pipelines artifacts that they can assemble and reuse like Legos.<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Adapt<\/b><span style=\"font-weight: 400\">. Data teams must monitor a consistent set of key performance indicators, tracking both technical operations and business health. Data observability tools can help them track KPIs as they optimize pipeline performance and data quality. When it comes time to adjust pipelines, data teams should make modular changes so as to minimize risk.<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400\">Business and data teams must govern all three of these phases by defining the roles of various stakeholders and creating and enforcing rules that control their processes. Data pipelines, and the processes managing them, must integrate with governance-related tools such as catalogs, master data management platforms, and data access management tools.<\/span><\/p>\n\t\t\t<\/div>\n\t\t\t\t\t\t<div class=\"sub\">\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<div class=\"quote\">\n\t\t\t\t\t<p>To design effective data pipelines, data engineers and architects must first align with their company\u2019s business priorities by defining stack ranking use cases according to their expected business value <\/p>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t\t\t\n\n\t\t\t\t\n\n\n\n\n\t\t\t\t\t\t\t\t<div class=\"vidimg\">\n\t\t\t\t\t<div class=\"img\">\n\t\t\t\t\t\t\t\t\t\t\t<div class=\"iframelink\">youtu.be\/qA1QLKHy0eU<\/div>\n\t\t\t\t\t\t<iframe class=\"videoem\" src=\"https:\/\/www.youtube.com\/embed\/qA1QLKHy0eU?rel=0\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen=\"\"><\/iframe>\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<div class=\"txtlnk\">\n\t\t\t\t\t\t<p>Learn about the phases of pipeline design, operation, and adaptation.<\/p>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\t\n\t<\/div>\n<\/div>\t\n\n<div id=\"tile\" class=\"entry-content\">\n<\/div>\n\n<div id=\"figure\" class=\"entry-content\">\n<div style=\" z-index: 2\" class=\"figure wow fadeIn bggrey b60w topover footover\" data-wow-delay=\"0.4s\">\n\t\t<h2>Data Pipeline Management: Success Factors<\/h2>\n\t\t\n\t\t\t\n\n\t\n\t    <img decoding=\"async\" src=\"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-content\/uploads\/sites\/18\/2023\/06\/Screenshot-2023-06-01-at-1.18.35-PM.png\" alt=\"\" \/>\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\n\t<div class=\"ref\"><h3>Successful data pipelines depend on effective design, operation, and adaptation.<\/h3>\n<\/div>\n\t\n\t<a class=\"close\" href=\"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-content\/uploads\/sites\/18\/2023\/06\/Screenshot-2023-06-01-at-1.18.35-PM.png\">View Large<\/a>\n<\/div>\n<\/div>\n\n<div class=\"entry-content\">\n\t<div style=\"\" class=\"wow fadeInDown resources bgblue notoppover colsingle b60w\">\n\t<h2>Additional Resources<\/h2>\n\t<ul>\n\t\t\t\t<li><a class=\"icon sml iwebsite\" target=\"_blank\" href=\"https:\/\/www.eckerson.com\/register?content=flexible-data-pipelines-balancing-standard-and-custom-approaches\">Flexible Data Pipelines Balancing Standard and Custom Approaches<\/a><\/li>\n\t\t\t\t<li><a class=\"icon sml iwebsite\" target=\"_blank\" href=\"https:\/\/www.eckerson.com\/articles\/how-to-design-streaming-data-pipelines-for-open-distributed-and-elastic-cloud-platforms\">How to Design Streaming Data Pipelines for Open Distributed and Elastic Cloud Platforms<\/a><\/li>\n\t\t\t<\/ul>\n<\/div>\n\t<\/div>\n\n<div id=\"button\" class=\"entry-content\">\n<\/div>\n\n<div class=\"entry-content\">\n\t<\/div>\n\n<div class=\"entry-content\">\n\t<\/div>","protected":false},"excerpt":{"rendered":"","protected":false},"author":1,"featured_media":1347,"parent":0,"menu_order":8,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"class_list":["post-16","page","type-page","status-publish","has-post-thumbnail","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/16","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/comments?post=16"}],"version-history":[{"count":143,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/16\/revisions"}],"predecessor-version":[{"id":2335,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/16\/revisions\/2335"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/media\/1347"}],"wp:attachment":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/media?parent=16"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}