{"id":2134,"date":"2023-05-31T16:37:03","date_gmt":"2023-05-31T16:37:03","guid":{"rendered":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/?page_id=2134"},"modified":"2023-06-01T17:28:39","modified_gmt":"2023-06-01T17:28:39","slug":"technology-2","status":"publish","type":"page","link":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/technology-2\/","title":{"rendered":"Best Practices"},"content":{"rendered":"<div class=\"entry-content\">\n\t<div style=\" z-index: 1\" class=\"simplesub wow zoomIn bggrey b90w notoppover footunder ceven withtitle singleno\">\n\t\t\t\t<h1>Best Practices<\/h1>\n\t\t\n\t\t<div class=\"textblockwrap\">\n\n\t\t\t\t\t\t<div class=\"txt\">\n\t\t\t<p><span style=\"font-weight: 400\">Data practitioners and leaders should take an agile, incremental, and business-centered approach to modernizing their data pipelines and environment.<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><b>Adaptability<\/b><span style=\"font-weight: 400\">. Data engineers and other stakeholders should treat data pipelines as living organisms rather than static systems. New users, use cases, risks, and opportunities will require ongoing changes to your data environment. Such changes might include new sources, targets, platforms, and connections. Design your architecture and pipelines for adaptability by investing in open tools, APIs, and data formats.<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Incremental change<\/b><span style=\"font-weight: 400\">. Data teams must maintain uptime and resiliency. They must adapt to new business and technical requirements by making incremental changes. Rather than ripping and replacing a platform or set of pipelines, you should phase in new elements in a modular fashion, one at a time, gauging progress along the way.<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Business-driven evaluation criteria<\/b><span>. When it comes time to select a pipeline tool, be sure to have business objectives shape your evaluation criteria. Advanced, flashy tools might impress engineers, but drive up compute costs or training requirements for business users.\u00a0<\/span><\/li>\n<\/ul>\n\t\t\t<\/div>\n\t\t\t\t\t\t<div class=\"sub\">\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<div class=\"quote\">\n\t\t\t\t\t<p>Data engineers and other stakeholders should treat data pipelines as living organisms rather than static systems. This requires an adaptable architecture with open tools, APIs, and data formats<\/p>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t\t\t\n\n\t\t\t\t\n\n\n\n\n\t\t\t\t\t\t\t\t<div class=\"vidimg\">\n\t\t\t\t\t<div class=\"img\">\n\t\t\t\t\t\t\t\t\t\t\t<div class=\"iframelink\">youtu.be\/oidoOjCnLxg<\/div>\n\t\t\t\t\t\t<iframe class=\"videoem\" src=\"https:\/\/www.youtube.com\/embed\/oidoOjCnLxg?rel=0\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen=\"\"><\/iframe>\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<div class=\"txtlnk\">\n\t\t\t\t\t\t<p>Best Practices for Modern Data Pipelines<\/p>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\t\n\t<\/div>\n<\/div>\t\n\n<div id=\"figure\" class=\"entry-content\">\n<div style=\"\" class=\"figure wow fadeIn bgorange b60w topunder nofootpover\" data-wow-delay=\"0.4s\">\n\t\t<h2>Best Practices Conclusion<\/h2>\n\t\t\n\t\t\t\n\n\t\n\t    <img decoding=\"async\" src=\"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-content\/uploads\/sites\/18\/2023\/06\/Screenshot-2023-06-01-at-1.25.56-PM-1024x456.png\" alt=\"\" \/>\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\n\t<div class=\"ref\"><p>Take an agile, incremental, and business-centered approach to modernizing your data pipelines<\/p>\n<\/div>\n\t\n\t<a class=\"close\" href=\"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-content\/uploads\/sites\/18\/2023\/06\/Screenshot-2023-06-01-at-1.25.56-PM.png\">View Large<\/a>\n<\/div>\n<\/div>\n\n<div class=\"entry-content\">\n\t<div style=\" z-index: 2\" class=\"simplesub wow zoomIn bgnone b60w topover footunder cignore singleyes\">\n\t\t\n\t\t<div class=\"textblockwrap\">\n\n\t\t\t\t\t\t<div class=\"sub\">\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\n\n\t\t\t\t\n\n\n\n\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\t\n\t<\/div>\n<\/div>\t\n\n<div class=\"entry-content\">\n\t<div style=\" z-index: 3\" class=\"wow fadeInDown resources bgblue notoppover colsingle b60w\">\n\t<h2>Additional Resources<\/h2>\n\t<ul>\n\t\t\t\t<li><a class=\"icon sml iwebsite\" target=\"_blank\" href=\"https:\/\/www.eckerson.com\/register?content=best-practices-in-dataops-how-to-create-robust-automated-data-pipelines\">Best Practices in DataOps<\/a><\/li>\n\t\t\t<\/ul>\n<\/div>\n\t<\/div>\n\n<div id=\"figure\" class=\"entry-content\">\n<div style=\"\" class=\"figure wow fadeIn bg b\" data-wow-delay=\"0.4s\">\n\t\t\n\t\t\t\n\n\t\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\n\t<div class=\"ref\"><\/div>\n\t\n\t<a class=\"close\" href=\"\">View Large<\/a>\n<\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"","protected":false},"author":6,"featured_media":1351,"parent":0,"menu_order":10,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"class_list":["post-2134","page","type-page","status-publish","has-post-thumbnail","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/2134","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/comments?post=2134"}],"version-history":[{"count":16,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/2134\/revisions"}],"predecessor-version":[{"id":2351,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/2134\/revisions\/2351"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/media\/1351"}],"wp:attachment":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/media?parent=2134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}