{"id":30,"date":"2022-06-09T12:52:33","date_gmt":"2022-06-09T12:52:13","guid":{"rendered":"https:\/\/eck.uidev.site\/?page_id=30"},"modified":"2023-05-31T16:37:50","modified_gmt":"2023-05-31T16:37:50","slug":"takeaways","status":"publish","type":"page","link":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/takeaways\/","title":{"rendered":"Key Takeaways"},"content":{"rendered":"<div class=\"entry-content\">\n\t<div style=\"\" class=\"simplesub wow zoomIn bgorange b80w notoppover nofootpover ceven withtitle singleno\">\n\t\t\t\t<h1>Key Takeaways<\/h1>\n\t\t\n\t\t<div class=\"textblockwrap\">\n\n\t\t\t\t\t\t<div class=\"txt\">\n\t\t\t<p>Wayne Eckerson and Kevin Petrie offered their key takeaways from the March 30 event on Modern Data Pipelines hosted by Eckerson Group.<\/p>\n\t\t\t<\/div>\n\t\t\t\t\t\t<div class=\"sub\">\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\n\n\t\t\t\t\n\n\n\n\n\t\t\t\t\t\t\t\t<div class=\"vidimg\">\n\t\t\t\t\t<div class=\"img\">\n\t\t\t\t\t\t\t\t\t\t\t<div class=\"iframelink\">youtu.be\/GRTHccOvrq4<\/div>\n\t\t\t\t\t\t<iframe class=\"videoem\" src=\"https:\/\/www.youtube.com\/embed\/GRTHccOvrq4?rel=0\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen=\"\"><\/iframe>\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<div class=\"txtlnk\">\n\t\t\t\t\t\t<p><\/p>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\t\n\t<\/div>\n<\/div>\t\n\n<div class=\"entry-content\">\n\t<div style=\"\" class=\"simplesub wow zoomIn bgblue b80w notoppover nofootpover ceven withtitle singleno\">\n\t\t\t\t<h1>Key Takeaways<\/h1>\n\t\t\n\t\t<div class=\"textblockwrap\">\n\n\t\t\t\t\t\t<div class=\"txt\">\n\t\t\t<p><b>Keynote<\/b><\/p>\n<ul>\n<li><span style=\"font-weight: 400\">Data environments get more complex by the day. New users, use cases, sources, targets, etc. put strain on the pipelines that must deliver timely, accurate data for analytics<\/span><\/li>\n<li><span style=\"font-weight: 400\">The market for data pipeline management includes four segments:<\/span>\n<ul>\n<li>Ingest: extract and load data from source to target<\/li>\n<li>Transform: filter, merge, and format data for consumption<\/li>\n<li>DataOps: optimize pipelines with CI\/CD, testing, and monitoring<\/li>\n<li>Orchestrate: schedule and execute workflows across pipelines and applications<\/li>\n<\/ul>\n<\/li>\n<li><span style=\"font-weight: 400\">Users of data pipelines include data engineers that specialize in pipelines and data analysts and scientists that prepare data to support analytics. They also include analytics engineers that transform and validate data for consumption by analysts.<\/span><\/li>\n<li><span style=\"font-weight: 400\">Data pipeline tools divide along two axes: specialty vs. suite, and cloud vs. hybrid focus.<\/span><\/li>\n<\/ul>\n<blockquote>\n<p style=\"text-align: left\"><b>Suite cloud products <\/b><span>help manage ingestion, transformation, DataOps, and pipeline orchestration with a suite for cloud environments<\/span><\/p>\n<p><b>Suite hybrid products <\/b><span style=\"font-weight: 400\">offer these capabilities for hybrid environments<\/span><\/p>\n<p><b>Specialty cloud products <\/b><span style=\"font-weight: 400\">focus on specific aspects of data pipeline management for cloud environments<\/span><\/p>\n<p><span style=\"font-weight: 400\"><strong>Specialty hybrid products<\/strong> have a similar focus on specific aspects of pipeline management for hybrid environments<\/span><\/p><\/blockquote>\n\t\t\t<\/div>\n\t\t\t\t\t\t<div class=\"sub\">\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<div class=\"moretxt\">\n\t\t\t\t\t<p><b>Practitioners<\/b><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Jennifer: agility, need for real time, unstructured, low code<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Ranajay: focus on managing costs, adopting prescriptive and predictive analytics<\/span><\/li>\n<\/ul>\n<p><b>Panel<\/b><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">\u201cTable stakes\u201d requirements include security, sources\/targets, and low code\/no code<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Data products must support any source, target, or channel, and be managed with metadata<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Many enterprises and vendors focus their data warehouse initiatives on Snowflake<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Shift happens! This means data drift, referring to changes in schema, metadata, infrastructure, and possibly ML model outputs over time. Users must adapt to this, preferably with automated tools.<\/span><\/li>\n<\/ul>\n\t\t\t\t<\/div>\n\t\t\t\t\n\n\t\t\t\t\n\n\n\n\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\t\n\t<\/div>\n<\/div>\t\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-2 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-1 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\"><\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\"><\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n<div id=\"tile\" class=\"entry-content\">\n\t\t<div class=\"tile info bgnone fourcol numbnoni\">\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal1\">\n\t\t\t\t1\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal2\">\n\t\t\t\t2\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal3\">\n\t\t\t\t3\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal4\">\n\t\t\t\t4\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal5\">\n\t\t\t\t5\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal6\">\n\t\t\t\t6\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal7\">\n\t\t\t\t7\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal8\">\n\t\t\t\t8\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t<\/div>\n\t<\/div>\n<\/div>\n\n<div id=\"tile\" class=\"entry-content\">\n\t\t<div class=\"tile info bgnone fourcol numbnoni\">\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal1\">\n\t\t\t\t1\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal2\">\n\t\t\t\t2\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal3\">\n\t\t\t\t3\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal4\">\n\t\t\t\t4\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal5\">\n\t\t\t\t5\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal6\">\n\t\t\t\t6\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\n\t\t<div class=\"item wow fadeIn\">\n\t\t\n\t\t\n\t\t\t\t\t\n\t\t\t<div class=\"numb wow fadeIn bgpal7\">\n\t\t\t\t7\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"txt\">\n\t\t\t\t<h2><\/h2>\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\t\t\t\t\n\t\t\t\t<\/div>\n\t<\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":16,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"class_list":["post-30","page","type-page","status-publish","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/30","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/comments?post=30"}],"version-history":[{"count":54,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/30\/revisions"}],"predecessor-version":[{"id":2243,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/30\/revisions\/2243"}],"wp:attachment":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/media?parent=30"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}