{"id":1615,"date":"2023-01-30T22:02:33","date_gmt":"2023-01-30T22:02:08","guid":{"rendered":"https:\/\/markets.eckerson.com\/data-marketplace\/technology-2\/"},"modified":"2023-06-01T17:13:40","modified_gmt":"2023-06-01T17:13:40","slug":"technology-2-2","status":"publish","type":"page","link":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/technology-2-2\/","title":{"rendered":"Market Definition"},"content":{"rendered":"<div class=\"entry-content\">\n\t<div style=\" z-index: 1\" class=\"simplesub wow zoomIn bggrey b90w notoppover footunder ceven withtitle singleno\">\n\t\t\t\t<h1>Market Definition<\/h1>\n\t\t\n\t\t<div class=\"textblockwrap\">\n\n\t\t\t\t\t\t<div class=\"txt\">\n\t\t\t<p><span style=\"font-weight: 400\">Now we define the four segments of the market for data pipeline management. They include data ingestion, data transformation, DataOps, and pipeline orchestration. A variety of commercial and open-source tools address one or more of these four market segments. Many commercial tools take a no code\/low code approach by providing a graphical user interface that reduces or eliminates the need for manual scripting.<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><b>Data ingestion<\/b> <b>tools<\/b><span style=\"font-weight: 400\"> help configure, manage, and monitor pipelines that ingest data using the various tasks described earlier.<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Transformation tools<\/b><span style=\"font-weight: 400\"> help design, build, and execute jobs that transform data in the ways described earlier.<\/span><\/li>\n<li style=\"font-weight: 400\"><b>DataOps tools<\/b><span style=\"font-weight: 400\"> help continuously integrate and continuously deploy (CI\/CD) pipeline code, test code functionality, and observe both pipeline performance and data quality.<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Orchestration tools <\/b><span>automate the workflows that stitch together pipelines and the tools and applications that consume their outputs.<\/span><\/li>\n<\/ul>\n\t\t\t<\/div>\n\t\t\t\t\t\t<div class=\"sub\">\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t<div class=\"quote\">\n\t\t\t\t\t<p>The market for data pipeline management includes data ingestion, data transformation, DataOps, and pipeline orchestration<\/p>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\n\t\t\t\t\n\n\t\t\t\t\n\n\n\n\n\t\t\t\t\t\t\t\t<div class=\"vidimg\">\n\t\t\t\t\t<div class=\"img\">\n\t\t\t\t\t\t\t\t\t\t\t<div class=\"iframelink\">youtu.be\/zLwl43c8EMs<\/div>\n\t\t\t\t\t\t<iframe class=\"videoem\" src=\"https:\/\/www.youtube.com\/embed\/zLwl43c8EMs?rel=0\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen=\"\"><\/iframe>\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<div class=\"txtlnk\">\n\t\t\t\t\t\t<p>Learn about four segments of the market<\/p>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\t\n\t<\/div>\n<\/div>\t\n\n<div class=\"entry-content\">\n\t<div style=\" z-index: 2\" class=\"simplesub wow zoomIn bgdblue b60w topover footover cignore withtitle singleyes\">\n\t\t\t\t<h1>Market Segments<\/h1>\n\t\t\n\t\t<div class=\"textblockwrap\">\n\n\t\t\t\t\t\t<div class=\"sub\">\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\n\n\t\t\t\t\t\t\t\t<div class=\"cfigure bgorange\">\n\t\t\t\t\t<h2>Data pipeline management comprises data ingestion and transformation, DataOps, and orchestration<\/h2>\n\t\t\t\t\t\n\n\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-content\/uploads\/sites\/18\/2023\/06\/Screenshot-2023-06-01-at-1.13.23-PM.png\" alt=\"\" \/>\n\t\t\t\t\t\n\n\n\n\n\t\t\t\t\t\t\t\t\t\t<a class=\"close\" href=\"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-content\/uploads\/sites\/18\/2023\/06\/Screenshot-2023-06-01-at-1.13.23-PM.png\">Close<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\n\n\n\n\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\t\n\t<\/div>\n<\/div>\t\n\n<div class=\"entry-content\">\n\t<div style=\" z-index: 3\" class=\"wow fadeInDown resources bgblue topover colsingle b60w\">\n\t<h2>Additional Resources<\/h2>\n\t<ul>\n\t\t\t\t<li><a class=\"icon sml iwebsite\" target=\"_blank\" href=\"https:\/\/www.eckerson.com\/articles\/modern-data-pipelines-three-principles-for-success\">Three Principles for Success<\/a><\/li>\n\t\t\t\t<li><a class=\"icon sml iwebsite\" target=\"_blank\" href=\"https:\/\/www.eckerson.com\/articles\/why-enterprises-should-implement-the-data-mesh-with-dataops\">Why Enterprises Should Implement the Data Mesh with DataOps<\/a><\/li>\n\t\t\t<\/ul>\n<\/div>\n\t<\/div>","protected":false},"excerpt":{"rendered":"","protected":false},"author":6,"featured_media":1351,"parent":0,"menu_order":7,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"class_list":["post-1615","page","type-page","status-publish","has-post-thumbnail","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/1615","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/comments?post=1615"}],"version-history":[{"count":26,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/1615\/revisions"}],"predecessor-version":[{"id":2315,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/pages\/1615\/revisions\/2315"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/media\/1351"}],"wp:attachment":[{"href":"https:\/\/markets.eckerson.com\/modern-data-pipelines\/wp-json\/wp\/v2\/media?parent=1615"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}