Now we define the four segments of the market for data pipeline management. They include data ingestion, data transformation, DataOps, and pipeline orchestration. A variety of commercial and open-source tools address one or more of these four market segments. Many commercial tools take a no code/low code approach by providing a graphical user interface that reduces or eliminates the need for manual scripting.

• Data ingestion tools help configure, manage, and monitor pipelines that ingest data using the various tasks described earlier.
• Transformation tools help design, build, and execute jobs that transform data in the ways described earlier.
• DataOps tools help continuously integrate and continuously deploy (CI/CD) pipeline code, test code functionality, and observe both pipeline performance and data quality.
• Orchestration tools automate the workflows that stitch together pipelines and the tools and applications that consume their outputs.