“Modern Data Pipelines: Accelerating, Automating, and Validating the Delivery of Complex Data”
Jennifer Achberger, Director of Data Engineering at LendingTree, and Ranajay Nandy, VP of Data Analytics, Citizen Watch America, described their team’s roles and objectives as well as the environments in which they manage data pipelines. Both their companies now demand real-time data, for example to support live operational reporting at LendingTree and predictive or prescriptive analytics at Citizen Watch. Jennifer emphasized the need for her team to “turn on a dime” and accommodate sudden shifts in business requirements by adding, changing, or removing data pipelines with dbt on Snowflake. Her team also continues to connect to new unstructured data sources requested by the business. Like Jennifer, Ranajay emphasized the need for agility as well as data validation by data engineers. He also described how business objectives guide his team’s priorities and activities.