As your data evolves, you need a way to track the who, what, when, why, and how of those changes. You need a data lineage system ... Query a SQL or NoSQL database for what the data looked like ...

Information is captured for tables ... upstream and downstream data flows. Additionally, lineage works across all languages supported by Databricks — including SQL, Python, R and Scala ...

For more details and examples, check out this blog post on `Use the Metadata API to Track SQL Executed Against Snowflake (Shared Job)` What’s next for data lineage in Matillion ETL? From our user ...

Bigeye’s end-to-end lineage capability maps the customer’s complete hybrid data pipeline, then allows business users and data analysts to deploy targeting monitoring on specific tables and columns.