This article has examined a number of access patterns to Azure Data Lake gen2 available from Azure Databricks. There are merits and disadvantages of each, and most likely it will be a combination of ...

Mount the data from the Azure Data Lake Storage Gen2 to Databricks. Use Pyspark within Databricks to perform data transformations using DELTA TABLES. Load the transformed data back to the Azure Data ...

Overall, Databricks Lakehouse Platform is almost excellent as a data lake that can replace a data warehouse. It’s not for everyone, however, and it’s not quite feature complete.

Delta Lake also uses the Spark engine to handle the metadata of the data lake (which by itself is often a big data problem). Over time, Databricks also plans to add an audit trail, among other things.

Data lakehouses combine the storage and analytics features of data warehouses and data lakes, with vendors like Databricks offering solutions for specific industries. After launching industry ...