-
Delivering Interoperability For Analytics in Healthcare with Databricks (Part 1 – Ingestion & Analysis)
This article, part of a series focused on Healthcare analytics, discusses how Databricks can be used to effectively manage interoperability in healthcare systems. By leveraging standardized protocols, such as FHIR…
-
Lakehouse Federation – Data Virtualisation for the Databricks Age
I’ve always been a fan of data virtualisation. I know it’s not the silver bullet for every kind of data integration scenarios (I still love my ETL/ELT), but for those…
-
Change of role!
It’s been a couple of months since my last post and that’s largely down to the fact that I’ve done something I never imagined doing – I’ve left Microsoft! There’s…
-
Data Quality ELT With Azure Data Factory
Been a while since my last post – got a few in the backlog now though so will have a trickle of articles coming through in the next few months.…
-
Building the Lakehouse Architecture With Azure Synapse Analytics
We’re all largely familiar with the common modern data warehouse pattern in the cloud, which essentially delivers a platform comprising a data lake (based on a cloud storage account like…
-
Detecting And Anonymizing PII Data With Azure Synapse Analytics
This isn’t a deep technical post, but more of an attempt to increase awareness of a utility created by Microsoft’s Commercial Software Engineering (CSE) team that allows data engineers to…
-
Change Data Capture Upsert Patterns With Azure Synapse Analytics and Databricks
Change Data Capture (Referred to as CDC for the rest of this article) is a common pattern used to capture change events from source databases and push them to a…
-
Cross Database Queries With Azure Synapse (Serverless)
I had a recent challenge from a customer that involved the need to join together datasets from different regions and present them as a unified view of data WITHOUT the…
-
Putting the “T” into “ELT”
In my last blog post (From Warehouse To Lakehouse – ELT/ETL Design Patterns With Azure Data Services) I talked about the different patterns that exist for Extract, Transform and Load…
-
From Warehouse To Lakehouse – ELT/ETL Design Patterns With Azure Data Services
I like a good pattern. I also like a good framework. From a data perspective, nothing lends itself better to having both of these as ETL/ELT. For those not familiar…
Lakehouses, data & AI
