#data-engineering

Articles tagged with #data-engineering

The Case for AI in Data Engineering
Series: AI-Augmented Data Engineering | Article 1 of 7 There is a quiet crisis in most data engineering teams. Pipelines fail at 2 AM. The on-call engineer spends three hours tracing a root cause tha
May 31, 20267 min read12
The Art of Program Visibility: Managing Databricks + Azure Data Programs at Scale
The Invisible Failure Most data platform programs don't fail loudly. They fail quietly — one missed dependency, one unreported pipeline issue, one status update that said 'green' while the underlying
May 14, 202611 min read9
Unity Catalog - the Unified Self
How Databricks Unity Catalog Mirrors the Advaitic Vision of One Undivided Reality
Apr 28, 202615 min read12
Migrating SPC Run Rules from SAS to Databricks
A Pharma Supply Chain Engineering Perspective · tech4nirvana.com Why This Migration Is Non-Trivial Earlier, I worked as Product Owner and Data Architect on a SAS to Databricks migration for a Pharma
Apr 22, 202618 min read51
The Data Engineer's Vedanta: Ancient Wisdom for Modern Data Pipelines
Introduction There is an ancient Sanskrit phrase that has guided seekers of truth for over a thousand years: "Tat Tvam Asi" — You are That. At its core, Advaita Vedanta, the non-dualist school of Indi
Apr 19, 20267 min read66