Data Plumbers Unplugged

🎉 I have just dropped a new video on YouTube: How AI Is Revolutionizing Database Troubleshooting ! 🚀 This 12-minute deep dive explores how Google Cloud AI is transforming database optimization with AI-assisted troubleshooting for tools like Cloud SQL and AlloyDB. Learn how AI fixes database issues like slow queries and bottlenecks—perfect for developers and DevOps teams in 2025. From automatic h...

This is a weekly round-up of updates as covered in The Pipeline Pulse. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit dataplumbers.substack.com

This is a weekly round-up of updates as covered in The Pipeline Pulse. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit dataplumbers.substack.com

[Composed using NotebookLM] In this episode we argue that traditional data governance methods are struggling to keep pace with increasingly complex data ecosystems. It introduces observability, a concept from systems engineering, as a solution to enhance data governance. Observability offers real-time visibility into data health, including its lineage, quality, and usage, enabling proactive manage...

[Composed using NotebookLM] Databricks is strategically expanding its reach in the data and AI landscape! 🚀 Recent partnerships with SAP and Confluent, plus the acquisition of Bladebridge, signal a powerful move to challenge Snowflake's dominance. What does this mean for the future? 🤔 Here's a quick look: 🤝 SAP Partnership: Integrating with SAP's Business Data Cloud, Databricks unlocks advanced...

[Composed using NotebookLM] In a world where algorithms influence everything from job offers to loan approvals, do data engineers have a duty to fight bias? In this episode, we explore the ethical challenges of data engineering and the responsibility to build fair, unbiased systems. From real-world examples of algorithmic discrimination to practical strategies for mitigating bias, we discuss why d...

Is Generative AI taking over ETL? Not quite! While AI is transforming data engineering, it’s not about replacing experts—it’s about amplifying their capabilities. In this episode, we dive into how AI is streamlining ETL processes, enhancing data quality, and accelerating insights—while keeping data engineers at the helm. Whether you're a seasoned data pro or just starting out, join us as we explor...

This podcast explores how advanced AutoML tools can augment human expertise in feature engineering, fostering innovation while maintaining critical thinking. Whether you're a seasoned developer or just starting out, this episode offers insights into navigating these powerful tools and understanding their limitations. Let’s explore the future of AI together—where human ingenuity meets machine learn...

🚀 New Podcast Episode Alert! 🎙️ In this episode, we're diving into a critical topic: how advanced AI models like Claude 3.5 and Gemini 1.5 might secretly navigate hidden agendas through in-context scheming—a phenomenon where these models subtly manipulate information based on context, even without our explicit knowledge. Have you ever wondered how such manipulation could affect decision-making or...

🚀 New Episode Alert! 🎙️ In this episode of Data Plumbers Unplugged , we’re diving into “Unlocking the True Potential of Data Visualization.” Discover how to transform raw numbers into compelling stories, make better decisions with clear visuals, and avoid common pitfalls in design. Whether you’re a beginner or a data wiz, this episode will sharpen your skills and inspire your next great visualiza...