Pipeline Conversations

Learn how leading companies like Dropbox, NVIDIA, and Slack tackle LLM security in production. This comprehensive guide covers practical strategies for preventing prompt injection, securing RAG systems, and implementing multi-layered defenses, based on real-world case studies from the LLMOps database. Discover battle-tested approaches to input validation, data privacy, and monitoring for building...

In this episode, we dive deep into the world of LLM optimization and cost management - a critical challenge facing AI teams today. Join us as we explore real-world strategies from companies like Dropbox, Meta, and Replit who are pushing the boundaries of what's possible with large language models. From clever model selection techniques and knowledge distillation to advanced inference optimization...

A comprehensive exploration of real-world lessons in LLM evaluation and quality assurance, examining how industry leaders tackle the challenges of assessing language models in production. Through diverse case studies, we cover the transition from traditional ML evaluation, establishing clear metrics, combining automated and human evaluation strategies, and implementing continuous improvement cycle...

Prompt engineering is the art and science of crafting instructions that unlock the potential of large language models (LLMs). It's a critical skill for anyone working with LLMs, whether you're building cutting-edge applications or conducting fundamental research. But what does effective prompt engineering look like in practice, and how can we systematically improve our prompts over time? To answer...

An in-depth exploration of LLM agents in production environments, covering key architectures, practical challenges, and best practices. Drawing from real-world case studies, this article examines the current state of AI agent deployment, infrastructure requirements, and critical considerations for organizations looking to implement these systems safely and effectively. Please read the full blog po...

Discover how embeddings power modern search and recommendation systems with LLMs, using case studies from the LLMOps Database. From RAG systems to personalized recommendations, learn key strategies and best practices for building intelligent applications that truly understand user intent and deliver relevant results. Please read the full blog post here and the associated LLMOps database entries he...

Explore real-world applications of Retrieval Augmented Generation (RAG) through case studies from leading companies. Learn how RAG enhances LLM applications with external knowledge sources, examining implementation strategies, challenges, and best practices for building more accurate and informed AI systems. Please read the full blog post [here]( www.zenml.io/blog/building-llm-applications-that-kn...

The LLMOps Database offers a curated collection of 300+ real-world generative AI implementations, providing technical teams with practical insights into successful LLM deployments. This searchable resource includes detailed case studies, architectural decisions, and AI-generated summaries of technical presentations to help bridge the gap between demos and production systems. Please read the full b...

This week I spoke with Daniel van Strien, a digital curator working at the British Library. Daniel has worked on a number of projects at the intersection of archives, libraries and machine learning and I was really happy to have the chance to get to unpack some of the ways he's finding to apply these techniques and tools. In particular, I found it interesting how important the annotation process i...

This week I spoke with Lak Lakhshmanan, who worked for years at Google on ML and AI projects and products at a senior level and he also brings years of experience working on meteorology and other scientific projects previously. Lak brings a ton of experience to the table and it was interesting to hear his suggestions around when it is and isn't appropriate to bring the full set of MLOps tools to t...

This week I spoke with Charles Frye. Not only has Charles volunteered to be a judge on our Month of MLOps competition happening right now, he's part of the core team working on the Full Stack Deep Learning course. Naturally, we get into education for practitioners as well as the things that Charles has seen in his own prior background working on production use cases. We also discuss the ways that...

In today's conversation, I'm speaking with Goku Mohandas, founder and creator of the amazing online resource MadeWithML . Goku has a bunch of practical experience, from working with Apple to a startup in the oncology space and much more. In this conversation we continued to unpack the theme of education in ML, the challenges when it comes to working across the full stack of ML applications, and wh...

So excited to be able to announce our 🔥 AMAZING 🔥 external judges for the ZenML Month of MLOps competition! We have a stellar panel of ✨ ML and MLOps heroes ✨ to help select the best pipelines from all of your submissions! 💥 Charles Frye, core instructor at the amazing Full Stack Deep Learning course 💥 Anthony Goldbloom, co-founder and former CEO of Kaggle 💥 Chip Huyen, author of 'Designing...

This week I spoke with Eric Landau, co-founder of Encord, a platform for data-centric computer vision. This podcast contains a lot of geekery about annotation, and even though Encord aren't an annotation tool per se, Eric and his team have tackled a bunch of quite complicated problems relating to that domain. We also discuss the much-used term 'data-centric AI' and consider where it's useful and w...

This week we dive into the abstractions that we're all trying to layer on top of the core ML processes and workflows. I spoke with Phil Howes, co-founder and chief scientist at BaseTen. BaseTen is a platform that allows data scientists to go from an initial model to an MVP web app quickly. We got into some of the big challenges he had working to build out the platform, as well as the core issue of...

This week I spoke with Savin Goyal and Hugo Bowne-Anderson from Outerbounds. They both work on leading, building and helping people put models into production through Metaflow, and I'm sure current users of ZenML will find this conversation interesting to hear how they think through the broader questions and engineering problems involved with MLOps. Above all, we spoke about the challenges involve...

This week I spoke with Mateo Rojas-Carulla, the CTO and a co-founder of Lakera and Matthias Kraft, also a co-founder and the CPO there. Lakera is an AI safety company that does a lot of work in the computer vision domain, building a platform and tools for users to gain more confidence in the output and functionality of their models. We discuss how they think about the testing of machine learning m...

This week I spoke with Robin Cole, a senior data scientist at Satellite Vu , a company that's about to launch a thermal imaging satellite into space in order to provide new ways of seeing the earth from above. Robin generously took the time to discuss his day to day work involving satellite data, the stack they work with at Satellite Vu as well as some of the difficulties that come up in the domai...

This week on the podcast I spoke with Gerard Kruisheer, the CTO and co-founder of Captain AI , a company based in the Netherlands working on autonomous shipping out of the busy Rotterdam port. We discussed the unique problems that come with building autonomous vehicles, the extent to which the latest and greatest research informs their work, their production stack and how they handle deployment fo...

I'll be having some conversations with the people behind the tools that ZenML offers as integrations. We spoke with Ben Wilson a few weeks back, and today I'm pleased to publish this conversation with Emeli Dral, co-founder and CTO of Evidently, an open-source tool tackling the problem of monitoring of models and data for machine learning. We discussed the challenges around building a tool that is...

This week I spoke with Karthik Kannan, cofounder and CTO of Envision , a company that builds on top of the Google Glass and using Augmented Reality features of phones to allow visually impaired people to better sense the environment or objects around them. Their software and devices are pretty popular and as you'll hear in this conversation, they've been on a real journey to get to where they are...

In this episode, I'm really happy to be able to continue the dialogue we've been having with our users and community around the role of data annotation and labeling in MLOps. We were lucky to get to talk to Iva Gumnishka , the founder of Humans in the Loop . They are an organisation that provides data annotation and collection services. Their teams are primarily made up of those who have been affe...

We took a few weeks break to reach out to some new guests and so I think we can go so far as declaring this next series of episodes as season 2 of Pipeline Conversations. Today, I'm extremely excited to present this conversation I had with Ben Wilson who works over at Databricks and who has also just released a new book called ' Machine Learning Engineering in Action '. It's a jam-backed guide to...

Adam and Hamza return for a short discussion of what we've been busy working on during the previous few months, where we're going with ZenML and why it's so amazing to be building an open-source tool.

I enthusiastically read Kush Varshney's book when it was released for free to the world several months back. Trustworthy Machine Learning is a concise and clear overview of many of the ways that machine learning can go wrong, and so I was especially keen to get Kush on to talk more about his work and research. I also got a stronger sense of appreciation for how good MLOps practices and workflows o...