How AI Is Built

Many companies use Elastic or OpenSearch and use 10% of the capacity. They have to build ETL pipelines. Get data Normalized. Worry about race conditions. All in all. At the moment, when you want to do search on top of your transactional data, you are forced to build a distributed systems. Not anymore. ParadeDB is building an open-source PostgreSQL extension to enable search within your database. T...

RAG isn't a magic fix for search problems. While it works well at first, most teams find it's not good enough for production out of the box. The key is to make it better step by step, using good testing and smart data creation. Today, we are talking to Saahil Ognawala from Jina AI to start to understand RAG. To build a good RAG system, you need three things: ways to test it, methods to create trai...

Documentation quality is the silent killer of RAG systems. A single ambiguous sentence might corrupt an entire set of responses. But the hardest part isn't fixing errors - it's finding them. Today we are talking to Max Buckley on how to find and fix these errors. Max works at Google and has built a lot of interesting experiments with LLMs on using them to improve knowledge bases for generation. We...

Ever wondered why vector search isn't always the best path for information retrieval? Join us as we dive deep into BM25 and its unmatched efficiency in our latest podcast episode with David Tippett from GitHub. Discover how BM25 transforms search efficiency, even at GitHub's immense scale. BM25, short for Best Match 25, use term frequency (TF) and inverse document frequency (IDF) to score document...

Ever wondered why your vector search becomes painfully slow after scaling past a million vectors? You're not alone - even tech giants struggle with this. Charles Xie, founder of Zilliz (company behind Milvus), shares how they solved vector database scaling challenges at 100B+ vector scale: Key Insights: Multi-tier storage strategy:  GPU memory (1% of data, fastest) RAM (10% of data) Local SSD Obje...

Modern search systems face a complex balancing act between performance, relevancy, and cost, requiring careful architectural decisions at each layer. While vector search generates buzz, hybrid approaches combining traditional text search with vector capabilities yield better results. The architecture typically splits into three core components: ingestion/indexing (requiring decisions between batch...

Today we are talking to Michael Günther, a senior machine learning scientist at Jina about his work on JINA Clip. Some key points: Uni-modal embeddings convert a single type of input (text, images, audio) into vectors Multimodal embeddings learn a joint embedding space that can handle multiple types of input, enabling cross-modal search (e.g., searching images with text) Multimodal models can pote...

Imagine a world where data bottlenecks, slow data loaders, or memory issues on the VM don't hold back machine learning. Machine learning and AI success depends on the speed you can iterate. LanceDB is here to to enable fast experiments on top of terabytes of unstructured data. It is the database for AI. Dive with us into how LanceDB was built, what went into the decision to use Rust as the main im...

Today’s guest is Mór Kapronczay. Mór is the Head of ML at superlinked. Superlinked is a compute framework for your information retrieval and feature engineering systems, where they turn anything into embeddings. When most people think about embeddings, they think about ada, openai. You just take your text and throw it in there. But that’s too crude. OpenAI embeddings are trained on the internet. B...

Today we have Jessica Talisman with us, who is working as an Information Architect at Adobe. She is (in my opinion) the expert on taxonomies and ontologies. That’s what you will learn today in this episode of How AI Is Built. Taxonomies, ontologies, knowledge graphs. Everyone is talking about them no-one knows how to build them. But before we look into that, what are they good for in search? Imagi...

ColPali makes us rethink how we approach document processing. ColPali revolutionizes visual document search by combining late interaction scoring with visual language models. This approach eliminates the need for extensive text extraction and preprocessing, handling messy real-world data more effectively than traditional methods. In this episode, Jo Bergum, chief scientist at Vespa, shares his ins...

Today, we're talking to Aamir Shakir , the founder and baker at mixedbread.ai , where he's building some of the best embedding and re-ranking models out there. We go into the world of rerankers, looking at how they can classify, deduplicate documents, prioritize LLM outputs, and delve into models like ColBERT. We discuss: The role of rerankers in retrieval pipelines Advantages of late interaction...

Text embeddings have limitations when it comes to handling long documents and out-of-domain data. Today, we are talking to Nils Reimers. He is one of the researchers who kickstarted the field of dense embeddings, developed sentence transformers, started HuggingFace’s Neural Search team and now leads the development of search foundational models at Cohere. Tbh, he has too many accolades to count of...

Hey! Welcome back. Today we look at how we can get our RAG system ready for scale. We discuss common problems and their solutions, when you introduce more users and more requests to your system. For this we are joined by Nirant Kasliwal, the author of fastembed. Nirant shares practical insights on metadata extraction, evaluation strategies, and emerging technologies like Colipali. This episode is...

In this episode of How AI is Built, Nicolay Gerold interviews Doug Turnbull, a search engineer at Reddit and author on “Relevant Search”. They discuss how methods and technologies, including large language models (LLMs) and semantic search, contribute to relevant search results. Key Highlights: Defining relevance is challenging and depends heavily on user intent and context Combining multiple sear...

In this episode, we talk data-driven search optimizations with Charlie Hull. Charlie is a search expert from Open Source Connections. He has built Flax, one of the leading open source search companies in the UK, has written “Searching the Enterprise”, and is one of the main voices on data-driven search. We discuss strategies to improve search systems quantitatively and much more. Key Points: Relev...

Welcome back to How AI Is Built.  We have got a very special episode to kick off season two.  Daniel Tunkelang is a search consultant currently working with Algolia. He is a leader in the field of information retrieval, recommender systems, and AI-powered search. He worked for Canva, Algolia, Cisco, Gartner, Handshake, to pick a few.  His core focus is query understanding.   **Query understanding...

Today we are launching the season 2 of How AI Is Built. The last few weeks, we spoke to a lot of regular listeners and past guests and collected feedback. Analyzed our episode data. And we will be applying the learnings to season 2. This season will be all about search. We are trying to make it better, more actionable, and more in-depth. The goal is that at the end of this season, you have a full-...

In this episode of "How AI is Built," host Nicolay Gerold interviews Jonathan Yarkoni, founder of Reach Latent. Jonathan shares his expertise in extracting value from unstructured data using AI, discussing challenging projects, the impact of ChatGPT, and the future of generative AI. From weather prediction to legal tech, Jonathan provides valuable insights into the practical applications of AI acr...

This episode of "How AI Is Built" is all about data processing for AI. Abhishek Choudhary and Nicolay discuss Spark and alternatives to process data so it is AI-ready. Spark is a distributed system that allows for fast data processing by utilizing memory. It uses a dataframe representation "RDD" to simplify data processing. When should you use Spark to process your data for your AI Systems? → Use...

In this episode, Nicolay talks with Rahul Parundekar, founder of AI Hero, about the current state and future of AI agents. Drawing from over a decade of experience working on agent technology at companies like Toyota, Rahul emphasizes the importance of focusing on realistic, bounded use cases rather than chasing full autonomy. They dive into the key challenges, like effectively capturing expert wo...

In this conversation, Nicolay and Richmond Alake discuss various topics related to building AI agents and using MongoDB in the AI space. They cover the use of agents and multi-agents, the challenges of controlling agent behavior, and the importance of prompt compression. When you are building agents. Build them iteratively. Start with simple LLM calls before moving to multi-agent systems. Main Tak...

In this episode, Kirk Marple, CEO and founder of Graphlit, shares his expertise on building efficient data integrations. Kirk breaks down his approach using relatable concepts: The "Two-Sided Funnel": This model streamlines data flow by converting various data sources into a standard format before distributing it. Universal Data Streams: Kirk explains how he transforms diverse data into a single,...

In our latest episode, we sit down with Derek Tu, Founder and CEO of Carbon, a cutting-edge ETL tool designed specifically for large language models (LLMs). Carbon is streamlining AI development by providing a platform for integrating unstructured data from various sources, enabling businesses to build innovative AI applications more efficiently while addressing data privacy and ethical concerns....

In this episode, Nicolay sits down with Hugo Lu, founder and CEO of Orchestra, a modern data orchestration platform. As data pipelines and analytics workflows become increasingly complex, spanning multiple teams, tools and cloud services, the need for unified orchestration and visibility has never been greater. Orchestra is a serverless data orchestration tool that aims to provide a unified contro...