Yannic Kilcher Videos (Audio Only)

#llm #ai #chatgpt How does one run inference for a generative autoregressive language model that has been trained with a fixed context size? Streaming LLMs combine the performance of windowed attention, but avoid the drop in performance by using attention sinks - an interesting phenomenon where the token at position 0 acts as an absorber of "extra" attention. OUTLINE: 0:00 - Introduction...

#ai #promptengineering #evolution Promptbreeder is a self-improving self-referential system for automated prompt engineering. Give it a task description and a dataset, and it will automatically come up with appropriate prompts for the task. This is achieved by an evolutionary algorithm where not only the prompts, but also the mutation-prompts are improved over time in a population-based, diversity...

#ai #retnet #transformers Retention is an alternative to Attention in Transformers that can both be written in a parallel and in a recurrent fashion. This means the architecture achieves training parallelism while maintaining low-cost inference. Experiments in the paper look very promising. OUTLINE: 0:00 - Intro 2:40 - The impossible triangle 6:55 - Parallel vs sequential 15:35 - Retention mechani...

#ai #rlhf #llm ReST uses a bootsrap-like method to produce its own extended dataset and trains on ever higher-quality subsets of it to improve its own reward. The method allows for re-using the same generated data multiple times and thus has an efficiency advantage with respect to Online RL techniques like PPO. Paper: https://arxiv.org/abs/2308.08998 Abstract: Reinforcement learning from human fee...

#mlnews #llama2 #openai Your regular irregular update on the world of Machine Learning. References: https://twitter.com/ylecun/status/1681336284453781505 https://ai.meta.com/llama/ https://about.fb.com/news/2023/07/llama-2-statement-of-support/ https://247wallst.com/special-report/2023/08/12/this-is-the-biggest-social-media-platform-ranking-the-worlds-largest-networking-sites/4/ https://github.com...

#cybercrime #chatgpt #security An interview with Sergey Shykevich, Threat Intelligence Group Manager at Check Point, about how models like ChatGPT have impacted the realm of cyber crime. https://threatmap.checkpoint.com/ Links: Homepage: https://ykilcher.com Merch: https://ykilcher.com/merch YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://yki...

#llm #safety #gpt4 A prime example of intellectual dishonesty of journalists and AI critics. Article: https://gizmodo.com/paknsave-ai-savey-recipe-bot-chlorine-gas-1850725057 My Recipe AI: https://github.com/yk/recipe-ai Links: Homepage: https://ykilcher.com Merch: https://ykilcher.com/merch YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://yki...

#ai #diffusion #stabilityai An interview with DeepFloyd members Misha Konstantinov and Daria Bakshandaeva on the release of the model IF, an open-source model following Google's implementation of Imagen. References: https://www.deepfloyd.ai/deepfloyd-if https://huggingface.co/DeepFloyd https://twitter.com/_gugutse_ https://twitter.com/_bra_ket Links: Homepage: https://ykilcher.com Merch: https...

#gpt4 #mit #ai A new paper claims to use GPT-4 to solve 100% of a set of MIT university exercises. Some people are skeptic and their investigations reveal more than one problem with this paper... OUTLINE: 0:00 - ChatGPT gives out Windows 10 keys 0:30 - MIT exam paper 2:50 - Prompt engineering 5:30 - Automatic grading 6:45 - Response by other MIT students 8:30 - Unsolvable questions 10:50 - Duplica...

#stablediffusion #ai #watermark Watermarking the outputs of generative models is usually done as a post-processing step on the model outputs. Tree-Ring Watermarks are applied in the latent space at the beginning of a diffusion process, which makes them nearly undetectable, robust to strong distortions, and only recoverable by the model author. It is a very promising technique with applications pot...

#gpt4 #rwkv #transformer We take a look at RWKV, a highly scalable architecture between Transformers and RNNs. Fully Connected (June 7th in SF) Promo Link: https://www.fullyconnected.com/?promo=ynnc OUTLINE: 0:00 - Introduction 1:50 - Fully Connected In-Person Conference in SF June 7th 3:00 - Transformers vs RNNs 8:00 - RWKV: Best of both worlds 12:30 - LSTMs 17:15 - Evolution of RWKV's Linear...

#gpt4 #ai #prompt Tree-of-Thought improves prompting of large language models (LLMs) by generalizing the concept of Chain-of-Thought prompting and introduces a tree search across language model thoughts, including state evaluation and backtracking. Experiments on toy tasks show large improvements over both classic and Chain-of-Thought prompting. OUTLINE: 0:00 - Introduction 1:20 - From Chain-of-Th...

#ai #openai #gpt4 US Senate hearing on AI regulation. MLST video on the hearing: https://www.youtube.com/watch?v=DeSXnESGxr4 Links: Homepage: https://ykilcher.com Merch: https://ykilcher.com/merch YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ykilcher.com/discord LinkedIn: https://www.linkedin.com/in/ykilcher If you want to support me, the...

#google #openai #mlnews Updates from the world of Machine Learning and AI Great AI memes here: https://twitter.com/untitled01ipynb OUTLINE: 0:00 - Google I/O 2023: Generative AI in everything 0:20 - Anthropic announces 100k tokens context 0:35 - Intro 1:20 - Geoff Hinton leaves Google 7:00 - Google memo leaked: we have no moat 11:30 - OpenAI loses 540M 12:30 - Google AI: Product first 15:50 - Ilya...

#ai #transformer #gpt4 This paper promises to scale transformers to 1 million tokens and beyond. We take a look at the technique behind it: The Recurrent Memory Transformer, and what its strenghts and weaknesses are. OUTLINE: 0:00 - Intro 2:15 - Transformers on long sequences 4:30 - Tasks considered 8:00 - Recurrent Memory Transformer 19:40 - Experiments on scaling and attention maps 24:00 - Concl...

#openassistant #chatgpt #mlnews Try the chat: https://open-assistant.io/chat Homepage: https://open-assistant.io Dataset: https://huggingface.co/datasets/OpenAssistant/oasst1 Code: https://github.com/LAION-AI/Open-Assistant Paper (temporary): https://ykilcher.com/oa-paper Links: Homepage: https://ykilcher.com Merch: https://ykilcher.com/merch YouTube: https://www.youtube.com/c/yannickilcher Twitte...

#openassistant #chatgpt #gpt4https://open-assistant.io/chathttps://huggingface.co/OpenAssistanthttps://github.com/LAION-AI/Open-Assistant Links: Homepage: https://ykilcher.com Merch: https://ykilcher.com/merch YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ykilcher.com/discord LinkedIn: https://www.linkedin.com/in/ykilcher If you want to sup...

#mlnews #gpt4 #copilot Your weekly news all around the AI world Check out W&B courses (free): https://wandb.courses/ OUTLINE: 0:00 - Intro 0:20 - GPT-4 announced! 4:30 - GigaGAN: The comeback of Generative Adversarial Networks 7:55 - ChoppedAI: AI Recipes 8:45 - Samsung accused of faking space zoom effect 14:00 - Weights & Biases courses are free 16:55 - Data Portraits 18:50 - Data2Vec 2.0...

#gpt4 #chatgpt #openai References: https://openai.com/product/gpt-4https://openai.com/research/gpt-4https://cdn.openai.com/papers/gpt-4.pdf Links: Homepage: https://ykilcher.com Merch: https://ykilcher.com/merch YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ykilcher.com/discord LinkedIn: https://www.linkedin.com/in/ykilcher If you want to s...

#mlnews #chatgpt #llama ChatGPT goes around the world and is finally available via API. Stunning mind-reading performed using fMRI and Stable Diffusion. LLaMA weights leak and hilarity ensues. GTC23 is around the corner! ERRATA: It's a 4090, not a 4090 ti 🙃 OUTLINE: 0:00 - Introduction 0:20 - GTC 23 on March 20 1:55 - ChatGPT API is out! 4:50 - OpenAI becomes more business-friendly 7:15 - Ope...

#ai #meta #languagemodel LLaMA is a series of large language models from 7B to 65B parameters, trained by Meta AI. They train for longer on more data and show that something like gpt-3 can be outperformed by significantly smaller models when trained like this. Meta also releases the trained models to the research community. OUTLINE: 0:00 - Introduction & Paper Overview 4:30 - Rant on Open-Sour...

#ai #huggingface #coding Join me as I build streaming inference into the Hugging Face text generation server, going through cuda, python, rust, grpc, websockets, server-sent events, and more... Original repo is here: https://github.com/huggingface/text-generation-inference OpenAssistant repo is here: https://github.com/LAION-AI/Open-Assistant (see inference/) Check out https://www.wandb.courses/ f...

#openassistant #chatgpt #ai Help us collect data for OpenAssistant, the largest and most open alternative to ChatGPT. https://open-assistant.io OUTLINE: 0:00 - Intro 0:30 - The Project 2:05 - Getting to Minimum Viable Prototype 5:30 - First Tasks 10:00 - Leaderboard 11:45 - Playing the Assistant 14:40 - Tricky Facts 16:25 - What if humans had wings? 17:05 - Can foxes be tamed? 23:45 - Can zebras b...

#chatgpt #ai #openai  ChatGPT, OpenAI's newest model is a GPT-3 variant that has been fine-tuned using Reinforcement Learning from Human Feedback, and it is taking the world by storm! Sponsor: Weights & Biases https://wandb.me/yannic OUTLINE: 0:00 - Intro 0:40 - Sponsor: Weights & Biases 3:20 - ChatGPT: How does it work? 5:20 - Reinforcement Learning from Human Feedback 7:10 - ChatGPT...

#ai #mlnews #gpt4 Your weekly news from the AI & Machine Learning world. OUTLINE: 0:00 - Introduction 0:25 - AI reads brain signals to predict what you're thinking 3:00 - Closed-form solution for neuron interactions 4:15 - GPT-4 rumors 6:50 - Cerebras supercomputer 7:45 - Meta releases metagenomics atlas 9:15 - AI advances in theorem proving 10:40 - Better diffusion models with expert denoiser...