Yannic Kilcher Videos (Audio Only)

#wikipedia #reinforcementlearning #languagemodels Original paper review here: https://youtu.be/XHGh19Hbx48 Machel Reid and Yutaro Yamada join me to discuss their recent paper on langauge model pre-training for decision transformers in offline reinforcement learning. OUTLINE: 0:00 - Intro 1:00 - Brief paper, setup & idea recap 7:30 - Main experimental results & high standard deviations 10:0...

#wikipedia #reinforcementlearning #languagemodels Transformers have come to overtake many domain-targeted custom models in a wide variety of fields, such as Natural Language Processing, Computer Vision, Generative Modelling, and recently also Reinforcement Learning. This paper looks at the Decision Transformer and shows that, surprisingly, pre-training the model on a language-modelling task signif...

#mlnews #rsc #gpt3 Some things we've missed in recent weeks! OUTLINE: 0:00 - Intro & Overview 0:40 - Meta builds AI Research Supercluster (RSC) 2:25 - OpenAI trains GPT-3 to follow instructions 4:10 - Meta AI releases multilingual language models 4:50 - Google LaMDA dialogue models 5:50 - Helpful Things 8:25 - Training the alpha matte generator for Pixel 6 10:15 - Drones used to deter pigeons...

#mlnews #muzero #nerf Your regularly irregular updates on everything new in the ML world! Merch: store.ykilcher.com OUTLINE: 0:00 - Intro 0:15 - Sponsor: Weights & Biases 2:15 - Uber switches from XGBoost to Deep Learning for ETA prediction 5:45 - MuZero advances video compression 10:10 - Learned Soft Prompts can steer large language models 12:45 - Block-NeRF captures entire city blocks 14:15...

#mlnews #kilcher #withtheauthors Many of you have given me feedback on what you did and didn't like about the recent "with the authors" videos. Here's the result of that feedback and an outlook into the future. Merch: store.ykilcher.com Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Dis...

#ai #gpu #tpu This video is an interview with Adi Fuchs, author of a series called "AI Accelerators", and an expert in modern AI acceleration technology. Accelerators like GPUs and TPUs are an integral part of today's AI landscape. Deep Neural Network training can be sped up by orders of magnitudes by making good use of these specialized pieces of hardware. However, GPUs and TPUs are only the begi...

#cm3 #languagemodel #transformer This video contains a paper explanation and an incredibly informative interview with first author Armen Aghajanyan. Autoregressive Transformers have come to dominate many fields in Machine Learning, from text generation to image creation and many more. However, there are two problems. First, the collected data is usually scraped from the web and uni- or bi-modal an...

#security #censorship #ai Most of us conceive the internet as a free and open space where we are able to send traffic between any two nodes, but for large parts of the world this is not the case. Entire nations have large machinery in place to survey all internet traffic and automated procedures to block any undesirable connections. Evading such censorship has been largely a cat-and-mouse game bet...

#hypertransformer #metalearning #deeplearning This video contains a paper explanation and an interview with author Andrey Zhmoginov! Few-shot learning is an interesting sub-field in meta-learning, with wide applications, such as creating personalized models based on just a handful of data points. Traditionally, approaches have followed the BERT approach where a large model is pre-trained and then...

#mlnews #alphacode #openai The latest and greatest from the world of Machine Learning! Merch: store.ykilcher.com Sponsor: Weights & Biases https://wandb.me/yannic OUTLINE: 0:00 - Intro 0:15 - Sponsor: Weights & Biases 3:15 - DeepMind's AlphaCode: AI competitive programmer 11:30 - OpenAI uses language models to prove math theorems 14:30 - StyleGAN XL: Scaling StyleGAN to diverse datasets 16...

#gpt3 #embodied #planning In this video: Paper explanation, followed by first author interview with Wenlong Huang. Large language models contain extraordinary amounts of world knowledge that can be queried in various ways. But their output format is largely uncontrollable. This paper investigates the VirtualHome environment, which expects a particular set of actions, objects, and verbs to be used....

#mlnews #openai #embeddings COMMENTS DIRECTLY FROM THE AUTHOR (thanks a lot for reaching out Arvind :) ): 1. The FIQA results you share also have code to reproduce the results in the paper using the API: https://twitter.com/arvind_io/status/... There's no discrepancy AFAIK. 2. We leave out 6 not 7 BEIR datasets. Results on msmarco, nq and triviaqa are in a separate table (Table 5 in the paper). NQ...

#deeplearning #brain #neuroscience Originally, Deep Learning sprang into existence inspired by how the brain processes information, but the two fields have diverged ever since. However, given that deep models can solve many perception tasks with remarkable accuracy, is it possible that we might be able to learn something about how the brain works by inspecting our models? I speak to Patrick Mineau...

#eleuther #gptneo #gptj EleutherAI announces GPT-NeoX-20B, a 20 billion parameter open-source language model, inspired by GPT-3. Connor joins me to discuss the process of training, how the group got their hands on the necessary hardware, what the new model can do, and how anyone can try it out! OUTLINE: 0:00 - Intro 1:00 - Start of interview 2:00 - How did you get all the hardware? 3:50 - What's t...

#deeplearning #symbolic #research This video includes an interview with first author Stéphane d'Ascoli (https://sdascoli.github.io/). Deep neural networks are typically excellent at numeric regression, but using them for symbolic computation has largely been ignored so far. This paper uses transformers to do symbolic regression on integer and floating point number sequences, which means that given...

LIMITED TIME MERCH DEAL: http://store.ykilcher.com Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitchute.com/channel/yann... LinkedIn: https://www.linkedin.com/in/ykilcher BiliBili: https://space.bilibili.com/2017636191 If you...

#mlnews #convnext #mt3 Your update on what's new in the Machine Learning world! OUTLINE: 0:00 - Intro 0:15 - ConvNeXt: Return of the Convolutions 2:50 - Investigating Saliency Cropping Algorithms 9:40 - YourTTS: SOTA zero-shot Text-to-Speech 10:40 - MT3: Multi-Track Music Transcription 11:35 - China regulates addictive algorithms 13:00 - A collection of Deep Learning interview questions & solu...

#deeplearning #neuralinterpreter #ai This video includes an interview with the paper's authors! What if we treated deep networks like modular programs? Neural Interpreters divide computation into small modules and route data to them via a dynamic type inference system. The resulting model combines recurrent elements, weight sharing, attention, and more to tackle both abstract reasoning, as well as...

#deeplearning #noether #symmetries This video includes an interview with first author Ferran Alet! Encoding inductive biases has been a long established methods to provide deep networks with the ability to learn from less data. Especially useful are encodings of symmetry properties of the data, such as the convolution's translation invariance. But such symmetries are often hard to program explicit...

#minerl #minecraft #deeplearning The MineRL BASALT challenge has no reward functions or technical descriptions of what's to be achieved. Instead, the goal of each task is given as a short natural language string, and the agent is evaluated by a team of human judges who rate both how well the goal has been fulfilled, as well as how human-like the agent behaved. In this video, I interview KAIROS, th...

#minerl #minecraft #deeplearning The MineRL BASALT challenge has no reward functions or technical descriptions of what's to be achieved. Instead, the goal of each task is given as a short natural language string, and the agent is evaluated by a team of human judges who rate both how well the goal has been fulfilled, as well as how human-like the agent behaved. In this video, I interview KAIROS, th...

#tesla #fsd #elon Watch the original podcast: https://www.youtube.com/watch?v=DxREm... An analysis of Elon's appearance on Lex Fridman. Very interesting conversation and a good overview of past, current, and future versions of Tesla's Autopilot system. OUTLINE: 0:00 - Intro 0:40 - Tesla Autopilot: How hard is it? 9:05 - Building an accurate understanding of the world 16:25 - History of Tesla's neu...

#playerofgames #deepmind #alphazero Special Guest: First author Martin Schmid (https://twitter.com/Lifrordi) Games have been used throughout research as testbeds for AI algorithms, such as reinforcement learning agents. However, different types of games usually require different solution approaches, such as AlphaZero for Go or Chess, and Counterfactual Regret Minimization (CFR) for Poker. Player o...

#glide #openai #diffusion Diffusion models learn to iteratively reverse a noising process that is applied repeatedly during training. The result can be used for conditional generation as well as various other tasks such as inpainting. OpenAI's GLIDE builds on recent advances in diffusion models and combines text-conditional diffusion with classifier-free guidance and upsampling to achieve unpreced...

#mlnews #gopher #glam Your updates on everything going on in the Machine Learning world. Sponsor: Weights & Biases https://wandb.me/yannic OUTLINE: 0:00 - Intro & Overview 0:20 - Sponsor: Weights & Biases 3:05 - DeepMind releases 3 papers on large language models 11:45 - Hugging Face Blog: Training CodeParrot from scratch 14:25 - Paper: Pre-Training vision systems with noise 15:45 - De...