Yannic Kilcher Videos (Audio Only)

#lama #inpainting #deeplearning At the end of the video is an interview with the paper authors! LaMa is a system that is amazing at removing foreground objects from images, especially when those objects cover a large part of the image itself. LaMa is specifically trained to reconstruct large masked areas and includes global information throughout its forward propagation by using Fourier Convolutio...

#mlnews #deepmind #ai The most trusted model in News! Get started with Weights & Biases here: https://wandb.me/yannic (it's free forever for personal use) OUTLINE: 0:00 - Intro 0:15 - Sponsor: Weights & Biases 3:10 - DeepMind tackles fundamental math 6:45 - Microsoft focuses on scaling effectively and efficiently 10:15 - NeurIPS Anthology Visualization 13:30 - Timnit Gebru launches researc...

#nuwa #microsoft #generative NÜWA is a unifying architecture that can ingest text, images, and videos and brings all of them into a quantized latent representation to support a multitude of visual generation tasks, such as text-to-image, text-guided video manipulation, or sketch-to-video. This paper details how the encoders for the different modalities are constructed, and how the latent represent...

#mlnews #gaugan #gpt-3 Your weekly dose of ML News! More GauGAN images here: https://drive.google.com/drive/folder... OUTLINE: 0:00 - Intro 0:20 - Sponsor: Weights & Biases 2:20 - OpenAI's removes GPT-3 Waitlist 4:55 - NVIDIA releases GauGAN2 Webapp 9:45 - Everyday Robots tackles real-life tasks 12:15 - MetNet-2: 12-hour Rain Forecasting 14:45 - TinyML Dog Bark Stopper 15:55 - AI learns to dri...

#scalingtransformers #terraformer #sparsity Transformers keep pushing the state of the art in language and other domains, mainly due to their ability to scale to ever more parameters. However, this scaling has made it prohibitively expensive to run a lot of inference requests against a Transformer, both in terms of compute and memory requirements. Scaling Transformers are a new kind of architectur...

#ext5 #transferlearning #exmix The T5 model has been a staple for NLP research for the last years. Both its size and its approach to formulate all NLP tasks as prompt-based language modeling make it a convenient choice to tackle new challenges and provides a strong baseline for most current datasets. ExT5 pushes T5 to its limits by pre-training not only on self-supervised mask filling, but also at...

#imle #backpropagation #discrete Backpropagation is the workhorse of deep learning, but unfortunately, it only works for continuous functions that are amenable to the chain rule of differentiation. Since discrete algorithms have no continuous derivative, deep networks with such algorithms as part of them cannot be effectively trained using backpropagation. This paper presents a method to incorpora...

#neurips #peerreview #machinelearning A look at the results of the 2021 NeurIPS peer review experiment. https://arxiv.org/abs/2109.09774 https://www.reddit.com/r/MachineLearn... Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitc...

#deeplearning #neuralarchitecturesearch #metalearning Deep Neural Networks are usually trained from a given parameter initialization using SGD until convergence at a local optimum. This paper goes a different route: Given a novel network architecture for a known dataset, can we predict the final network parameters without ever training them? The authors build a Graph-Hypernetwork and train on a no...

#grafting #adam #sgd The last years in deep learning research have given rise to a plethora of different optimization algorithms, such as SGD, AdaGrad, Adam, LARS, LAMB, etc. which all claim to have their special peculiarities and advantages. In general, all algorithms modify two major things: The (implicit) learning rate schedule, and a correction to the gradient direction. This paper introduces...

#mlnews #cedille #wmt Only the greatest of news from the world of Machine Learning. OUTLINE: 0:00 - Sponsor: Weights & Biases 1:50 - Cedille - French Language Model 3:55 - Facebook AI Multilingual model wins WMT 5:50 - YOU private search engine 10:35 - DeepMind's Open-Source Arnheim 12:10 - Company sued for using AI to make website more accessible 18:05 - Alibaba DAMO Academy creates 10 Trilli...

#deeplearning #backpropagation #simulation More and more systems are made differentiable, which means that accurate gradients of these systems' dynamics can be computed exactly. While this development has led to a lot of advances, there are also distinct situations where backpropagation can be a very bad idea. This paper characterizes a few such systems in the domain of iterated dynamical systems,...

#mlnews #turing #reskin The latest and greatest from the Machine Learning world Sponsor: Weights & Biases https://wandb.com References: Microsoft Turing Bletchley: Universal Image Language Representation Model https://www.microsoft.com/en-us/resea... https://turing.microsoft.com/bletchley Meta AI Tactile Sensing https://ai.facebook.com/blog/teaching... https://ai.facebook.com/blog/reskin-a......

#machinelearning #ardm #generativemodels Diffusion models have made large advances in recent months as a new type of generative models. This paper introduces Autoregressive Diffusion Models (ARDMs), which are a mix between autoregressive generative models and diffusion models. ARDMs are trained to be agnostic to the order of autoregressive decoding and give the user a dynamic tradeoff between spee...

#pathways #mlnews #ego4d Your irregular dose of Machine Learning News. OUTLINE: 0:00 - Intro 0:20 - Sponsor: Weights & Biases 2:10 - Google Introduces Pathways AI Architecture 6:30 - OpenAI trains Language Models to do High School Math 8:25 - Sam Altman says Neural Networks truly learn 9:35 - Google AI researchers frustrated with lawyers 12:10 - DeepMind RL Lecture Series 2021 12:40 - Fashion...

#efficientzero #muzero #atari Reinforcement Learning methods are notoriously data-hungry. Notably, MuZero learns a latent world model just from scalar feedback of reward- and policy-predictions, and therefore relies on scale to perform well. However, most RL algorithms fail when presented with very little data. EfficientZero makes several improvements over MuZero that allows it to learn from aston...

#ytalks #siraj #plagiarism A conversation with Siraj Raval about his journey on YouTube, and the perils of fame. OUTLINE: 0:00 - Intro 1:30 - Welcome 3:15 - Starting out: From Economics to YouTube 13:00 - More Views: Plagiarizing Video Content 23:30 - One Step Up: Copying A Research Paper 29:15 - Was there another way? 39:00 - Clickbait Course: Make Money with Machine Learning 50:30 - Rock Bottom...

#gtc21 #mlnews #mujoco Registriere für GTC'21 und gewinne eine RTX 3090: https://nvda.ws/2Y2B5ni OUTLINE: 0:00 - Intro 0:15 - Sponsor: NVIDIA GTC'21 6:10 - DeepMind kauft & Open-Sourct MuJoCo 9:05 - PyTorch 1.10 Veröffentlicht 11:25 - Google Lernt Spreadsheet Formeln 14:15 - handtracking.io 15:25 - Zellinstanzsegmentierungswettbewerb 16:15 - Hilfreiche Bibliotheken 23:15 - Waymo autos verirren...

#gtc21 #mlnews #mujoco Register to GTC'21 and Win a RTX 3090: https://nvda.ws/2Y2B5ni OUTLINE: 0:00 - Intro 0:15 - Sponsor: NVIDIA GTC'21 5:35 - DeepMind buys & Open-Sources MuJoCo 7:25 - PyTorch 1.10 Released 9:10 - Google Predicts Spreadsheet Formulas 11:25 - handtracking.io 12:25 - Cell Instance Segmentation Challenge 13:00 - Helpful Libraries 17:50 - Waymo cars keep turning into same dead-...

#aiia #ai #art A trip report from the AiiA Festival in Geneva organized by the ImpactAI foundation. OUTLINE: 0:00 - Intro 1:50 - Laura Tocmacov: The Festival 4:10 - Timothy O'Hear: The Tech 6:50 - Jonathan O'Hear: The Robot 11:50 - Cléa Chopard: The Artist 17:45 - Final Words Website: https://aiiafestival.org/en/ Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: htt...

#gpt3 #knowledge #symbolic Symbolic knowledge models are usually trained on human-generated corpora that are cumbersome and expensive to create. Such corpora consist of structured triples of symbolic knowledge. This paper takes a different approach and attempts to generate such a corpus by prompting GPT-3. Results show that clever prompting, combined with targeted small critic models trained on hu...

#sbb #seatreview #travel A friendly parody of Travel Vloggers and Airplane Seat Reviews :) No, SBB did not pay me for this (but they should ;) ) Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitchute.com/channel/yann... Minds: h...

#mlnews #turingnlg #convmixer Your latest upates on what's happening in the Machine Learning world. OUTLINE: 0:00 - Intro 0:16 - Weights & Biases raises on 1B valuation (sponsored) 2:30 - Microsoft trains 530 billion parameter model 5:15 - StyleGAN v3 released 6:45 - A few more examples may be worth billions of parameters 8:30 - ConvMixer fits into a tweet 9:45 - Improved VQGAN 11:25 - William...

#deepmind #nowcasting #machinelearning Your holy update on what's new in the Machine Learning world. OUTLINE: 0:00 - Intro 0:30 - DeepMind tackles Nowcasting 3:30 - The Guardian's shady reporting on TruthfulQA 6:15 - Stochastic training not necessary for generalization 7:35 - Google AI's efficient partitioning of road networks 9:15 - MiniHack Reinforcement Learning Environment 10:45 - Plato XL 11B...

#grokking #openai #deeplearning Grokking is a phenomenon when a neural network suddenly learns a pattern in the dataset and jumps from random chance generalization to perfect generalization very suddenly. This paper demonstrates grokking on small algorithmic datasets where a network has to fill in binary tables. Interestingly, the learned latent spaces show an emergence of the underlying binary op...