Arxiv Papers

The paper proposes PagedAttention, an attention algorithm inspired by virtual memory and paging techniques, to address the memory inefficiencies in large language model serving systems. The proposed system, vLLM, achieves near-zero waste in memory and improves throughput by 2-4 times compared to existing systems. https://arxiv.org/abs//2309.06180 YouTube: https://www.youtube.com/@ArxivPapers PODCA...

The paper proposes that the superior performance of Transformers in deep learning is due to an architectural bias towards mesa-optimization, a learned process within the forward pass. They reverse-engineer Transformers and show that the learned optimization algorithm can be used for few-shot tasks. They also propose a new self-attention layer that improves performance. https://arxiv.org/abs//2309....

This paper introduces speculative decoding, a method to sample from autoregressive models faster by running them in parallel on the outputs of approximation models. It achieves 2X-3X acceleration without changing the distribution. https://arxiv.org/abs//2211.17192 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016...

The paper analyzes large language models and finds that many neurons are "dead" and never activate, while others act as token and n-gram detectors. The models become more sparse with scale, and some neurons are positional. https://arxiv.org/abs//2309.04827 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id16924760...

The paper proposes a new learning paradigm called Fiat that combines in-context learning and fine-tuning for large language models. It shows that Fiat outperforms both paradigms on multilingual tasks with limited training data. https://arxiv.org/abs//2309.04663 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Sp...

ImageBind-LLM is a multi-modality instruction tuning method for large language models. It can respond to audio, 3D point clouds, video, and their embedding-space arithmetic using only image-text alignment training. The model uses a learnable bind network to align the embedding space between LLaMA and ImageBind's image encoder, allowing visual instructions to be injected via a gating mechanism....

The paper proposes a decoding strategy called DoLa to reduce hallucinations in large language models (LLMs) without the need for external knowledge or additional fine-tuning. DoLa improves truthfulness in LLMs across multiple tasks. https://arxiv.org/abs//2309.03883 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id16924760...

The paper proposes SLiMe, a method that leverages large vision-language models to segment images with as few as one annotated sample. SLiMe outperforms existing one-shot and few-shot segmentation methods. https://arxiv.org/abs//2309.03179 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcaste...

CM3Leon ("Chameleon”) is a retrieval-augmented, token-based, decoder-only multi-modal language model capable of generating and infilling text and images. It achieves state-of-the-art performance in text-to-image generation and demonstrates high controllability in various tasks. https://arxiv.org/abs//2309.02591 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://po...

YaRN is a compute-efficient method to extend the context window of transformer-based language models, allowing them to effectively utilize and extrapolate to longer context lengths. It surpasses previous methods and can extrapolate beyond the limited context of a fine-tuning dataset. https://arxiv.org/abs//2309.00071 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://p...

MVDream is a multi-view diffusion model that generates consistent multi-view images from text prompts. It improves 3D generation stability and can be fine-tuned for personalized 3D generation. https://arxiv.org/abs//2308.16512 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.c...

This paper introduces LLaSM, a large multi-modal speech-language model capable of following speech-and-language instructions. The model demonstrates a more natural way for humans to interact with AI. A dataset and code are also provided. https://arxiv.org/abs//2308.15930 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id169...

The paper addresses the length generalization failure of Transformer-based Large Language Models (LLMs) on long sequences. They propose a solution called "LM-Infinite" that involves an attention mask and a distance limit, which allows LLMs to generate fluent texts and carry out downstream tasks on longer contexts. The solution is computationally efficient and demonstrates consistent flue...

The paper explores the capabilities of linear probing and visual prompting/reprogramming methods for computer vision tasks, focusing on data sparsity and model sparsity. It finds that lottery ticket models may not always perform as well as dense models and that dense models have superior calibration. https://arxiv.org/abs//2308.14969 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Po...

The paper introduces the fast feedforward (FFF) architecture, which offers comparable performance to feedforward networks at a fraction of the inference cost. FFFs can replace feedforward networks and mixture-of-expert networks in transformers. A vision transformer trained with FFF achieves single-neuron inferences with only a 5.8% performance decrease. https://arxiv.org/abs//2308.14711 YouTube: h...

The paper introduces Eventful Transformers, a method for reducing the computational costs of Vision Transformers in video recognition tasks by identifying and re-processing only significant changes between subsequent frames. The approach achieves significant computational savings with minimal accuracy reduction. https://arxiv.org/abs//2308.13494 YouTube: https://www.youtube.com/@ArxivPapers PODCAS...

Code Llama is a family of large language models for code that provide state-of-the-art performance, infilling capabilities, support for large input contexts, and instruction following ability. The models are released under a permissive license. https://arxiv.org/abs//2308.12950 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-paper...

This paper explores critical learning periods in deep linear network models and shows that these periods depend on the depth of the model and structure of the data distribution. The study also examines the impact of pre-training on transfer performance in multi-task learning. https://arxiv.org/abs//2308.12221 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts....

The paper surveys existing methods for extending the context length of large language models and introduces a new truncation strategy. The authors test these methods on various evaluation tasks and find that linear scaling is the most effective method. They also release new long-context models and code for replication. https://arxiv.org/abs//2308.10882 YouTube: https://www.youtube.com/@ArxivPapers...

Reinforced Self-Training (ReST) is a simple algorithm that aligns large language models (LLMs) with human preferences using reinforcement learning from human feedback (RLHF). It improves translation quality efficiently and effectively. https://arxiv.org/abs//2308.08998 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id16924...

The paper proposes an efficient weight-only quantization method for large language models (LLMs) to reduce memory consumption and accelerate inference. The method utilizes a heuristic approach that only uses the model weights of a pre-trained model, without requiring additional fine-tuning. The approach addresses the challenges and issues associated with LLM quantization and achieves higher throug...

The paper introduces Graph of Thoughts (GoT), a framework that enhances prompting capabilities in large language models by modeling information as a graph. GoT offers advantages over existing paradigms and can be extended for new prompting schemes. https://arxiv.org/abs//2308.09687 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-p...

The paper analyzes the emergence of new skills in language models when scaled up, using a statistical framework and mathematical analysis. It introduces the concept of "slingshot generalization" where competence levels violate usual generalization theory. https://arxiv.org/abs//2307.15936 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/u...

This paper introduces Bayesian Flow Networks (BFNs), a generative model that uses Bayesian inference to modify independent distributions and passes them through a neural network to generate interdependent distributions. BFNs achieve competitive results in image and language modeling tasks. https://arxiv.org/abs//2308.07037 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: htt...

The paper explores the use of code in enhancing the reasoning capability of large language models (LLMs) for math problems. They propose a prompting method called explicit code-based self-verification (CSV) that significantly improves accuracy on math datasets. https://arxiv.org/abs//2308.07921 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/po...