Arxiv Papers

The paper investigates training instabilities in large Transformer-based models and explores ways to reproduce and study these instabilities at smaller scales. It examines sources of instability, explores the impact of learning rate and other interventions, and studies cases where instabilities can be predicted. https://arxiv.org/abs//2309.14322 YouTube: https://www.youtube.com/@ArxivPapers PODCAS...

Decomposed Prompt Tuning (DEPT) is a method for parameter-efficient fine-tuning of language models. It achieves better performance while saving memory and time costs compared to other approaches, and is adaptable to different model architectures and sizes. https://arxiv.org/abs//2309.05173 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast...

Decomposed Prompt Tuning (DEPT) is a method for parameter-efficient fine-tuning of language models. It achieves better performance while saving memory and time costs compared to other approaches, and is adaptable to different model architectures and sizes. https://arxiv.org/abs//2309.05173 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast...

Text2Reward is a data-free framework using large language models to automate the creation of dense reward functions in reinforcement learning from natural language goals. It outperforms expert-written codes in multiple benchmarks and supports real-world deployment and human feedback refinement. https://arxiv.org/abs/2309.11489 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts:...

This paper offers a comprehensive survey on large language model (LLM)-based AI agents, discussing their foundation, a general framework, and applications in various scenarios. It further delves into agent societies and highlights key challenges. https://arxiv.org/abs//2309.07864 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-pap...

Large language models (LLMs) exhibit a failure of generalization known as the Reversal Curse, where they struggle to answer questions in the reverse direction of their training data. This failure persists across different model sizes and families and is not improved by data augmentation. https://arxiv.org/abs//2309.12288 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https...

Large language models (LLMs) exhibit a failure of generalization known as the Reversal Curse, where they struggle to answer questions in the reverse direction of their training data. This failure persists across different model sizes and families and is not improved by data augmentation. https://arxiv.org/abs//2309.12288 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https...

The paper discusses the use of machine learning algorithms to predict the risk of cardiovascular disease in patients with diabetes. https://arxiv.org/abs//2309.12307 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers

The paper discusses the use of machine learning algorithms to predict the risk of cardiovascular disease in patients with diabetes. https://arxiv.org/abs//2309.12307 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers

The paper introduces BTLM-3B-8K, a new state-of-the-art 3 billion parameter language model that outperforms existing models and provides excellent long context performance. It is compact, requiring less memory and compute, making it accessible for mobile and edge devices. https://arxiv.org/abs//2309.11568 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.appl...

The paper introduces BTLM-3B-8K, a new state-of-the-art 3 billion parameter language model that outperforms existing models and provides excellent long context performance. It is compact, requiring less memory and compute, making it accessible for mobile and edge devices. https://arxiv.org/abs//2309.11568 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.appl...

The paper introduces a method called "FreeU" that improves generation quality in diffusion models without additional training or finetuning. It strategically re-weights the contributions from skip connections and backbone feature maps. https://arxiv.org/abs//2309.11497 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-pape...

The paper introduces a method called "FreeU" that improves generation quality in diffusion models without additional training or finetuning. It strategically re-weights the contributions from skip connections and backbone feature maps. https://arxiv.org/abs//2309.11497 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-pape...

The study addresses hallucinations in language models, introducing the Chain-of-Verification (CoVe) method. This process drafts, fact-checks, and verifies responses, effectively reducing hallucinations in various tasks. https://arxiv.org/abs//2309.11495 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: h...

The study addresses hallucinations in language models, introducing the Chain-of-Verification (CoVe) method. This process drafts, fact-checks, and verifies responses, effectively reducing hallucinations in various tasks. https://arxiv.org/abs//2309.11495 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: h...

https://arxiv.org/abs//2309.10668 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers

https://arxiv.org/abs//2309.10668 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers

The paper introduces PDFTriage, an approach to improve document question answering by considering the structure and content of structured documents. Experimental results show its effectiveness, and a benchmark dataset is released for further research. https://arxiv.org/abs//2309.08872 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxi...

The paper introduces PDFTriage, an approach to improve document question answering by considering the structure and content of structured documents. Experimental results show its effectiveness, and a benchmark dataset is released for further research. https://arxiv.org/abs//2309.08872 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxi...

Contrastive Decoding, a training-free text generation method, improves reasoning tasks by maximizing the difference in likelihood between strong and weak models, outperforming other methods on various benchmarks. https://arxiv.org/abs//2309.09117 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://...

Contrastive Decoding, a training-free text generation method, improves reasoning tasks by maximizing the difference in likelihood between strong and weak models, outperforming other methods on various benchmarks. https://arxiv.org/abs//2309.09117 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://...

The paper proposes a method to identify and interpret the directions in activation space of neural networks, addressing the issue of polysemanticity. The method uses sparse autoencoders to reconstruct internal activations and achieves more interpretable and monosemantic results compared to alternative approaches. This method can enable precise model editing and improve model transparency and steer...

The paper explores the impact of parameter sparsity on the scaling behavior of Transformers trained on massive datasets. It identifies a scaling law that describes the relationship between weight sparsity, number of non-zero parameters, and amount of training data. The findings provide insights into the optimal sparsity level for computational efficiency improvements. https://arxiv.org/abs//2309.0...

The paper introduces Agents, an open-source library that allows non-specialists to build and deploy autonomous language agents with advanced features, and provides a link to access the library. https://arxiv.org/abs//2309.07870 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify....

This paper introduces a new approach called Statistical Rejection Sampling Optimization (RSO) to improve the alignment of language models with human preferences. RSO outperforms existing methods on evaluations from both language models and human raters. https://arxiv.org/abs//2309.06657 YouTube: https://www.youtube.com/@ArxivPapers PODCASTS: Apple Podcasts: https://podcasts.apple.com/us/podcast/ar...