Arxiv Papers

Deeper transformer language models tend to generalize better for compositional tasks, even when the total number of parameters is kept constant. The benefits of depth for generalization cannot be solely attributed to better performance on language modeling or in-distribution data. https://arxiv.org/abs//2310.19956 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_...

Deeper transformer language models tend to generalize better for compositional tasks, even when the total number of parameters is kept constant. The benefits of depth for generalization cannot be solely attributed to better performance on language modeling or in-distribution data. https://arxiv.org/abs//2310.19956 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_...

The paper presents Battle of the Backbones (BoB), a benchmarking framework that evaluates the performance of various pretrained models on computer vision tasks. The results show that supervised pretrained convolutional neural networks still perform best, but self-supervised learning backbones are competitive and should be explored further. https://arxiv.org/abs//2310.19909 YouTube: https://www.you...

The paper presents Battle of the Backbones (BoB), a benchmarking framework that evaluates the performance of various pretrained models on computer vision tasks. The results show that supervised pretrained convolutional neural networks still perform best, but self-supervised learning backbones are competitive and should be explored further. https://arxiv.org/abs//2310.19909 YouTube: https://www.you...

This paper surveys the emerging field of knowledge editing in deep neural networks, which aims to enable efficient and effective changes to pre-trained models without affecting previous tasks. It reviews relevant approaches and outlines potential future directions. https://arxiv.org/abs//2310.19704 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Pod...

This paper surveys the emerging field of knowledge editing in deep neural networks, which aims to enable efficient and effective changes to pre-trained models without affecting previous tasks. It reviews relevant approaches and outlines potential future directions. https://arxiv.org/abs//2310.19704 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Pod...

MM-VID is an integrated system that combines GPT-4V with specialized tools to facilitate advanced video understanding, addressing challenges in long-form videos and complex tasks. It uses video-to-script generation to transcribe multimodal elements and enables advanced capabilities in video comprehension. Experimental results demonstrate its effectiveness across different video genres and lengths....

MM-VID is an integrated system that combines GPT-4V with specialized tools to facilitate advanced video understanding, addressing challenges in long-form videos and complex tasks. It uses video-to-script generation to transcribe multimodal elements and enables advanced capabilities in video comprehension. Experimental results demonstrate its effectiveness across different video genres and lengths....

The paper introduces \system, a pre-trained diffusion code generation model that addresses the limitation of auto-regressive models by iteratively denoising a complete program. It performs on par with state-of-the-art models in accuracy and outperforms them in diversity versus quality. https://arxiv.org/abs//2310.17680 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@a...

The paper introduces Davidsonian Scene Graph (DSG), an evaluation framework for text-to-image models. DSG addresses reliability challenges in question generation and visual question answering, and includes an open-sourced evaluation benchmark called DSG-1k. https://arxiv.org/abs//2310.18235 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: h...

The paper introduces Davidsonian Scene Graph (DSG), an evaluation framework for text-to-image models. DSG addresses reliability challenges in question generation and visual question answering, and includes an open-sourced evaluation benchmark called DSG-1k. https://arxiv.org/abs//2310.18235 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: h...

The paper presents ZEPHYR-7B, a language model that achieves state-of-the-art performance on chat benchmarks by using distilled direct preference optimization (dDPO) and AI Feedback (AIF) data. https://arxiv.org/abs//2310.16944 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 S...

The paper presents ZEPHYR-7B, a language model that achieves state-of-the-art performance on chat benchmarks by using distilled direct preference optimization (dDPO) and AI Feedback (AIF) data. https://arxiv.org/abs//2310.16944 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 S...

The paper proposes a method called JudgeLM to evaluate large language models (LLMs) in open-ended scenarios. They fine-tune LLMs as scalable judges and introduce techniques to address biases. JudgeLM achieves state-of-the-art performance and high agreement with human judges. https://arxiv.org/abs//2310.17631 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers...

The paper proposes a method called JudgeLM to evaluate large language models (LLMs) in open-ended scenarios. They fine-tune LLMs as scalable judges and introduce techniques to address biases. JudgeLM achieves state-of-the-art performance and high agreement with human judges. https://arxiv.org/abs//2310.17631 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers...

The paper analyzes language models and identifies a mechanism for binding entities to their attributes. It shows that language models represent binding information through internal activations and that binding vectors form a continuous subspace. This provides insights into how language models represent symbolic knowledge in-context. https://arxiv.org/abs//2310.17191 YouTube: https://www.youtube.co...

Controlled decoding (CD) is a novel off-policy reinforcement learning method that uses a value function called a prefix scorer to steer autoregressive generation towards high reward outcomes. CD is effective in controlling language models and can handle multiple rewards without additional complexity. It can also be applied in a blockwise fashion at inference-time, making it a promising approach fo...

Controlled decoding (CD) is a novel off-policy reinforcement learning method that uses a value function called a prefix scorer to steer autoregressive generation towards high reward outcomes. CD is effective in controlling language models and can handle multiple rewards without additional complexity. It can also be applied in a blockwise fashion at inference-time, making it a promising approach fo...

This paper explores the transfer of "complementary" knowledge between pretrained deep learning models without performance degradation. The authors propose a data partitioning technique for successful transfer and assess the scalability and impact of model properties on knowledge transfer. https://arxiv.org/abs//2310.17653 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www....

This paper explores the transfer of "complementary" knowledge between pretrained deep learning models without performance degradation. The authors propose a data partitioning technique for successful transfer and assess the scalability and impact of model properties on knowledge transfer. https://arxiv.org/abs//2310.17653 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www....

https://arxiv.org/abs//2310.16656 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers

https://arxiv.org/abs//2310.16656 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers

This paper introduces a method, MIN-K% PROB, to detect if a large language model was trained on a given text without knowing the pretraining data. It achieves better results than previous methods and is effective in various real-world scenarios. https://arxiv.org/abs//2310.16789 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podca...

This paper introduces a method, MIN-K% PROB, to detect if a large language model was trained on a given text without knowing the pretraining data. It achieves better results than previous methods and is effective in various real-world scenarios. https://arxiv.org/abs//2310.16789 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podca...

The paper introduces QMoE, a compression and execution framework that allows trillion-parameter language models to be run efficiently on affordable hardware with minimal accuracy loss. https://arxiv.org/abs//2310.16795 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: h...