Arxiv Papers

MobileVLM is a multimodal vision language model designed for mobile devices. It achieves competitive performance compared to larger models and demonstrates state-of-the-art inference speed on both CPU and GPU. The models are available on GitHub. https://arxiv.org/abs//2312.16886 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podca...

MobileVLM is a multimodal vision language model designed for mobile devices. It achieves competitive performance compared to larger models and demonstrates state-of-the-art inference speed on both CPU and GPU. The models are available on GitHub. https://arxiv.org/abs//2312.16886 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podca...

This survey examines the impact of Mixture of Experts (MoE), multimodal learning, and Artificial General Intelligence (AGI) on generative AI, including their applications, challenges, and ethical considerations. It also discusses the influence of these technologies on research priorities and academic communication. https://arxiv.org/abs//2312.10868 YouTube: https://www.youtube.com/@ArxivPapers Tik...

This survey examines the impact of Mixture of Experts (MoE), multimodal learning, and Artificial General Intelligence (AGI) on generative AI, including their applications, challenges, and ethical considerations. It also discusses the influence of these technologies on research priorities and academic communication. https://arxiv.org/abs//2312.10868 YouTube: https://www.youtube.com/@ArxivPapers Tik...

The paper introduces a framework that improves the generalizability and factuality of Large Language Models (LLMs) through task-specific fine-tuning and discriminative models, resulting in enhanced performance on various language tasks. https://arxiv.org/abs//2312.15918 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple...

The paper introduces a framework that improves the generalizability and factuality of Large Language Models (LLMs) through task-specific fine-tuning and discriminative models, resulting in enhanced performance on various language tasks. https://arxiv.org/abs//2312.15918 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple...

The paper proposes a method for accelerating large-scale pre-training by using model-based data selection policies. The method reduces computation needed for training while still achieving the same performance as models trained with uniform sampling. The approach is shown to be effective across datasets and tasks, and also improves performance in multimodal transfer tasks and pretraining regimes....

The paper proposes a method for accelerating large-scale pre-training by using model-based data selection policies. The method reduces computation needed for training while still achieving the same performance as models trained with uniform sampling. The approach is shown to be effective across datasets and tasks, and also improves performance in multimodal transfer tasks and pretraining regimes....

The paper introduces a simple technique called LAyer-SElective Rank reduction (LASER) that improves the performance of large language models by selectively removing higher-order components of their weight matrices. The technique is shown to be effective across different language models and datasets. https://arxiv.org/abs//2312.13558 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www...

The paper introduces a simple technique called LAyer-SElective Rank reduction (LASER) that improves the performance of large language models by selectively removing higher-order components of their weight matrices. The technique is shown to be effective across different language models and datasets. https://arxiv.org/abs//2312.13558 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www...

The paper introduces LoRAMoE, a plugin version of Mixture of Experts (MoE), to address the challenge of world knowledge forgetting during fine-tuning of large language models. Experimental results show that LoRAMoE can coordinate experts based on data type and prevent knowledge forgetting. https://arxiv.org/abs//2312.09979 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.co...

The paper introduces LoRAMoE, a plugin version of Mixture of Experts (MoE), to address the challenge of world knowledge forgetting during fine-tuning of large language models. Experimental results show that LoRAMoE can coordinate experts based on data type and prevent knowledge forgetting. https://arxiv.org/abs//2312.09979 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.co...

The paper introduces time vectors, a method to customize language models to specific time periods. By finetuning the model on data from a single time period and subtracting the weights of the original model, time vectors improve performance on text from that time period. Interpolating between time vectors allows for better performance on intervening and future time periods. The findings are consis...

The paper introduces time vectors, a method to customize language models to specific time periods. By finetuning the model on data from a single time period and subtracting the weights of the original model, time vectors improve performance on text from that time period. Interpolating between time vectors allows for better performance on intervening and future time periods. The findings are consis...

The paper introduces DREAM-Talk, a two-stage diffusion-based framework for generating emotional talking faces. It achieves both expressive emotional talking and accurate lip-sync by using a novel diffusion module and a video-to-video rendering module. DREAM-Talk outperforms state-of-the-art methods in terms of expressiveness, lip-sync accuracy, and perceptual quality. https://arxiv.org/abs//2312.1...

The paper introduces DREAM-Talk, a two-stage diffusion-based framework for generating emotional talking faces. It achieves both expressive emotional talking and accurate lip-sync by using a novel diffusion module and a video-to-video rendering module. DREAM-Talk outperforms state-of-the-art methods in terms of expressiveness, lip-sync accuracy, and perceptual quality. https://arxiv.org/abs//2312.1...

This paper introduces a novel approach to optimizing Large Language Models (LLMs) through contextual pruning, resulting in smaller, domain-specific LLMs that maintain core functionalities. The method is effective across diverse datasets and has potential for future development. https://arxiv.org/abs//2312.12682 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_pap...

This paper introduces a novel approach to optimizing Large Language Models (LLMs) through contextual pruning, resulting in smaller, domain-specific LLMs that maintain core functionalities. The method is effective across diverse datasets and has potential for future development. https://arxiv.org/abs//2312.12682 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_pap...

This paper addresses the challenge of efficiently running large language models (LLMs) on devices with limited DRAM capacity by storing model parameters on flash memory and bringing them on demand to DRAM. The authors propose two techniques, "windowing" and "row-column bundling," which enable running models up to twice the size of available DRAM with significant increases in in...

This paper addresses the challenge of efficiently running large language models (LLMs) on devices with limited DRAM capacity by storing model parameters on flash memory and bringing them on demand to DRAM. The authors propose two techniques, "windowing" and "row-column bundling," which enable running models up to twice the size of available DRAM with significant increases in in...

The paper discusses the use of machine learning algorithms to predict the risk of cardiovascular disease based on electronic health records. https://arxiv.org/abs//2312.11370 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv...

The paper discusses the use of machine learning algorithms to predict the risk of cardiovascular disease based on electronic health records. https://arxiv.org/abs//2312.11370 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv...

The paper explores the concept of weak model supervision and its ability to elicit the full capabilities of a stronger model. The authors test this using pretrained language models and find that simple methods can improve weak-to-strong generalization, making progress on aligning superhuman models. https://arxiv.org/abs//2312.09390 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www....

The paper explores the concept of weak model supervision and its ability to elicit the full capabilities of a stronger model. The authors test this using pretrained language models and find that simple methods can improve weak-to-strong generalization, making progress on aligning superhuman models. https://arxiv.org/abs//2312.09390 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www....

This paper explores the use of token-level self-evaluation to improve the accuracy and quality of generated content by large language models. Experimental results show that self-evaluation based scores are effective in selective generation. https://arxiv.org/abs//2312.09300 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.a...