Arxiv Papers

The paper proposes a new approach called prompt-aligned personalization for creating personalized images using complex textual prompts. The method improves text alignment and can handle intricate prompts, outperforming existing techniques. https://arxiv.org/abs//2401.06105 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.ap...

The paper addresses challenges in Reinforcement Learning from Human Feedback (RLHF) by proposing methods to mitigate incorrect and ambiguous preferences in the dataset and improve model generalization using contrastive learning and meta-learning. Open-source code and datasets are provided. https://arxiv.org/abs//2401.06080 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.co...

The paper investigates whether current safety training techniques can detect and remove deceptive behavior in AI systems. The study finds that backdoored behavior can persist in large language models, even with standard safety training techniques, and adversarial training can hide the unsafe behavior. https://arxiv.org/abs//2401.05566 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://w...

The paper investigates whether current safety training techniques can detect and remove deceptive behavior in AI systems. The study finds that backdoored behavior can persist in large language models, even with standard safety training techniques, and adversarial training can hide the unsafe behavior. https://arxiv.org/abs//2401.05566 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://w...

The paper introduces a framework called Patchscopes that leverages large language models to explain their internal representations in natural language, addressing shortcomings of prior interpretability methods and opening up new possibilities and applications. https://arxiv.org/abs//2401.06102 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts...

AMIE, an AI system optimized for diagnostic dialogue, outperformed primary care physicians in a study of text-based consultations, demonstrating greater diagnostic accuracy and superior performance on multiple axes. https://arxiv.org/abs//2401.05654 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv...

AMIE, an AI system optimized for diagnostic dialogue, outperformed primary care physicians in a study of text-based consultations, demonstrating greater diagnostic accuracy and superior performance on multiple axes. https://arxiv.org/abs//2401.05654 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv...

This paper shows that decoder-only transformers can be seen as infinite multi-state RNNs, and pretrained transformers can be converted into finite multi-state RNNs. The proposed TOVA policy outperforms other policies in long-range tasks. https://arxiv.org/abs//2401.06104 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.appl...

This paper shows that decoder-only transformers can be seen as infinite multi-state RNNs, and pretrained transformers can be converted into finite multi-state RNNs. The proposed TOVA policy outperforms other policies in long-range tasks. https://arxiv.org/abs//2401.06104 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.appl...

The paper introduces PIXART-, a text-to-image synthesis framework that integrates the Latent Consistency Model (LCM) and ControlNet. PIXART- generates high-quality images of 1024px resolution with efficient training and fast inference speed. It offers fine-grained control over image generation and is a promising alternative to other models in the field. https://arxiv.org/abs//2401.05252 YouTube: h...

The paper introduces PIXART-, a text-to-image synthesis framework that integrates the Latent Consistency Model (LCM) and ControlNet. PIXART- generates high-quality images of 1024px resolution with efficient training and fast inference speed. It offers fine-grained control over image generation and is a promising alternative to other models in the field. https://arxiv.org/abs//2401.05252 YouTube: h...

Lengthening the reasoning steps in prompts improves the reasoning abilities of large language models (LLMs), while shortening the steps diminishes their abilities. Incorrect rationales can still yield favorable outcomes if they maintain the required length of inference. The advantages of increasing reasoning steps are task-dependent. https://arxiv.org/abs//2401.04925 YouTube: https://www.youtube.c...

Lengthening the reasoning steps in prompts improves the reasoning abilities of large language models (LLMs), while shortening the steps diminishes their abilities. Incorrect rationales can still yield favorable outcomes if they maintain the required length of inference. The advantages of increasing reasoning steps are task-dependent. https://arxiv.org/abs//2401.04925 YouTube: https://www.youtube.c...

This paper introduces Lightning Attention-2, the first linear attention implementation that overcomes the issue with cumulative summation, enabling linear attention to realize its theoretical computational benefits. It achieves this through a tiling technique and is significantly faster than other attention mechanisms. https://arxiv.org/abs//2401.04658 YouTube: https://www.youtube.com/@ArxivPapers...

This paper introduces Lightning Attention-2, the first linear attention implementation that overcomes the issue with cumulative summation, enabling linear attention to realize its theoretical computational benefits. It achieves this through a tiling technique and is significantly faster than other attention mechanisms. https://arxiv.org/abs//2401.04658 YouTube: https://www.youtube.com/@ArxivPapers...

This study explores whether combining smaller conversational AI models can achieve performance comparable to larger models. The results suggest that blending multiple models can potentially outperform or match the capabilities of larger models without increased computational demands. https://arxiv.org/abs//2401.02994 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arx...

This study explores whether combining smaller conversational AI models can achieve performance comparable to larger models. The results suggest that blending multiple models can potentially outperform or match the capabilities of larger models without increased computational demands. https://arxiv.org/abs//2401.02994 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arx...

Activation Beacon is a plug-and-play module for large language models that allows them to process longer contexts with a limited context window, while preserving their original capabilities. It achieves competitive memory and time efficiency and can be trained efficiently with short-sequence data. Experimental results show improved performance on long-context language modeling and understanding ta...

Mixtral 8x7B is a Sparse Mixture of Experts (SMoE) language model that outperforms other models on various benchmarks, including mathematics, code generation, and multilingual tasks. It also introduces Mixtral 8x7B - Instruct, a model that surpasses several other models on human benchmarks. Both models are available under the Apache 2.0 license. https://arxiv.org/abs//2401.04088 YouTube: https://w...

The paper proposes combining State Space Models (SSMs) with Mixture of Experts (MoE) to unlock the scaling potential of SSMs. The resulting model, MoE-Mamba, outperforms both Mamba and Transformer-MoE in terms of performance and training steps. https://arxiv.org/abs//2401.04081 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcas...

This paper introduces two scaled-down variants of the Stable Diffusion XL (SDXL) text-to-image model, achieved through progressive removal of layers and losses. These models effectively emulate the original SDXL while reducing parameters and latency, making them more accessible for deployment in resource-constrained environments. https://arxiv.org/abs//2401.02677 YouTube: https://www.youtube.com/@...

This paper introduces two scaled-down variants of the Stable Diffusion XL (SDXL) text-to-image model, achieved through progressive removal of layers and losses. These models effectively emulate the original SDXL while reducing parameters and latency, making them more accessible for deployment in resource-constrained environments. https://arxiv.org/abs//2401.02677 YouTube: https://www.youtube.com/@...

This paper introduces DistAttention, a distributed attention algorithm, and DistKV-LLM, a distributed LLM serving system, to improve the performance and resource management of cloud-based LLM services. The system achieved significant throughput improvements and supported longer context lengths compared to existing systems. https://arxiv.org/abs//2401.02669 YouTube: https://www.youtube.com/@ArxivPa...

This paper introduces DistAttention, a distributed attention algorithm, and DistKV-LLM, a distributed LLM serving system, to improve the performance and resource management of cloud-based LLM services. The system achieved significant throughput improvements and supported longer context lengths compared to existing systems. https://arxiv.org/abs//2401.02669 YouTube: https://www.youtube.com/@ArxivPa...

This paper introduces the concept of Relative Creativity to evaluate the creative potential of AI models. It proposes Statistical Creativity as a quantifiable measure and provides a training guideline for fostering creativity in AI. https://arxiv.org/abs//2401.01623 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com...