Daily Paper Cast

🤗 Upvotes: 14 | cs. AI, cs. CY, cs. LG Authors: Clément Desroches, Martin Chauvin, Louis Ladan, Caroline Vateau, Simon Gosset, Philippe Cordier Title: Exploring the sustainable scaling of AI dilemma: A projective study of corporations' AI environmental impacts Arxiv: http://arxiv.org/abs/2501.14334v2 Abstract: The rapid growth of artificial intelligence (AI), particularly Large Language Models (L...

🤗 Upvotes: 8 | cs. SE, cs. AI Authors: Aitor Arrieta, Miriam Ugarte, Pablo Valle, José Antonio Parejo, Sergio Segura Title: Early External Safety Testing of OpenAI's o3-mini: Insights from the Pre-Deployment Evaluation Arxiv: http://arxiv.org/abs/2501.17749v1 Abstract: Large Language Models (LLMs) have become an integral part of our daily lives. However, they impose certain risks, including those...

🤗 Upvotes: 8 | cs. CV Authors: Hailong Guo, Bohan Zeng, Yiren Song, Wentao Zhang, Chuang Zhang, Jiaming Liu Title: Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks Arxiv: http://arxiv.org/abs/2501.15891v1 Abstract: Image-based virtual try-on (VTON) aims to generate a virtual try-on result by transferring an input garment onto a target person's image. Howe...

🤗 Upvotes: 6 | cs. CR, cs. AI, cs. CL, cs. LG Authors: Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, Ling Liu Title: Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderation Arxiv: http://arxiv.org/abs/2501.17433v1 Abstract: Recent research shows that Large Language Models (LLMs) are vulnerable to harmful fine-tuning attacks -- models lose their saf...

🤗 Upvotes: 6 | cs. CL, cs. AI Authors: Jenna Russell, Marzena Karpinska, Mohit Iyyer Title: People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text Arxiv: http://arxiv.org/abs/2501.15654v1 Abstract: In this paper, we study how well humans can detect text generated by commercial LLMs (GPT-4o, Claude, o1). We hire annotators to read 300 non-fiction...

🤗 Upvotes: 29 | cs. AI, cs. CV, cs. LG Authors: Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, Yi Ma Title: SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training Arxiv: http://arxiv.org/abs/2501.17161v1 Abstract: Supervised fine-tuning (SFT) and reinforcement learning (RL) are widely used post-trainin...

🤗 Upvotes: 15 | cs. LG, cs. CL Authors: Ruizhe Wang, Yeyun Gong, Xiao Liu, Guoshuai Zhao, Ziyue Yang, Baining Guo, Zhengjun Zha, Peng Cheng Title: Optimizing Large Language Model Training Using FP4 Quantization Arxiv: http://arxiv.org/abs/2501.17116v1 Abstract: The growing computational demands of training large language models (LLMs) necessitate more efficient methods. Quantized training present...

🤗 Upvotes: 11 | cs. CV Authors: Chenguo Lin, Panwang Pan, Bangbang Yang, Zeming Li, Yadong Mu Title: DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation Arxiv: http://arxiv.org/abs/2501.16764v1 Abstract: Recent advancements in 3D content generation from text or a single image struggle with limited high-quality 3D datasets and inconsistency from 2D multi-view gener...

🤗 Upvotes: 10 | cs. CL, cs. LG Authors: Hongzhi Huang, Defa Zhu, Banggu Wu, Yutao Zeng, Ya Wang, Qiyang Min, Xun Zhou Title: Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling Arxiv: http://arxiv.org/abs/2501.16975v1 Abstract: Tokenization is a fundamental component of large language models (LLMs), yet its influence on model scaling and performance is not fully explored. In this pa...

🤗 Upvotes: 10 | cs. LG Authors: Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, Stella Biderman, Adria Garriga-Alonso, Arthur Conmy, Neel Nanda, Jessica Rumbelow, Martin Wattenberg, Nandi Schoots, Joseph Miller, Eric J. Michaud, Stephen Casper, Max Tegmark, William Saunders, David Bau,...

🤗 Upvotes: 5 | cs. LG, cs. AI, cs. CL Authors: J. Pablo Muñoz, Jinjie Yuan, Nilesh Jain Title: Low-Rank Adapters Meet Neural Architecture Search for LLM Compression Arxiv: http://arxiv.org/abs/2501.16372v1 Abstract: The rapid expansion of Large Language Models (LLMs) has posed significant challenges regarding the computational resources required for fine-tuning and deployment. Recent advancements...

🤗 Upvotes: 4 | cs. CL, cs. AI Authors: Sankalp KJ, Ashutosh Kumar, Laxmaan Balaji, Nikunj Kotecha, Vinija Jain, Aman Chadha, Sreyoshi Bhaduri Title: IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding Arxiv: http://arxiv.org/abs/2501.15747v2 Abstract: Known by more than 1.5 billion people in the Indian subcontinent, Indic languages present unique challenge...

🤗 Upvotes: 3 | cs. CL, cs. AI Authors: Thibaud Leteno, Irina Proskurina, Antoine Gourru, Julien Velcin, Charlotte Laclau, Guillaume Metzler, Christophe Gravier Title: Histoires Morales: A French Dataset for Assessing Moral Alignment Arxiv: http://arxiv.org/abs/2501.17117v1 Abstract: Aligning language models with human values is crucial, especially as they become more integrated into everyday life...

🤗 Upvotes: 26 | cs. CL Authors: An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, Junyang Lin, Kai Dang, Kexin Yang, Le Yu, Mei Li, Minmin Sun, Qin Zhu, Rui Men, Tao He, Weijia Xu, Wenbiao Yin, Wenyuan Yu, Xiafei Qiu, Xingzhang Ren, Xinlong Yang, Yong Li, Zhiying Xu, Zipeng Zhang Title: Qwen2.5-1M Technical Report Arx...

🤗 Upvotes: 13 | cs. CL Authors: Lin Yueyu, Li Zhiyuan, Peter Yue, Liu Xiao Title: ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer Arxiv: http://arxiv.org/abs/2501.15570v1 Abstract: As is known, hybrid quadratic and subquadratic attention models in multi-head architectures have surpassed both Transformer and Linear RNN models , with these works prim...

🤗 Upvotes: 13 | cs. LG, cs. AI Authors: Scott Fujimoto, Pierluca D'Oro, Amy Zhang, Yuandong Tian, Michael Rabbat Title: Towards General-Purpose Model-Free Reinforcement Learning Arxiv: http://arxiv.org/abs/2501.16142v1 Abstract: Reinforcement learning (RL) promises a framework for near-universal problem-solving. In practice however, RL algorithms are often tailored to specific benchmarks, relying...

🤗 Upvotes: 11 | cs. SD, cs. CL, eess. AS Authors: Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, Zhizheng Wu Title: Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation Arxiv: http://arxiv.org/abs/2501.15907v1 Abstract: Recent advancements in speec...

🤗 Upvotes: 9 | cs. CV, cs. AI Authors: Chuanyang Zheng Title: iFormer: Integrating ConvNet and Transformer for Mobile Application Arxiv: http://arxiv.org/abs/2501.15369v1 Abstract: We present a new family of mobile hybrid vision networks, called iFormer, with a focus on optimizing latency and accuracy on mobile applications. iFormer effectively integrates the fast local representation capacity of...

🤗 Upvotes: 7 | cs. CV, cs. AI, cs. LG, q-bio. NC Authors: Paul Gavrikov, Jovita Lukasik, Steffen Jung, Robert Geirhos, Bianca Lamm, Muhammad Jehanzeb Mirza, Margret Keuper, Janis Keuper Title: Are Vision Language Models Texture or Shape Biased and Can We Steer Them? Arxiv: http://arxiv.org/abs/2403.09193v1 Abstract: Vision language models (VLMs) have drastically changed the computer vision model...

🤗 Upvotes: 5 | cs. LG Authors: Ryan Ehrlich, Bradley Brown, Jordan Juravsky, Ronald Clark, Christopher Ré, Azalia Mirhoseini Title: CodeMonkeys: Scaling Test-Time Compute for Software Engineering Arxiv: http://arxiv.org/abs/2501.14723v1 Abstract: Scaling test-time compute is a promising axis for improving LLM capabilities. However, test-time compute can be scaled in a variety of ways, and effecti...

🤗 Upvotes: 4 | cs. LG, cs. AI Authors: Samira Abnar, Harshay Shah, Dan Busbridge, Alaaeldin Mohamed Elnouby Ali, Josh Susskind, Vimal Thilak Title: Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models Arxiv: http://arxiv.org/abs/2501.12370v2 Abstract: Scaling the capacity of language models has consistently proven to be a reliable approach for improving pe...

🤗 Upvotes: 33 | cs. LG, cs. AI, cs. CL Authors: Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja, Ryan Kim, Jason Hausenloy, Oliver Zhang, Mantas Mazeika, Daron Anderson, Tung Nguyen, Mobeen Mahmood, Fiona Feng, Steven Y. Feng, Haoran Zhao, Michael Yu, Varun Gangal, Chelsea Zou, Zihan Wang, Jessica P. Wang,...

🤗 Upvotes: 26 | cs. IR, cs. CL Authors: Liang Wang, Haonan Chen, Nan Yang, Xiaolong Huang, Zhicheng Dou, Furu Wei Title: Chain-of-Retrieval Augmented Generation Arxiv: http://arxiv.org/abs/2501.14342v1 Abstract: This paper introduces an approach for training o1-like RAG models that retrieve and reason over relevant information step by step before generating the final answer. Conventional RAG meth...

🤗 Upvotes: 22 | cs. CL, cs. AI Authors: Zicheng Zhang, Xiangyu Zhao, Xinyu Fang, Chunyi Li, Xiaohong Liu, Xiongkuo Min, Haodong Duan, Kai Chen, Guangtao Zhai Title: Redundancy Principles for MLLMs Benchmarks Arxiv: http://arxiv.org/abs/2501.13953v1 Abstract: With the rapid iteration of Multi-modality Large Language Models (MLLMs) and the evolving demands of the field, the number of benchmarks pro...

🤗 Upvotes: 13 | cs. CL, cs. AI, cs. LG Authors: Zhengyang Tang, Ziniu Li, Zhenyang Xiao, Tian Ding, Ruoyu Sun, Benyou Wang, Dayiheng Liu, Fei Huang, Tianyu Liu, Bowen Yu, Junyang Lin Title: RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques Arxiv: http://arxiv.org/abs/2501.14492v1 Abstract: Critiques are important for enhancing the performance of Large Language Models...