Daily Paper Cast

🤗 Upvotes: 122 | cs. CL, cs. AI, cs. LG Authors: Deepak Nathani, Lovish Madaan, Nicholas Roberts, Nikolay Bashlykov, Ajay Menon, Vincent Moens, Amar Budhiraja, Despoina Magka, Vladislav Vorotilov, Gaurav Chaurasia, Dieuwke Hupkes, Ricardo Silveira Cabral, Tatiana Shavrina, Jakob Foerster, Yoram Bachrach, William Yang Wang, Roberta Raileanu Title: MLGym: A New Framework and Benchmark for Advancing...

🤗 Upvotes: 82 | cs. CV, cs. AI Authors: Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hénaff, Jeremiah Harmsen, Andreas Steiner, Xiaohua Zhai Title: SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features Arx...

🤗 Upvotes: 81 | cs. CL Authors: M-A-P Team, Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, Kang Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, Chujie Zheng, Kaixing Deng, Shuyue Guo, Shian Jia, Sichao Jiang, Yiyan Liao, Rui Li, Qinrui Li, Sirun Li, Yizhi Li, Yunwen Li, Dehua Ma, Yuansheng Ni, Haoran Que, Qiyao Wang, Zhoufutu Wen, Siwei Wu, Tianshun Xing, Ming Xu, Zhenzhu...

🤗 Upvotes: 51 | cs. CL Authors: Sergey Pletenev, Maria Marina, Daniil Moskovskiy, Vasily Konovalov, Pavel Braslavski, Alexander Panchenko, Mikhail Salnikov Title: How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM? Arxiv: http://arxiv.org/abs/2502.14502v1 Abstract: The performance of Large Language Models (LLMs) on many tasks is greatly limited by the knowledge learned during...

🤗 Upvotes: 39 | cs. LG, cs. AI Authors: Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph E. Gonzalez, Ion Stoica Title: S*: Test Time Scaling for Code Generation Arxiv: http://arxiv.org/abs/2502.14382v1 Abstract: Increasing test-time compute for LLMs shows promise across domains but remains underexplored in code generation, despite extensive study in...

🤗 Upvotes: 24 | cs. CL, cs. AI Authors: Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, Chong Luo Title: Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning Arxiv: http://arxiv.org/abs/2502.14768v1 Abstract: Inspired by the success of DeepSeek-R1, we explore the potential of rule-based reinforcement learning (RL) in lar...

🤗 Upvotes: 22 | quant-ph, cs. AI, cs. IT, cs. LG, math. IT Authors: Austin Yubo He, Zi-Wen Liu Title: Discovering highly efficient low-weight quantum error-correcting codes with reinforcement learning Arxiv: http://arxiv.org/abs/2502.14372v1 Abstract: The realization of scalable fault-tolerant quantum computing is expected to hinge on quantum error-correcting codes. In the quest for more efficien...

🤗 Upvotes: 20 | cs. CV, cs. AI, cs. CL Authors: Shangqing Tu, Yucheng Wang, Daniel Zhang-Li, Yushi Bai, Jifan Yu, Yuhao Wu, Lei Hou, Huiqin Liu, Zhiyuan Liu, Bin Xu, Juanzi Li Title: LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models Arxiv: http://arxiv.org/abs/2502.14834v1 Abstract: Existing Large Vision-Language Models (LVLMs) can process inputs with contex...

🤗 Upvotes: 18 | cs. CL, cs. AI Authors: Yein Park, Chanwoong Yoon, Jungwoo Park, Minbyul Jeong, Jaewoo Kang Title: Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Information Arxiv: http://arxiv.org/abs/2502.14258v1 Abstract: While the ability of language models to elicit facts has been widely investigated, how they handle temporally changing facts remains und...

🤗 Upvotes: 15 | cs. CL, cs. LG Authors: Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, Bang Zhang, Xin Zhou, Nan Du, Jia Li Title: S$^2$R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning Arxiv: http://arxiv.org/abs/2502.12853v1 Abstract: Recent studies have demonstrated the effectiveness of LLM test-time scaling. However, existing approaches to incentivize...

🤗 Upvotes: 97 | cs. CV, cs. CL Authors: Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, Junyang Lin Title: Qwen2.5-VL Technical Report Ar...

🤗 Upvotes: 31 | cs. CV, cs. RO Authors: Hao Gao, Shaoyu Chen, Bo Jiang, Bencheng Liao, Yiang Shi, Xiaoyang Guo, Yuechuan Pu, Haoran Yin, Xiangyu Li, Xinbang Zhang, Ying Zhang, Wenyu Liu, Qian Zhang, Xinggang Wang Title: RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning Arxiv: http://arxiv.org/abs/2502.13144v1 Abstract: Existing end-to-end autonomous driv...

🤗 Upvotes: 28 | cs. SD, cs. AI Authors: Zihan Liu, Shuangrui Ding, Zhixiong Zhang, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, Jiaqi Wang Title: SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation Arxiv: http://arxiv.org/abs/2502.13128v1 Abstract: Text-to-song generation, the task of creating vocals and accompaniment from textual inputs, poses significa...

🤗 Upvotes: 22 | cs. CL, cs. AI, cs. LG Authors: Jusen Du, Weigao Sun, Disen Lan, Jiaxi Hu, Yu Cheng Title: MoM: Linear Sequence Modeling with Mixture-of-Memories Arxiv: http://arxiv.org/abs/2502.13685v1 Abstract: Linear sequence modeling methods, such as linear attention, state space modeling, and linear RNNs, offer significant efficiency improvements by reducing the complexity of training and in...

🤗 Upvotes: 22 | cs. CL Authors: William Jurayj, Jeffrey Cheng, Benjamin Van Durme Title: Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answering Arxiv: http://arxiv.org/abs/2502.13962v1 Abstract: Scaling the test-time compute of large language models has demonstrated impressive performance on reasoning benchmarks. However, existing evaluations of test-time scaling make...

🤗 Upvotes: 21 | cs. CL Authors: Shi Yu, Zhiyuan Liu, Chenyan Xiong Title: Craw4LLM: Efficient Web Crawling for LLM Pretraining Arxiv: http://arxiv.org/abs/2502.13347v1 Abstract: Web crawl is a main source of large language models' (LLMs) pretraining data, but the majority of crawled web pages are discarded in pretraining due to low data quality. This paper presents Crawl4LLM, an efficient web cra...

🤗 Upvotes: 19 | cs. CL, cs. LG Authors: Guanzheng Chen, Xin Li, Michael Qizhe Shieh, Lidong Bing Title: LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization Arxiv: http://arxiv.org/abs/2502.13922v2 Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities through pretraining and alignment. However, superior short-context...

🤗 Upvotes: 17 | cs. AI Authors: Yuetai Li, Xiang Yue, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ramasubramanian, Radha Poovendran Title: Small Models Struggle to Learn from Strong Reasoners Arxiv: http://arxiv.org/abs/2502.12143v1 Abstract: Large language models (LLMs) excel in complex reasoning tasks, and distilling their reasoning capabilities into smaller models has sho...

🤗 Upvotes: 15 | cs. LG, cs. AI, cs. DC Authors: Michael Luo, Xiaoxiang Shi, Colin Cai, Tianjun Zhang, Justin Wong, Yichuan Wang, Chi Wang, Yanping Huang, Zhifeng Chen, Joseph E. Gonzalez, Ion Stoica Title: Autellix: An Efficient Serving Engine for LLM Agents as General Programs Arxiv: http://arxiv.org/abs/2502.13965v1 Abstract: Large language model (LLM) applications are evolving beyond simple ch...

🤗 Upvotes: 10 | cs. CL, cs. AI, cs. IR, cs. IT, math. IT Authors: Yucheng Shi, Tianze Yang, Canyu Chen, Quanzheng Li, Tianming Liu, Xiang Li, Ninghao Liu Title: SearchRAG: Can Search Engines Be Helpful for LLM-based Medical Question Answering? Arxiv: http://arxiv.org/abs/2502.13233v1 Abstract: Large Language Models (LLMs) have shown remarkable capabilities in general domains but often struggle wi...

🤗 Upvotes: 65 | cs. CL, cs. AI, cs. SD Authors: Yuhao Zhang, Zhiheng Liu, Fan Bu, Ruiyu Zhang, Benyou Wang, Haizhou Li Title: Soundwave: Less is More for Speech-Text Alignment in LLMs Arxiv: http://arxiv.org/abs/2502.12900v1 Abstract: Existing end-to-end speech large language models (LLMs) usually rely on large-scale annotated data for training, while data-efficient training has not been discusse...

🤗 Upvotes: 51 | cs. CL, cs. LG Authors: Yuri Kuratov, Mikhail Arkhipov, Aydar Bulatov, Mikhail Burtsev Title: Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity Arxiv: http://arxiv.org/abs/2502.13063v1 Abstract: A range of recent works addresses the problem of compression of sequence of tokens into a shorter sequence of real-valued vectors t...

🤗 Upvotes: 44 | cs. LG Authors: Jaehyeong Jo, Sung Ju Hwang Title: Continuous Diffusion Model for Language Modeling Arxiv: http://arxiv.org/abs/2502.11564v1 Abstract: Diffusion models have emerged as a promising alternative to autoregressive models in modeling discrete categorical data. Yet diffusion models that directly work on discrete data space do not fully exploit the power of iterative refi...

🤗 Upvotes: 42 | cs. CV, cs. AI Authors: Lijie Liu, Tianxiang Ma, Bingchuan Li, Zhuowei Chen, Jiawei Liu, Qian He, Xinglong Wu Title: Phantom: Subject-consistent video generation via cross-modal alignment Arxiv: http://arxiv.org/abs/2502.11079v1 Abstract: The continuous development of foundational models for video generation is evolving into various applications, with subject-consistent video gene...

🤗 Upvotes: 33 | cs. AI, cs. CL Authors: Feng Luo, Rui Yang, Hao Sun, Chunyuan Deng, Jiarui Yao, Jingyan Shen, Huan Zhang, Hanjie Chen Title: Rethinking Diverse Human Preference Learning through Principal Component Analysis Arxiv: http://arxiv.org/abs/2502.13131v1 Abstract: Understanding human preferences is crucial for improving foundation models and building personalized AI systems. However, pre...