Arxiv Papers

This paper explores grokking in deep learning, linking delayed generalization to Softmax Collapse and proposing solutions to enable grokking without regularization through new activation functions and training algorithms. https://arxiv.org/abs//2501.04697 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast...

This paper explores grokking in deep learning, linking delayed generalization to Softmax Collapse and proposing solutions to enable grokking without regularization through new activation functions and training algorithms. https://arxiv.org/abs//2501.04697 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast...

ComMer is a framework that efficiently personalizes Large Language Models by compressing user documents into compact representations, improving performance in skill learning tasks while facing challenges in knowledge-intensive applications. https://arxiv.org/abs//2501.03276 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.a...

ComMer is a framework that efficiently personalizes Large Language Models by compressing user documents into compact representations, improving performance in skill learning tasks while facing challenges in knowledge-intensive applications. https://arxiv.org/abs//2501.03276 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.a...

This paper addresses privacy concerns in proprietary language models by optimizing transformer architectures for private inference, focusing on the role of nonlinearities and introducing entropy-guided mechanisms for improved performance. https://arxiv.org/abs//2501.03489 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.app...

This paper addresses privacy concerns in proprietary language models by optimizing transformer architectures for private inference, focusing on the role of nonlinearities and introducing entropy-guided mechanisms for improved performance. https://arxiv.org/abs//2501.03489 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.app...

The paper explores using mechanistic interpretability to enhance gradient descent training in AI, aiming to reduce compute costs and mitigate harmful behaviors through efficient learning curricula. https://arxiv.org/abs//2501.02362 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id16924760...

The paper explores using mechanistic interpretability to enhance gradient descent training in AI, aiming to reduce compute costs and mitigate harmful behaviors through efficient learning curricula. https://arxiv.org/abs//2501.02362 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id16924760...

This study evaluates LLMs' language understanding using novel tasks from deterministic finite automata, revealing they struggle compared to basic models when faced with unfamiliar languages. https://arxiv.org/abs//2501.02825 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spot...

This study evaluates LLMs' language understanding using novel tasks from deterministic finite automata, revealing they struggle compared to basic models when faced with unfamiliar languages. https://arxiv.org/abs//2501.02825 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spot...

This study enhances large language models' reasoning abilities using Monte Carlo Tree Search for process supervision, significantly improving performance on mathematical reasoning tasks and demonstrating transferability across datasets. https://arxiv.org/abs//2501.01478 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple...

This study enhances large language models' reasoning abilities using Monte Carlo Tree Search for process supervision, significantly improving performance on mathematical reasoning tasks and demonstrating transferability across datasets. https://arxiv.org/abs//2501.01478 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple...

This paper presents a method to predict large language model errors using black-box feature extraction, outperforming traditional approaches and enabling nuanced evaluations of model performance and architecture. https://arxiv.org/abs//2501.01558 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-pa...

This paper presents a method to predict large language model errors using black-box feature extraction, outperforming traditional approaches and enabling nuanced evaluations of model performance and architecture. https://arxiv.org/abs//2501.01558 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-pa...

The paper introduces JOG3R, a unified model that enhances video generation and camera pose estimation, demonstrating improved accuracy in 3D awareness through fine-tuning and feature repurposing. https://arxiv.org/abs//2501.01409 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016...

The paper introduces JOG3R, a unified model that enhances video generation and camera pose estimation, demonstrating improved accuracy in 3D awareness through fine-tuning and feature repurposing. https://arxiv.org/abs//2501.01409 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016...

The paper presents a predictive theory of creativity in convolutional diffusion models, identifying inductive biases that enable novel image generation beyond training data through local patch combinations. https://arxiv.org/abs//2412.20292 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/i...

The paper presents a predictive theory of creativity in convolutional diffusion models, identifying inductive biases that enable novel image generation beyond training data through local patch combinations. https://arxiv.org/abs//2412.20292 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/i...

https://arxiv.org/abs//2501.00655 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers

https://arxiv.org/abs//2501.00655 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers

The paper introduces Titans, a new architecture combining neural long-term memory and attention, outperforming Transformers in various tasks while efficiently handling larger context windows. https://arxiv.org/abs//2501.00663 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spo...

The paper introduces Titans, a new architecture combining neural long-term memory and attention, outperforming Transformers in various tasks while efficiently handling larger context windows. https://arxiv.org/abs//2501.00663 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spo...

This paper proposes adaptive batch size schedules for large-scale language model training, enhancing efficiency and generalization, while outperforming traditional methods in pretraining models, particularly smaller ones. https://arxiv.org/abs//2412.21124 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast...

This paper proposes adaptive batch size schedules for large-scale language model training, enhancing efficiency and generalization, while outperforming traditional methods in pretraining models, particularly smaller ones. https://arxiv.org/abs//2412.21124 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast...

The paper introduces Functional Risk Minimization (FRM), a framework improving performance in various learning tasks by comparing functions instead of outputs, enhancing understanding of generalization in over-parameterized models. https://arxiv.org/abs//2412.21149 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/...