Eye on AI Weekly Research Watch

Training robots via reinforcement learning is expensive because each physical trial is costly and only reveals one outcome path. WorldSample addresses this by combining real robot rollouts with a learned "world model" that generates additional high-fidelity synthetic experience, reducing the need for physical trials. A technique called Policy-Paced Learning carefully schedules how synthetic data i...

Grading command-line exams at scale is difficult because rule-based autograders can't handle partial credit or syntactic variation, while manual grading doesn't scale with rising enrollments. This study tests whether four frontier LLMs (GPT, Claude Opus, Gemini, GLM) can approximate expert human grading of Linux/bash responses, using a four-level cognitive taxonomy from basic file operations to ad...

This observational study challenges the assumption that giving coding AI agents more tools (like browser testing) automatically improves output quality. Across ninety independent runs building the same application, the dominant factor in performance was model capability tier and reasoning effort—not extra tools, which raised costs without improving reliability. Notably, increasing reasoning effort...

Autonomous AI agents are expected to iteratively improve executable policies through feedback, but existing evaluations often reduce this complex process to a single final score, obscuring how the improvement actually happens. EvoPolicyGym introduces a controlled setting where an agent repeatedly edits a policy within compact interactive reinforcement learning environments under a fixed budget. Be...

Improving specialized-domain LLMs typically requires costly human-labeled data or real-world feedback, which is often unavailable. This paper proposes Neuron-OPSD, a method that uses the model's own internal neuron activation patterns—rather than external labels—to select useful training data and build a "teacher" context for self-distillation via majority-vote pseudo-labels. Compared to existing...

Diffusion transformers producing state-of-the-art images and videos are computationally expensive, and standard quantization techniques for compressing them must be re-calibrated for every new model or modality since activation patterns shift constantly. OrbitQuant solves this by rotating activations into a normalized basis where their statistical distribution becomes fixed and predictable, enabli...

Vision-Language-Action (VLA) models for robotics are bottlenecked by scarce, costly expert demonstrations that combine observations, instructions, and actions together. This paper argues that physical competence (how to move) and semantic task understanding (what to do) can be learned separately, with only the latter needing costly labeled data. Their proposed Task-Agnostic Pretraining (TAP) first...

This study investigates what actually determines whether pairing a human with an AI model improves forecasting accuracy, using real-money prediction markets on Polymarket as an objective benchmark. Rather than a single average effect, the results reveal three distinct behavioral patterns: some people simply defer to the AI, others misuse it to confirm their own biases (performing worse than the AI...

As code evolves, its test suite must evolve alongside it, but existing benchmarks for evaluating AI coding agents often treat tests and code changes in isolation using unverified static metadata. TestEvo-Bench fixes this by mining real commit histories across 152 open-source Java projects, creating executable tasks for both generating new tests and updating existing ones, evaluated with concrete m...

Vision-language models (VLMs) process images as many small "visual tokens," and pruning redundant ones speeds up inference—but existing pruning methods often discard important details when instructions are dense or fine-grained. This paper identifies two causes: noisy textual signals corrupting relevance scoring, and fragmented feature selection. Their proposed method, EADP, uses statistical entro...

Classical symbolic solvers excel at guaranteeing correct solutions to constraint satisfaction problems but can be slow on large search spaces, while neural models are fast but not always reliable. G-RRM combines both by using symbol-equivariant recurrent reasoning models to generate solution proposals that guide symbolic solvers like backtracking algorithms and SAT solvers. The paper finds that ne...

Machine learning interatomic potentials (MLIPs) are increasingly used to simulate molecular and material behavior for scientific discovery, yet the optimizers used to train them have remained an overlooked design choice, with most researchers defaulting to Adam. This paper systematically benchmarks newer matrix-structured optimizers—Muon, SOAP, and a SOAP-Muon hybrid—on training NequIP and Allegro...

Self-distillation—where a single LLM acts as both teacher and student—is a practical way to train reasoning models, but it risks "privileged information leakage," where the student learns shortcuts based on information it won't have at test time. DemoPSD addresses this by steering the student toward a balanced target between its own distribution and the teacher's, dynamically adjusting the blend b...

Following complex storylines in long-form TV dramas requires accurately identifying who is speaking each line, a task complicated by multiple characters, unclear audio, and short utterances where voice alone is unreliable. This paper introduces DramaSR-532K, a massive benchmark of 532,000 annotated dialogue lines across 900+ characters, and DramaSR-LRM, a reasoning-based model that combines audio,...

As LLM agents increasingly operate in socially structured environments—with roles, audiences, and reputational stakes—this paper asks whether that social context causes agents to say different things publicly than they privately "believe." Using a dual-channel debate setup with public statements and hidden off-the-record responses, the researchers find dramatic divergence (rising to ~40%) when soc...

Modern LLMs can technically process very long documents, but they often fail to actually use relevant details buried within them—a gap between having access to information and effectively reasoning over it. ReContext tackles this with a training-free method that uses the model's own internal attention signals to identify and "replay" the most relevant evidence before generating a final answer, wit...

Even after alignment training, deployed LLMs can still produce unsafe outputs, making real-time monitoring essential for catching failures as they happen. This paper studies a straightforward monitoring approach: taking a safety score from an external verifier model and triggering an alarm once it crosses a calibrated threshold. Tested on mathematical reasoning and red-teaming datasets, this simpl...

Many practical programming tasks—like flagging important log entries or fixing malformed JSON—don't fit clean rules but are usually solved by expensive, non-reproducible calls to large LLM APIs. This paper proposes compiling such "fuzzy" natural-language specifications into small, locally-run neural adapters instead. Using a compact 4B compiler and a lightweight 0.6B interpreter, the resulting pro...

When companies need to remove sensitive personal data memorized by an LLM, current "unlearning" techniques are typically judged only by whether the model stops outputting the information—not whether the underlying knowledge is actually erased from its parameters. This creates risk, since obfuscated knowledge can often be recovered through resurfacing attacks. LACUNA addresses this by embedding syn...

As AI coding agents gain autonomy and work across persistent codebases over time, a troubling vulnerability emerges: a compromised agent could spread a malicious "side task" across many pull requests rather than concentrating it in one, making detection harder. This paper introduces "Iterative VibeCoding," a benchmark simulating this exact scenario across CLI tools and Flask services. The authors...

Financial fraud detection in transaction networks faces a fundamental challenge: fraudulent activity is rare, well-disguised, and often underrepresented in labeled data. Standard graph neural networks tend to smooth out the very irregularities that signal fraud. ADC-GNN tackles this with three complementary mechanisms: diffusion-guided feature augmentation that stabilizes node representations agai...

In-context segmentation asks a model to identify target regions in new images using only a handful of labeled reference examples — no retraining required. Current approaches work by matching low-level visual features between references and queries, making them brittle when references vary in viewpoint, lighting, or appearance. CG-ICS instead extracts high-level semantic concepts from references us...

Jailbreak attacks — prompts engineered to make safety-aligned LLMs produce harmful outputs — are a persistent concern, but exactly how they work mechanistically has remained murky. This paper provides evidence that successful attacks don't erase safety representations; they selectively suppress specific "Adversarially Compromised Heads" in early attention layers while leaving "Safety-Aligned Heads...

Reinforcement learning has dramatically improved LLM reasoning on tasks like competition math — but the resulting models often reason in ways that are difficult for weaker models or humans to follow, limiting their real-world utility. Tandem Reinforcement Learning (TRL) addresses this by co-training a strong "senior" model alongside a frozen "junior" model: both contribute to generating reasoning...

Large-scale studies linking heart imaging measurements to disease risk typically rely on pre-defined, single-variable features chosen by experts — an approach that may miss important non-linear relationships or interactions between measurements. CPAgents automates the discovery of richer, composite phenotypes (ratios, polynomial combinations, interaction terms) through a three-agent loop: an Analy...