Johann L.

What we often chalk up to “expected randomness” in LLMs turns out to be partly a byproduct of infrastructure choices—batch sizes, request load, how attention kernels are implemented. The idea of batch invariance stood out. If your kernel changes reduction order when batch size or KV cache state changes, then your “same prompt” no longer means “same output.” For applications that depend on reproducibility, audit trails, or consistency (e.g. RL training, regulated domains), that’s not just academic. What do you think: for your stack, how big is the gap between “probabilistic slack acceptable” vs “exact repeatability required”? Are we shifting toward expecting determinism even in deployed systems? https://lnkd.in/gbKTt9-g #AIEngineering #Reproducibility #LLMInference

12

New to the PyTorch Ecosystem Landscape: Kubetorch. Kubetorch enables ML research and development on Kubernetes across training, inference, RL, evals, data processing, and more, in a simple and unopinionated package. Learn more: https://lnkd.in/gXtGfq44 #PyTorch #Kubernetes #MLOps #AIInfrastructure

330

2 Comments

Stanford and AWS researchers released a new playbook for smarter RL training and it’s a game-changer for reasoning models. Reinforcement learning with verifiable rewards (RLVR) has powered breakthroughs in reasoning, but it’s hitting a wall, training longer no longer makes models think smarter. DeepSearch, a new framework from Stanford, University of Tokyo, and AWS, takes a bold step forward: instead of just scaling training depth, it scales training breadth. Their insight? Exploration, not brute force, is the real bottleneck. Traditional RLVR models rely on shallow rollouts and hope the model stumbles upon good reasoning paths. DeepSearch changes that by embedding Monte Carlo Tree Search (MCTS) directly into the training loop, not just at inference time. Here’s how DeepSearch redefines RLVR: - Global Frontier Selection: Prioritizes the most promising reasoning paths across the entire search tree instead of following rigid root-to-leaf traversals. - Entropy-Guided Supervision: Identifies confidently wrong reasoning paths, the ones models are sure about but still get wrong, to improve feedback and calibration. - Adaptive Replay Buffers: Stores verified solutions and hard examples, ensuring the model learns efficiently without forgetting prior reasoning steps. - Tree-GRPO Objective: A fine-grained reward mechanism that gives credit to intermediate reasoning steps, not just final answers. The results? DeepSearch-1.5B sets a new SOTA on reasoning benchmarks (62.95%), outperforming Nemotron-Research-Reasoning-Qwen-1.5B v2 and doing it 5.7× faster. What this means for AI research is profound: Instead of throwing more GPUs and steps at the problem, DeepSearch shows that structured exploration and verifiable feedback can push reasoning models forward, smarter, not harder. A must-read for anyone working on RL for reasoning, test-time search, or scaling agentic intelligence. #DeepSearch #RLVR #ReinforcementLearning #AIResearch #ReasoningModels #AgenticAI #ArtificialIntelligence #MonteCarloTreeSearch #AIInnovation #LLMReasoning #ScientificAI #AITraining #AIMethods #FutureOfAI #StanfordAI

3