Keshav Ramji

This paper explains the quiet reason LLMs feel smart right up until they miss the point. The issue isn’t missing knowledge. It’s missing intent. Even when all the relevant information is right there, models often misread what the user is actually trying to do. The authors show that LLMs are excellent at mapping text to plausible replies, but much weaker at inferring goals. Two prompts can look nearly identical on the surface while implying very different outcomes. Humans pick up that difference instantly. Models often don’t. The paper draws a clear boundary between understanding context and understanding intent. Context is what’s written: facts, constraints, instructions. Intent lives underneath that: priorities, tradeoffs, what matters most when things conflict. Today’s models optimize for surface alignment, not goal reconstruction. One experiment makes the failure obvious. Users asked questions that could be read as exploratory or decision-oriented. Models answered confidently but picked the wrong mode far too often. They explained when a recommendation was expected, or gave a hard answer when the user was clearly still thinking. The facts were right. The response missed the point. Another pattern shows up in instruction following. When users expect the model to fill gaps or challenge assumptions, the model treats the prompt as a complete specification. The output looks obedient but unhelpful. This isn’t hallucination. It’s misaligned helpfulness. Paraphrasing makes the issue clearer. When the same intent is phrased differently, model behavior shifts noticeably. That tells us the model is reacting to wording, not reconstructing an underlying goal. Humans normalize phrasing. Models anchor to it. More context doesn’t fix this. In many cases, it makes things worse. Extra background pushes the model toward local relevance instead of global purpose. Longer answers create the illusion of understanding while diluting what the user actually wants. The paper argues this won’t be solved by larger context windows or better prompts alone. Intent is rarely stated outright. It has to be inferred, tracked, and sometimes revised mid-conversation. That requires reasoning about the user, not just the text. The implication for agents is uncomfortable. If a system can’t reliably infer intent, autonomy becomes risky. Tool use amplifies mistakes. Confident execution based on the wrong goal is worse than slowing down to ask a clarifying question. The authors suggest treating intent as a first-class object: something to model, update, and verify explicitly. Not just what was said, but what outcome is being optimized. Read the full paper: https://lnkd.in/gF5aSZJB I share AI updates here, but I build the tools at getoutbox.ai the fastest way to create your own AI voice agent without code. Join our Skool community to learn, share, and get early access to AI voice strategies → https://lnkd.in/gH-vxvf5

8

Something I kept thinking about while working on RL for LLMs Reward hacking isn't the model being sneaky. It's the model doing exactly what you told it to do. That distinction matters a lot. If your reward signal is "humans prefer this response", the model will learn to sound confident, be agreeable, and write longer answers. Not because those things are correct, but because raters happened to like them. You didn't get a smarter model. You got a model that's better at looking smart. The tricky part is your reward model is just an approximation of what you actually want. And any approximation has blind spots. The policy WILL find them, not intentionally, just through optimization pressure over thousands of steps. This is partly why process reward models are interesting. Instead of just scoring the final answer, you score the reasoning steps. Harder to game a process than an output. But even that's not a complete fix. A model can learn to write steps that look reasonable and still get to the wrong answer. I think the real lesson is: reward hacking is a specification problem, not a model problem. The model is doing its job. We just haven't defined the job precisely enough yet. Still one of the most interesting open problems in the space.

18

Stop calling it "AI Engineering" if you only know how to prompt an LLM. (Harsh? Maybe. True? Absolutely.) I’m currently exploring LLM and data science I spend my days dealing with Abstract Algebra, Probability, and Calculus. I spend my nights researching LLM workflows, RAG pipelines, and tokenization strategies. Sitting at the intersection of Math and Code, I’ve noticed a dangerous trend: The market is flooded with people building "wrappers". But looking at my network, 17% of you are Founders and Co-Founders. And I know you aren't looking for wrapper-builders. You are looking for: Engineers who understand the 'Why', not just the 'How'. Researchers who can optimize latency, not just import a library. Talent that can read a research paper, not just a documentation page. Frameworks change weekly. The hype cycle resets monthly. But the Math? The Math stays the same. If you are a student or early in your career like me, here is my advice: Don't just learn the tool. Learn the logic behind the tool. We are here to build the future, not just use it. To the founders and tech leaders here are you hiring for depth (Math/Logic) or width (Tools/APIs) in 2026? Let’s argue in the comments. #ArtificialIntelligence #Mathematics #GenZ #StartupIndia #MachineLearning #DeepTech

53

19 Comments

Found a course by my home school, CMU, called Machine Learning in Production This course bridges the gap between AI research and software engineering, focusing entirely on the messy reality of production. It covers the entire lifecycle from a prototype ML model to an entire system deployed in production. CMU has open-sourced essentially everything, and the companion MIT Press textbook is completely free to read online. Thank you Carnegie Mellon University . I have new stuff to learn now. Book: https://lnkd.in/gCw86h6r Course website: https://lnkd.in/gCZa7CR7

130

5 Comments