Kaileigh Stopa

AI is good at things analytics is bad at, and vice versa? I recanted a recent AI finding to Matt Regan today over lunch today. I passed Claude Code a delimited file and asked it to "tell me how column H differs when column J has a value or not." Column H was an extended email exchange between two parties. Column J was more or less a success flag. I more specifically asked it to tokenize the emails, n-gram the results, and find good positive prediction indicators. I was at least smart enough to tell it to exclude reply text (repetitive). The results were wholly uninteresting. I asked it to try some more filtering methods (TF-IDF, etc.). Still, the results were lame. It seemed to be too focused on success indication and the top 100 results were largely unusable (names, generic business jargon, etc.). I decided to change tack ... "Tell me what you find in the email exchanges where the success flag was on." Now, here we go. I got some clear and interesting findings, with stats to back them up. I'd actually studied this data myself beforehand and could verify (for the most part). I kept mulling it over the weekend, and come this morning I decided to push a bit further. "Is there anything particular about the first few emails in the chain that indicates a higher possibility of the success flag and vice versa?" All of a sudden I had a half dozen good points on each side. What led here? Getting out of the way. I'd probably still be sparring with Claude, trying to get decent lists of positively and negatively correlated n-grams, if I hadn't stepped back. I'm super happy I rolled with the AI tide rather than swimming against it.

10

AI is only as good as the context you give it. I've been refining a Tableau chatbot, and my biggest takeaway has been the power of Context Engineering. LLMs have a finite canvas. If you don't manage that space effectively, then you run into issues like context drift and degraded performance. To solve this, I’ve focused on learning ways to maximize the efficiency of the initial tokens passed so no critical insights are lost in the noise. As a first step I have been learning about context engineering in the form of role based tailoring. I have enhanced the chatbot to recognize whether the user is a Viewer, a Manager, or an Executive. - Smart Data Usage: We make the best use of the initial context window to prevent information loss. - Token Efficiency: We aren't just "dumping" data; we are curating the most relevant facts to keep the output sharp and cost effective. - Tailored Insights: The output matches the user's specific expectations and goals. I can now fine-tune these personas in minutes, which historically would have taken days or weeks. The speed at which we can now iterate is a total game changer!

15