Blenheim Chalcot

Shipping GenAI without automated evaluation is flying blind. Over a decade of engineering automation and DevOps pipelines has taught me one absolute truth: relying on manual testing or "hope" isn't a scalable deployment strategy. This is especially true for top-tier LLMs that can still hallucinate, miss user intent, and drift off-brand. To solve this, I recently engineered a 5-Tier Enterprise AI Evaluation Framework that catches model failures before they ever hit production. Here is how you build a real safety net: 1. Retrieval & RAG Validation: Checking context precision and recall to ensure the model pulls the exact right data before generating an answer. 2. Deterministic Rules: Hard-coded safeguards to instantly block known prompt injections, PII leaks, and formatting violations. 3. LLM-as-a-Judge: Semantic scoring to evaluate complex nuances like tone, hidden bias, and intent accuracy. 4. Ground Truth Validation: Automated baseline comparisons to track response trajectories and prevent version regressions. 5. Continuous Red Teaming: Automated adversarial testing to expose jailbreaks and boundary vulnerabilities continuously. The result? Predictable, safe, and truly enterprise-ready AI. #AIEvaluation #AIAutomation #GenAITesting #AIML #GenA

Sat down with Anna Haffner from Reddit, Inc. at Shoptalk Spring. She said something that reframed the whole AI search conversation for me: "Reddit is the fitting room - where people talk about the real real of how a product works for you." The word "Reddit" gets appended to a Google search 250 times every second! Not because people distrust Google, but because people want a human answer at the end of it. That behaviour is already there, at scale, whether a brand has a strategy for it or not. What surprised me most was the measurement story. Anna was direct — Reddit is full funnel now, not just an awareness channel. It's where people go right before they decide to buy something. Measurement partners like Fospha are proving it out, with their State of Retail Commerce 2026 report showing revenue influenced by Reddit grew 257% in 2025 and cost per purchase improving 34% for brands scaling their investment on the platform. https://lnkd.in/eBpjsi-B The other thing she kept coming back to: brands walk in asking about their generative AI strategy, and completely miss 20+ years of real, searchable human conversation about their products that's already sitting there on Reddit. Full interview on #ClickZ → https://lnkd.in/eD-fkibD Interview clips dropping on ClickZ.com soon. https://lnkd.in/eEV5tveA ClickZ's Shoptalk coverage is sponsored by Fospha.com - the Measurement OS for online retail. Teagan Allen Amanda Feldman Lee Arthur Tamara Karsten William Sandford Jamie Bolton Harry Forsyth Snezhina Kashukeeva

One of the most common questions I hear from brands at the moment: "We're already on PMAX and Brand Search - is there really more to get from Google?" There is. Quite a lot more. We ran a program with Google across Q4 2025 with 25 retail eCommerce brands to understand what actually happens when brands invest across Google's full funnel, not just the bottom of it. The results were pretty striking. Key takeaways? 📊 Diversifying your Google mix compounds. Adding one new channel to your Google mix - say, Demand Gen alongside existing PMAX and Search - drove 14% higher ROAS. Adding two channels pushed that to 37%. The gap between brands with a broad mix and those running PMAX alone keeps widening. 💡 Most brands are massively underinvesting in Demand Gen. Brands putting 10–20% of their Google budget into Demand Gen achieved double the ROAS of those at 0–5%. The majority of brands we see are still well below that threshold - which means there's real headroom sitting there untapped. 🔄 Upper-funnel spend makes your lower-funnel spend more efficient. This is the finding that tends to land hardest in conversations. Scaling Demand Gen and YouTube didn't eat into PMAX or Brand Search performance, it lifted it. ROAS improved, CAC fell. Once you see it in the data it makes complete sense, but most measurement frameworks make it impossible to see. Finisterre scaled their Demand Gen spend 5.7X into peak and saw total Google revenue up 27% YoY. Derek Rose ran a two-phase approach - awareness pre-peak, capture during it - and came out with Google ROAS up 44% and Brand Search ROAS up 111%. The reason most brands don't invest full funnel in Google has less to do with budget and more with their current measurement system. Last Click attribution doesn't give Demand Gen and YouTube the credit they deserve, so the confidence to scale never builds. Full-funnel measurement fixes that - you can actually see what upper-funnel spend is doing downstream, and the investment case becomes obvious pretty quickly. If you're thinking about your Google strategy for the rest of 2026, worth a read. Link in comments 👇