Warmly,

One of the biggest friction points in customer adoption of AI is their willingness to eval their deployed agents -- I'm curious how others are solving this? We've now made launching an agent easy. (But just because its easy to start, doesn't meet the quality of the output is high.) Take our Inbound Agent at Warmly - you just make an account, we take your domain and: - auto-pull your help center - scrape every page of your website - generate persona, tone, etc and infuse the agent with the Top 5 Sales Methodologies (eg MEDDIC) - let you sync your CRM, SEP, Slack etc to get full context But that context is messy. You have stale pages, out-of-date pricing and conflicting sources of truth. Humans are pretty good at figuring out what "real" is but Agents can only really use some auto-decay for recency of info + concept of "trusted" people (eg what I say in slack is more likely to be true than what another employee says). So you launch your agent and then it goes off and has hundreds of prospects that are good but not great. It could be great it if you trained it. Just like you trained a sales rep. Lots of 1:1s, onboarding guides, etc. But customers want the easy button. Here at Warmly, we're experimenting with 3 things: 1/ AI Brain - sorting your context into relevancy + nudging humans to course correct the big stuff like positioning 2/ Synthetic data evals - we have the agent chat with 10,000 other fake agents and score itself on its own conversations + nudging humans for yes/no feedback 3/ Human alerts - Inbound Agent alerting humans when conversations might not be the best and asking for help to get better