AI Sales Agents Fail at Trust, Not Just Selling

When AI agents are deployed in high-stakes sales conversations, their success is typically measured by multi-dimensional quality scores that assess factors like coherence, empathy, and memory. However, a new study from a major Chinese matchmaking platform s this approach, showing that these scores may not reliably predict actual business outcomes. The research, conducted by Liang Chen, Qi Liu, Wenhuan Lin, and Feng Liang, investigates the criterion validity of such evaluation rubrics—whether the scores align with verified conversion rates, such as completed purchases. This gap between measurement and real-world success is not just an academic concern; it risks optimizing AI systems for metrics that don't translate to tangible , potentially wasting resources and eroding user trust in commercial AI applications.

The core finding of the study is dimension-level heterogeneity: individual quality dimensions vary dramatically in their association with business conversion. In an expanded Phase 2 analysis involving 60 human conversations with verified conversion labels, two dimensions showed significant positive correlations after statistical correction. Need Elicitation (D1) had a Spearman correlation of ρ = 0.368 with conversion, and Pacing Strategy (D3) had ρ = 0.354, both with p-values below 0.01 and medium-to-large effect sizes (Cohen's d around 0.75). In contrast, Contextual Memory (D5) showed no detectable association (ρ = 0.018), indicating that remembering conversation details did not influence purchase decisions. This heterogeneity causes a composite dilution effect, where an equal-weighted total score (ρ = 0.272) underperforms its best components because non-predictive dimensions dilute the signal from predictive ones.

Ology involved a two-phase design to address initial confounding issues. Phase 1 was a pilot study with 14 purposively sampled conversations mixing human and AI agents, which suggested an evaluation-outcome paradox—higher quality scores were linked to worse outcomes. Phase 2 expanded to a stratified random sample of 60 human-only conversations to eliminate agent-type confounds, using verified binary conversion labels from operational records. The evaluation rubric comprised seven dimensions scored on a 1–5 scale by an LLM judge (Claude Opus 4.6), with chain-of-thought reasoning to reduce biases like verbosity preference. Statistical analyses included Spearman correlations, Cohen's d for effect sizes, Bonferroni corrections for multiple comparisons, and logistic regression to control for conversation length, ensuring robust despite the modest sample size.

Analysis reveals that conversion-informed reweighting can partially restore criterion validity. By adjusting dimension weights based on empirical associations—such as boosting Pacing Strategy to 40% and reducing Contextual Memory to 0%—the composite correlation improved from ρ = 0.272 to ρ = 0.351, with a p-value of 0.006 in Phase 2. Logistic regression confirmed that D3's association with conversion strengthens when controlling for conversation length, with an odds ratio of 3.18. Complementary behavioral analysis of 130 conversations through a Trust-Funnel framework identified a candidate mechanism: AI agents executed sales behaviors effectively, with 72% reaching the closing stage, but 0% reached the trust threshold required for conversion, indicating a desynchronization between sales actions and user trust building.

For everyday readers, this research matters because it highlights a fundamental flaw in how AI performance is often assessed in real-world applications like customer service, sales, and support. If companies rely on quality scores that don't predict outcomes, they might deploy AI systems that appear competent but fail to deliver , leading to wasted investments and frustrated users. The study advocates for a three-layer evaluation architecture—safety, quality, and business layers—where criterion validity testing becomes standard practice. This means validating metrics against hard outcomes like sales conversions, not just human preferences, to ensure AI improvements translate to tangible benefits. In contexts like matchmaking or high-emotion sales, trust calibration dimensions like pacing and need elicitation prove more critical than technical capabilities, suggesting that AI development should prioritize user-centered strategies over raw performance metrics.

Despite its insights, the study has several limitations. The sample size of 60 conversations, while improved from the pilot, remains modest, with wide confidence intervals for effect estimates. The Trust Ladder annotations were generated by an LLM without human validation, introducing potential measurement error. are specific to a Chinese matchmaking platform and may not generalize directly to other cultural or commercial contexts. Additionally, the weight scheme optimization involved some circular analysis, as the same data were used for and evaluation, though temporal cross-validation provided partial mitigation. Future work should include human-annotated validation, pre-registered replications with larger samples, and tests in diverse domains to confirm the structural risk of composite dilution and explore causal pathways between quality dimensions and outcomes.