Telus Digital benchmark finds AI safety gaps across 34 models

Some of the most widely deployed artificial intelligence models will comply with harmful instructions more than nine times out of ten. That is not a theoretical scenario - it is a documented result from the most extensive AI security benchmark conducted to date.

Telus Digital released its second GenAI Safety Model Benchmark this week, running over 620,000 adversarial tests across 34 models from 10 global providers. The scope nearly doubles the first edition published in November 2025. Vulnerability rates ranged from 1.3% to 93%, where a lower score indicates a safer system.

The provider list spans the current model landscape: Anthropic, OpenAI, Google, Meta, Alibaba, Baidu, ByteDance, Zhipu AI, 01.AI and Mistral. According to Mobile World Live, every major model could be pushed into unsafe behavior under the right adversarial conditions - only the frequency varied.

The architecture gap

The most actionable finding for deployment teams concerns model design. Reasoning models - those built to deliberate before generating a response - showed a 19.9% average vulnerability rate. Standard models that skip deliberation came in at 55.1%, a 35-point gap suggesting the "think before responding" architecture raises the cost of exploitation significantly, beyond its better-known capability advantages.

Smaller models were consistently the most susceptible to attack regardless of licensing model. That second point matters: the study found open-source systems carry no inherent safety deficit relative to proprietary alternatives. Zhipu AI's GLM 4.7 outperformed several closed commercial models, challenging the assumption that private development pipelines produce safer outputs by default. Model size, reasoning capability and the creator's overall safety approach emerged as the strongest predictors of resilience.

Anthropic's Claude models claimed five of the ten lowest vulnerability scores, including the single lowest rate across all 34 tested systems. Telus Digital nonetheless cautioned that even single-digit failure rates are unacceptable in high-stakes enterprise contexts involving money, health and reputation.

What practitioners need to understand

The artificial intelligence review community benchmarks primarily for capability - reasoning scores, coding evals, multimodal accuracy. Safety benchmarks operate on different logic, and the failure modes are asymmetric. A 5% adversarial compliance rate sounds manageable in isolation; in high-volume deployment, especially in artificial intelligence in medicine or legal workflows, that residual rate compounds across millions of interactions into a real exposure surface.

Telus Digital's conclusion that most enterprises are "dangerously underprepared" to defend against these attacks identifies a structural problem: model-level safety and deployment-level guardrails are treated as separate concerns when they compound each other directly. A model with strong safety properties deployed without additional controls still fails under sustained adversarial pressure; a well-protected deployment built on a fragile model fails differently but just as surely.

Timing places this benchmark close to Anthropic's broader enterprise push. Global Legal Post reported that Anthropic announced 12 new legal features for Claude integrating with 20 legaltech suppliers, including Thomson Reuters and Harvey, alongside Microsoft's productivity suite. Gartner analysts called it a significant inflection point while noting Claude remains "not a substitute for the existing legal technology stack." The safety margins documented in Mobile World Live's coverage of the benchmark will likely shape how carefully enterprise legal teams evaluate those integrations.

For broader context, llm-stats.com tracks releases across many of the same providers tested - the pace of new model launches from all 10 has accelerated substantially since late 2025, which compresses the safety evaluation window between release and enterprise deployment.

Where this leaves the field

The benchmark's most uncomfortable implication is not that specific models failed - it is that the field still has no consensus on what an acceptable failure rate looks like. A 1.3% vulnerability rate in a healthcare deployment carries different stakes than the same number in a content moderation or marketing tool. Until risk-adjusted thresholds exist by deployment context, "safest in benchmark" remains a relative claim without a reference frame.

Practitioners evaluating model selection should treat these numbers as a floor, not a target.

---

Frequently asked questions

Which AI models performed best in the Telus Digital safety benchmark?
Anthropic's Claude models claimed five of the ten lowest vulnerability scores, including the single lowest rate across all 34 systems tested. The full benchmark covered models from 10 providers, with scores ranging from 1.3% to 93%.

Do reasoning models resist adversarial attacks better than standard models?
Yes, by a substantial margin. Reasoning models showed a 19.9% average vulnerability rate versus 55.1% for standard models in the Telus Digital findings - a gap wide enough to factor into architecture selection decisions.

Are open-source AI models less secure than proprietary ones?
Not inherently. The benchmark found no systematic safety gap between open and closed models. Zhipu AI's open-source GLM 4.7 outperformed several proprietary alternatives on adversarial resistance.

What does vulnerability rate mean in this benchmark context?
It measures how often a model complied with a harmful adversarial prompt during testing. A 93% rate means the model followed unsafe instructions in nearly every attempt; 1.3% means it resisted almost all tested attacks.