AI Game Characters Need Role-Specific Prompts

Artificial intelligence is transforming video games by powering non-player characters (NPCs) that can converse in unscripted, improvisational ways, promising more dynamic and engaging experiences. However, a new study reveals that simply refining AI prompts does not guarantee players will notice or value these improvements, challenging common assumptions in game design and AI research.

Researchers from the University of Regina discovered that increasing the specificity of prompts for large language models (LLMs) like GPT-4o does not reliably enhance players' experiences. In a study with 10 participants, they compared high-constraint prompts, which included detailed instructions and behavioral rules, with low-constraint prompts that offered minimal guidance. Despite technical differences, quantitative ratings showed no significant improvement in engagement, immersion, or usability between the two conditions. This indicates that players are largely indifferent to underlying prompt refinements unless breakdowns in believability or coherence occur.

The team developed a voice-based game prototype called Interview, where players interrogate AI-powered NPCs in a detective scenario. Using a within-subjects design, each participant experienced both prompt conditions, with interactions mediated by speech-to-text and text-to-speech pipelines to simulate natural dialogue. The high-constraint prompts incorporated JSON schemas with character traits, emotional states, and rules to minimize contradictions, while low-constraint prompts allowed greater improvisational freedom. This setup enabled direct comparison of how prompt specificity influences real-time player interactions.

Analysis of the results uncovered a critical pattern: the effects of prompt scaffolding are role-dependent. For quest-giver NPCs, such as an interviewer who anchors the narrative, high-constraint prompts improved stability and reliability. However, for suspect NPCs that require spontaneity and deception, the same constraints reduced improvisational believability. In a follow-up evaluation using an LLM judge, high-constraint methods scored higher on relevance but lower on variation for suspects, highlighting a trade-off where enforcing rules can suppress the surprise essential for engaging roles. Qualitative feedback from players reinforced this, with some describing high-constraint NPCs as 'steady' but occasionally 'on-rails,' while low-constraint ones were 'livelier' yet risked confusion.

This research matters because it shifts the focus from technical optimization to user-centered design in AI-driven games. For developers and designers, it underscores that players prioritize perceptible qualities like responsiveness and coherence over hidden prompt refinements. In real-world applications, this means AI characters should be tailored to their roles—using tight constraints for rule-enforcers but allowing flexibility for characters that thrive on unpredictability. This approach can lead to more believable and enjoyable gaming experiences, potentially influencing other interactive AI systems in education or social simulations.

The study has limitations, including a small sample size of 10 participants, which may not capture long-term preferences or diverse player backgrounds. Additionally, the evaluation relied on short, first-play sessions, so repeated interactions might reveal different insights. The researchers note that future work should explore longitudinal studies and varied genres to generalize these findings.

By introducing the Play framework, which formalizes role-sensitive prompts with fuzzy-symbolic boundaries, the study provides a blueprint for balancing stability and freedom in AI interactions. This reframes AI evaluation not just as a technical challenge but as a usability problem, emphasizing that what players experience ultimately defines success.