AI Research Lacks Standards for Robot Gender

When you interact with a voice assistant like Siri or a social robot like Pepper, do you perceive it as male, female, or something else? This question is more than academic—it shapes how people trust, stereotype, and behave toward artificial agents in everyday life. Yet, a comprehensive new study shows that research on how humans gender AI lacks basic standards, with one-third of studies manipulating agent gender without measuring whether participants actually perceived it as intended. This gap undermines scientific rigor and risks embedding harmful stereotypes into the technology that increasingly mediates our social world.

The researchers conducted a systematic scoping review of 51 papers from the last decade, focusing on human-agent interaction (HAI) studies that measured perceptions of agent gender. They found that 54 papers were excluded because they manipulated gender—through voice, body design, or names—but did not measure it, meaning almost half of quantitative research in this area may have invalid . The study, detailed in a paper by Seaborn et al., highlights that without clear operationalization—defining and measuring gender perceptions—on how gender affects interactions with robots or virtual agents are unreliable. For instance, if a robot is designed with a female voice but users don't perceive it as female, any conclusions about gender effects are flawed.

To uncover how agent gender has been studied, the team analyzed papers from venues like the ACM/IEEE International Conference on Human-Robot Interaction and journals such as the International Journal of Social Robotics. They extracted data on terms, definitions, theories, and measures used, following the PRISMA-ScR protocol for scoping reviews. ology involved screening 1,709 records, with 51 meeting eligibility criteria after exclusions. Researchers charted items like study goals, manipulation checks, and measurement tools, using reflexive thematic analysis to identify patterns and gaps. This approach allowed them to map the current state of research, revealing a lack of consensus on even basic concepts like what "gender" means in the context of non-human agents.

Show a field in disarray. Most studies (84.3%) used only binary gender options like "female" and "male," with just 15.7% including categories like "neutral" or "ambiguous." Over time, options have diversified slightly, with terms like "agender" appearing in 2023 and 2024, but the binary model remains dominant. In terms of measurement, 56 different measures were found across 50 papers, with no standardization. Common s included Likert scales for gendered traits (e.g., masculinity/femininity) and semantic differential scales, but these often treated gender as a simple binary, ignoring non-binary perceptions. Notably, 45 manipulation checks were conducted across studies, but seven papers did not report any, and many used inconsistent tools, making comparisons across research impossible.

Are significant for both science and society. Without standards, researchers cannot reliably synthesize or understand how agent gender influences real-world outcomes like trust, stereotyping, or toxicity in interactions. For example, the paper cites studies where voice assistants designed with female voices were subjected to verbal abuse, potentially reinforcing sexist norms. The researchers propose a meta-level framework based on gender as a social structure, emphasizing transparency in operationalization—defining concepts, linking them to theory, specifying indicators, and detailing measurement. This approach, they argue, would allow for diverse epistemological perspectives while enabling rigorous, comparable research.

However, the study has limitations. It concluded in summer 2024, so newer work is not included, and the analysis was conducted by solo researchers without multiple coders, risking unchecked bias. The review also focused on English-language materials, reflecting a WEIRD (Western, Educated, Industrialized, Rich, Democratic) bias common in HCI research. The authors note that gender is a sensitive, political topic, and measuring it can reveal marginalized identities, requiring ethical handling of data to protect participants, especially gender minorities. They call for future work to explore intersectionality and non-Western contexts to build a more inclusive understanding of agent gendering.