Artificial intelligence safeguards meant to curb bias may be creating another form of exclusion, according to a new University of Washington study. Researchers testing leading AI models across nearly 24,000 story completions found that female characters were often erased from generated text.
Instead of presenting female characters, the systems overwhelmingly produced male animals or used the ungendered pronoun “it.” The findings raise questions about whether current safety measures reduce harmful stereotypes or simply remove visible references to women and girls.
Large Test Reveals a Gender Imbalance
The study examined nearly 24,000 story completions, giving researchers a broad sample of AI-generated writing. Its scale suggests that the result was not limited to a handful of unusual responses.
The tested systems were described as leading AI models. However, the available findings do not identify individual models or provide separate performance results for each one.
The central pattern was clear: safeguards designed to reduce bias appeared to shift generated characters away from female identities. Models instead favored two alternatives:
- Animals described with male pronouns
- Characters described with the ungendered pronoun “it”
This outcome may reflect how developers train models to avoid offensive or stereotypical depictions. If a system treats gendered descriptions as risky, it may choose language that appears safer. Yet that approach can make female characters less visible.
Guardrails Can Produce Unintended Results
AI guardrails are rules and training methods that shape what a model will generate. They may block harmful content, discourage stereotypes, or guide systems toward neutral language.
Such controls can reduce direct bias. But the Washington findings point to a problem with evaluating safety through avoidance alone. A model may produce fewer stereotypes while also producing fewer women.
The safeguards “have accidentally erased female characters,” according to the study’s central finding.
Erasure differs from overt discrimination, but it can still affect representation. Stories help children and adults form ideas about who can lead, solve problems, or take part in public life. Repeatedly assigning male identities to animals may preserve the assumption that male is the default.
Using “it” creates a separate concern. The pronoun is common for objects and sometimes for animals of unknown sex. Applied too broadly, it removes identity rather than creating balanced representation.
Bias Testing May Need Wider Measures
The results suggest that developers should examine what appears after a safety intervention, not only what disappears. A successful system should avoid degrading portrayals without making one gender scarce.
Testing could measure the share of female, male, and ungendered characters across many prompts. Reviewers could also assess whether each group receives similar roles, traits, and levels of agency.
Model makers may need to separate harmful stereotyping from ordinary gender identification. Mentioning that a character is female is not itself biased. The concern lies in how that character is described and treated.
Questions Remain About Model Performance
Further detail will be needed to determine whether some models performed better than others. Researchers and developers will also need to test whether the pattern changes across story genres, age groups, languages, and prompt styles.
The study does not suggest that AI safeguards should be abandoned. It shows why those controls require close review after deployment. Rules built around neutral wording can still produce unequal outcomes at scale.
With nearly 24,000 completions examined, the University of Washington findings offer a warning for AI developers and users. Reducing stereotypes is only one part of fair representation. Future audits will need to track whether safety systems give women equal presence, rather than quietly writing them out.
Rashan is a seasoned technology journalist and visionary leader serving as the Editor-in-Chief of DevX.com, a leading online publication focused on software development, programming languages, and emerging technologies. With his deep expertise in the tech industry and her passion for empowering developers, Rashan has transformed DevX.com into a vibrant hub of knowledge and innovation. Reach out to Rashan at [email protected]
