It's How You Ask: Gender-Associated Linguistic Bias in LLMs

Prompts phrased in ways more common among women get plainer, less formal AI-written emails.

0.988

probe accuracy

Decoding prompt style from activations

Style is encoded far more strongly than sign-off name gender.

How it works

Hidden user bias
Bias studies check how models depict people, not whether they serve users with different phrasing habits differently.
Paired prompt rewrites
427 real WildChat writing requests rewritten with GPT-4 in women-associated and men-associated phrasing, sent to four models.
Plainer replies, early layers
Women-associated phrasing got less sophisticated, less formal replies. Names had no effect. Style is encoded in layers 0–7.

From the paper