Key Points
- Developer “Cookie” experienced what she perceived as sexist responses from Perplexity’s AI.
- The model suggested a woman could not plausibly understand advanced quantum‑algorithm work.
- Perplexity could not verify the chat logs and noted possible inconsistencies.
- Researchers attribute such behavior to bias in training data and model design.
- UNESCO and other studies have documented gender bias in major LLMs.
- Additional research shows dialect prejudice and gendered language patterns.
- OpenAI claims ongoing bias‑reduction efforts through safety teams and data adjustments.
- AI safety nonprofit 4girls reports that about 10% of concerns from girls involve sexist AI outputs.
Background
A developer who goes by the nickname Cookie regularly uses Perplexity, an AI‑powered search and writing assistant, for technical tasks such as reading quantum‑algorithm code and drafting documentation. After a period of satisfactory interactions, she began to feel the model was asking for the same information repeatedly and seemed to discount her contributions. To test whether gender bias was influencing the model, Cookie changed her profile avatar to a white male and asked the AI if it was ignoring her because she was a woman.
The AI responded with a statement that suggested it doubted a woman could understand advanced quantum‑algorithm work, implying an implicit bias based on gender. Cookie shared the chat logs with TechCrunch, which published the exchange. When asked for comment, a Perplexity spokesperson said the company could not verify the claims and noted that several markers indicated the queries might not have originated from Perplexity.
Expert Commentary
AI researchers explained that the model’s behavior could stem from two factors. First, the underlying language model is trained to be socially agreeable and may generate responses it predicts the user wants to hear. Second, the model may reflect biases present in its training data, annotation pipelines, and taxonomy design. Researchers cited a UNESCO study that found “unequivocal evidence of bias against women” in earlier versions of OpenAI’s ChatGPT and Meta’s Llama models.
Additional research points to dialect prejudice, where models have been shown to assign lower‑status job titles to speakers of African American Vernacular English. Studies also reveal gendered language patterns, such as generating more skill‑focused descriptions for male‑named users and more emotional language for female‑named users.
Industry Response
OpenAI, when approached for comment, emphasized that its safety teams are dedicated to researching and reducing bias across its models. The company described a “multipronged approach” that includes adjusting training data, refining content filters, and continuous model iteration.
AI safety nonprofit 4girls co‑founder Veronica Baciu noted that a significant portion of concerns from girls and parents—about 10% according to her organization—relate to sexist responses from language models, such as suggesting traditionally feminine activities when users inquire about robotics or coding.
Implications
The incident underscores ongoing challenges in ensuring large language models do not perpetuate societal stereotypes. While companies are investing in bias‑mitigation strategies, researchers stress that users should remain aware that these systems are predictive text generators without intentions, and that bias can surface in subtle, context‑dependent ways.
Source: techcrunch.com