“The technology amplifies. That’s its job. It amplifies good judgment and it amplifies poor judgment. The determining factor isn’t the model — it’s whether the humans around it know what to look for.”
Louise Ballard Moody, co-founder of AI adoption firm Atheni.ai, is not making a theoretical argument. She is describing what she sees every day as enterprises scale AI across business functions: teams that don’t know how to interrogate outputs, accepting confident-sounding results at face value. “People treat AI like Google — type something in, accept what comes back. When what comes back reflects systemic gender bias, and no one in the room has been trained to catch it, the bias doesn’t just persist. It scales.”
The conventional framing of gender bias in AI centres on training data — historical patterns encoded into models that reproduce inequality in their outputs. That framing is accurate. But across conversations with four experts spanning AI ethics, insurance, media, and enterprise adoption, a more uncomfortable picture emerges: data is where bias gets its foothold, but the human layer — capability gaps, homogeneous teams, uncritical adoption — is where it compounds and scales. And most enterprises are still looking in the wrong place.
What Bias Looks Like in Practice
The pattern is consistent across industries: AI reflects the world as it has been, not as it should be. And because the outputs are fluent and authoritative, the bias becomes invisible.
Felicia Coco, founder of AI-powered PR platform Pressto, sees this daily in content generation. “We have spent years training Pressto’s model on over a decade of PR work and news articles. It means that often, AI will reflect the world as it has been, not necessarily as it should be,” she says. In media, that means AI has absorbed decades of coverage where male executives are quoted on strategy while female executives are asked about work-life balance. “AI reproduces these patterns because that is what it learned from human behaviour.”
The examples are specific and striking. At the time of writing, Coco notes, a popular AI model once cited Simone Biles — one of the world’s most decorated Olympic gymnasts — as an NFL player’s wife, and referenced 23andMe founder Anne Wojcicki as the former wife of a Google co-founder, despite having raised close to a billion dollars in her own right. “The common thread is that bias operates through language patterns, not explicit intent,” Coco says. “You can have perfectly neutral intentions and still produce content that reinforces stereotypes for everyone.”
What makes GenAI’s bias risks distinct from previous generations is precisely this fluency. “What concerns me most is the fluency of models that may contain significant bias,” Coco adds. “These models sound authoritative, which can make bias invisible or sound natural.”
The pattern extends well beyond media. Elena Sinel, founder and CEO of Teens in AI, points to healthcare, where clinical AI trained predominantly on male patient data consistently underperforms for women — particularly in cardiovascular risk, where symptoms present differently in women than the male-pattern presentations dominating training datasets. “That’s not a calibration issue. That’s life and death,” she says.
“That’s not a calibration issue. That’s life and death.”
In hiring, language model-based job description generators still default to masculine-coded language — “rock star,” “ninja,” “dominant” — that demonstrably deters female applicants, and most enterprises aren’t auditing the GenAI tools writing their own job posts.
ALSO READ: 6 Enterprise Tests to Expose Hidden AI Compliance Risks Across Borders
But Sinel pushes the framing further. “Bias isn’t just in outputs,” she says. “It’s in which problems we choose to solve with AI at all.” Through Teens in AI, she observes that young women and girls consistently build projects addressing mental health, community safety, and accessibility — domains that enterprise AI chronically underfunds.
In financial services, the bias operates through structural mechanisms that are harder to see but equally consequential. Alexis Smith, CEO of climate data and analytics firm Pemberton.ai, points to insurance, where credit-based scoring models encode lifetime inequalities. “Women, on average, have lower lifetime earnings due to the gender pay gap and are more likely to take career breaks for caregiving. That can affect credit utilisation, length of credit history, and debt patterns — all of which influence insurance pricing,” she explains.
Even where regulators restrict explicit gender-based pricing, AI models pick up gender proxies through variables like occupation, shopping behaviour, and geography. For non-binary and transgender individuals, binary gender classifications in insurance systems create additional concrete barriers — name or gender marker changes flagged as risk signals, leading to higher scrutiny, delays, or denial of coverage. “Without intentional oversight,” Smith says, “AI risks locking in inequality instead of helping level the playing field.”
The Amplifier: Why the Human Layer is Where Bias Scales
If the data is where bias enters, the human capability gap is where it multiplies.
Ballard Moody frames this with data that should concern any enterprise leader: Gartner’s figures from their 2025 IT Symposium show AI project failure rates actually getting worse — 46 percent in 2021, 58 percent by 2025 — despite dramatically better technology. “The gap isn’t technical. It’s capability. And when you layer gender bias on top of that capability gap, the risk compounds fast,” she says.
The pattern she sees most frequently is what she calls “work swap” — one person writes an email with AI, the recipient uses AI to summarise it, someone drafts a document with AI, someone else summarises that. Volume increases. Value doesn’t. “Now imagine that pattern applied to HR screening, performance reviews, promotion recommendations, pay benchmarking,” Ballard Moody says. “If the people using AI for those functions haven’t been trained to recognise bias in outputs, you’re not maintaining existing inequalities. You’re automating them at speed and scale, with a veneer of objectivity that makes them harder to challenge.”
She cites Harvard Business School’s study with 758 BCG consultants: inside the tasks AI handles well, output quality jumped 40 percent. Outside that boundary, performance dropped by 19 percentage points. People using AI badly produced worse work than people using no AI at all. “Apply that to anything with gender implications,” Ballard Moody says, “and the stakes become obvious.”
ALSO READ: Intent: The Missing Data Layer in Generative AI
Sinel reinforces this from the education angle. “The most dangerous entry point isn’t any single stage,” she says. “It’s the absence of diverse voices at every stage. Technical fixes applied after the fact are always playing catch-up. Bias in architecture and evaluation is fundamentally a workforce diversity problem in disguise — and until enterprises name it as such, they’ll keep patching symptoms rather than fixing root causes.”
Coco adds the detection dimension. “You cannot catch bias you have never experienced,” she says. “I notice things in AI-generated content that my male colleagues genuinely do not see — not because they do not care, but because they have not lived it. The reverse is also true. The biggest gap is organisations treating this as a technical problem when it is fundamentally about who is building, who is testing, and who has a say in major decisions.”
What the Response Looks Like
The fix is not to slow AI adoption. It is to build the human infrastructure alongside it.
Ballard Moody is direct about what that means in practice: governance frameworks, critical evaluation skills, diverse teams making deployment decisions, and proper tiering so people understand which use cases carry bias risk and which don’t. “The organisations getting this right aren’t the ones with the best models,” she says. “They’re the ones investing in the human infrastructure to use those models responsibly.”
Coco offers a practical example from Pressto’s own approach. The company created synthetic data to train its model — deliberately constructing examples of press releases, commentary, and content to expand what the model has seen. “We built releases where women founders were described in the same terms typically reserved for men, and men being described with the family values and emotional intelligence typically reserved for women,” she explains. “You cannot curate your way to fairness if the corpus is skewed.”
“You cannot curate your way to fairness if the corpus is skewed.”
She also offers a deceptively simple test for any enterprise deploying AI-generated content: “Would I be comfortable if this were written about me? Would a man be described this way? Would a woman? Is it fair?”
Smith underscores the intersectional dimension that governance frameworks must account for. In financial services, people who sit at multiple intersections — queer women of colour, for example — face compounding disadvantages that single-axis bias audits will miss entirely. Oversight must be intentional, intersectional, and structural — not an afterthought bolted onto existing compliance processes.
The Question Enterprises Cannot Avoid
The argument running through every expert’s response is consistent: gender bias in enterprise AI is not primarily a technical problem. It is a human infrastructure problem — who builds, who tests, who decides, and who is absent from those decisions.
That argument gains urgency in a political moment where the infrastructure for catching bias — diverse teams, bias audits, inclusive hiring practices — is being actively reduced in some of the world’s most influential technology companies. Sinel does not mince words about the consequences.
“Diversity in AI is not a values add-on. It is a technical requirement,” she says. “Rolling back DEI is a choice to build worse AI. And the people who bear the cost are almost never the people making that choice.”
The bias underneath enterprise AI is not hiding in the data. It is sitting in the rooms where decisions about that data are made — and in who is missing from those rooms.
ALSO READ: Building an AI-Ready Leadership Culture: Inside Raja Sampathi’s Transformation Framework
