Far and wide, AI has flourished. Its generative counterpart, GenAI, has taken industries and individuals by storm, boasting an adoption rate faster than that of the Internet itself. Of the 5 billion people online today1, 180 million — which is about 1 in 28 users — use GenAI every day2.
Without doubt, GenAI holds immense value for its users, delivering reliable information and insight instantaneously at just the click of a prompt. At its core, GenAI is built on data from the Internet, strewn over millions of websites including popular apps and platforms populated with infinite user-generated content, virtual libraries, databases, files, and various other text and nontext inputs. The fact that the accuracy of GenAI outputs corresponds directly with the accuracy of the data it is trained upon raises a big question — what kind of biases and misinformation exist on public Internet data today?
Fundamentally, Internet data suffers its largest bias from an incomplete data population — stemming from huge divides in digital access across the world. Not only does this data systematically misrepresent certain regions and demographics, typically those from developing or hard-to-reach areas, but it is also heavily influenced by psychographics relating to application usage and activity patterns — simply giving more weight to those who use the Internet more.
The largest gap here, of course, comes down to the biggest demographic segment in the world — female users. While Internet penetration for women has been steadily increasing over the last decade, this hasn’t always been the case. Historically, the Internet has been dominated by male users, owing to existing differences in digital access. This means that the bulk of historical data fed into GenAI platforms may focus disproportionately more on male-written content and applications, and male-coded online behaviour. Even now, disparities persist — take LinkedIn for instance, where only 43% of its users are women compared to 57% who are men3. As a result, a much smaller fraction of the conversations, discussions and thought leadership that exists on this platform comes from women — a classic example of how current data sources can still be heavily skewed.
And with GenAI echoing the voices already on the Internet, this bias could have a significant cascading effect, perpetuating existing gender inequalities, and leaving little room for a change in narratives.
Incomplete data training and how it leads to GenAI’s failure to address the needs of half the population
A common example of such bias is healthcare data. Though ChatGPT isn’t a conventionally recommended source to direct healthcare questions, AI in itself has proven immensely accurate in identifying diseases and providing clinical advice4 — but for women, the bias still lingers. GenAI is likely to misdiagnose female patients due to a difference in mainstream symptoms and those that present in women specifically — ADHD is a good case in point. While hyperactivity and lack of focus are common signs of this condition, it is known that women and girls only showcase more discreet symptoms such as emotional sensitivity and forgetfulness, making it harder to diagnose them compared to men.
Gender bias can even show up in the most ordinary prompts. For example, when asked which type of cars are most preferred by Americans, ChatGPT suggests that pickup trucks top the charts in terms of popularity. Further investigation, however, reveals that this trend is actually driven by an overwhelming preference for pickups by male drivers, while female drivers continue to opt for more compact, practical vehicles such as crossover SUVs and non-luxury sedans. Why does this happen? Simply because Internet conversation on cars may be largely fuelled by men discussing their next purchase options and vehicle preferences, while women — quite literally — continue to take the back seat.
One-sided logics and why problem-solving becomes tougher
GenAI is part fact, and part analysis. This means that the data it is fed not only affects the outputs it produces, but also influences the way it thinks and makes decisions. From how it processes information to the language it chooses to present its conclusions, GenAI will inevitably mirror real-world Internet behaviour — again, one that possibly leans further towards men. This greatly changes the way in which its underlying generative models work, giving more weight to some parameters over others and consistently fitting data to trends that present more obviously in the majority segment i.e. male users. Ultimately, this means that GenAI — without conscious programming or deliberate direction — could end up hard-wired to think more like a man, rather than a woman or neutral entity.
A good analogy to this can be drawn from the popular book Men Are from Mars, Women Are from Venus, which emphasizes that some of the biggest incompatibilities between men and women are a result of inherent psychological differences. Here, one of the primary examples cited are, “men’s complaint that if they offer solutions to problems that women bring up in conversation, the women are not necessarily interested in solving those problems, but talking about them”5. This draws great parallels to the answering mechanisms of GenAI chatbots, which often default to producing solutions rather than engaging in empathetic or exploratory conversation. Think about what these tools do when presented with a problem — they swoop in to deliver an answer, sometimes endlessly listing possible options and scenarios, and then either conclude with a decision or prompt the user for a decision. It is an uncommon sight for GenAI chatbots to, instead, invite the user to engage in further conversation about the prompt itself, as well as encourage them to share the emotions and perspectives behind it, in order to seek a deeper understanding of the problem. GenAI chatbots also tend to be more assertive and confident in their language, which leans towards male-coded behaviour, rather than be consensus-seeking and thoughtful as more frequently seen in women-coded behaviour.
Skewed predictions and how this reinforces gender gaps
GenAI is credited with its powerful predictive capabilities, which it derives from sophisticated deep learning models and neural networks. When tasked with outputting reliable predictions for relatively new subject areas, however, a lack of data remains a formidable challenge. Where data isn’t available for certain topics and industries, GenAI is tempted to extrapolate its findings from historical information across other topics and industries to provide a reliable guesstimation. This is an especially relevant challenge for new and emerging professional areas such as cybersecurity, space technology, quantum computing or even AI itself, where there isn’t adequate data available yet on women’s participation, opportunities and success stories. As a result, when trying to better understand these fields, GenAI might causally (pun intended) reproduce old narratives and assumptions, reflecting historical information rather than contemporary realities.
In some ways, this is a catch-22 — because of historically gender-biased information dominating certain major fields, GenAI is motivated to apply this to new industries and generalise past stereotypes. This creates biased narratives for fields where information on women-centric opportunities hasn’t even been adequately established yet, further hindering growth and career ambitions for those venturing into such areas. For instance, a quick search on ChatGPT on “common challenges for girls choosing data science courses” might yield answers that point to women’s historically lower uptake of STEM subjects, suggesting that this is still a relevant challenge. In many parts of the world today, however, this reasoning no longer holds, as women increasingly pursue STEM subjects and qualifications due to better access and more diverse education programs. The reality is that this is no longer a factor to consider, and yet when prompted, GenAI may very well reinforce a narrative that has already been overcome.
Greater inclusion is the way forward
Though the consequences are far-reaching, the root of the challenge is clear: inadequate data. As long as information about women and by women remains limited, generative AI will struggle to produce accurate insights, apply unbiased logics, and make reliable predictions. The onus, therefore, is now on us to address gender bias in one of the most important facets of artificial intelligence — its datasets. Whether it is inclusion in digitally-available research studies, equal participation in online forums, or representation in published knowledge sources, we have to consciously encourage women’s participation in ways that will contribute to a more fair and unbiased information pool, so that our fellow GenAI systems can learn from data that teaches them to be truly equal. Not only will this reshape narratives and outcomes for women, but it will greatly contribute to the reliability and accuracy of GenAI as a whole.
Coming back to our title: is ChatGPT a woman or a man? The tool itself believes that, “It doesn’t have a gender, and is just an AI.” Unlike its other answers, this is hard-coded. Which means that it is now up to us, the rest of the Internet, to shape AI’s identity, voice — and pronouns.
Bio
Tara Neal is the Executive Editor and Telecom Strategist of The Fast Mode. She has more than 26 years of experience in research, analysis and strategy planning. Tara has worked on various strategy projects covering business strategies and performance management. She has written extensively on various topics relating to the latest technologies, digital services innovations, operator strategies, market trends and the development of various emerging sectors. Tara holds a First Class Honours in BSc Accounting and Finance from The London School of Economics, UK and is a CFA charterholder from the CFA Institute, United States.
- https://www.statista.com/statistics/617136/digital-population-worldwide/ ↩︎
- https://www.technollama.co.uk/a-gemini-report-how-many-people-are-using-generative-ai-on-a-daily-basis-a-gemini-report ↩︎
- https://www.statista.com/statistics/933964/distribution-of-users-on-linkedin-worldwide-gender/ ↩︎
- https://hai.stanford.edu/news/can-ai-improve-medical-diagnostic-accuracy ↩︎
- https://archive.org/details/MenAreFromMarsAndWomenAreFromVenus/page/n7/mode/2up ↩︎