Unmasking AI

1. The Coded Gaze: Uncovering Bias in AI Systems

"Defaults are not neutral. They often reflect the coded gaze—the preferences of those who have the power to choose what subjects to focus on."

The coded gaze refers to how the priorities, preferences, and prejudices of those who create technology can propagate harm through discrimination and erasure. Joy Buolamwini discovered this concept while working on a facial recognition project at MIT, where she had to wear a white mask for the system to detect her face. This experience led her to investigate bias in AI systems, particularly in facial recognition technologies.

Key findings:

• AI systems often perform poorly on darker-skinned individuals and women
• Benchmark datasets used to train AI models are frequently skewed towards lighter-skinned males
• These biases can lead to real-world consequences, from false arrests to denied opportunities

Buolamwini's research revealed that major tech companies' facial analysis systems had significant accuracy disparities based on skin type and gender, with error rates up to 34.4% between lighter males and darker females.

2. From Art Project to Global Movement: The Birth of the Algorithmic Justice League

"I was not going to hold back. A graduate student going up against tech giants was not the typical path, but neither was coding in whiteface to be seen by a machine."

The Algorithmic Justice League (AJL) emerged from Buolamwini's master's thesis project at MIT. What began as an art installation exploring the limitations of facial recognition technology evolved into a global movement for algorithmic accountability and justice.

AJL's mission:

• Raise awareness about the impact of AI bias
• Advocate for more inclusive and equitable AI systems
• Develop tools and methodologies to audit AI systems for bias
• Engage with policymakers and industry leaders to promote responsible AI development

The organization's work has influenced policy decisions, corporate practices, and public discourse on AI ethics, demonstrating the power of combining academic research with activism and art.

3. Algorithmic Audits: Exposing Flaws in Commercial AI Products

"Even if my class project didn't work on me? My light-skinned classmates seemed to enjoy using it. And of course, there could certainly be an advantage to not having one's face detected, considering the consequences of cameras tracking individuals, and the dangers of mass surveillance."

Algorithmic audits are systematic evaluations of AI systems to identify biases and performance disparities across different demographic groups. Buolamwini's "Gender Shades" project was a groundbreaking algorithmic audit that exposed significant accuracy gaps in commercial gender classification systems.

Key findings of the Gender Shades audit:

• All tested systems performed worst on darker-skinned females
• The largest accuracy gap was 34.4% between lighter males and darker females
• The audit revealed that even leading tech companies' AI products had substantial biases

The Gender Shades project and subsequent audits have led to improvements in commercial AI systems and increased awareness of the need for diverse testing datasets and rigorous evaluation methods.

4. The Power of Evocative Audits: Humanizing AI's Impact

"Can machines ever see my queens as I view them? Can machines ever see our grandmothers as we knew them?"

Evocative audits use artistic expression and personal narratives to illustrate the human impact of algorithmic bias. Buolamwini's spoken word piece "AI, Ain't I A Woman?" is a prime example, showcasing how AI systems misclassified images of prominent Black women.

Impact of evocative audits:

• Humanize the consequences of AI bias
• Reach broader audiences beyond academic circles
• Inspire action and policy change

The "AI, Ain't I A Woman?" video went viral and was featured in the documentary "Coded Bias," helping to raise public awareness about AI bias and its real-world implications.

5. Fighting Erasure: Amplifying Marginalized Voices in Tech

"Being quiet about my findings would not have prevented harm, because these systems were already in development. My speaking out provided an opportunity to consider alternative pathways, including nonuse."

Combating erasure in AI and tech involves actively amplifying marginalized voices and challenging the status quo. Buolamwini's experiences with media erasure and academic gatekeeping highlight the importance of diverse perspectives in AI research and development.

Strategies for fighting erasure:

• Collaborate with and support underrepresented researchers and practitioners
• Use media platforms to highlight diverse voices and experiences
• Challenge institutions and companies to address systemic biases

Buolamwini's work with the documentary "Coded Bias" and her advocacy efforts have helped bring attention to the contributions of women and people of color in AI ethics and research.

6. Beyond Academia: Engaging Policymakers and the Public

"Congress will do something about this."

Engaging with policymakers is crucial for translating research findings into real-world change. Buolamwini's congressional testimonies and work with government agencies demonstrate the impact researchers can have on policy decisions.

Key policy engagements:

• Testified before Congress on facial recognition technology
• Contributed to the development of the AI Bill of Rights
• Supported local efforts to regulate facial recognition use by law enforcement

These efforts have led to increased scrutiny of AI systems, proposed legislation, and policy changes at various levels of government.

7. The Costs of Inclusion and Exclusion in AI Development

"There are costs of inclusion and costs of exclusion to be considered in the design and deployment of AI systems that must be contextualized."

Balancing inclusion and exclusion in AI development requires careful consideration of potential benefits and harms. While diverse datasets can improve AI performance, they may also enable more pervasive surveillance and control.

Considerations:

• Improving AI accuracy can enhance beneficial applications (e.g., medical diagnostics)
• More accurate facial recognition could also enable mass surveillance
• Excluding certain groups from datasets may protect privacy but lead to underperformance for those groups

Buolamwini advocates for a nuanced approach that considers the broader societal implications of AI systems, rather than focusing solely on technical performance metrics.

8. Toward Algorithmic Justice: From Research to Real-World Change

"We need laws. Over the years draft legislation on algorithmic accountability, remote biometric technologies, and data privacy has been introduced. With growing awareness about the impact of AI on our lives, we need to know that our government institutions will protect our civil rights regardless of how technology evolves."

Algorithmic justice requires a multifaceted approach combining research, advocacy, policy change, and public engagement. Buolamwini's journey from graduate student to influential AI ethicist illustrates the potential for individuals to drive systemic change.

Key components of the fight for algorithmic justice:

• Rigorous research and audits of AI systems
• Public education and awareness campaigns
• Collaboration with policymakers and industry leaders
• Support for grassroots efforts and community organizing

The release of the AI Bill of Rights and growing public awareness of AI ethics issues demonstrate progress, but ongoing vigilance and advocacy are necessary to ensure that AI systems are developed and deployed in ways that respect human rights and promote equity.

Last updated: July 30, 2024