Lentis, Algorithmic Bias

Algorithmic bias refers to undesirable results from a computer system that incorrectly or unfairly prioritizes one group over another. This chapter currently focuses on algorithmic bias in the United States.

Background

Of particular importance is algorithmic bias in Machine Learning (ML) and Artificial Intelligence (AI).

Companies and individuals are increasing their reliance on AI systems. Search engine results, social media recommendations, hiring decisions, stock market predictions, and policing practices use information from AI predictive modeling. [1] Algorithmic bias in these models is particularly dangerous due to human Automation Bias, the tendency for humans to disregard contradictory information when presented with a computer-generated solution [2]. This increases the likelihood that bias in the AI system will result in unfair or inequitable outcomes where the system is in use.

How does Algorithmic Bias occur in AI systems? Often, AI systems operate on massive data sets amalgamated from existing sources without refinement. This leads to any existing bias, often institutional or implicit, being passed along to the AI system. [3] Consider an AI hiring algorithm created to give a competitive advantage in finding the best possible candidates for a computer science position. Because computer science is currently male dominated field, the hiring algorithm might erroneously prioritize male applicants. An example of a similar case is the Amazon Hiring Algorithm. In this way, AI systems will replicate existing bias and perpetuate existing prejudice in the status quo.

Assembled bias, a proposed type of Algorithmic Bias unique to AI and ML systems, describes novel biases introduced by AI systems that do not currently exist in society.[4] Assembled bias arises from the idea that AI models are generative rather than purely statistical. A toy example to better understand this phenomenon is AI generation of realistic images.[5] When asked to create an image of a spider, an AI model might produce a highly detailed, realistically textured image of an eleven-legged spider. Even though the AI has only ever trained with images of eight-legged spiders, the AI has no concept of how to count legs – instead, it creates its own metrics for what defines a spider.

Diversity is notably lacking from AI development and research positions. Only about 20% of AI researchers are women, and other minority groups are similarly underrepresented.[6] Because the end goal when developing an AI system is often quantifying a nebulous concept, the perspectives of those present and working on the system strongly influence the outcome. The lack of diversity in AI positions represents another possible source of bias because the majority perspective will be overvalued in the outcome.

Currently, the unfair outcomes resulting from Algorithmic Bias in AI systems appear to be disadvantaging primarily women and sexual or racial minorities. Combating this unintended result of AI and ML is an ongoing field of research.[7]

Case Studies and Social Impacts

Amazon's Hiring Algorithm

In 2014, Amazon initiated a program that used AI to review job resumes. The purpose of the program was to reduce the time spent on finding good candidates to fill job openings. The AI was trained on resumes submitted to the company during the previous 10 years. The demographics of the submitted resumes were similar to that of most tech companies: composed mostly of men, especially in technical roles. Consequently, the AI produced an algorithm that favored male resumes more than female resumes, as most of the previously successful candidates were male. This manifested itself in the algorithm as it penalized resumes that included the word “women’s”, or rewarding those with words more commonly found on male resumes such as “captured” or “executed”. The algorithm was also found to downgrade candidates who graduated from all-women’s colleges.[8]

To combat the gender-bias outcome of the tool, Amazon made the program neutral to gender-correlated terms. By doing so, Amazon engaged in internal auditing, a technique designed to reduce algorithmic bias by ensuring any emerging bias in a machine learning model is caught and stopped. However, it did not address the crux of the problem, which was the underlying data. A different approach would have been to analyze the existing data set to predict where the algorithm could be biased, evaluating and modifying assumptions about the data as necessary. This approach has been successful in other AI applications. In this case, however, Amazon left the underlying data unaddressed. This abetted the algorithm to find additional discriminatory ways of sorting the candidates. The tool was retired in 2017 after “executives lost hope for the project.”[8]

Critics of Amazon argue this algorithm has perpetuated the gender disparity in their workforce. Amazon does not dispute that their recruiters looked at the recommendations generated by the tool, but maintain the tool “was never used by [them] to evaluate candidates.”[8] Opponents of Amazon argue the two are mutually exclusive; recruiters were surely influenced by a tool that rates candidates from 1-5 stars, even if they claim they were not. Some critics, like ACLU Attorney Rachel Goodman, further contend that AI hiring tools “are not eliminating human bias – they are merely laundering it through software.”[9] Proponents of AI hiring technology are more optimistic: In a 2017 survey from CareerBuilder, 55% of U.S. human resources managers said AI would be a regular part of their work within the next five years.[10] However, as Vice President of LinkedIn Talent Solutions John Jersin explains, the technology has much room for improvement before it can “make a hiring decision on its own.”[8]

Amazon has since revived the project, hoping that emphasizing diversity will lead to more equitable algorithmic outcomes. However, in the absence of critically evaluating the data the models are trained on, it is unclear why a certain algorithmic emphasis will lead to a non-discriminatory outcome of the algorithm.

Vendors, Regulators, and the Hiring Pipeline

While Amazon's algorithm represents one company's failed attempt at automated recruiting, automated hiring tools are now widespread. As of 2025, an estimated 99% of Fortune 500 companies use some form of applicant tracking system, and a growing share rely on AI for screening, ranking, and video assessment.[11] Vendors such as HireVue, Pymetrics, and Eightfold supply these tools to employers across industries. HireVue, one of the largest, reports having facilitated over 70 million video interviews.[12]

Vendors frame their tools as standardizing fairness. HireVue states that its AI provides "a standard, structured, and fair way to screen many candidates… in a shorter time and at lower cost."[13] Critics argue this framing conflates algorithmic consistency with equity. The Electronic Privacy Information Center filed an FTC complaint against HireVue in 2019, alleging that the company's facial analysis features lacked scientific basis and produced biased outcomes.[14] HireVue announced in January 2021 that it had removed facial analysis from its assessments, citing public concern, but retained voice and language analysis.[15]

Employers and HR departments occupy a distinct position in this pipeline. The Society for Human Resource Management reports that among organizations using AI in hiring, nearly nine in ten do so to save time or increase efficiency.[16] However, most HR departments lack the technical capacity to independently audit the tools they purchase, creating a diffusion of responsibility: vendors maintain that employers control how the tools are used, while employers defer to algorithmic outputs they cannot meaningfully evaluate.

The Regulatory Landscape

Governance of automated hiring is fragmented. In the United States, no federal law specifically addresses AI in employment. The Equal Employment Opportunity Commission has issued guidance noting that automated screening can violate the Americans with Disabilities Act and Title VII, but this guidance is non-binding.[17] In the absence of federal action, individual states and cities have legislated.

Illinois passed the Artificial Intelligence Video Interview Act in 2019, requiring employers to notify applicants when AI evaluates video interviews and obtain their consent.[18] New York City's Local Law 144, effective in 2023, requires employers using automated employment decision tools to commission annual independent bias audits and publish the results.[19] The European Union has taken a different approach: the EU AI Act, which entered into force in 2024, classifies hiring AI as "high-risk" and imposes mandatory pre-deployment impact assessments.[20] This product-safety framework contrasts with the U.S. civil-rights model, which generally requires harm to occur before legal action can proceed.

The trade group BSA | The Software Alliance, whose members include Microsoft, Oracle, and Workday, has actively lobbied to narrow these regulations. In comments on Local Law 144, BSA argued that "requiring third-party audits is not a feasible or optimal approach," citing the absence of common AI auditing standards and professional oversight bodies.[21] The result is a contested regulatory environment in which the standards meant to govern these tools are themselves shaped by the industries they are intended to constrain.

Facial Recognition Algorithms

Several companies, including IBM and Microsoft, have developed facial recognition algorithms to recognize people only using their face22]. These algorithms are available to the public and have been used by police departments in conjunction with [video surveillance programs 23]. In 2018, Joy Buolamwini studied biases in gender classification for facial recognition algorithms from IBM, Microsoft, and Face++, and discovered that all of them perform better on "lighter faces than darker faces", and performed worst for "darker female faces "[24]. All of the algorithms had an accuracy gap between "lighter males and darker females" of over 20% with IBM's algorithm performing the worst with a gap of 34.4%[24]. In 2019, the National Institute of Standards and Technology (NIST) confirmed the issue of [algorithmic gender bias and racial bias is an industry wide problem with 189 of the studied facial recognition algorithms being least accurate for people of color, and in particular, women of color[25]. In response to the research IBM stopped working on their facial recognition algorithm[26], Microsoft and Face++ released improved versions of their facial recognition algorithms and reduced their accuracy gap by over 19%[27], and Amazon stopped police departments from using its facial recognition algorithm[28].

Wrongful Arrests and the Limits of Corporate Response

While companies like IBM and Amazon paused or withdrew their facial recognition products in response to research findings, deployment by law enforcement continued through other vendors. In January 2020, Detroit police arrested Robert Williams in front of his wife and two daughters after a facial recognition system incorrectly matched him to surveillance footage of a shoplifting suspect.[29] Williams spent thirty hours in jail before officers acknowledged the match was wrong. His was the first publicly reported case of a wrongful arrest caused by facial recognition technology.[30]

Williams's case was not isolated. By 2025, the American Civil Liberties Union had documented more than a dozen publicly known wrongful arrests in the United States caused by police reliance on facial recognition matches, with nearly all of the affected individuals being Black.[31] Cases include Nijeer Parks in New Jersey, Porcha Woodruff in Detroit (arrested while eight months pregnant), and Randal Reid in Georgia.[32] The American Civil Liberties Union, working with the University of Michigan Civil Rights Litigation Initiative, filed suit on Williams's behalf in 2021, arguing that Detroit's use of the technology violated his civil rights.[33] The case settled in June 2024, with Detroit agreeing to what advocates described as the strongest police facial recognition policy in the United States, including a prohibition on arrests based solely on algorithmic matches.[34]

The Williams case illustrates a recurring pattern in algorithmic bias advocacy. Researchers such as Joy Buolamwini generate quantitative evidence of bias; advocacy organizations identify specific victims whose stories make the harm legible to non-technical audiences; journalists amplify these cases; and litigators or legislators use the resulting political pressure to force change. Buolamwini's Gender Shades findings were published in 2018, but the most consequential policy responses came only after individual harms became public. This dynamic raises questions about the time lag between technical research and accountability: tools shown to be biased often remain in deployment for years before specific cases generate enough visibility to compel response.

Loan Lending Algorithms

Financial technology (Fintech) is a field in which loan lending algorithms have begun to replace face-to-face meetings. In the U.S., 45% of the largest mortgage lenders offer software-based loan solutions but they have been scrutinized for discriminative pricing. A study by Bartlett et al. found that through online platforms and face-to-face meetings Black and Latinx borrowers paid, on average, 7.9 basis points higher interest on purchase loans than comparable borrowers and paid 3.6 basis points more on refinance loans. This difference costs Black and Latinx borrowers $756M annually. Researchers found the discrimination in algorithms to be about 40 percent less than face-to-face lenders. Specifically, when Fintech algorithms are used, underrepresented borrowers pay 5.3 basis points higher than their counterparts (2.6 points lower than traditional methods)[35]. The study analyzed 30-year, fixed-rate, single-family residential loans issued by Fannie Mae and Freddie Mac between 2008 and 2015, and lenders were found to make 11 percent to 17 percent higher profits from purchase loans to underrepresented groups[36]. Although loaning algorithms demonstrate less bias than traditional face-to-face lenders, the persistence of bias of any degree emphasizes the need to examine loaning algorithms and determine the source of the algorithmic bias.

Loaning algorithms are based on machine learning and big data, which use a wide array of customer attributes to set prices. For example, geography can play a major role since the algorithm can target locations where comparison shopping is less likely. The algorithm may determine areas that are financial deserts where financial reserves are low and applicants are faced with monopoly pricing instead of having access to many options. The algorithm may not specifically target underrepresented applicants, but its logic may set a higher price knowing the applicant is more likely to accept it[36].

Underlying historical discrimination in training data may also lead algorithms to disfavor underrepresented groups and further wealth disparity. Fintech loaning algorithms use the prospective borrower’s credit histories, employment status, assets, debts, and the size of the loan requested to set interest rates. “If the data that you’re putting in is based on historical discrimination, then you’re basically cementing the discrimination at the other end,” says Aracely Panameño, director of Latino affairs for the Center for Responsible Lending. Research also shows that payday loan sellers often prey on neighborhoods predominantly populated with people of color since they typically have fewer bank branches. Banks report both positive and negative credit behavior while payday loan services only report missed payments. As a result, underrepresented groups from these neighborhoods find themselves with incomplete or skewed credit histories that are later fed into loan financing algorithms[37].

The Algorithm

The Correctional Offender Management Profiling for Alternative Sanctions algorithm (COMPAS), is a machine learning algorithm for judicial decision-making during criminal sentencing. Used in Wisconsin, New York, California, and Florida, COMPAS predicts the potential risk of recidivism. Created by Northpointe, Inc. (now Equivant), this commercial algorithm uses a questionnaire to categorize defendants as "low-risk", "medium-risk", or "high-risk", with respective scores of 1-4, 5-7, or 8-10.[38] Factors such as age, gender, and criminal history are used while race is not.[39] Used still today to advise on bail, sentencing, and early release, many are questioning the validity and fairness of COMPAS in providing objective advice, particularly by race.[40]

In 2016 the Pulitzer Prize-winning nonprofit news organization ProPublica[41] conducted a study to assess COMPAS for racial bias. A COMPAS score of medium or high risk and a defendant recidivating in 2 years, and a COMPAS score of low and a defendant not recidivating in 2 years are considered correct predictions[42]. ProPublica found that a correct prediction of recidivism was made for white and black defendants 59 percent and 63 percent of the time, respectively. This is roughly the same rate. Upon questioning, the company said "it had devised the algorithm to achieve this goal. A test that is correct in equal proportions for all groups cannot be biased.”[40] Northpointe Inc. even cites other studies within their Practitioner's Guide confirming the success and neutrality of their algorithm.[43]

ProPublica found that Black defendants had uniform scores from 1-10, while White defendants had predominantly lower scores. After adjusting for Black defendants having higher recidivism rates overall, ProPublica found that COMPAS, while maintaining a similar accuracy, is "more likely to misclassify a black defendant as higher risk than a white defendant...the test tended to make the opposite mistake with whites."[42]

Perpetuated Systemic Racism

COMPAS perpetuates the historic and structural bias that is found in the criminal system. For example, COMPAS considers one's area of residence. Minority-dominated areas are often more policed, which inflates arrest statistics. If geography correlates to recidivism, then race indirectly does too. Thus, these correlations can be dangerous when not taking validity measures. As a private company, Northpointe has no legal obligation to share details on how COMPAS calculates its score or weighs its variables.[38]

Defendants labeled high/medium risk may be subject to harsher sentencing. The United States Sentencing Commission found that offenders who were sentenced to an incarceration length between 60 to 120 months and over 120 months were significantly less likely to recidivate.[44] If a defendant is suspected of being high-risk, they may be given a longer sentence, and be less likely to receive bail or early release. COMPAS’s mislabeling of Black defendants may result in negative consequences despite reported racial neutrality, and could further perpetuate racial inequity in the criminal justice system.[41]

The Algorithm

Several U.S. health care systems use commercial algorithms to guide health decisions and target patients for "high-risk care management" programs to help ensure that appropriate care is provided. Most of these programs are beneficial to patients with critical conditions because they have teams of specialized nurses, extra primary care appointment slots, and other scarce resources.[45] As a result, hospitals and insurance companies have relied on one specific algorithm to minimize the cost for patients while targeting sicker patients that would benefit most from these programs.[46]

In 2019, Obermeyer et al. conducted a study funded by the National Institute for Health Care Management Foundation on this commercial care management algorithm in response to “the growing concern that algorithms may reproduce racial and gender disparities via the people building them or through the data used to train them." This predictive-risk algorithm generated a risk score for each patient to infer their medical needs, which was primarily based on the patient's previous health care spending. Based on the score, patients would be automatically enrolled to the program, referred to their primary care physician for consultation, or no recommendations were made.[45]

The study found that Black patients with the same level of algorithm-predicted risk as White patients had 26.3% more chronic illnesses. Furthermore, researchers found that when looking at specific biomarkers that index the severity of various chronic illnesses (hypertension, diabetes, bad cholesterol, etc.), Black patients had more severe illnesses than White patients with the same risk score. This was due to Black patients generating lower expected medical costs than White patients with the same chronic illnesses. These spending differences resulted in disparities between the perceived level of sicknesses between Black and White patients since the algorithm focused on patient health care spending as it's primary mechanism for producing risk scores.[45]

Perpetuated Systemic Racism

The study identifies two systemic causes for reduced health care spending by Blacks that contributed to this algorithm's bias. First, poor patients face several barriers that can impede their access to healthcare despite having health insurance (geography, transportation access, competing demands from jobs or child care, knowledge of reasons to seek care, etc.). Second, race can impact patient spending through either direct discrimination by physicians or changes to the doctor-patient relationship, causing patients to not be recommended for or choosing not to seek further care. These observations demonstrate that accurate methods for decision-making models can indirectly disadvantage specific groups in society and have the ability to perpetuate existing systemic racism.[45]

The Dutch Childcare Benefit Scandal (Toeslagenaffaire)

The Dutch Childcare Benefit Scandal, known in the Netherlands as the Toeslagenaffaire, serves as a landmark case of how algorithmic bias can lead to systemic human rights violations outside of the United States. Starting in 2013, the Dutch Tax and Customs Administration (Belastingdienst) employed a self-learning algorithm to create risk profiles of individuals suspected of childcare benefit fraud. The algorithm utilized "foreign-sounding names" and "dual nationality" as high-risk indicators for fraud.[47]

Because of the "High Risk" designation generated by the algorithm, tens of thousands of families—predominantly from low-income and ethnic minority backgrounds—were subjected to aggressive debt collection. The tax authorities demanded the immediate repayment of entire years of benefits based on minor clerical errors or mere suspicion. This led to severe financial ruin, divorces, and the illegal removal of children from their homes by social services. The scandal eventually led to the mass resignation of the Dutch cabinet in January 2021.[48] This case highlights the dangers of automation bias, where civil servants deferred to the "objective" output of the risk-scoring system, ignoring the obvious discriminatory variables embedded in the training data.

Generative AI and "Assembled Bias" (Google Gemini)

In early 2024, the launch of Google’s Gemini image generation tool introduced a new dimension of algorithmic bias: over-correction leading to historical inaccuracy. When users prompted the model to generate images of historically specific groups—such as the Founding Fathers of the United States or 1940s German soldiers—the model produced images featuring diverse racial and gender identities that did not align with the historical record.[49]

This phenomenon is a practical example of "Assembled Bias" discussed earlier in this chapter. The bias was not necessarily present in the original historical training data; rather, it was "assembled" by the developers' safety filters and diversity-tuning layers intended to prevent the exclusion of minority groups in contemporary contexts. While these guardrails were designed to combat the historical underrepresentation of minorities in AI, their application to historical prompts resulted in what critics described as an erasure of historical facts.[50] Google temporarily suspended the tool's ability to generate images of people, illustrating the sociotechnical challenge of balancing the need for diverse representation with the requirement for factual and contextual accuracy.

Regulatory Models and the Sociotechnical Divide

The failures of Amazon's hiring algorithm[8], HireVue's facial analysis tools[14], and the healthcare management algorithm[45] described above have a common feature: internal auditing and voluntary corporate response did not prevent discriminatory outcomes. What regulatory structures are necessary to address this at a systemic level is contested, and the European Union and United States have taken markedly different approaches.

The EU AI Act

The EU AI Act, which entered into force in 2024, does not wait for individual algorithmic failures before intervening[51]. It assigns risk categories to AI systems prior to deployment. By August 2026, systems classified as "high-risk," including those used in employment, loan lending, and law enforcement, must meet requirements for fundamental rights impact assessments, data governance standards, and human oversight[52]. Systems classified as posing "unacceptable risk," including biometric categorization based on sensitive traits and predictive policing algorithms, are banned from deployment[53].

The U.S. Regulatory Environment

The United States has no comparable federal law. The EEOC has issued guidance noting that automated screening tools can violate the ADA and Title VII, but this guidance is non-binding. In the absence of federal legislation, state lawmakers have introduced over 1,500 AI-related bills as of 2026[54]. Legislation like the Colorado AI Act targets developers of high-risk systems that make consequential decisions about healthcare, housing, and employment [55].

These state-level efforts face resistance from tech industry lobbying groups. Arguing that a fragmented compliance environment is unworkable for developers, industry organizations have actively advocated for a unified federal standard that would preempt state AI laws[56]. This creates a volatile sociotechnical dynamic, pitting state legislatures, the primary source of binding regulation in this area, in direct conflict with an industry approach that prioritizes removing local barriers to AI development.

Regulatory Models and Accountability

The two regulatory models differ in when they require accountability. The EU treats high-risk AI as a product requiring pre-deployment safety assessment, similar to how medical devices are regulated. The U.S. model has generally required harm to occur before legal action follows. The facial recognition cases are illustrative: Buolamwini's findings were published in 2018, but the most consequential policy responses did not come until after Robert Williams's wrongful arrest became public in 2020. Whether pre-deployment regulation or a reactive civil-rights framework more effectively reduces algorithmic harm is an open question, and one that the case studies in this article do not definitively resolve.

Military AI

A January executive order by the Trump administration made void a Biden executive order that wanted to establish guardrails for AI by developing processes that protected citizens against harm, specifically protecting citizens' privacy and civil rights [57]. The executive order stated the official policy of the United States was to sustain and enhance the global AI dominance through a minimally burdensome national policy framework for AI[58]. Based on this policy, Secretary of Defense Pete Hegseth addressed senior members of the U.S. military talking of how the Department of Defense would become an "AI-first" warfighting force and that the department would unleash experimentation with America's leading AI models[59]. Organizations such as the International Committee of the Red Cross (ICRC) are concerned of AI integration with military operations which they argue could misclassify individuals as "targets" if they perceive linkage to the adversary combatants, however remote or irrelevant. For instance, if both the adversary and non-combatant studied at the same school or, shared a mutual connection, or even invents inexistent patterns, that prompts the targeting of innocent civilians [60] AI company Anthropic has also come out concerned, with a statement by CEO Dario Amodei which brings up the concern of fully autonomous weapons being used in war. Specifically, that without proper oversight, fully autonomous weapons cannot be relied upon to exercise the critical judgement that our highly troops exhibit, and that fully autonomous weapons need to be deployed with guardrails, which don't currently exist.[61] The "AI-first" U.S. military directive is still relatively new, where the effects of such policy won't be clearly seen or understood until the integrated system is used in real world battle conditions. The push to quickly integrate AI that has no safety measures in U.S. military operations, combined with high stakes missions where situations change rapidly, could lead to unintentional consequences such as the mistaken killing of non-combatants.

Conclusion

In the future, algorithmic biases will have an increased impact on human activity.[62] It is clear that unmonitored, unrestrained production and application of algorithms can be detrimental to various groups of people. The ease with which companies and industries can utilize these algorithms creates an environment where inequitable outcomes are perpetuated. Possible avenues for reducing algorithmic biases include legislation, internal auditing, and community adjudication, as well as increased awareness and a culture of responsibility.[62] Many of these avenues are primarily social and non-technical, which accords with the socio-technical nature of algorithmic bias. The increased prevalence of algorithmic biases is simply one way in which technology is impacting human life in the U.S, and more broadly, the world. Many other novel technologies with seemingly unlimited potential must be examined carefully before their widespread adoption, as failure to do so can result in harmful consequences, which is the case for many applied algorithms today. Further research is recommended on the impact of legislation on algorithmic bias, the impact of algorithmic bias in countries outside the U.S., and new bias-mitigation techniques.

References

• ↑ IBM. (n.d.). Big Data Analytics. https://www.ibm.com/analytics/hadoop/big-data-analytics.
• ↑ Cummings, Mary (2004). "Automation Bias in Intelligent Time Critical Decision Support Systems" (PDF). AIAA 1st Intelligent Systems Technical Conference (PDF). doi:10.2514/6.2004-6313. ISBN 978-1-62410-080-2. Archived from the original on 2014-11-01.
• ↑ Hao, K. (2019, February 4). This is how AI bias really happens-and why it's so hard to fix. MIT Technology Review. https://www.technologyreview.com/2019/02/04/137602/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/.
• ↑ Waller, R.R., Waller, R.L. Assembled Bias: Beyond Transparent Algorithmic Bias. Minds & Machines 32, 533–562 (2022). https://doi-org.proxy1.library.virginia.edu/10.1007/s11023-022-09605-x
• ↑ Gray, Richard. (2018, November 27). Why machines dream of spiders with 15 legs. https://www.bbc.com/future/article/20181127-the-weird-way-machines-with-ai-see-the-world
• ↑ Kumar, S., Choudhury, S. Gender and feminist considerations in artificial intelligence from a developing-world perspective, with India as a case study. Humanit Soc Sci Commun 9, 31 (2022). https://doi.org/10.1057/s41599-022-01043-5
• ↑ Hao, K. (2019, February 4). This is how AI bias really happens-and why it's so hard to fix. MIT Technology Review. https://www.technologyreview.com/2019/02/04/137602/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/.
• ↑ a b c d e Dastin, J. (2018, Oct. 10). Amazon Scraps secret AI recruiting tool that showed bias against women. Reuters. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G
• ↑ Goodman, R. (2018, Oct. 12). Why Amazon’s Automated Hiring Tool Discriminated Against Women. American Civil Liberties Union. https://www.aclu.org/blog/womens-rights/womens-rights-workplace/why-amazons-automated-hiring-tool-discriminated-against
• ↑ CareerBuilder. (2017, May 18). More than half of HR managers say artificial intelligence will become a regular part of HR in next 5 years [press release]. https://www.scribbr.com/apa-examples/press-release/
• ↑ Jobscan. (2025). Fortune 500 ATS usage report. https://www.jobscan.co/blog/fortune-500-use-applicant-tracking-systems/
• ↑ HireVue. (2025). AI-powered skill validation, video interviewing, assessments and more. https://www.hirevue.com/
• ↑ HireVue. (2022, April). AI explainability statement (short form). https://www.hirevue.com/wp-content/uploads/2022/04/HV_AI_Short-Form_Explainability_1pager.pdf
• ↑ a b Electronic Privacy Information Center. (2019, November 6). EPIC files FTC complaint against HireVue. https://epic.org/epic-files-complaint-with-ftc-about-employment-screening-firm-hirevue/
• ↑ Maurer, R. (2021, February 3). HireVue discontinues facial analysis screening. SHRM. https://www.shrm.org/topics-tools/news/talent-acquisition/hirevue-discontinues-facial-analysis-screening
• ↑ SHRM. (2024). 2024 Talent Trends Survey: Artificial intelligence findings. https://shrm-res.cloudinary.com/image/upload/AI/2024-Talent-Trends-Survey_Artificial-Intelligence-Findings.pdf
• ↑ U.S. Equal Employment Opportunity Commission. (2022, May 12). U.S. EEOC and U.S. Department of Justice warn against disability discrimination. https://www.eeoc.gov/newsroom/us-eeoc-and-us-department-justice-warn-against-disability-discrimination
• ↑ Illinois Artificial Intelligence Video Interview Act, 820 ILCS 42 (2019). https://www.ilga.gov/Legislation/ILCS/Articles?ActID=4015&ChapterID=68
• ↑ New York City Local Law 144 of 2021, Administrative Code § 20-871 (2023).
• ↑ European Parliament. (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
• ↑ BSA | The Software Alliance. (2023, January 23). Comments on the New York City Department of Consumer and Worker Protection's revised proposed rules implementing Local Law 144. https://www.bsa.org/files/policy-filings/01232023nyctools.pdf
• ↑ Najibi, A. (2020). Racial Discrimination in Face Recognition Technology. Science in the News; Harvard University. https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/
• ↑ Hao, K. (2020, June 12). The two-year fight to stop Amazon from selling face recognition to the police. MIT Technology Review. https://www.technologyreview.com/2020/06/12/1003482/amazon-stopped-selling-police-face-recognition-fight/
• ↑ a b Buolamwini, J., Gebru, T., Friedler, S., & Wilson, C. (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification *. Proceedings of Machine Learning Research, 81, 1–15. http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf
• ↑ Grother, P., Ngan, M., & Hanaoka, K. (2019). Face recognition vendor test part 3: Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects. https://doi.org/10.6028/nist.ir.8280

‌

• ↑ Jee, C. (2020, June 9). IBM says it is no longer working on face recognition because it’s used for racial profiling. Technology Review; MIT Technology Review. https://www.technologyreview.com/2020/06/09/1002947/ibm-says-it-is-no-longer-working-on-face-recognition-because-its-used-for-racial-profiling/
• ↑ Buolamwini, J. (2019). Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products – MIT Media Lab. MIT Media Lab. https://www.media.mit.edu/publications/actionable-auditing-investigating-the-impact-of-publicly-naming-biased-performance-results-of-commercial-ai-products/
• ↑ Hao, K. (2020, June 12). The two-year fight to stop Amazon from selling face recognition to the police. MIT Technology Review. https://www.technologyreview.com/2020/06/12/1003482/amazon-stopped-selling-police-face-recognition-fight/
• ↑ Hill, K. (2020, June 24). Wrongfully accused by an algorithm. The New York Times. https://www.nytimes.com/2020/06/24/technology/facial-recognition-arrest.html
• ↑ American Civil Liberties Union of Michigan. (2024). Williams v. City of Detroit. https://www.aclumich.org/en/cases/facial-recognition
• ↑ American Civil Liberties Union. (2025). More than a dozen wrongful arrests due to police reliance on facial recognition technology. https://www.aclu.org/news/privacy-technology/more-than-a-dozen-wrongful-arrests-due-to-police-reliance-on-facial-recognition-technology
• ↑ Innocence Project. (2024, January 17). Artificial intelligence is putting innocent people at risk of being incarcerated. https://innocenceproject.org/news/artificial-intelligence-is-putting-innocent-people-at-risk-of-being-incarcerated/
• ↑ American Civil Liberties Union. (2024, June 28). Williams v. City of Detroit. https://www.aclu.org/cases/williams-v-city-of-detroit-face-recognition-false-arrest
• ↑ Michigan Public Radio. (2024, June 28). Wrongful facial recognition arrest in Detroit leads to landmark settlement. https://www.michiganpublic.org/criminal-justice-legal-system/2024-06-28/it-didnt-make-sense-at-all-wrongful-facial-recognition-arrest-leads-to-landmark-settlement
• ↑ Gunn, D. (2019, Oct.). Minority Borrowers Pay More, Even under Algorithmic Lending. The Digest No. 10. www.nber.org/digest/oct19/minority-borrowers-pay-more-even-under-algorithmic-lending
• ↑ a b Counts, L. (2018, Nov. 13). Minority homebuyers face widespread statistical lending discrimination, study finds. BerkeleyHaas. newsroom.haas.berkeley.edu/minority-homebuyers-face-widespread-statistical-lending-discrimination-study-finds
• ↑ Martinez, E., & Kirchner, L. (2021, Aug. 25). The Secret Bias Hidden in Mortgage-Approval Algorithms. The Markup. themarkup.org/denied/2021/08/25/the-secret-bias-hidden-in-mortgage-approval-algorithms
• ↑ a b Park, A. L. (2019, February 19). Injustice Ex Machina: Predictive Algorithms in Criminal Sentencing. Law Meets World. https://www.uclalawreview.org/injustice-ex-machina-predictive-algorithms-in-criminal-sentencing/#_ftn2.
• ↑ Rahman, F. (2020, September 7). Compas case study: Fairness of a Machine Learning Model. Towards Data Science. https://towardsdatascience.com/compas-case-study-fairness-of-a-machine-learning-model-f0f804108751.
• ↑ a b Angwin, J., & Larson, J. (2016, December 30). BIAS in Criminal Risk Scores is Mathematically Inevitable, Researchers Say. ProPublica. https://www.propublica.org/article/bias-in-criminal-risk-scores-is-mathematically-inevitable-researchers-say.
• ↑ a b Spielkamp, M. (2017, June 12). Inspecting Algorithms for Bias. MIT Technology Review. https://www.technologyreview.com/2017/06/12/105804/inspecting-algorithms-for-bias/.
• ↑ a b Larson, J., Mattu, S., Kirchner, L., & Angwin, J. (2016, May 23). How we Analyzed the COMPAS Recidivism Algorithm. ProPublica. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm.
• ↑ Northpointe Inc. (2015, March 19). Practitioner's Guide to COMPAS Core. http://www.northpointeinc.com/downloads/compas/Practitioners-Guide-COMPAS-Core-_031915.pdf
• ↑ ORD (2020, April). Office of Research and Data, United States Sentencing Commission. Length of Incarceration and Recidivism. https://www.ussc.gov/sites/default/files/pdf/research-and-publications/research-publications/2020/20200429_Recidivism-SentLength.pdf
• ↑ a b c d e Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453. https://doi.org/10.1126/science.aax2342
• ↑ Vartan, S. (2019, October 24). Racial bias found in a major health care risk algorithm. Scientific American. https://www.scientificamerican.com/article/racial-bias-found-in-a-major-health-care-risk-algorithm/.
• ↑ Amnesty International. (2021, October 25). Xenophobic machines: Discriminatory algorithmic detector in the Dutch childcare benefit scandal. https://www.amnesty.org/en/documents/eur35/4686/2021/en/
• ↑ Heikkilä, M. (2022, March 29). Dutch environment of 'institutional racism' led to childcare benefits scandal, report says. Politico. https://www.politico.eu/article/dutch-institutional-racism-childcare-benefits-scandal-report/
• ↑ Milmo, D. (2024, February 22). Google pauses Gemini AI chatbot’s ability to generate images of people. The Guardian. https://www.theguardian.com/technology/2024/feb/22/google-pauses-gemini-ai-chatbot-image-generation-people
• ↑ Knight, W. (2024, February 27). Google’s Gemini AI image bias is a sign of a much larger problem. Wired. https://www.wired.com/story/google-gemini-ai-image-bias-historical-accuracy/
• ↑ European Parliament. (2024, March 13). Artificial Intelligence Act: MEPs adopt landmark law. European Parliament News. https://www.europarl.europa.eu/news/en/press-room/20240308IPR19015/artificial-intelligence-act-meps-adopt-landmark-law
• ↑ European Commission. (2024). AI Act. Shaping Europe's Digital Future. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
• ↑ Future of Privacy Forum. (2024). FPF Resources on the EU AI Act. FPF Resources. https://fpf.org/fpf-resources-on-the-eu-ai-act/
• ↑ MultiState. (2026). State AI Legislation Tracker 2026: All 50 States. https://www.multistate.ai/artificial-intelligence-ai-legislation
• ↑ Colorado General Assembly. (2024). Senate Bill 24-205: Consumer Protections for Artificial Intelligence. https://leg.colorado.gov/bills/sb24-205
• ↑ BSA | The Software Alliance. (2024). 2025 State AI Wave Building After 700 Bills in 2024. BSA News & Events. https://www.bsa.org/news-events/news/2025-state-ai-wave-building-after-700-bills-in-2024
• ↑ "Trump administration rolls back Biden AI executive order and launches Stargate project". American Psychological Association. February 28, 2025.
• ↑ Trump, Donald (December 11, 2025). "ENSURING A NATIONAL POLICY FRAMEWORK FOR ARTIFICIAL INTELLIGENCE". THE WHITE HOUSE.
• ↑ Hegseth, Pete (January 9, 2025). "MEMORANDUM FOR SENIOR PENTAGON LEADERSHIP COMMANDERS OF THE COMBATANT COMMANDS DEFENSE AGENCY AND DOW FIELD ACTIVITY DIRECTORS" (PDF). {{cite web}}: line feed character in |title= at position 42 (help)
• ↑ Alvarez, Jimena Sofia Viveros (September 4, 2024). "The risks and inefficacies of AI systems in military targeting support". blogs.icrc.org. {{cite web}}: |archive-date= requires |archive-url= (help); Check date values in: |archive-date= (help)
• ↑ Amodei, Dario (February 26, 2026). "Statement from Dario Amodei on our discussions with the Department of War". Anthropic.
• ↑ a b Garcia, M. (2016, Dec. 1). Racist in the Machine: The Disturbing Implications of Algorithmic Bias. World Policy Journal, 33(4), 111 - 117.

Where this page came from

This page was imported from Wikibooks. From “Lentis” on Wikibooks, by its contributors, under CC BY-SA 4.0. Changed here: set as a page; navigation and edit links left out; each image under its own licence, credited in its caption.

Nobody has written it yet — it is the source material at a new address, which is why search engines are asked to skip it and why no one earns from it. It is up for grabs: take it on, and it is yours to rewrite and to earn from.