FairCompass: A Visual Tool That Guides Practitioners Through Auditing a Model for Fairness

Many tools exist for finding unfairness in machine learning models, yet the paper behind FairCompass says there is reportedly little practical use of them in real applications. Its proposed fix is a human-in-the-loop prototype that does not just compute fairness numbers: it also helps the person doing the audit choose which fairness definition fits the job.

Why existing tools fall short

The paper reviews tools such as Aequitas, AI Fairness 360, Google's What-If Tool, Fairlearn, FairSight and Microsoft's fairness checklists, and names four recurring criticisms:

• Too technical. They stress statistical and algorithmic definitions, although fairness is also a social problem that needs human values.
• Fairness is subjective. One survey defines 23 types of bias and 10 fairness metrics, and some definitions cannot hold at the same time.
• Hard to put into practice. Studies suggest the tools often miss what practitioners need.
• Little support. Practitioners can feel overwhelmed, unqualified to decide, or afraid of missing a bias they do not know how to look for.

How FairCompass works

The system is built around two main tabs and a collapsible sidebar. In the sidebar the user views how each feature is distributed, generates subgroups (for example by sex) and saves sets of subgroups to revisit. The groups carry across both tabs.

• Subgroup Exploration gives an overview of the subgroups, suggests similar ones and compares them in detail on metrics such as false positive and false negative rates.
• Fairness Compass shows a decision tree of fairness definitions. Answering its questions, such as whether there is a relevant policy or whether base rates are equal, leads to a suitable definition. Selecting a metric draws bar or scatter plots for the active subgroups.

Figure from the paper: the Subgroup Exploration tab.

Figure from the paper: the Fairness Compass tab.

The authors tie the two tabs together with an Exploration, Guidance and Informed Analysis loop, built on the Knowledge Generation Model for Visual Analytics. The user forms a hypothesis in exploration, learns which definitions apply in guidance, and then returns to the data with that knowledge.

Figure from the paper: the extended model.

A walkthrough on an income model

The evaluation is a single use case. A practitioner at a financial institution must judge whether an income-prediction model, a simple two-layer neural network trained on the Adult Income dataset from the UCI Machine Learning Repository, is fair enough to feed a loan-size application.

1. She finds that the sample has twice as many men as women and splits the data into subgroups by sex.
2. In the decision tree, she works through policy, equal base rates and explaining variables, and lands on conditional statistical parity: equal proportions of favourable outcomes for men and women with the same occupation and working hours.
3. For every occupation, the model predicted proportionally fewer women to earn more than $50,000. For Exec-managerial staff the gap in negative rate was 34%. Women in that group had an almost 25% lower false negative rate and an almost 20% higher false positive rate.
4. In a second round she compares men and women who work 40, 45 or 50 hours. The negative rate for men is higher than for women by more than 15% at all three, and the error-rate pattern repeats.

She concludes the model is unfair and should not be used for home-loan decisions, and plans to check other sensitive attributes before reporting. The authors stress that another practitioner could follow a different path.

Limits the authors name

• The guidance stage covers only part of the fairness landscape, and the decision tree may not fit every context. For recidivism prediction, for example, it ends at demographic parity, which has been argued to be flawed in practice.
• Over-reliance on the tree could itself become a bias, if users assume its answer is always right.
• Related tools it builds on, such as FairVis, handle only binary classifiers, so rankings or recommendations cannot be audited this way.
• Responsibility for fairness should sit with organisations, not only with individual practitioners.

Sources

• "FairCompass: Operationalising Fairness in Machine Learning," arXiv:2312.16726, CC BY 4.0 (authors not listed in the imported text): https://arxiv.org/abs/2312.16726. Changed here: condensed and rewritten in hubnx's own words; three figures from the paper are shown.
• Authors of the original: Jessica Liu, Huaming Chen, Jun Shen.