Artificial intelligence is transforming how organisations make decisions, from approving bank loans to determining which patients receive medical care. While these technologies promise greater efficiency and consistency, they also raise troubling questions about fairness. As AI becomes more influential in everyday life, researchers are exploring how organisations can prevent automated systems from reinforcing existing inequalities.
A new study involving researchers at Penn State University introduces a framework designed to help organisations identify and manage bias in AI-powered decision-making. Published in the journal MIS Quarterly, the research proposes treating AI fairness as an ongoing responsibility rather than a problem that can be solved through a single technical adjustment.
According to Jing Tian, assistant professor of information systems at Penn State’s Smeal College of Business, achieving fairness requires balancing competing interests. Although automated systems can process information quickly and consistently, they may also reproduce discrimination embedded in historical data. Previous applications in healthcare and recruitment have demonstrated how algorithms can disadvantage particular demographic groups.
The researchers argue that fairness is far more complicated than simply eliminating biased information. Different stakeholders often have conflicting expectations about what constitutes an equitable outcome. Treating everyone identically, for example, may fail to address disadvantages experienced by certain communities, while prioritising equal outcomes could create tensions with efficiency, accuracy and financial objectives.
These challenges become particularly apparent in healthcare. An emergency department physician might favour applying identical assessment criteria to every patient, while an advocate might emphasise improving access for underserved populations. Hospital administrators, meanwhile, must consider limited resources and the number of patients requiring treatment. Each perspective introduces legitimate concerns that cannot always be reconciled.
To address these competing priorities, the researchers developed a design theory called Fairness Adaptation through AI-augmented Responsiveness, or FAIR. The approach divides automated decision-making into three interconnected components: the information systems they collect, the predictive models they employ and the rules used to convert predictions into actual decisions.
Rather than examining AI as an impenetrable system, FAIR encourages organisations to investigate each component individually. Automated monitoring tools can detect emerging disparities and suggest possible improvements, while human specialists assess their significance, consider potential consequences and approve appropriate adjustments. This collaboration combines technological efficiency with human judgement and accountability.
The framework also recommends establishing organisation-wide standards for monitoring, documentation and oversight while allowing individual departments to adapt procedures to their circumstances. Importantly, the researchers emphasise that human supervision should increase alongside the potential consequences of automated decisions. Recommending consumer products, for instance, requires less scrutiny than determining access to essential medical treatment.
Successful implementation would require investment in employee training, dedicated governance teams and accessible channels for public feedback. Organisations would also need clearly defined responsibilities and procedures for responding when unfair outcomes emerge. Such measures could help businesses recognise problems before they become deeply embedded in automated processes.
Ultimately, the study suggests that AI fairness should be managed much like cybersecurity: through continuous monitoring, adaptation and accountability. Although implementing these safeguards may initially demand considerable resources, the researchers believe sustained human involvement could help organisations develop more trustworthy automated systems as technology and societal expectations evolve.
More information: Arun Rai et al, FAIR: A Design Theory for Artificial Intelligence Fairness, MIS Quarterly. DOI: 10.25300/MISQ/2026/17971
Journal information: MIS Quarterly Provided by Penn State