Artificial intelligence is quietly becoming a gatekeeper in many parts of everyday life. From deciding which job applicants make it through an initial screening to helping lenders assess borrowers, algorithms are increasingly influencing who gets opportunities — sometimes before a human decision-maker ever enters the picture.

That growing reliance on automated decision-making has raised concerns about what researchers call “algorithmic monoculture”: a situation in which many organisations rely on the same algorithm to make similar decisions. Critics worry that if one hiring algorithm rejects a candidate, for example, the same person could face rejection everywhere the algorithm is used.

But researchers at MIT argue that algorithmic monoculture is not necessarily harmful in every situation. After examining several common objections through mathematical models and simulations, they found that many of the potential problems depend heavily on how the algorithm is designed, how accurate it is and where it is being used. The research appears in Philosophical Perspectives.

“A trend toward algorithmic monoculture is a realistic scenario, and a really important issue that is being brought about by the use of AI, but it is hard to say in the abstract whether monoculture would be a bad thing,” says study co-author Brian Hedden, a professor at MIT. “It depends on the details.”

One frequently cited concern is systematic exclusion. If companies use the same hiring algorithm, applicants rejected by one employer might also be rejected by others. Yet the researchers found that monoculture does not necessarily reduce the overall number of people hired. In some circumstances, it could even strengthen applicants’ bargaining power because employers may find themselves competing for the same candidates, potentially driving up wages.

Other concerns also depend on how a system operates. A monoculture could limit applicants’ ability to adapt if the same résumé is automatically circulated among employers. But that problem could be reduced if candidates are allowed to revise and resubmit their applications. Similarly, although a common algorithm might encourage people to “game” the system, the researchers note that applicants could also try to exploit several different algorithms in a more diverse system.

A potentially more serious problem involves what the researchers describe as informational echo chambers. Independent decision-makers can benefit from the “wisdom of crowds,” with different approaches uncovering candidates that others overlook. If every employer uses the same system, hiring decisions may become increasingly similar, reducing experimentation and discovery. Introducing some randomness into an algorithm could help preserve exploration.

Accuracy may ultimately be crucial. A single highly effective algorithm could sometimes outperform several weaker ones. The researchers found that an “ensemble algorithm,” combining the assessments of multiple hiring algorithms into one system, could in some simulations perform as well as or better than a system in which employers use different algorithms. Whether such an approach would work effectively in the real world remains uncertain. The researchers say more empirical work is needed to understand when algorithmic monoculture creates risks — and when sharing a common system might actually offer advantages.

More information: Brian Hedden et al, Algorithmic Monoculture and Its Critics, Philosophical Perspectives. DOI: 10.1111/phpe.70018

Journal information: Philosophical Perspectives Provided by Massachusetts Institute of Technology