Image: gremlin/Getty Images.
After an experiment to design artificial intelligence models, a team of computer science researchers from the University of Toronto and MIT warn: AI model design presents a problem that, if not addressed quickly, could have catastrophic consequences for humans .
In summary, all AI models need to be trained on large amounts of data. However, their research shows that the current approach is deeply flawed.
AI is already influencing our daily lives
Look around and see how AI has already found its way into our everyday lives and society: Alexa reminds you of your appointments, machines diagnose the cause of your fatigue, algorithms suggest work penalties, prison sentences, etc. Without forgetting that certain professions use AI tools, for example to filter bank loan applications. In 10 years, virtually everything we do will be controlled by an algorithm.
So if you’ve applied to rent accommodation or a loan or applied for jobs tailored to your needs and you’re only getting rejections, it may not just be bad luck. It may be that these negative results are actually due to poor training of the artificial intelligence algorithms.
As Aparna Balagopalan, David Madras, David H. Yang, Dylan Hadfield-Menell, Gillian Hadfield, and Marzyeh Ghassemi point out in their article in Science, AI systems trained on descriptive data invariably make much better decisions than humans make.
And if these results are not corrected, these AI systems could have devastating effects in areas where important decisions are made.
A notable difference in judgment
Normative or descriptive?
As part of a project examining how AI models justify their predictions, the aforementioned scientists found that people in the study sometimes gave different answers when asked to assign descriptive or normative labels to data.
A “normative statement” involves a value judgment because it states what should be. For example: “He should work harder to pass his exam. » A description, for its part, is objective because it describes what is. For example: “The rose is red. »
The team was stumped and decided to investigate the question further with another experiment. This time she collected four data sets to test additional configurations.
A judgment that takes context into account
Among the data sets, the scientists chose one with photos of dogs and a regulation that prohibits aggressive dogs from entering an apartment. They then asked several groups to label normative or descriptive data using a process that reflected how the data was formed. This is where things get interesting.
Those responsible for “descriptive” labels had to ask themselves whether certain characteristics were actually present or not: aggressiveness, poor hygiene, etc., without knowing the context. In case of a positive reaction, these people unknowingly indicated that the rule had been broken and the dog was banned from the apartment. At the same time, another group was tasked with applying “normative” labels to the same images after being informed of the aggressive dog rule.
This study found that participants were more likely to judge dogs when they didn’t know the rules.
The difference in assessment is also significant. The group that advocated for “descriptive” labels (unknowingly) condemned 20% more dogs than the group that was aware of the policy.
Towards a reproduction of inequalities?
AIs have prejudices…
The results of this experiment could have serious consequences for our daily lives, especially for the less privileged. This is even more true when we add it to the already known biases of artificial intelligence.
For example, let’s analyze the danger of a “machine learning loop” that feeds an algorithm for evaluating doctoral applications. Using thousands of previous applications and data selected by supervisors, the algorithm learns which candidate profiles are generally selected: candidates with good grades, a good record, a good school… and who are white.
This does not mean that the algorithm is racist, but rather that the data used to train it is biased. Lawyer Francisco de Abreu Duarte draws a parallel to the situation of people in poverty when faced with credit: “Poor people have no access to credit because they are poor.” And since they are not recognized, they remain poor. »
Today, this bias problem is ubiquitous in machine learning technologies. This is not just about prejudices that could be described as racist, but also about discrimination, for example based on gender, age or disability.
…and judge more harshly
“Most researchers interested in artificial intelligence and machine learning take into account that human judgments are biased [car empreints de préjugés]”But this result reveals something much worse,” warns Marzyeh Ghassemi, an assistant professor of electrical engineering and computer science at MIT.
When human judgments are already biased, these models not only reproduce the already problematic biases, they go even further. In fact, the data on which they are trained has a flaw: people do not characterize a situation or a person in the same way when they know that their opinion will be factored into a judgment.
Ultimately, artificial intelligence turns out to be much tougher than humans, even if we take into account their biases. For this reason, its use to classify data could ultimately be a ticking time bomb. Especially if the AI models are not properly trained.
Source: ZDNet.com