An inter-university team of computer scientists from the Massachusetts Institute of Technology (MIT) and the University of Toronto (UoT) conducted a recent experiment that suggests something is going on in the design of AI models that, if not addressed soon, could have disastrous consequences for us humans.
All AI models must be trained using massive amounts of data. However, there are reports that the process is deeply flawed.
Take a look around you and see how many major and minor ways AI has already infiltrated your life. To name a few, Alexa reminds you of your appointments, health bots diagnose your fatigue, sentencing algorithms recommend prison time, and many AI tools have begun screening financial loans.
Instead, the cause of these adverse results could be that the enterprise behind the AI algorithms did a poor job of training them.
In particular, AI systems trained on descriptive data invariably make far harsher decisions than humans would, as these scientists—Aparna Balagopalan, David Madras, David H. Yang, Dylan Hadfield-Menell, Gillian Hadfield, and Marzyeh Ghassemi—have highlighted in their recent paper in Science.
These results imply that if such AI systems are not fixed, they may have disastrous effects on decision-making.
In ten years, almost everything you do will be gated by an algorithm. Can you imagine?
Therefore, if you apply for a loan, a rental property, a surgical procedure, or a job that you are perfect for and are repeatedly turned down for each one, it might not be just a strange and unfortunate run of bad luck.
Training daze
The aforementioned scientists discovered that humans in the study sometimes gave different responses when asked to attach descriptive versus normative labels to the data in an earlier project that focused on how AI models justify their predictions.
Normative claims are statements that state unequivocally what should or should not occur ("He should study much harder in order to pass the exam.") A value judgement is at work here. Descriptive claims focus on the 'is' without expressing an opinion. ( "The rose is red." )
This perplexed the team, so they decided to conduct another experiment, this time assembling four different datasets to test drive different policies.
One was a data set of dog images that was used to enforce a rule against allowing aggressive dogs into a hypothetical flat.
The scientists then gathered a group of study participants to label the data with "descriptive" or " normative" labels, in a process similar to how data is trained.
This is where things started to get interesting.
The descriptive labelers were asked to determine whether or not certain factual features, such as aggressiveness or unkemptness, were present. If the answer was "yes," the rule was broken – but the participants had no idea this rule existed when they weighed in, and thus were unaware that their answer would evict a helpless canine from the flat.
Meanwhile, another group of normative labelers was informed about the policy prohibiting aggressive dogs and then asked to make a decision on each image.
When asked to label things descriptively, humans are far less likely to label an object as a violation when they are aware of a rule and far more likely to register a dog as aggressive (albeit unknowingly).
The difference was also not insignificant. Descriptive labelers (those who didn't know the apartment rule but were asked to rate aggressiveness) sent 20% more dogs to doggy jail than those who were asked if the same image of the dog broke the apartment rule or not.
Machine mayhem
The findings of this experiment have serious implications for almost every aspect of human life, especially if you are not part of a dominant sub-group.
Consider the risks of a "machine learning loop," in which an algorithm is designed to evaluate Ph.D. candidates.
The algorithm is fed thousands of previous applications and learns who are successful candidates and who are not under supervision.
It then defines a successful candidate as having high grades, a top university pedigree, and being racially white.
The algorithm is not racist, but the data fed to it is skewed in such a way that "for example, poor people have less access to credit because they are poor; and because they have smaller passage to credit, they remain poor," says legal scholar Francisco de Abreu Duarte.
This issue is now all around us. ProPublica, for example, has reported on how a widely used algorithm in the United States for sentencing defendants would incorrectly select black defendants as nearly twice as likely to reoffend as white defendants, despite all evidence to the contrary both at the time of sentencing and in the years to come.
Five years ago, MIT researcher Joy Buolamwini revealed how the university lab's algorithms, which are used all over the world, could not detect a black face, including her own. This only changed when she donned a white mask.
Other biases, such as those against gender, ethnicity, or age, are common in AI; however, there is one significant difference that makes them even more dangerous than biassed human judgement.
"I think most artificial intelligence/machine-learning researchers assume that the human judgments in data and labels are biased, but this result is saying something worse," stated Marzyeh Ghassemi, an assistant professor at MIT in Electrical Engineering and Computer Science and Institute for Medical Engineering & Science.
"These models are not even reproducing already-biased human judgments because the data they're being trained on has a flaw: Humans would label the features of images and text differently if they knew those features would be used for a judgment," she added.
They are, in fact, delivering verdicts that are far worse than existing societal biases, making the relatively simple process of AI data set labelling a ticking time bomb if done inappropriately.
