Artificial intelligence tools are more susceptible to targeted attacks than previously anticipated, effectively forcing AI systems to make poor choices.
The term "adversarial attacks" refers to the manipulation of data being fed into an AI system in order to create confusion in the system. For example, someone might know that putting a specific type of sticker at a specific spot on a stop sign could effectively make the stop sign invisible to an AI system. Hackers can also install code on an X-ray machine that alters image data, leading an AI system to make inaccurate diagnoses.
“For the most part, you can make all sorts of changes to a stop sign, and an AI that has been trained to identify stop signs will still know it’s a stop sign,” stated Tianfu Wu, coauthor of a paper on the new work and an associate professor of electrical and computer engineering at North Carolina State University. “However, if the AI has a vulnerability, and an attacker knows the vulnerability, the attacker could take advantage of the vulnerability and cause an accident.”
Wu and his colleagues' latest study aims to determine the prevalence of adversarial vulnerabilities in AI deep neural networks. They discover that the vulnerabilities are far more common than previously believed.
What's more, we found that attackers can take advantage of these vulnerabilities to force the AI to interpret the data to be whatever they want. Using the stop sign as an example, you could trick the AI system into thinking the stop sign is a mailbox, a speed limit sign, a green light, and so on, simply by using slightly different stickers—or whatever the vulnerability is, Wu added.
This is incredibly important, because if an AI system is not dependable against these sorts of attacks, you don't want to put the system into operational use—particularly for applications that can affect human lives.
The researchers created a piece of software called QuadAttacK to study the sensitivity of deep neural networks to adversarial attacks. The software may be used to detect adversarial flaws in any deep neural network.
In general, if you have a trained AI system and test it with clean data, the AI system will behave as expected. QuadAttacK observes these activities to learn how the AI makes data-related judgements. This enables QuadAttacK to figure out how the data can be modified to trick the AI. QuadAttack then starts delivering altered data to the AI system to observe how it reacts. If QuadAttacK discovers a vulnerability, it can swiftly make the AI see whatever QuadAttacK desires.
The researchers employed QuadAttacK to assess four deep neural networks in proof-of-concept testing: two convolutional neural networks (ResNet-50 and DenseNet-121) and two vision transformers (ViT-B and DEiT-S). These four networks were picked because they are widely used in AI systems across the globe.
“We were surprised to find that all four of these networks were very vulnerable to adversarial attacks,” Wu stated. “We were particularly surprised at the extent to which we could fine-tune the attacks to make the networks see what we wanted them to see.”
QuadAttacK has been made accessible by the research team so that the research community can use it to test neural networks for shortcomings.
