What is data poisoning? Attacks thatcorrupt machine learning models

Artificial Intelligence
Image credit: source

Machine learning adoption exploded over the past decade, driven in part by the rise of cloud computing, which has made high performance computing and storage more accessible to all businesses. As vendors integrate machine learning into products across industries, and users rely on the output of its algorithms in their decision making, security experts warn of adversarial attacks designed to abuse the technology.

Most social networking platforms, online video platforms, large shopping sites, search engines and other services  have some sort of recommendation system based on machine learning. The movies and shows that people like on Netflix, the content that people like or share on Facebook, the hashtags and likes on Twitter, the products consumers buy or view on Amazon, the queries users type in Google Search are all fed back into these sites’ machine learning models to make better and more accurate recommendations.

It’s not news that attackers try to influence and skew these recommendation systems by using fake accounts to upvote, downvote, share or promote certain products or content. Users can buy services to perform such manipulation on the underground market as well as “troll farms” used in disinformation campaigns to spread fake news.

“In theory, if an adversary has knowledge about how a specific user has interacted with a system, an attack can be crafted to target that user with a recommendation such as a YouTube video, malicious app, or imposter account to follow,” Andrew Patel, a researcher with the Artificial Intelligence Center of Excellence at security vendor F-Secure explained in a blog post. “As such, algorithmic manipulation can be used for a variety of purposes including disinformation, phishing scams, altering of public opinion, promotion of unwanted content, and discrediting individuals or brands. You can even pay someone to manipulate Google’s search autocomplete functionality.”

What is data poisoning?

Data poisoning or model poisoning attacks involve polluting a machine learning model’s training data. Data poisoning is considered an integrity attack because tampering with the training data impacts the model’s ability to output correct predictions. Other types of attacks can be similarly classified based on their impact:

  • Confidentiality, where the attackers can infer potentially confidential information about the training data by feeding inputs to the model
  • Availability, where the attackers disguise their inputs to trick the model in order to evade correct classification
  • Replication, where attackers can reverse-engineer the model in order to replicate it and analyze it locally to prepare attacks or exploit it for their own financial gain

The difference between an attack that is meant to evade a model’s prediction or classification and a poisoning attack is persistence: with poisoning, the attacker’s goal is to get their inputs to be accepted as training data. The length of the attack also differs because it depends on the model’s training cycle; it might take weeks for the attacker to achieve their poisoning goal.

Copyright © 2021 IDG Communications, Inc.

(Excerpt) Read more Here | 2021-04-12 14:00:00

Leave a Reply

Your email address will not be published. Required fields are marked *