IT Security: Training Weaknesses in Artificial Intelligence
Sources of error in machine learning are too often overlooked and results are distorted as consequence (Source: Karlsruhe Institute of Technology – Press Releases)
Machine learning (ML) is successfully used in many digital application areas, including computer security. Artificial intelligence (AI) methods are also used in IT security research. However, there are weaknesses in training AIs, as researchers at the Karlsruhe Institute of Technology (KIT) and other international research institutions have discovered.
“Self-learning algorithms are often superior to traditional methods. For example, ML is used to investigate novel attacks on computer systems and adapt defensive measures to the threats,” says Christian Wressnegger of the KASTEL Security Research Labs at KIT. However, results could be skewed, for example by feeding the AI unrepresentative data that misrepresents the real-world occurrence of viruses. “In app stores of the major providers, apps with security risks occur less often than in alternative sources with lower security standards,” the junior professor explains.
In addition, he says, one must take into account that attackers usually try to circumvent or even attack defenses. “For example, a learning antivirus program trained on such incomplete data could prove useless in practice.”
Lack of awareness of proper use of machine learning
Wressnegger and his team worked with international partners to examine 30 recent research papers that used ML for IT security and were published at prestigious computer and systems security conferences. All had failed to address one or more sources of error. “There is a lack of awareness of the difficulties of applying machine learning correctly,” the cybersecurity expert states.
The original press release can be found at:
IT-Sicherheit: Trainingsschwächen bei Künstlicher Intelligenz (only in german)
The original publication can be found at (Open Access):
Ahmed E. Zoheir, Laura Meisch, Marta Velaz Martín, Christoph Bickmann, Alexei Kiselev, Florian Lenk, Anne-Kristin Kaster, Kersten S. Rabe, und Christof M. Niemeyer: Macroporous Silicone Chips for Decoding Microbial Dark Matter in Environmental Microbiomes. ACS Applied Materials & Interfaces, 2022. Weblink
Localization in the Helmholtz Research Field Information:
Helmholtz Research Field Information, Program 1: Engineering Digital Futures, Topic 3: Engineering Secure Systems
Contact:
Jun.-Prof. Dr. Christian Wressnegger
Institute of Information Security and Dependability (KASTEL)
Karlsruher Institute of Technology (KIT)
Phone: +49 721 608-41330
E-Mail: c.wressnegger@kit.edu
