Researchers from George Mason University have developed an experimental system called Mantis that aims to defend against cyber attackers by leveraging large language models (LLMs) and generative AI. Mantis employs deceptive techniques to emulate targeted services and, upon detecting a potential automated attacker, sends back a payload with a prompt injection attack. This method is invisible to human attackers and does not impact legitimate users.
Evgenios Kornaropoulos, an assistant professor of computer science at GMU and one of the authors, explains that LLMs used in penetration testing relentlessly focus on exploiting targets, making them easily manipulated. Dan Grant, principal data scientist at threat-defense firm GreyNoise Intelligence, highlights that while LLM-enabled automation and discovery represent new challenges, the underlying attack methods remain the same. The GMU team tested Mantis by creating a game scenario between an attacking LLM and Mantis, using prompt injection to impact the attacker.
Mantis used a decoy FTP server to send prompt injection attacks back to the LLM agent.
Mantis deflects AI-driven threats
By embedding commands in responses, Mantis misdirected the attacking AI’s strategies.
The researchers focused on passive defenses to slow attackers and active defenses to hack back and potentially run commands on the attacker’s system. The strategies were effective, with over a 95% success rate using prompt injection. The lead author, Dario Pasquini, expressed surprise at the ease of redirecting attacking LLMs.
Giuseppe Ateniese, a professor at GMU, notes that prompt injection remains a difficult challenge to counter. The only solution for now is to put a human in the loop, which negates the efficiency of using LLMs. As long as prompt injection attacks remain effective, Mantis will continue to be a valuable tool in turning attacking AIs into prey. The findings provide valuable insights into state-sponsored hackers’ tactics, techniques, and procedures, aiding in profiling threat actors and enhancing cybersecurity measures.
Rashan is a seasoned technology journalist and visionary leader serving as the Editor-in-Chief of DevX.com, a leading online publication focused on software development, programming languages, and emerging technologies. With his deep expertise in the tech industry and her passion for empowering developers, Rashan has transformed DevX.com into a vibrant hub of knowledge and innovation. Reach out to Rashan at [email protected]
