The Google Threat Intelligence Group (GTIG) recently published a new report “Adversarial Misuse of Generative AI,” which is well worth the read. The report shares findings on how government-backed threat actors use and misuse the Gemini web application. Although the GTIG is committed to countering threats across Google’s platforms, it is also committed to sharing findings “to raise awareness and enable stronger protections across the wider ecosystem.” This is an excellent mission.

GTIG found government adversaries, including the People’s Republic of China (PRC), Russia, Iran, and North Korea, are attempting to misuse Gemini through jailbreak attempts, “coding and scripting tasks, gathering information about potential targets, researching publicly known vulnerabilities and enabling post-compromise activities, such as defense evasion in a target environment.”

According to the report, Iranian threat actors used Gemini the most, for “crafting phishing campaigns, conducting reconnaissance on defense experts and organizations, and generating content with cybersecurity themes.” Over ten Iran-backed groups were using Gemini for these purposes.

PRC threat actors used Gemini the second most to “conduct reconnaissance, for scripting and development, to troubleshoot code, and to research how to obtain deeper access to target networks. They focused on topics such as lateral movement, privilege escalation, data exfiltration, and detection evasion.” GTIG found over 20 China-backed groups were using and misusing Gemini.

Nine North Korean-backed groups “used Gemini to support several phases of the attack lifecycle, including researching potential infrastructure and free hosting providers, reconnaissance on target organizations, payload development, and assistance with malicious scripting and evasion techniques. They also used Gemini to research topics of strategic interest to the North Korean government, such as the South Korean military and cryptocurrency. Of note, North Korean actors also used Gemini to draft cover letters and research jobs—activities that would likely support North Korea’s efforts to place clandestine IT workers at Western companies.”

Russian threat actors are using Gemini the least. Three Russia-backed groups focused on coding tasks, including converting publicly available malware into another coding language and adding encryption functions to existing code.

This research confirms our previous suspicions. Google has “shared best practices for implementing safeguards, evaluating model safety and red teaming to test and secure AI systems.” They are also actively sharing threat intelligence that will assist all users of AI tools to understand and mitigate risks of threat actors misusing AI.