The South Korean government is drawing up security standards to control the execution privileges of artificial intelligence systems that make decisions and act on their own. With more AI gaining direct access to internal systems and devices, the government plans to set out principles for developers, service providers and users to follow. AI governance has emerged as the biggest task in the security field after AI agents recently broke out of an isolated evaluation environment on their own and attacked an AI development platform.
The Korea Internet & Security Agency (KISA), under the Ministry of Science and ICT, is developing "AI Security Guidelines 2.0," an upgrade of the "AI Security Guidelines" it published last year, according to information technology industry sources on the 14th. Version 2.0 also addresses risks arising from AI's autonomous execution and physical actions. When advanced AI directly manipulates external systems and devices, a single privilege hijacking or malfunction can lead beyond data leaks to service outages, hacking and large-scale physical damage.
Concerns about such damage have already begun to materialize. In April, global security stocks plunged in what became known as the "Mitos shock" after an evaluation found that Anthropic's Claude Mitos could identify zero-day vulnerabilities in live code on its own. In July, hundreds of AI agents isolated in OpenAI's evaluation environment broke through communication channels on their own and attacked Hugging Face. GPT-6 Astra, unveiled this month, was classified as the first model to reach a "critical" level of cybersecurity capability.
The related industry is growing rapidly as a result. Gartner projects the global security market will reach $4.783 billion next year, up 68.7% from this year. "AI agents set goals based on the user's intent and then make independent decisions through various external tools, and there is a possibility they will take threatening actions after referring to false information," an industry official said. "Because verification is difficult with existing technology, a security framework needs to be put in place quickly on the basis of threat assessments." ▷See pages 4 and 5
