KT has unveiled safety evaluation standards for a multimodal artificial intelligence (AI) model that reflect the cultural context of Korean society.
On Tuesday, KT introduced "KSAFE-MM," a Korean-style safety evaluation benchmark for multimodal large language models (MLLM) developed jointly with Korea University.
Multimodal AI is a model that understands and processes various forms of data together, including not only text but also images and voice. As AI services have recently evolved to receive both images and sentences as input simultaneously, the importance of standards to verify how models respond to harmful or sensitive questions has also grown.
KSAFE-MM consists of "KSAFE-MM-G," which converts globally common risk factors to fit Korean culture, and "KSAFE-MM-C," which reflects issues unique to Korean society. A distinctive feature is the inclusion of sensitive topics that domestic users may actually encounter, such as jeonse fraud (involving a Korean lease system requiring a large lump-sum deposit instead of monthly rent) and the Dokdo dispute. The total number of evaluation samples is 14,135, making it the largest Korean-language multimodal safety evaluation dataset in the country.
Another feature of the benchmark is its automated construction process. Many existing safety benchmarks rely on experts manually creating and reviewing questions, which carries significant time and cost burdens. In contrast, KSAFE-MM collects sensitive topics from local communities, generates questions, and then creates synthetic images. It also applies a four-stage automated pipeline that even generates "jailbreak questions" intended to bypass AI safety mechanisms. This presents a framework that can quickly build benchmarks reflecting the social context of each region, even without experts from a specific culture.
KSAFE-MM can be used for safety verification before the launch of actual AI services, red team testing, and guardrail model evaluation. The research results and dataset have been released on arXiv, a preprint repository for academic papers, and Hugging Face, an AI model sharing platform, making them available for anyone to use. KT believes the same method can be expanded to other cultures, having applied it on a trial basis to building a Japanese-language dataset.
Park Jae-hyung, head of the Frontier AI Lab at KT AX Future Technology Institute (executive vice president), said, "Releasing a safety benchmark goes beyond simply distributing data; it creates a foundation for the AI safety research ecosystem to develop together. We expect KSAFE-MM to establish itself as a common standard for verifying AI safety in the Korean language and Korean cultural context across academia and industry."
