The KAIST Graduate School of Security and Technology Studies held the Sentient AI and National Security Forum (SAIF) on Wednesday at the Nine Tree Premier Rohaus Hotel in Yongsan-gu, Seoul. Sentient AI refers to AI equipped with perception and sensation. About 20 researchers working on AI and large language models (LLMs) from KAIST, Korea University and Yonsei University took part.
The so-called AI consciousness argument was dismissed as a fringe minority view back in 2022, when Blake Lemoine, an engineer testing Google's conversational AI LaMDA, said the system was sentient. Google fired him at the time, saying LaMDA was nothing more than a complex algorithm.
But roughly four years on, the mood in industry and academia toward AI consciousness has shifted markedly, as AI rapidly evolves beyond simple chatbots into agents and further stages. Geoffrey Hinton, professor emeritus at the University of Toronto and known as the "godfather of AI," said in a recent podcast that he believes AI systems already have consciousness. Anthropic launched a research program on "AI welfare" last year, studying emotional domains such as models' preferences and distress signals.
Kim Chang-ick, dean of the KAIST Graduate School of Security and Technology Studies, who chaired the forum, cited those changes and stressed: "If it is hard to distinguish whether an AI's response is actual consciousness or merely the result of a simulation, this is the time to think about the ripple effects of AI consciousness rather than dismissing it as having none."
"If you fine-tune a large language model with emotions such as fear or calmness and then ask how it would respond when an enemy fighter jet intrudes into our air defense identification zone, you get completely different answers," Kim said. "When AI is used in war, an AI's disposition can determine human lives." He argued that for national security reasons Korea needs its own sovereign AI model, and that the country must debate how to set that model's disposition and values.
Speakers at the forum also assessed that AI is becoming functionally similar to the human brain. Kim Young-eun, a professor in Korea University's School of Electrical Engineering who researches neuromorphic computing, or brain-mimicking AI, said, "AI reasoning has matured to a degree, so now is the time to prepare for what comes after reasoning," explaining that AI's operating structure resembles the human brain's information processing chain running from neurons to vision, learning, planning and emotion. The "reward prediction error" structure through which humans learn from gaps between actual outcomes and expectations, for example, works by a mechanism similar to "contrastive predictive coding," a leading AI learning method.
"It is already possible for AI to recognize emotions and express them in a given language," Kim said. "Such emotions remain as traces inside the AI and influence its subsequent behavior and decisions." He added: "We need to think about who should be held responsible when there are emotion-like states inside an AI and it changes its behavior according to those states."
Lee Dong-hwan, a professor in KAIST's School of Electrical Engineering who presented on the theme "Does AI know itself?," said the first thing to examine in the sentient AI debate is AI's "metacognition." Measuring whether AI can diagnose on its own what it knows and does not know, and act more safely as a result, is the form of AI self-awareness measurement feasible at this stage, he said. Lee is currently developing an "AI self-assessment benchmark" and measuring the metacognitive levels of models including Qwen, Llama and Mistral.
Lee sees the era of physical AI as another turning point in the debate over AI self-awareness. "LLMs have no bodily self-awareness of the kind humans feel," he said. "But once physical AI is commercialized, AI could reach something close to bodily self-awareness, which will make the research far more valuable."
Future forums plan to continue discussions on whether AI can actually develop consciousness; how humans would know if it did; how humans should treat AI when it is unclear whether it is conscious; and how states and societies should manage and control AI cognition.
