The Ministry of Science and ICT has released additional details from the second-stage evaluation of its "Sovereign AI Foundation Model" project. The government had initially announced only the eliminated team, without disclosing the top performer or scores in each category. But amid disputes over the transparency of the evaluation, it decided to disclose the top company and highest score in each category, as it had done in the first-stage evaluation.
The LG AI Research institute posted the highest scores in two categories — expert review and public rating — in the project's second evaluation, the ministry said on the 21st. In the expert review, which carried a maximum of 35 points, LG AI Research ranked first among the four teams with 29.5 points. The overall average was 28.75 points. The expert review involved 10 outside experts — three from industry, five from academia and two from research institutions — who conducted about five days of written assessment and Q&A.
The experts did not judge the models solely on benchmark performance. They allotted 10 points to development strategy and technology, 10 points to development outcomes and future plans, and 15 points to the domestic AI ecosystem, global impact and contribution plans. The assessment covered technological independence, resource-use efficiency, the usability and practicality of the models, and their potential to expand into various industries and services, as well as plans to support universities, startups and the domestic AI chip ecosystem and to enter global markets.
LG AI Research also received the highest score in the public rating, with 7.6 points, against an average of 7.03. The government set gender and age quotas based on national resident registration statistics, then used a system to randomly select a 200-member public evaluation panel, of whom 185 took part in the actual rating. The evaluators used the websites running each team's AI model firsthand, focusing not on the UI and UX but on the substance and quality of the content the models generated.
Other companies, by contrast, led on the benchmarks that measure the models' objective performance. On the AAII benchmark, which uses the evaluation methods of major global leaderboards, the four teams averaged 9.48 points, with Motif Technologies ranking first at 11.9 points. On the NIA benchmark, which assessed seven areas — mathematics, knowledge, long-text comprehension, safety, reliability, Korean language and instruction-following — SK Telecom (017670) ranked first with 13.4 points, against an average of 13.05.
SK Telecom also stood out in the professional user assessment, which involved startup chief executives who had used AI models extensively in actual development work. Of 50 people selected after checks for conflicts of interest and other issues, 49 took part in the assessment, and SK Telecom received the highest score at 11.6 out of 15 points. The four teams averaged 10.53 points.
When the ministry announced the second-stage evaluation results on the 18th, it did not disclose the top company or scores in each category, citing concerns that doing so could stigmatize lower-ranked companies. But as calls continued for greater transparency in the evaluation process and results, it decided to broaden the scope of disclosure. "We balanced the transparency of the evaluation results against unexpected direct and indirect harm to some companies from disclosing them," the ministry said. "We decided to disclose the top company and its score in each evaluation category at the same level as in the first-stage evaluation announcement."
Meanwhile, the second evaluation advanced three teams — Upstage, SK Telecom and LG AI Research — to the next stage, while Motif Technologies was eliminated.
