A startup has overtaken larger companies to rank first on a global performance index in the second-round evaluation of South Korea's government-backed project to build homegrown AI foundation models. Still, some caution that the result is difficult to treat as final, since it covers only part of the overall assessment and does not reflect Korean-language performance.
Motif Technologies' "Motif 3" scored 47 on the latest Artificial Analysis Intelligence Index (AAII), the highest among the four companies participating in the homegrown foundation model project, according to Artificial Analysis, an overseas AI performance evaluation firm, on the 13th. Upstage's "Solar Open2" followed with 37, ahead of SK Telecom's "A.X-K2" at 35 and the LG AI Research's "K-EXAONE 2.0" at 31. Artificial Analysis is a global AI analysis firm that comprehensively evaluates AI models across a range of capabilities, including mathematics, science, coding and reasoning.
Motif 3 is a large language model with 314 billion parameters that Motif Technologies developed entirely from scratch, from pretraining through post-processing, in about five months. "In the AI model market, borders are meaningless, and only demonstrating world-class performance gives a homegrown foundation model its true meaning," CEO Lim Jung-hwan said. "Building on the confidence we have gained this time, we will complete a top-tier AI model that can compete on equal footing with frontier models from the United States and China."
The AAII score does not represent the final result of the second-round evaluation, however. In that evaluation, benchmark testing accounts for 40 of a possible 100 points, of which the AAII contributes 25 and a benchmark assessment by the National Information Society Agency (NIA) contributes 15. The remaining points come from expert evaluation (35 points) and user evaluation (25 points).
Industry officials cite the absence of Korean-language benchmarks from the AAII as a limitation. There is also debate over the reliability of benchmark scores themselves, as so-called "benchmaxxing" — intensively boosting scores through data and post-training aimed at specific benchmarks — spreads across the global AI industry.
The government plans to announce the results of the second-round evaluation soon. In the final assessment, three of the four participating teams will survive to advance to the next stage.
