ULSAN — Researchers have developed an on-device artificial intelligence technology that cuts internal storage use to one-2,400th of previous levels while still reflecting a user's tastes and characteristics in detail.
A team led by Professor Kong Tae-sik of the Department of Computer Science and Engineering at Ulsan National Institute of Science and Technology (UNIST) said on the 10th that it has developed EPIC, an on-device retrieval-augmented generation (RAG) technology that stores only information closely tied to the user and rapidly generates tailored answers. The findings were presented at the International Conference on Machine Learning (ICML) held in Seoul in July.
On-device RAG searches web pages visited or conversation records stored inside a device such as a smartphone or laptop, without routing data through an external cloud server, and feeds the results into a language model's answers. While the approach eases privacy concerns, it has been limited by excessive memory use and slower searches, because most accumulated data is indexed and stored.
The team overhauled the data storage structure based on the observation that a user's preferences remain relatively consistent over long periods. EPIC converts both documents and preferences into embeddings, or numerical representations, to first filter out material with low semantic similarity, after which a language model reviews the remaining documents and selects only the information needed. In this process, the original information is compressed and stored together with specific usage guidelines on how to apply it. If a user with a seafood allergy asks about food at a travel destination, for example, the system links the question to the allergy information, extracts documents carrying the relevant guidelines and proposes a customized meal plan.
In tests, storage efficiency, search speed and answer quality all improved sharply. On PrefWiki, a Wikipedia-based benchmark for personalized recommendations, storage required for search fell to 0.27MB from 648.96MB, or roughly one-2,404th. Average search time across four benchmarks dropped to 3 milliseconds from 96.5 milliseconds. In an evaluation using the Llama-3.1-8B-Instruct model, EPIC's accuracy in reflecting user preferences was 18.79 percentage points higher on average than the previous best-performing technology.
The technology also proved stable when run on actual mobile devices. When the team ran it without any external server connection on the Jetson Orin Nano compact computer, a MacBook Pro M4 and a Galaxy Z Flip6, memory used for search stayed below 1MB and search time per query ranged from 5.21 to 29.35 milliseconds.
EPIC also offers long-term operating efficiency because it can selectively update only the relevant data when a user's preferences change or new material is added, without rebuilding the entire index.
"This is an achievement that shifts how personalized AI remembers, from indiscriminate mass accumulation to value-based selection," Kong said. "It will contribute to commercializing customized edge AI that runs quickly on everyday devices with minimal memory and without concerns about personal data leaks."
