Memorilla: Latent Semantic Memory for LLMs

[Submitted on 15 Oct 2026 (v1), final revised 3 Nov 2026 (this version, v3)]

View PDF HTML (experimental)

Abstract:Memory enables ample tongue models (LLMs) to use data beyond their parameters, specified as former person conversations, lengthy documents, or an agent's before actions. For a icy LLM, admission to this accumulated cognition is constricted to what a recollection scheme places in its environment window. However, gathering and combining applicable data needed for all query during keeping the model's environment abbreviated remains challenging for existing systems. Here we current Memorilla, a recollection scheme that gathers data from an complete document gathering into a fixed figure of query-dependent 'memory tokens' for a icy LLM. Memorilla encodes documents (such as person chats, publish chapters) formerly into reusable embeddings, afterward learns to recover and location query-specific data in the LLM's input embedding space. Using lone 16 recollection tokens, Memorilla achieves the finest achievement against baselines including Mem0 and MemoRAG on five of eight benchmarks spanning personalization, long-document and knowledge-intensive inquiry answering. Notably, on FactKG, it reaches 84.33% accuracy versus 80.43% alongside full-context inference, during using 108x small tokens to portray the origin documents. Furthermore, Memorilla can facilitate continual learning in agentic systems anywhere the document figure (the agent's former turns) increases throughout the trajectory. In a TextWorld environment, we display that Memorilla reduces the agent's inclination to roam between rooms it has already explored, and improves its average achievement charge from 18.6% alongside full-history in-context learning to 24.2%. Overall, Memorilla hiv icy LLMs in accumulating and using cognition efficiently throughout long-context interactions and takes a stage towards construction continually learning AI systems.