Conditional Memory via Scalable Lookup: A New Dimension of Sparsity for Large Language Models

Author: Xin Cheng; Wangding Zeng; Damai Dai; Qinyu Chen; Bingxuan Wang; Zhenda Xie; Kezhao Huang; Xingkai Yu; Zhewen Hao; Yukun Li; Han Zhang; Huishuai Zhang; Dongyan Zhao; Wenfeng Liang

Summary

#

WHAT: Engram is a conditional memory module that modernizes classic N-gram embeddings for O(1) lookup, integrated into large language models (LLMs) to enhance the efficiency of managing static knowledge and local dependencies.

WHY: Engram addresses the inefficiency of Transformer models in simulating knowledge retrieval through computation, offering a new axis of sparsity complementary to the conditional computation paradigm (MoE). This improves performance across various domains, including knowledge retrieval, general reasoning, and coding and math tasks.

WHO: Key players include researchers and engineers from DeepSeek-AI and Peking University, who developed Engram, and the AI research community studying and implementing advanced language models.

WHERE: Engram positions itself in the market of large language models (LLMs), integrating with existing architectures like Mixture-of-Experts (MoE) to enhance efficiency and performance.

WHEN: Engram is an emerging technology gaining attention for its potential to improve language model performance. Its maturity is in the development phase, with ongoing studies and implementations.

BUSINESS IMPACT:

• Opportunities: Engram can be integrated into the existing stack to improve language model performance, reducing computational costs and enhancing knowledge retrieval efficiency.
• Risks: Competition with other conditional memory technologies and the adoption of new language model architectures could pose a threat.
• Integration: Engram can be easily integrated with existing MoE architectures, offering immediate performance improvements without the need to completely re-train models.

TECHNICAL SUMMARY:

• Core Technology Stack: Engram uses modernized N-gram embeddings, tokenizer compression, multi-head hashing, contextualized gating, and multi-branch integration. The model is implemented in Python and uses deep learning frameworks like PyTorch.
• Scalability and Architectural Limits: Engram can scale up to billions of parameters, with a model size of 175B parameters. Its efficiency is demonstrated in large-scale pre-training and inference scenarios.
• Key Technical Differentiators: Engram offers O(1) lookup for static patterns, reduces the computational depth required for knowledge retrieval, and frees attention capacity for global context. Its infrastructure efficiency allows for asynchronous prefetching of embeddings, reducing communication overhead.

Technical Details:

• Engram Pipeline: The Engram pipeline includes two main phases: retrieval and fusion. In the retrieval phase, local contexts are mapped to static memory entries via deterministic hashing. In the fusion phase, the retrieved embeddings are dynamically modulated by the current hidden state and refined through light convolution.
• Application Examples:
  • Knowledge Retrieval: Engram improves knowledge retrieval in benchmarks like MMLU, CMMLU, and MMLU-Pro.
  • General Reasoning: Shows significant gains in general reasoning benchmarks like BBH, ARC-Challenge, and DROP.
  • Coding and Math: Improves performance in coding and math benchmarks like HumanEval, MATH, and GSMK.
  • Long Context: Enhances retrieval and reasoning capabilities in long contexts, as demonstrated in benchmarks like LongPPL and RULER.
• Usage Examples:
  • Pre-training: Engram has been used in large-scale pre-training models, such as Engram-B and Engram-B, which have shown significant improvements over MoE baselines.
  • Inference: During inference, Engram allows for asynchronous prefetching of embeddings, reducing communication overhead and improving efficiency.
  • Gating Visualization: The visualization of Engram’s gating mechanism shows that the module effectively identifies and retrieves stereotypical linguistic patterns, such as multi-token entities and formulaic phrases.

Use Cases

#

• Private AI Stack: Integration into proprietary pipelines
• Client Solutions: Implementation for client projects

Resources

#

Original Links

#

Article recommended and selected by the Human Technology eXcellence team, processed through artificial intelligence (in this case with LLM HTX-EU-Mistral3.1Small) on 2026-01-27 12:30 Original source:

Related Articles

#

• Kimi K2: Open Agentic Intelligence - AI Agent, Foundation Model
• "🚀 Hello, Kimi K2 Thinking! The Open-Source Thinking Agent Model is here" - Natural Language Processing, AI Agent, Foundation Model
• Reimagining LLM Memory: Using Context as Training Data Unlocks Models That Learn at Test-Time - Natural Language Processing, AI, Foundation Model

The HTX Take

#

This topic is at the heart of what we build at HTX. The technology discussed here — whether it’s about AI agents, language models, or document processing — represents exactly the kind of capability that European businesses need, but deployed on their own terms.

The challenge isn’t whether this technology works. It does. The challenge is deploying it without sending your company data to US servers, without violating GDPR, and without creating vendor dependencies you can’t escape.

That’s why we built ORCA — a private enterprise chatbot that brings these capabilities to your infrastructure. Same power as ChatGPT, but your data never leaves your perimeter. No per-user pricing, no data leakage, no compliance headaches.

Want to see how ready your company is for AI? Take our free AI Readiness Assessment — 5 minutes, personalized report, actionable roadmap.