In January 2026, DeepSeek released Engram (Conditional Memory), a technology that caused significant ripples within the large language model (LLM) community.

Its core idea is straightforward: instead of compelling LLMs to memorize general knowledge through rote learning, the approach provides them with a "plug-in memory".

The implementation involves indexing common N-grams like artificial intelligence  and photosynthesis into a hash table that enables on-demand checkup, thereby conserving computing power for reasoning tasks.

A natural question arises: can this approach be applied to the models in other domains? The answer is yes—and the results are remarkable.

Just 16 days after the Engram paper (titled Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models) was published, a Hangzhou-based R&D team launched Gengram (Genomic Engram), a module that adapts the "plug-in dictionary" concept to the field of genomics.

Code Link: https://github.com/zhejianglab/Gengram

Model Link: https://huggingface.co/ZhejiangLab/Gengram

Paper Link: https://arxiv.org/abs/2601.22203

How Does Gengram Work?

Inspired by Engram, Gengram adopts its core logic: decoupling static motif identification from dynamic context reasoning.

It constructs a learnable hash table that stores semantic vectors for all DNA fragments of length 1 to 6 (known as k-mer, e.g., ATG and CGTA). Many of these k-mers correspond to known functional elements in biology (e.g., transcription factor binding sites), effectively serving as a "Practical Genomics Phrasebook" for the model.

Unlike other domains, DNA sequences are represented by just four letters (A/T/C/G) plus an N for unknown bases, resulting in an extremely small character set. This enables Gengram to retrieve information efficiently without complex tokenization, achieving fast lookups with negligible computational overhead.

In fact, not all motifs require support from this "dictionary" due to their varying functional significance. To address this, Gengram incorporates a dynamic gating mechanism.

The model autonomously decides when to "consult the dictionary" based on context: it enables retrieval upon detecting critical motifs like exons and promoters, and disables it in non-coding regions, relying instead on intrinsic reasoning to optimize resource usage.

Testing reveals that the gating mechanism has learned to determine "when to consult references and when to think independently".

A Small Module but a Big Leap in Performance

While Gengram is a lightweight plug-in with approximately 20 million parameters, which is negligible for models with tens of billions of parameters, it delivers impressive performance gains.

Under identical training settings, Gengram-based models (8k/32k context) outperform non-Gengram ones across nearly all tasks.

Specifically, the area under curve (AUC) for predicting splice sites improved by 16.1% (from 0.776 to 0.901), while the AUC for predicting chromatin profiles (H3K36me3) increased by 22.6% (from 0.656 to 0.804).

This leap in performance endows the models with remarkable data leverage.

In a horizontal benchmarking against multiple mainstream DNA foundation models, the Gengram-integrated model achieved performance on core tasks that was comparable or even superior to models trained on data scale dozens of times larger than its own, despite using minimal training data and a small number of activated parameters. This demonstrates a significant improvement in training data efficiency.

Moreover, Gengram exhibits exceptional compatibility, enabling seamless deployment across various architectures like Dense and MoE (Mixture of Experts).

Paradigm Implications: Gengram Paves New Paths for AI‑Powered Scientific Models

Gengram's success extends well beyond addressing specific problems in genome modeling; it represents an elegant proof of concept for new paradigms in developing next-generation AI that truly understands science.

From brute-force memorization to structured knowledge plug-ins: a paradigm shift in efficiency. Traditional AI models boost capabilities by scaling up parameters and data, essentially forcing the network to memorize everything through sheer computing power. In contrast, Gengram encapsulates well-grounded structured apriori domain knowledge (such as functional motifs) into a lightweight, queryable external knowledge base, allowing the core model to offload tedious pattern memorization and instead focus on higher-level context reasoning and combinatorial innovation. This suggests a future architecture for science foundation models: a general-purpose model core + multiple domain-specific plugins, working synergistically.

Injecting inductive biases: hard-coding of biophysical laws. Gengram incorporates the structural property of B-form DNA—where the helix makes one complete turn approximately every 10.5 base pairs (corresponding to a periodicity of ~21 bp for the double helix)—into its local window mechanism. This spatial phase periodicity is then encoded as a priori knowledge, enabling the model to capture phase-specific stereochemical patterns and protein-binding preferences.

Intrinsic interpretability: bringing AI's "thought process" into view. The model moves beyond implicit statistical fitting. Thanks to explicit hash queries and gated memory pathways, it demonstrates high sensitivity to key motifs like TATA-box and poly(T) even in its shallow layers. Consequently, its internal residual intensity peaks align precisely with genomic functional boundaries, marking a transition from operating as a "black box" to exhibiting "biological cognitive footprints".

A novel path for solving long-range dependencies: from local optima to global insight. Experiments demonstrate that Gengram allows a model trained on 8K tokens to handle 32K sequences effectively. This opens new avenues for tackling intricate, long-sequence problems—such as gene regulatory element (GRE) prediction, epigenetic analysis, cross-species evolutionary analysis, and multi-omics modeling— by proving that fine-grained local perception can drive global understanding.

Gengram introduces a novel paradigm that encodes domain-specific laws into explicit architectural constraints

Gengram introduces a novel paradigm that encodes domain-specific laws into explicit architectural constraints. It demonstrates that fine-grained local structure perception can effectively overcome nominal context length limitations, enabling cost-efficient modeling of long-range dependencies.

What's the Story behind the Under-the-radar Genos Team?

Although the paper is modestly attributed to "Genos Team", the underlying powerhouse institutions can be inferred from Zhejianglab and BGI-HangzhouAI in open-source code repositories: the former points to Zhejiang Lab, a new R&D institution in Hangzhou dedicated to intelligent computing, while the latter is BGI Research, Hangzhou.

The collaboration between the two institutions has created a cross-disciplinary innovation barrier in "AI + Life Science"—a unique edge that neither pure computer science (CS) teams nor genomics-focused teams can achieve independently.

The majority of experiments in this study were built upon Genos, a human-centric genomic foundation model. According to publicly available information, Genos outperforms the state-of-the-art Evo-2 model in most metrics.