Deep Fusion Memory learns chunk-level parametric memory and fuses it into intermediate hidden states of a frozen LLM, enabling inference without explicit retrieval. Across six QA tasks, it outperforms the strongest baseline by 12.71% on Mistral-7B-v0.3 and 11.19% on Qwen3-8B-Base. Domain adaptation across code, mathematics, and law yields average gains of 6.51 and 4.38 points over Qwen3-4B-Base and Qwen3-8B-Base, respectively.Deep Fusion Memory 学习chunk-level 参数化记忆，并将其融合到冻结大语言模型的中间隐状态，推理时无需显式检索。在六项问答任务上，相较最强基线，Mistral-7B-v0.3 和 Qwen3-8B-Base 的性能分别提升 12.71% 和 11.19%；在代码、数学与法律领域适应任务上，相较 Qwen3-4B-Base 和 Qwen3-8B-Base，平均得分分别提升 6.51 和 4.38 分。
Under review at WSDM 2027 · arXiv:2608.25521投稿至 WSDM 2027，审稿中 · arXiv:2608.25521
We show that how query expansions are integrated into dense retrievers is itself a critical research variable, not merely a downstream implementation detail. AnchorQE isolates this integration choice and improves retrieval effectiveness by up to 12.89% across TREC-DL, LoTTE, and BEIR without task-specific training.我们表明，查询扩展如何被整合进稠密检索器，本身就是一个关键的研究变量，而不仅仅是生成之后的实现细节。AnchorQE 将这一 integration choice 单独建模，在无需任务特定训练的情况下，于 TREC-DL、LoTTE 与 BEIR 上最高提升 12.89% 的检索有效性。
Large language models (LLMs) can generate query expansions without task-specific training, yet the same expansions often make a frozen dense retriever worse. We identify an underexplored factor: prior work has often focused on what text is generated, while how generated text is incorporated into dense retrievers has received less systematic attention. By holding generated expansions fixed, we show that performance degradation can often be attributed to the integration method itself. We introduce AnchorQE, a training-free method that separately encodes the original query and its expansion before interpolating them. The interpolation factor is estimated using an unsupervised online strategy that operates over a small part of the unlabeled test stream. Intuitively, our strategy assigns high expansion trust only when expansions are both retrieval-strong and consistent with the original query's retrieved evidence. We show that AnchorQE improves retrieval effectiveness by up to 12.89% when compared to widely-used expansion-only or text-level concatenation baselines across TREC-DL, LoTTE, and BEIR. Further, we show that our online strategy to estimate the interpolation factor outperforms a fixed weight tuned on a development partition by up to 3.81%.大语言模型（LLM）无需任务特定训练即可生成查询扩展，但相同的扩展文本往往会降低冻结稠密检索器的效果。我们指出了一个尚未被充分研究的因素：既有工作通常关注生成了什么文本，而对生成文本如何接入稠密检索器缺少系统性分析。通过固定生成的扩展内容，我们发现性能下降往往源于整合方式本身。为此，我们提出 AnchorQE：一种无需训练的方法，先分别编码原始查询与扩展，再对二者进行插值。插值系数由无监督在线策略估计，该策略仅使用一小部分无标签测试流；只有当扩展既具备较强检索能力，又与原始查询检索到的证据一致时，系统才会赋予其较高权重。与常用的仅扩展输入或文本级拼接基线相比，AnchorQE 在 TREC-DL、LoTTE 和 BEIR 上的检索有效性最高提升 12.89%；其在线权重估计策略也比在开发集上调优的固定权重最高提升 3.81%。
BibTeX
@misc{sun2026queryexpansion,
title = {Query Expansion Is More Than Generation: Improving Dense Retrieval through Better Integration},
author = {Sun, Siyuan and Surdeanu, Mihai},
year = {2026},
eprint = {2608.25521},
archivePrefix = {arXiv},
primaryClass = {cs.IR},
url = {https://arxiv.org/abs/2608.25521}
}
Information RetrievalDense RetrievalQuery ExpansionLLMs