A project by ServiceNow Research

ReTreever: Hierarchical Retrieval at Scale

Hierarchical retrieval that keeps the interpretability of tree-based search without giving up the speed of dense retrieval.

Shubham Gupta^1,2,3, Zichao Li^1,4,3, Tianyi Chen¹, Cem Subakan^2,3, Siva Reddy^1,4,3, Perouz Taslakian^1,4,3, Valentina Zantedeschi^1,2

¹ServiceNow Research · ²Université Laval · ³Mila - Québec AI Institute · ⁴McGill University

Paper PDF Interactive inspector Poster SlidesCode coming soon

ReTreever architecture — A frozen bi-encoder feeds a differentiable tree organizer; leaf-assignment vectors support retrieval, and paths expose structure.

Motivation

Dense retrieval is fast, but opaque. Hierarchical retrieval is interpretable, but slow.

ReTreever learns a binary tree directly over embeddings. The hierarchy is optimized for retrieval, not clustering, so it can produce strong nearest-neighbor representations while exposing semantic retrieval paths.

Embeddings only

No LLM calls are required for tree construction or navigation.

Coarse-to-fine

Any tree level can be used as a representation.

Inspectable paths

Queries and documents can be traced through the learned tree.

Scalable search

Single-index and multi-index retrieval both come from one learned hierarchy.

Key findings

High retrieval quality at low hierarchical-retrieval latency

160xfaster than RAPTOR on average

+20.7average NDCG@10 points vs RAPTOR

9.9 msaverage query latency

0LLM calls for construction or search

Tree-based text retrieval.
Dataset	RAPTOR latency	RAPTOR NDCG@10	ReTreever latency	ReTreever NDCG@10
NQ	1758.05	40.28	10.65	54.48
HotpotQA	2246.51	59.37	10.01	85.04
TopiOCQA	701.61	18.69	9.66	31.00
RepLiQA	1647.18	49.58	9.47	80.23

Showcase

Look inside the learned tree

The trees below are intentionally large: they are the main visual evidence for ReTreever's inspectability.

Interactive inspector

Browse paths and semantic groupings directly in the demo.

Open inspector

**Text tree.** Main-paper text hierarchy showing a learned semantic path through the corpus.

**Image tree.** ImageNet hierarchy example showing visually coherent subclusters.

Resources

Project resources

PaperPaper PDF

Open the ICML 2026 paper.

PosterICML poster PDF

View the final poster.

SlidesTalk slides PDF

Open the original provided slides.

InspectorInteractive tree demo

Explore learned paths and semantic groups.

CodeComing soon

Public release will be added here.

BibTeX

@inproceedings{gupta2026retreever,
  title     = {Hierarchical Retrieval at Scale: Bridging Interpretability and Efficiency},
  author    = {Gupta, Shubham and Li, Zichao and Chen, Tianyi and Subakan, Cem and Reddy, Siva and Taslakian, Perouz and Zantedeschi, Valentina},
  booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
  year      = {2026}
}