Slashing RAG index storage by orders of magnitude could make large-scale retrieval far cheaper.
LEANN is an open-source vector index from UC Berkeley that stores a pruned graph and recomputes embeddings at query time. It shrinks a 60M-chunk RAG index from 201GB to 6GB and matches HNSW accuracy at 90% recall, at the cost of seconds-level search.