Tight Bounds and Phase Transitions for Incremental and Dynamic Retrieval

Retrieval data structures are data structures that answer key-value queries without paying the space overhead of explicitly storing keys. The problem can be formulated in four settings (static, value-dynamic, incremental, or dynamic), each of which offers different levels of dynamism to the user. In this paper, we establish optimal bounds for the final two settings (incremental and dynamic) in the case of a polynomial universe. Our results complete a line of work that has spanned more than two decades, and also come with a surprise: the incremental setting, which has long been viewed as essentially equivalent to the dynamic one, actually has a phase transition, in which, as the value size \(v\) approaches \(\log n\), the optimal space redundancy actually begins to shrink, going from roughly \(n \log \log n\) (which has long been thought to be optimal) all the way down to \(\Theta(n)\) (which is the optimal bound even for the seemingly much-easier value-dynamic setting).