Recent research has highlighted the promise of scaling memory embeddings in LLM training. While Engram and STEM index memory by token identity or local n-grams, can we design more flexible memory routing that captures how each token’s meaning changes with context? This paper introduces Mixture-of-Memory Embeddings (MoME), which uses a learned router to sparsely select among multiple memory slots for each token. The architecture outperforms strong memory baselines across three model…