Google’s new neural-net LLM architecture separates memory components to control exploding costs of capacity and compute
Titans architecture complements attention layers with neural memory modules that select bits of information worth saving in the long term.
Google’s Titans Give AI Human-Like Memory
At the heart of Titans' design is a concerted effort to more closely emulate the functioning of the human brain.