Prune the History, Pack the Survivors
TL;DR for operators Persistent audiovisual assistants cannot keep extending their working history indefinitely on consumer hardware. As earlier inputs accumulate, the model stores reusable representations so later updates do not have to recompute the entire history; those stored representations consume memory and also increase the attention work required on every update. In one reported Apple M2 Pro profile, MiniCPM-o-4.5 approaches 80K cached tokens and about 23 GiB after 6.25 minutes, while per-chunk prefill latency rises from 28 to 69 seconds and attention time rises from roughly 2 to 39 seconds. ...