Design Long Term Memory in Drawhisper
LLM Limitations in Long Context Windows
Recent large language models (LLMs) have made remarkable progress in solving intricate problems, completing complex tasks, and sustaining multi-round conversations. Reasoning-centric models such as DeepSeek-R1, GPT-5, and Claude 4.5 Opus routinely reach—or even surpass—domain experts, giving teams unprecedented leverage.
At the same time, these models now ship with context windows measured in millions of tokens, allowing them to reference long-running conversations and early-stage knowledge. Yet even the strongest models still struggle to find the proverbial needle when the haystack is littered with noise. This loss of precision over long transcripts is known as context rot. The challenge compounds when a user starts a fresh session and discovers the model has forgotten everything learned previously.
Training new capabilities directly into the model also remains cost-prohibitive. Once fine-tuning finishes, the learned knowledge is locked into the weights, meaning the model cannot absorb newly surfaced facts in real time during a chat.