Your Company’s Memory Should Outlast Its AI Models

PUBLISHER ARTICLE PREVIEW

From the original article

People often ask which model powers my company. The answer matters: it sets quality, cost and limits, and I am happy to spend an hour on it. I also want to know a second thing: what the organisation will still have when that model changes.

Put it as a date. If your provider doubled its price on a Tuesday, or retired the endpoint you built on, what would you still own on Wednesday morning?

For a lot of companies, the honest answer is: an API key, an invoice, and a very long conversation history sitting on someone else’s servers under someone else’s retention schedule.

My background is chemistry. I spent years building grey-box models for chemical plants, wired into SCADA, lab results, and the operator’s own notes. That work teaches one rule fast: a number without its conditions is not a result. If someone swapped a sensor last week and nobody wrote it down, the reading on the screen explains nothing.

That is a lab notebook problem. It arrives now as a procurement problem, and it is rarely on the sheet where the AI budget is decided.

A larger context window lets a model use more information inside a single request. Durable storage determines what survives between requests. Portability determines whether that information remains usable when the provider changes. These are separate architectural questions, and the million-token window encouraged everyone to treat them as one.

Google’s own documentation offers the analogy directly: “An analogy for the context window is short-term memory.” Take that literally. Short-term memory is the thing you lose.

So every vendor selling an enormous window also sells something separate to persist state. Read those layers narrowly, in their own words, and note what each one actually covers.

OpenAI stores Response objects for 30 days by default; Conversation objects and their items are exempt from that TTL. Amazon’s 30-day deletion applies to the Bedrock Session Management API, and covers that API alone. Anthropic’s client-side memory tool illustrates one useful boundary: the model requests memory operations, the application controls the storage, while the same company also ships managed memory stores for its hosted agents.

All of these are ordinary engineering decisions, published openly. They also mean the retention rules for your company’s working context live in someone else’s release notes, and you should be able to name which rule applies to which of your data.

Swapping one text-generation call for another is a weekend. That is why “we’re multi-model” is such a cheap thing to say. The expensive layers are the ones nobody demos.

Embeddings. Changing the embedding model typically requires re-embedding the corpus and migrating or rebuilding the index. A change of generation model carries no such requirement, and the two get confused constantly — usually by whoever is promising the migration will be quick. The 2025 literature describes the standard path as “re-encoding the entire corpus and rebuilding the Approximate Nearest Neighbor (ANN) index, leading to significant operational disruption and computational cost”; that paper’s own contribution, Drift-Adapter, is a method for deferring the rebuild by learning a transform between embedding spaces.

Either way, the migration cost covers retrieval validation and operational work as well as the embedding calls themselves.

Fine-tuned behaviour. One sentence from Cohere’s September 2025 deprecation notice belongs above every procurement desk: “Previously fine-tuned models will no longer be accessible.” Behaviour you paid to create, retired alongside the endpoint that hosted it. Everybody followed the published process. Your model still went away.

Prompt and tool behaviour.