The AI doesn’t remember yesterday.
It doesn’t know what project you’re working on, what you decided last week, or why. It knows what you’ve told it since you opened this conversation, and nothing before that. That’s the context window, and it isn’t a file the model reaches into when it needs something. It’s whatever is already sitting in front of it.
Most people run into this and blame the tool. Something’s off today, it forgot what we talked about, it’s being inconsistent. Usually the tool didn’t change. What changed is what got told to it, or didn’t.
I work alongside an AI assistant for most of what I do now, building courses, writing, running the operational side of a small company. When I first started with AI, I ran into this every day. Every session started cold. I’d spend the first few minutes re-pasting the same background I’d loaded the day before, the same course details, the same project state, over and over. Groundhog Day with a keyboard.
So I built something to fix it. My own fleet of AI agents named it “the helm,” not me. But, it worked. (I’m good with anything sailing related.) The helm reads the first line I type, matches it against a list of topics, and loads the right file before I ask for anything. Ask a cardiology question and it pulls the course curriculum automatically, no prompting required. That was the whole idea: stop re-explaining myself every morning.
Building it was the easy part. Knowing whether it helped at all was harder.
I had Claude grade 622 past sessions against one question: when the system loaded something, was it the right thing to load? Version one of the helm fired on almost every session and was only useful three percent of the time. That’s not context management, it’s clutter with a mission statement.
Eight iterations later, it was useful 41 percent of the time. Not a number I’d brag about anywhere else. But real, and not for the reason I expected going in. The biggest gain didn’t come from getting better at picking the right file. It came from knowing when not to load anything at all. Sessions where the system correctly stayed quiet went from 25 to 262. Most of the tuning wasn’t clever. I split broad topics into narrower ones, built a list of near-misses that looked related but weren’t, and let the system stay quiet more often than it spoke up. Then the improvement flattened out. Keyword matching only gets you so far, and I’d found the ceiling.
I’d also built a manual override for exactly this failure mode. I could always type one keyword and force the correct file in myself. It was the safety net, and easily the part of the whole system I was proudest of. A bad guess from the machine would never cost me anything, because I’d catch it.
Then I checked the log for how many times I’d used it. Zero.
Not once, across months of daily use. The override I built specifically to correct the system’s mistakes sat there unused. Not because the system stopped making mistakes. Because by the time I might have reached for it, I’d usually already moved past whatever the mistake cost me and kept working. The safety net worked by never getting tested, which is a strange thing to call a success.
It’s the same failure mode I train students out of. Never trust your recollection of the chart when the chart is sitting right there. I built a system that does exactly what I tell every NP student not to do with a patient: trust memory over the source, assume you’d catch it if you were wrong. Then I checked, and I hadn’t caught anything, because there’d been nothing for me to catch. The machine had already done it.
That’s the part worth sitting with if you use AI daily and haven’t measured anything about it yet. You probably think you’d notice when the tool gets something wrong. I thought that too, and I’m the guy who built the fix for it. I fixed the machine’s blind spot before I noticed I was the harder problem to solve.
Fix what you can fix. I fixed the context window. The part of the system I never got around to auditing was me.