MODEL DRIFT MONITOR

Is Your AI Getting Dumber? — Ai Agents 2026

Reviewed October 18, 2026

Vendors ship updates silently. The model behind your product can lose capabilities between two Tuesdays — and nothing in your dashboard will tell you. Below is the public, dated evidence that this happens, plus the gaps where nobody is measuring.

Every point below is cited. Each score names the benchmark, the model version, the publication and the date. Where there is no public data, we label the gap — we never interpolate, smooth, or invent scores. If a point can't be sourced, it isn't plotted.

1. Same model, same task, three months apart

The 2023 drift study

Stanford & UC Berkeley researchers ("How is ChatGPT's behavior changing over time?", arXiv July 2023; published in Harvard Data Science Review, March 2024) tested the March 2023 and June 2023 versions of GPT-4 and GPT-3.5 on identical tasks. GPT-4 got measurably worse at three of four tasks — in one case catastrophically.

Blue = March 2023 · Red = June 2023. Sources: SiliconANGLE, Jul 20 2023 (prime identification, via Ars Technica); AI Business (code generation, sensitive questions). GPT-3.5 prime identification improved 7.4% → 86.8% — drift cuts both ways, which is exactly why continuous measurement matters.

2. Vendor lab scores are snapshots, not monitoring

MMLU (general knowledge) — vendor-reported flagship scores

Benchmarks announced at launch prove what a model could do on release day. They say nothing about the same model six months later. Note the protocol caveats under the chart.

Sources: GPT-4 technical report, Mar 2023 (MMLU benchmark history); Google, Dec 6 2023 (Techworm, Dec 7 2023); OpenAI, May 13 2024 (Search Engine Journal). ⚠ Gemini Ultra's 90.04% used a 32-shot chain-of-thought protocol; on standard 5-shot it scored 83.7% (same source) — vendor scores are not always directly comparable.

GAP — labeled, not interpolated: no vendor published a longitudinal same-model MMLU tracking series for 2024–2026. Lab release-day scores are all we get; the drift in between is invisible.

AIME (competition math) — reasoning-era scores

The reasoning-model era moved the headline benchmark from MMLU to AIME. Note these are different exams in different years — AIME 2024 and AIME 2025 are not the same test, so this is a trend of capability eras, not a like-for-like comparison.

Sources: OpenAI, Sep 2024 — o1 at 74.4% on AIME 2024 (WebProNews); OpenAI, Aug 7 2025 — GPT-5 at 94.6% on AIME 2025, 74.9% on SWE-bench Verified (Slashdot, Aug 7 2025; DataCamp).

3. The "lazy ChatGPT" episode

December 2023 — degradation you could feel but not measure

Through late 2023 users across Reddit and OpenAI's developer forums reported ChatGPT giving shorter answers, skipping tasks, and deflecting work back to the user. On December 8, 2023, OpenAI's @ChatGPTapp account publicly acknowledged it: "we've heard all your feedback about GPT4 getting lazier! we haven't updated the model since Nov 11th, and this certainly isn't intentional." OpenAI's January 2024 model update announcement said the new GPT-4 Turbo preview was intended to reduce cases of "laziness."

Sources: Dataconomy, Dec 12 2023; Guidantech (on the fix). This episode had no published numbers — which is itself the point: the most widely-felt degradation of 2023 was never quantified by anyone, vendor included.

4. The full cited record

Date	Model	Benchmark / task	Score	Source	Note
Mar 2023	GPT-4	MMLU (5-shot)	86.4%	GPT-4 Technical Report	Launch-day lab score
Mar 2023	GPT-4 (Mar version)	Prime identification	97.6%	SiliconANGLE, Jul 20 2023	Stanford/Berkeley drift study
Mar 2023	GPT-4 (Mar version)	Directly executable code	50%	AI Business	Stanford/Berkeley drift study
Jun 2023	GPT-4 (Jun version)	Prime identification	2.4%	SiliconANGLE, Jul 20 2023	−95.2 points in 3 months
Jun 2023	GPT-4 (Jun version)	Directly executable code	10%	AI Business	Same study
Jun 2023	GPT-4 (Jun version)	Sensitive Q answers	5% (was 21%)	AI Business	Answer rate, not accuracy
Jun 2023	GPT-3.5 (Jun version)	Prime identification	86.8% (was 7.4%)	AI Business	Drift improved this one
Dec 2023	ChatGPT (GPT-4)	User-reported "laziness"	—	Dataconomy, Dec 12 2023	Acknowledged by OpenAI Dec 8, 2023; never quantified
Dec 2023	Gemini Ultra	MMLU (CoT@32)	90.04%	Techworm, Dec 7 2023	83.7% on standard 5-shot
May 2024	GPT-4o	MMLU	88.7%	Search Engine Journal	Launch-day lab score
Sep 2024	o1	AIME 2024	74.4%	WebProNews	Launch-day lab score
Aug 2025	GPT-5	AIME 2025	94.6%	Slashdot, Aug 7 2025	Different exam than AIME 2024
Aug 2025	GPT-5	SWE-bench Verified	74.9%	DataCamp	Launch-day lab score

5. Why silent downgrades should scare you

Contracts you can't keep. If your product promises 95% extraction accuracy and the model silently drifts to 80%, you're in breach before anyone notices.
Reproducibility dies. A demo that worked in March fails in June on the "same" model. Debugging something that changed under you is pure cost.
Compliance exposure. Regulated outputs (finance, health, legal) evaluated against one model version may not hold for the version actually serving traffic.
Nobody else is watching. Vendors publish launch-day scores, not longitudinal monitoring. The June 2023 drift was caught by outside academics, not by OpenAI.

6. How we watch for it

ModelMinder is our model deprecation radar: it tracks the models your stack depends on and alerts you when capabilities shift, versions deprecate, or vendors change terms under you. Silent downgrade detection — continuous benchmark probes against the exact model versions in your pipeline — is the core of why we built it: the drift study above was published after the damage, by outsiders. We believe someone should be measuring continuously, with the numbers on the record.

FROM INGENIOUS AI

ModelMinder is the model deprecation radar — we watch your models so a silent downgrade never becomes your outage.

All 11 platforms: DisclosureGuard · ModelMinder · TermsWatch · Vibe-Code Audit · Rogue-Agent Defense · BreachHunter · ChatBreaker · Agent Flight Recorder · Agent Auth Gateway · DomainGuard · ContentCraft AI

Our platforms are being upgraded and return November 1 — get early access now:

Get early access on WhatsApp →

Why should AI agents 2026 builders monitor model downgrades?

Because your agent is only as good as the model under it. Vendors ship silent updates that can erode reasoning, instruction-following, or tool use between releases — and your agent's prompts don't change, so failures look like your bug. Tracking benchmark drift per model version tells you whether a bad week is your code or their weights. Pin versions, re-run evals, and alert on regression.

Frequently asked questions

What is AI model downgrade?

When a flagship AI model quietly performs worse over time — through silent updates, cost-cutting, or behavior shifts.

Do AI models get worse over time?

Sometimes: benchmark regressions after updates are publicly documented. This detector tracks cited benchmark points over time — gaps labeled, never interpolated.

How do you detect model degradation?

Re-run fixed benchmarks on a schedule and compare against pinned baselines — exactly what the tracker above does.