OpenAI Codex review: it deleted the rule that made it wrong

Contents

Is OpenAI Codex worth it?

C

Yes, and it is close to free to find out, which is the most useful thing about it. Codex is not sold separately — it comes bundled with a ChatGPT subscription, and there is a real free tier underneath that.

I gave it a job I expected it to bungle: rename a field end to end across a working repository, touching SQL, an n8n workflow JSON and Python. Thirty-three references. It found and changed all of them without a corrective prompt from me, and I checked that against disk rather than believing the summary on screen.

Then it did the thing that decided this rating. It edited my project rules file without asking, and the line it deleted was the rule that made its own change wrong.

Alley Rating: 4.4 / 5 — Power Tool. Capability is not what holds it there: it met every requirement on both tasks I set it. What holds it there is that its two faults are failures of trust rather than polish, and neither is fixed by being good at the work.

How much of Codex did I actually test?

Less than I would like, and the honest answer shapes everything below.

This review rests on two measured tasks, both on the free tier, plus a one-month $20 trial earlier in the year that I abandoned. The first was a hard refactor across an existing repository. The second was a greenfield build from a single prompt — the same six-requirement brief I have given Replit, Lovable, Bolt and Antigravity, so the results sit side by side. Together that is enough to judge how Codex handles two very different jobs. It is not enough to tell you how it holds up over a quarter, and I am not going to imply otherwise.

What makes those two tasks worth writing up is that every claim was checked against disk. The reference count was counted at HEAD across both tokens. The changed files came from git, not from the agent’s summary. The syntax of everything it touched was independently validated. Where a number here differs from what Codex reported, the number here is the one from the filesystem.

What does Codex actually do?

Codex is OpenAI’s coding agent. You describe a task, it works across your codebase, edits files, runs commands and reports back. It runs as a terminal agent — the Codex CLI — plus an IDE extension, a desktop app and a cloud runner, with GitHub-integrated code review as a capability across them, gated by tier.

One clarification worth making early, because stale sources still muddy it. The “OpenAI Codex” that appears in older references was a code-completion model from 2021, deprecated in 2023. Today’s Codex is an agent and shares nothing with it but a name.

OpenAI has also been widening the remit. Its own recent positioning describes Codex as becoming a productivity tool for everyone rather than strictly a developer product, which is worth knowing if you are betting a workflow on its current shape.

What happened when I gave Codex a hard refactor?

The brief I gave Codex was deliberately awkward: rename b_roll_suggestions to b_roll_scenes, along with the sibling field b_roll_prompts that travels with it, across a repository where those fields appear in three languages and one of the occurrences is buried inside a string.

It found all thirty-three. Twenty-three instances of one token and ten of another, across seven files: a SQL schema, an n8n workflow JSON, three Python scripts, a smoke test and a project rules file. Measured with git afterwards: seven files changed, sixteen insertions, thirty-four deletions.

The hardest edit was one I expected it to fail. Inside the n8n workflow JSON, a conditional lived in JavaScript stored as an escaped string. That is the kind of thing a regex mangles and a careless agent corrupts. Codex extracted it, edited it with a structured node -e script rather than a find-and-replace, and put it back. Then it noticed its own first pass had been incomplete and went round again, without being told.

One caveat the headline number hides, and our own Codex versus Claude Code write-up records it too. Thirty-three of thirty-three is the count inside the pipeline code. Repo-wide, 45 further occurrences survive across four frozen planning documents that Codex did not touch and did not flag. They were stale artefacts rather than live code, so leaving them was arguably right — but nobody told it that, and it did not raise the question.

I verified the result rather than trusting it. JSON.parse accepted the workflow (28 nodes intact), node --check passed on the extracted guard JavaScript, and py_compile passed on all four Python files.

Does Codex respect boundaries?

Partly, and the split is the most interesting thing I found.

Run	What it was told	What it did	Verdict
Run 1	An empty directory	Searched the filesystem, found my real repo, tried to write	Gate fired, denied, zero writes
Run 2	A clone plus an explicit path boundary	Stayed inside it throughout	Boundary held
Run 2, same session	Nothing about the rules file	Edited it without asking	Ungated, inside the workspace

On the first run I pointed it at an empty directory and gave it the same task. Codex searched the filesystem, located my actual production repository, and planned edits to a live schema file. When it went to write, the approval gate fired, because that path sat outside the workspace it had been given. I denied it, and git status confirmed it had written nothing.

That is the system working. The boundary it was told about held.

On the second run, with a throwaway clone and an explicit path boundary in the prompt, it stayed inside the lines throughout. Also correct.

The boundary it had to infer is where it went wrong. One of the seven files it edited was CLAUDE.md, my project rules file. Codex changed it without asking, because it sat inside the workspace and everything inside the workspace is ungated by default. The line it removed read “Don’t drop the legacy column” — which is to say, it deleted the rule explaining why its own rename was incomplete.

There is a defensible reading: documentation should track code, and if you rename a field the docs mentioning it should follow. But a rules file is not documentation, and an agent that edits its own constraints without a prompt is doing something categorically different from editing a source file.

Worth knowing here: Codex reads AGENTS.md, not CLAUDE.md. It touched mine only because that file contained the string it had been asked to rename. If you want Codex to follow house rules, they go in AGENTS.md.

Which parts of Codex did I not test?

Codex is four surfaces and a review capability wearing one name. Both my tests used the local driven agent; I have not touched the others.

Surface	Tested?
Local driven agent (desktop)	Both tasks
CLI	No
IDE extension	No
Cloud task runner	No
GitHub code review	No, and it is Pro or Business only

There is the Codex CLI in your terminal, an IDE extension, a desktop app, a cloud task runner, and GitHub-integrated code review. Both tests ran the local driven agent on my own machine — once against an existing checkout, once in a clean empty folder.

I did not test the cloud surface, where tasks run on OpenAI’s infrastructure rather than yours. That changes the trust question completely — the boundary findings above are about a process running under my own permissions, and none of them transfer.

I did not test GitHub code review, which is the surface most of the search traffic around this product is actually looking for. Note that neither cloud tasks nor code review are available on Plus, so trying them means Pro or Business.

I did not test it over time. Everything here comes from three driven runs across two sittings ten days apart. The failure modes that matter most in agentic tools — context degradation over long sessions, drift across many small tasks — take weeks to surface and I have not put weeks into it.

What I can tell you is how it behaves on two well-scoped jobs, measured properly. That is a narrower claim than most reviews of this product make, and it is the one the evidence supports.

Put your rules in AGENTS.md, not CLAUDE.md

Codex reads AGENTS.md for project instructions and never looks at CLAUDE.md. The practical consequence is worth separating out, because it produces a misleading picture.

Codex will happily edit a CLAUDE.md sitting in the workspace, as mine did. From the outside that looks like an agent reading your rules and overruling them. It is not. It is an agent that never read them treating the file as ordinary text.

You do not have to duplicate your instructions to fix this. A one-line @AGENTS.md import inside CLAUDE.md makes Claude Code read Codex’s file, which keeps one source of truth.

Can you trust what Codex tells you it did?

Not entirely, and this is the finding I would most want to know before relying on it.

When it finished, Codex reported a diff badge: six files, thirteen insertions, thirty-one deletions. Git disagreed.

Metric	Codex reported	Git on disk
Files changed	6	7
Insertions	13	16
Deletions	31	34

It undercounted mid-run too, showing four files and ten insertions at a point where the disk already held six files and fifteen. The work was right. The account of the work was wrong, consistently in the same direction. To be fair to it, the rules-file edit did appear in its written summary — it was the counts that were off, not the disclosure.

There is a version of this that is harmless: a badge that lags a write buffer. There is another where a reviewer skims the badge, sees six files, and never opens the seventh. I cannot tell you which from the time I have spent with it.

What it does when it cannot finish

The best thing Codex did in the whole test was fail honestly.

Network access is blocked by default. Partway through, it needed to fetch something to run a type-check, and asked. I denied it. It did not shrug and report a passing check, which is the failure mode I was watching for. It named the exact error — ENOTFOUND — said plainly that the check had not run, and then cleaned up the artifact it had created for the attempt.

That is a small moment and it counts for a lot. An agent that invents a green tick when it cannot reach the network is worse than useless, because it has taught you to trust a signal that is not there.

How does Codex compare on the same brief?

The refactor above is a good test and an unusual one. So I gave Codex the brief every other builder in this cluster has had: one message, six numbered requirements, no follow-ups, in an empty folder.

Build a single-page subscription tracker. Add a tool with name, cost and billing cycle. Annual plans must show a monthly-equivalent rather than the annual figure. A running monthly total pinned at the top. A cancelled state that leaves the total but stays visible in its own section. Persistence via localStorage. A dark theme.

Codex read the empty folder, concluded a self-contained static page was the right shape, and built it in 2 minutes 44 seconds. It met all six.

The trace is worth reading rather than skimming, because it revised itself twice without being asked. After creating the file it went back to normalise how annual entries display — “annual entries should not surface the yearly price in the list, only the monthly equivalent” — and then made what it called a tiny robustness pass, patching a browser API fallback that needed to check globalThis.crypto safely. Neither was in the brief. Both are the kind of thing you would raise in review.

It then checked its own work: “I’m doing a lightweight syntax check on the embedded JavaScript and a quick file review now.”

That is a weaker check than Antigravity’s unit tests, and it is more than the three hosted builders did, which was nothing. It also told me plainly that the folder was not a git repository so it had no git status to report — an admission of what it could not verify, unprompted.

I checked each requirement by using the app. A $2,400-a-year plan rendered as $200.00/mo under a “MONTHLY EQUIVALENT” label, with “Billed annually” as context. The pinned total moved from $310.00 to $110.00 when I cancelled that entry, which then dropped into a Cancelled section with a restore button. Both survived a hard reload.

The stored data is the part I liked most. Codex saved the raw 2400 alongside cycle: "annual" and computes the monthly figure at render, rather than storing the derived number. That is the correct model and not the obvious one, and it is the same choice Replit and Lovable made.

Tool	Met	Time	What it produced	Self-verification
Lovable	6/6	109 sec	Hosted app	None
Codex	6/6	2m 44s	585 lines on disk, one file	JS syntax check
Bolt	6/6	2–3 min	Hosted app	None
Antigravity	6/6	~2.5 min including an approval gate	2,894 lines, three files	Wrote and ran unit tests
Replit	6/6	~4 min	Hosted app	None

Capability did not separate them. All five met every requirement, and all five invented a restore button nobody asked for.

What separates Codex here is restraint. It produced the same six requirements in 585 lines on disk where Antigravity took 2,894. That is a like-for-like count of project code: Antigravity’s test file lived outside the project, in its own working directory, so none of its 2,894 lines are tests. It added cancel, restore and delete, and then stopped. Antigravity added a six-currency switcher, a category spend breakdown, a ⌘K search, grid and table views and JSON import/export. On a throwaway prototype that is free polish. On something you intend to maintain, it is five times the code to read.

It also ran unprompted without an approval gate, which Antigravity will not do. Whether that is a feature depends entirely on how much you trust it, and the refactor above is the reason to hold some of that trust back. Put the two together honestly and the gate looks less like friction: an approval step is exactly what would have caught Codex editing my rules file.

What does Codex cost?

Nothing on its own. Codex is included with ChatGPT rather than sold as a product, which makes the pricing question really a ChatGPT question.

Plan	Price	Notes
Free	$0	Real, and where I ran this test
Go	$8/mo
Plus	$20/mo	No cloud tasks, no code review
Pro	from $100/mo (5x) or $200/mo (20x)	Expressed as usage multiples
Business	$20/user/mo annual, $25 monthly

Three details matter more than the headline numbers. Usage above your plan runs on credits that expire after twelve months, are non-refundable and non-transferable, and draw from a pool shared with ChatGPT Work and Excel. Auto top-up exists with a settable maximum monthly spend, which is the setting to find before a heavy week. And cloud tasks and code review are not on Plus — if those are why you are here, Plus is not the tier.

One dated note: GPT-5.4 and GPT-5.4-mini retire in Codex on 31 August 2026 for ChatGPT-signed-in users. OpenAI’s Codex rate card lists what the credits buy per model. The full ladder, including the per-token rate card, is in the Codex pricing guide.

Purchasable credits are a Plus and Pro feature, which is the documented reason the free tier cannot bill you. That matches what I saw: I hit a limit and was told to wait rather than offered credits, though I could not tell whether that was the monthly cap or a shorter rolling window.

Who is Codex for?

Take it if you already pay for ChatGPT. It costs nothing extra, and on the evidence of two measured tasks it is genuinely capable. That is an unusually low barrier.

Take it for well-scoped mechanical work. A rename across languages, with one nasty case buried in a string, is exactly the shape it handled well.

Take it if you want to try an agent before committing. The free tier is real, and because credits are a Plus and Pro feature it has no way to bill you.

Skip it if you need the summary to be load-bearing. If your workflow is “read the agent’s report, approve, move on”, the undercounting badge is a problem rather than a quirk.

Skip it if your rules file is sacred. Put your instructions in AGENTS.md and understand that anything inside the workspace can be edited without a prompt.

The good

Codex got a genuinely hard refactor right. Thirty-three of thirty-three references, seven files, three languages, in a single run, verified against disk.

It writes less code than its rivals for the same result. On an identical six-requirement brief it met all six in 585 lines on disk and one file, where Antigravity took 2,894 across three, and it resisted adding features nobody asked for.

It reached for the right tool on the hardest edit. A structured node -e script to reach JavaScript stored inside JSON, rather than a regex over the raw file.

It caught its own incomplete pass. Unprompted, mid-task.

It failed honestly when the network was denied. Named the error, admitted the check had not run, cleaned up after itself.

The boundary it was told about held. The approval gate fired on the out-of-workspace write, and denying it left the repository untouched.

It is bundled, not sold. No separate subscription to justify.

The bad

The fault	How much it matters
Edited the rules file ungated	High, and it removed the rule that made it wrong
Diff badge undercounts	High if you rely on the summary
Searched outside the given directory	Medium, the gate did catch it
Cloud tasks and review absent on Plus	Medium, a tier surprise
Credits expire after 12 months	Low, but non-refundable

Codex edited my project rules without asking. Everything inside the workspace is ungated, and that includes the file describing what it may not do.

It undercounted its own work. Six files reported, seven on disk, and the same undercount showed mid-run.

It went looking outside the folder I gave it. Pointed at an empty directory, it found my real repository unprompted. The gate stopped the write, but the search itself was not something I asked for.

The tier boundaries surprise people. Cloud tasks and code review are not on Plus, which is not obvious from the pricing page.

My evidence is about twenty-five minutes deep. Everything above is measured, and there is not much of it. Treat this as a well-verified first impression rather than a verdict on a quarter of use.

What are the best Codex alternatives?

Where I stand, so you can weigh this section: this site earns referral credits on Claude signups and nothing at all on Codex or Antigravity, the two tools it is measured against most closely here. The editorial work here also runs on Claude Code, so treat any comparison between the two as coming from an interested party.

Claude Code is the tool I use daily on an existing codebase, at $20 a month on Pro or $100 and up on Max. It has been the deeper agent on long work in my own use, which is a different kind of evidence from the measured run above, and it is not free to try in the way Codex is.

Cursor is the answer if you want to stay in an editor you drive rather than hand work to an agent, and I have compared their cost shapes. Pro is $20 a month, though my own bill averaged $46 across eleven months.

Google Antigravity is the free one, and the only agent I have tested that wrote and ran its own unit tests. It will not run unattended, and it gives you no hosting, but it costs nothing at all.

For the wider field, the Claude Code alternatives guide covers the terminal agents including the open-source options.

The final word

Across both tests, Codex was better at the work than at describing the work. It took a refactor I expected to defeat it, handled the ugliest edit with the right tool, caught its own mistake, and told the truth when it could not finish — and then handed me a summary that was quietly wrong about how much it had touched.

The rules-file edit is the part I keep coming back to. It is defensible in isolation and unsettling in aggregate: an agent that treats its own constraints as just another file inside the workspace. Put your instructions in AGENTS.md, read the diff rather than the badge, and it is a strong tool that costs most people nothing extra.