Reducing AI Coding Token Usage: SunnyCode vs Codex

Where the token gap comes from

The largest recorded difference is input context. Cached input accounts for 96.5% of Codex’s additional tokens versus CodeMode. At the frozen rates, 70.0% of the estimated cost gap comes from cached input, 24.6% from uncached input, and 5.4% from output. This decomposition explains the accounting; it does not isolate the causal effect of a product design choice.

That is why token savings and cost savings differ. CodeMode used 77.2% fewer tokens on average, with 53.5% lower estimated cost. Cached input is discounted, but repeated context still adds up.

Results by task type

For feature work, both SunnyCode modes passed 14/15 primary checks, versus 12/15 for Codex. All three setups passed every bug-fix run. The chart shows mean recorded tokens per run, including failed runs.

Fewer tokens did not always mean less time

From submission to the saved final response, Codex averaged 222.6 seconds, CodeMode 232.2, and SunnyCode Base 297.1. Comparing three-run means for each task, Codex was slower than CodeMode on six of eight tasks. Faster results on D and H lowered its overall mean. Accumulated cached-input tokens do not directly predict generation time.

Applying the comparison to your own workflow

To compare agents, hold the model, reasoning level, tasks, and acceptance criteria constant. Record uncached input, cached input, and output tokens separately, retaining both successful and failed runs, and compare cost alongside completion time. These results make agent choice worth evaluating for token efficiency; they do not guarantee the same savings in your repository.

Limitations and unsuccessful checks

These are 72 runs on eight tasks, not a general quality ranking. Base was collected later; provider conditions, caches, and host load were not randomized. Half the tasks came from p-queue. Costs are API-equivalent estimates at frozen rates, not ChatGPT OAuth bills.

Supplemental checks retain the CodeMode C3 string-Iterable defect and the Codex C1 timing-sensitive failure. Pass rates use the original primary acceptance checks; they do not establish an absence of other defects.

Download the source data