Raw data
Logs
Every check our lab has saved: the good ones, the failed ones, and the tokens they cost us. This is the ledger behind every board on the site.
What our lab spent
Tokens the paid apps reported back to us for our own test runs. A token is a small chunk of text, about four letters. Total is new input + cache + output.
Pick a time window
Every count on this page follows the window you pick here. The default is the last 24 hours. 30d is as far back as we serve.
Total tokens · last 5 hours
310,504
65 samples · $0.8190 at published API list prices
Apps reported $0.2549 of their own. 25 of 65 samples report no cost at all — those rows are blank, not free.
15 samples are missing a token count, so the totals above are short by whatever they used.
How to read this
Two dollar columns, and they are not the same. Cost is the figure the app itself sent back with the answer — Claude and Grok send one, Codex sends none. API $ is our own recomputation of the same tokens at the vendor's published API list price, so all three apps sit on one scale. Neither is our bill: we pay flat monthly plans. A blank means not measured, never $0. The same API dollars, rolled up to a whole usage window, are what the quota board prices.
How to read this
New in is uncached prompt tokens. Cache is cache reads plus writes — cheaper, but still usage. Out is everything the model wrote, including think. Total adds those three. Bigger is not better here. It is what our own testing used.
By app
| Split | n | New in | Cache | Out | Total | API $ |
|---|---|---|---|---|---|---|
| Grok Build | 15 | 169.6k | 4,096 | 3,006 | 176.7k | $0.3590 |
| Codex | 25 | 112.1k | 5,632 | 2,004 | 119.8k | $0.3129 |
| Claude Code | 25 | 7,283 | 0 | 6,783 | 14.1k | $0.1472 |
By check
| Split | n | New in | Cache | Out | Total | API $ |
|---|---|---|---|---|---|---|
| speed | 50 | 216.6k | 8,320 | 11.3k | 236.2k | $0.7131 |
| ping | 15 | 72.4k | 1,408 | 541 | 74.3k | $0.1060 |
By model
| Split | n | New in | Cache | Out | Total | API $ |
|---|---|---|---|---|---|---|
| Grok 4.6 | 10 | 108.9k | 2,816 | 1,578 | 113.3k | $0.2286 |
| Grok 4.5 | 5 | 60.7k | 1,280 | 1,428 | 63.4k | $0.1303 |
| GPT-5.6 Luna | 10 | 44.4k | 0 | 520 | 44.9k | $0.0095 |
| GPT-5.6 Sol | 5 | 24.7k | 0 | 486 | 25.2k | $0.1381 |
| GPT-5.6 Terra | 5 | 24.7k | 0 | 486 | 25.2k | $0.0552 |
| GPT-5.5 | 5 | 18.4k | 5,632 | 512 | 24.5k | $0.1101 |
| Haiku 4.5 | 10 | 2,610 | 0 | 3,139 | 5,749 | $0.0183 |
| Sonnet 5 | 5 | 1,610 | 0 | 1,234 | 2,844 | $0.0156 |
| Fable 5 | 5 | 1,533 | 0 | 1,205 | 2,738 | $0.0756 |
| Opus 5 | 5 | 1,530 | 0 | 1,205 | 2,735 | $0.0378 |
Test runs
6 launches in this window · 17 failed samples. Open a run to see every model it touched and what that cost.
Aug 23, 6:00:01 PMpulsescheduled1m 23s · 13 rec · 0 skip · 5 fail · 38.1k tok
| Target | Result | Wall | New in | Cache | Out | Total | API $ |
|---|---|---|---|---|---|---|---|
| pingcodex | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 6.13s | 0 | 0 | 0 | 0 | $0.0000 |
| pinggrok | ok | 8.01s | 9,819 | 128 | 29 | 9,976 | $0.0199 |
| pingclaude | ok | 3.19s | 250 | 0 | 66 | 316 | $0.0006 |
| speedcodex/luna | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:29 PM. | 8.23s | 0 | 0 | 0 | 0 | $0.0000 |
| speedcodex/terra | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 6.12s | 0 | 0 | 0 | 0 | $0.0000 |
| speedcodex/sol | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 5.79s | 0 | 0 | 0 | 0 | $0.0000 |
| speedcodex/gpt-5.5 | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 6.00s | 0 | 0 | 0 | 0 | $0.0000 |
| speedgrok/grok-4.6 | ok | 5.85s | 12.1k | 256 | 283 | 12.7k | $0.0261 |
| speedgrok/grok-4.5 | ok | 5.35s | 12.1k | 256 | 283 | 12.7k | $0.0260 |
| speedclaude/haiku | ok | 8.11s | 272 | 0 | 560 | 832 | $0.0031 |
| speedclaude/sonnet | ok | 5.38s | 322 | 0 | 241 | 563 | $0.0031 |
| speedclaude/opus | ok | 4.55s | 306 | 0 | 241 | 547 | $0.0076 |
| speedclaude/fable | ok | 5.56s | 307 | 0 | 241 | 548 | $0.0151 |
Aug 23, 5:00:00 PMpulsescheduled1m 25s · 13 rec · 0 skip · 6 fail · 38.1k tok
| Target | Result | Wall | New in | Cache | Out | Total | API $ |
|---|---|---|---|---|---|---|---|
| pingclaude | ok | 3.48s | 250 | 0 | 75 | 325 | $0.0006 |
| pingcodex | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 6.91s | 0 | 0 | 0 | 0 | $0.0000 |
| pinggrok | ok | 6.61s | 9,691 | 256 | 29 | 9,976 | $0.0197 |
| speedclaude/fable | rate limit warningserved with a rate limit warning: seven_day_overage_included at 0.76 utilization | 6.46s | 306 | 0 | 241 | 547 | $0.0151 |
| speedcodex/luna | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 7.09s | 0 | 0 | 0 | 0 | $0.0000 |
| speedcodex/terra | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 6.42s | 0 | 0 | 0 | 0 | $0.0000 |
| speedcodex/sol | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 5.96s | 0 | 0 | 0 | 0 | $0.0000 |
| speedcodex/gpt-5.5 | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 6.33s | 0 | 0 | 0 | 0 | $0.0000 |
| speedgrok/grok-4.6 | ok | 7.25s | 12.3k | 128 | 293 | 12.7k | $0.0264 |
| speedgrok/grok-4.5 | ok | 6.25s | 12.1k | 256 | 285 | 12.7k | $0.0261 |
| speedclaude/haiku | ok | 6.62s | 272 | 0 | 531 | 803 | $0.0029 |
| speedclaude/sonnet | ok | 5.24s | 322 | 0 | 241 | 563 | $0.0031 |
| speedclaude/opus | ok | 4.80s | 306 | 0 | 241 | 547 | $0.0076 |
Aug 23, 4:10:14 PMpulsecatch-up5.80s · 0 rec · 13 skip · 0 fail · 0 tok
| Target | Result | Wall | New in | Cache | Out | Total | API $ |
|---|---|---|---|---|---|---|---|
| pinggrok | skipped · already recorded this slot | — | — | — | — | — | — |
| pingclaude | skipped · already recorded this slot | — | — | — | — | — | — |
| pingcodex | skipped · already recorded this slot | — | — | — | — | — | — |
| speedclaude/opus | skipped · already recorded this slot | — | — | — | — | — | — |
| speedclaude/fable | skipped · already recorded this slot | — | — | — | — | — | — |
| speedcodex/luna | skipped · already recorded this slot | — | — | — | — | — | — |
| speedcodex/terra | skipped · already recorded this slot | — | — | — | — | — | — |
| speedcodex/sol | skipped · already recorded this slot | — | — | — | — | — | — |
| speedcodex/gpt-5.5 | skipped · already recorded this slot | — | — | — | — | — | — |
| speedgrok/grok-4.6 | skipped · already recorded this slot | — | — | — | — | — | — |
| speedgrok/grok-4.5 | skipped · already recorded this slot | — | — | — | — | — | — |
| speedclaude/haiku | skipped · already recorded this slot | — | — | — | — | — | — |
| speedclaude/sonnet | skipped · already recorded this slot | — | — | — | — | — | — |
Aug 23, 4:00:01 PMpulsescheduled1m 38s · 13 rec · 0 skip · 6 fail · 38.1k tok
| Target | Result | Wall | New in | Cache | Out | Total | API $ |
|---|---|---|---|---|---|---|---|
| pinggrok | ok | 7.28s | 9,691 | 256 | 29 | 9,976 | $0.0197 |
| pingclaude | ok | 4.37s | 250 | 0 | 78 | 328 | $0.0006 |
| pingcodex | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 7.13s | 0 | 0 | 0 | 0 | $0.0000 |
| speedclaude/opus | ok | 6.84s | 306 | 0 | 241 | 547 | $0.0076 |
| speedclaude/fable | rate limit warningserved with a rate limit warning: seven_day_overage_included at 0.75 utilization | 8.43s | 306 | 0 | 241 | 547 | $0.0151 |
| speedcodex/luna | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 7.38s | 0 | 0 | 0 | 0 | $0.0000 |
| speedcodex/terra | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 6.00s | 0 | 0 | 0 | 0 | $0.0000 |
| speedcodex/sol | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 7.38s | 0 | 0 | 0 | 0 | $0.0000 |
| speedcodex/gpt-5.5 | failedYou've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Aug 24th, 2026 8:30 PM. | 6.32s | 0 | 0 | 0 | 0 | $0.0000 |
| speedgrok/grok-4.6 | ok | 6.63s | 12.1k | 256 | 293 | 12.7k | $0.0262 |
| speedgrok/grok-4.5 | ok | 6.54s | 12.1k | 256 | 285 | 12.7k | $0.0261 |
| speedclaude/haiku | ok | 9.44s | 272 | 0 | 550 | 822 | $0.0030 |
| speedclaude/sonnet | ok | 8.48s | 322 | 0 | 241 | 563 | $0.0031 |
Aug 23, 3:00:00 PMpulsescheduled2m 3s · 13 rec · 0 skip · 0 fail · 99.2k tok
| Target | Result | Wall | New in | Cache | Out | Total | API $ |
|---|---|---|---|---|---|---|---|
| pingcodex | ok | 8.52s | 12.0k | 0 | 5 | 12.0k | $0.0024 |
| pinggrok | ok | 5.80s | 9,690 | 256 | 17 | 9,963 | $0.0196 |
| pingclaude | ok | 4.94s | 250 | 0 | 73 | 323 | $0.0006 |
| speedclaude/sonnet | ok | 10.5s | 322 | 0 | 270 | 592 | $0.0033 |
| speedclaude/opus | ok | 7.25s | 306 | 0 | 241 | 547 | $0.0076 |
| speedclaude/fable | ok | 9.15s | 307 | 0 | 241 | 548 | $0.0151 |
| speedcodex/luna | ok | 13.7s | 10.8k | 0 | 255 | 11.0k | $0.0025 |
| speedcodex/terra | ok | 10.5s | 12.3k | 0 | 243 | 12.6k | $0.0276 |
| speedcodex/sol | ok | 12.4s | 12.3k | 0 | 243 | 12.6k | $0.0690 |
| speedcodex/gpt-5.5 | ok | 13.0s | 6,995 | 5,632 | 255 | 12.9k | $0.0454 |
| speedgrok/grok-4.6 | ok | 6.11s | 12.1k | 256 | 293 | 12.7k | $0.0261 |
| speedgrok/grok-4.5 | ok | 6.76s | 12.4k | 0 | 292 | 12.7k | $0.0265 |
| speedclaude/haiku | ok | 9.69s | 272 | 0 | 554 | 826 | $0.0030 |
Aug 23, 2:00:00 PMpulsescheduled1m 49s · 13 rec · 0 skip · 0 fail · 96.9k tok
| Target | Result | Wall | New in | Cache | Out | Total | API $ |
|---|---|---|---|---|---|---|---|
| pingclaude | ok | 5.14s | 250 | 0 | 106 | 356 | $0.0008 |
| pingcodex | ok | 6.40s | 10.8k | 0 | 5 | 10.8k | $0.0022 |
| pinggrok | ok | 5.19s | 9,436 | 512 | 29 | 9,977 | $0.0193 |
| speedclaude/haiku | ok | 8.54s | 272 | 0 | 546 | 818 | $0.0030 |
| speedclaude/sonnet | ok | 7.75s | 322 | 0 | 241 | 563 | $0.0031 |
| speedclaude/opus | ok | 7.24s | 306 | 0 | 241 | 547 | $0.0076 |
| speedclaude/fable | ok | 7.89s | 307 | 0 | 241 | 548 | $0.0151 |
| speedcodex/luna | ok | 12.4s | 10.8k | 0 | 255 | 11.0k | $0.0025 |
| speedcodex/terra | ok | 9.68s | 12.3k | 0 | 243 | 12.6k | $0.0276 |
| speedcodex/sol | ok | 10.9s | 12.3k | 0 | 243 | 12.6k | $0.0690 |
| speedcodex/gpt-5.5 | ok | 10.7s | 11.4k | 0 | 257 | 11.6k | $0.0646 |
| speedgrok/grok-4.6 | ok | 8.53s | 11.9k | 512 | 283 | 12.7k | $0.0258 |
| speedgrok/grok-4.5 | ok | 5.50s | 11.9k | 512 | 283 | 12.7k | $0.0257 |
Each row: start time · kind · trigger (cron means the timer, anything else means by hand) · wall time · rec / skip / fail · total tokens. Skipped means it was not that target's turn, or a rate limit stopped the app. Failed is a real error, and we keep it.
Sample log
Every measurement we saved, newest first. Filter by check or by app. Most old test-kit rows stay in Logs only. Older speed rows that can be rebuilt exactly from their raw files may return to the history. The counts above always cover the full window, not the filter.
This window, counted by outcome
- ok · 48
- failed · 15
- rate limit warning · 2
Every saved sample in the window, sorted into its label. The filters below change the table, never these counts.
The status words
- ok
- The app answered and the answer passed our grader.
- rate limit
- The vendor refused the request. We stop the rest of that app for the round. Busy plan, not a slow model.
- rate limit warning
- The vendor served the correct answer but flagged that the plan's usage window is getting full. Its real wait stays in experienced-speed summaries, and the warning is labeled. It never stops the round.
- timeout
- The app never finished in the time we allow.
- auth
- The sign-in was not accepted. We skip that app until it is fixed.
- used tools
- The app reached for a tool even though we said not to. The row does not count.
- wrong answer
- Text came back, but it was not what we asked for. Speed with a wrong answer never counts.
- remapped
- We asked for one model and the app served another. We save both names and mark the row, and the boards treat it with care.
- parse
- The app’s own output was not readable, so we could not trust the timings.
- overloaded / credits / failed
- The vendor said it was busy, out of credit, or gave a plain error.
| Started | Check | Target | Status | Wall | Total | Cost | API $ |
|---|---|---|---|---|---|---|---|
| Aug 23, 6:01:18 PM | speed | claude/fableclaude-fable-5 | ok | 5.56s | 548 | $0.0161 | $0.0151 |
| Aug 23, 6:01:13 PM | speed | claude/opusclaude-opus-5 | ok | 4.55s | 547 | $0.0085 | $0.0076 |
| Aug 23, 6:01:08 PM | speed | claude/sonnetclaude-sonnet-5 | ok | 5.38s | 563 | $0.0040 | $0.0031 |
| Aug 23, 6:01:00 PM | speed | claude/haikuclaude-haiku-4-5 | ok | 8.11s | 832 | $0.0041 | $0.0031 |
| Aug 23, 6:00:18 PM | ping | claude/haikuclaude-haiku-4-5 | ok | 3.19s | 316 | $0.0015 | $0.0006 |
| Aug 23, 5:01:21 PM | speed | claude/opusclaude-opus-5 | ok | 4.80s | 547 | $0.0085 | $0.0076 |
| Aug 23, 5:01:15 PM | speed | claude/sonnetclaude-sonnet-5 | ok | 5.24s | 563 | $0.0040 | $0.0031 |
| Aug 23, 5:01:09 PM | speed | claude/haikuclaude-haiku-4-5 | ok | 6.62s | 803 | $0.0039 | $0.0029 |
| Aug 23, 5:00:23 PM | speed | claude/fableclaude-fable-5served with a rate limit warning: seven_day_overage_included at 0.76 utilization | rate limit warning | 6.46s | 547 | $0.0161 | $0.0151 |
| Aug 23, 5:00:05 PM | ping | claude/haikuclaude-haiku-4-5 | ok | 3.48s | 325 | $0.0016 | $0.0006 |
| Aug 23, 4:01:30 PM | speed | claude/sonnetclaude-sonnet-5 | ok | 8.48s | 563 | $0.0040 | $0.0031 |
| Aug 23, 4:01:21 PM | speed | claude/haikuclaude-haiku-4-5 | ok | 9.44s | 822 | $0.0040 | $0.0030 |
| Aug 23, 4:00:32 PM | speed | claude/fableclaude-fable-5served with a rate limit warning: seven_day_overage_included at 0.75 utilization | rate limit warning | 8.43s | 547 | $0.0161 | $0.0151 |
| Aug 23, 4:00:25 PM | speed | claude/opusclaude-opus-5 | ok | 6.84s | 547 | $0.0085 | $0.0076 |
| Aug 23, 4:00:13 PM | ping | claude/haikuclaude-haiku-4-5 | ok | 4.37s | 328 | $0.0016 | $0.0006 |
| Aug 23, 3:01:54 PM | speed | claude/haikuclaude-haiku-4-5 | ok | 9.69s | 826 | $0.0040 | $0.0030 |
| Aug 23, 3:00:42 PM | speed | claude/fableclaude-fable-5 | ok | 9.15s | 548 | $0.0161 | $0.0151 |
| Aug 23, 3:00:34 PM | speed | claude/opusclaude-opus-5 | ok | 7.25s | 547 | $0.0085 | $0.0076 |
| Aug 23, 3:00:24 PM | speed | claude/sonnetclaude-sonnet-5 | ok | 10.5s | 592 | $0.0043 | $0.0033 |
| Aug 23, 3:00:19 PM | ping | claude/haikuclaude-haiku-4-5 | ok | 4.94s | 323 | $0.0016 | $0.0006 |
| Aug 23, 2:00:43 PM | speed | claude/fableclaude-fable-5 | ok | 7.89s | 548 | $0.0161 | $0.0151 |
| Aug 23, 2:00:36 PM | speed | claude/opusclaude-opus-5 | ok | 7.24s | 547 | $0.0086 | $0.0076 |
| Aug 23, 2:00:28 PM | speed | claude/sonnetclaude-sonnet-5 | ok | 7.75s | 563 | $0.0040 | $0.0031 |
| Aug 23, 2:00:20 PM | speed | claude/haikuclaude-haiku-4-5 | ok | 8.54s | 818 | $0.0040 | $0.0030 |
| Aug 23, 2:00:03 PM | ping | claude/haikuclaude-haiku-4-5 | ok | 5.14s | 356 | $0.0017 | $0.0008 |
Showing 25 of 25 filtered samples · 65 in the full window. Same rows as JSON: /api/probes · /api/speed · /api/runs
Read next
- How we test
What each check asks, and what every unit means.
- Quota
The same API dollars, rolled up to a whole week.
- Lab health
Is the lab fresh right now, and is it still signed in?
- Developers
The same data as JSON. No key, no login.