Skip to content
BenchAlert
Contact us
Front-end bench
IntelliBench is in development

Real tests of the AI coding tools you pay for.

Speed, cost and value, measured on our own lab machine. 100% free.

Front-end bench
Live boardsLast run Speed · Cost · Value

SpeedBench

ModelEffortTime
  1. Devin by CognitionSWE-2SUBhigh Effort6.54s Time
  2. OpenAIGPT-6 SolSUBhigh Effort8.89s Time
  3. OpenAIGPT-6 AstraSUBhigh Effort13.31s Time
  4. AnthropicOpus 5.5SUBhigh EffortUnavailable Time

    No verified successful attempt yet.

BenchAlert.com

Cost per task

Slice = share of $1 one task costs

  1. OpenAIGPT-6 SolSUB

    6.1 tasks per $1high Effort

    $0.17per task

  2. Devin by CognitionSWE-2SUB

    2.1 tasks per $1high Effort

    $0.47per task

  3. AnthropicOpus 5.5SUB

    1.5 tasks per $1high Effort

    $0.68per task

  4. OpenAIGPT-6 AstraSUB

    1.3 tasks per $1high Effort

    $0.75per task

BenchAlert.com

Subscription value · API estimate

  • AnthropicClaude Code
    You pay: $200.00/month
    API value/month: ≈$8,960
  • OpenAICodex
    You pay: $200.00/month
    API value/month: ≈$7,736
  • xAIGrok Build
    You pay: $300.00/month
    API value/month: ≈$5,738

Recorded workloads at API prices, cache priced separately. Grok uses the Sep 13 usage lab; its bounds cover rounding only. Total uncertainty remains unresolved. Monthly = 4.35 weekly pools.

BenchAlert.com

More models are coming to the boards.

One lab. The same task.

Every model on the speed board gets the same fixed job on the same machine. A time only counts when the output passes our checks.

Real paid plans

The subscriptions people actually buy, plus selected APIs. The two are timed separately and never mixed.

Verified output only

A time counts only when the output passes our checks. Everything else is marked, never hidden.

Every attempt on the record

Failed attempts and retry delays stay recorded. Each board shows its real measurement date.

Launch posts are not a benchmark

They rarely share a method, a date or the runs that failed. We publish all three.

A typical launch post

  • No method shared
  • No date on the numbers
  • Failed runs left out

Every board on BenchAlert

  • The same fixed task for every model
  • Only verified output counts
  • Every attempt stays on the record

More from the lab

Design tests, a benchmark in progress, release alerts and the raw data behind every board.

Front-end bench

One brief. Original designs. Compare models and effort levels side by side.

Compare designs
Designs
19
Models
14
Effort levels
5

IntelliBench

In development

Can it handle the change?

A benchmark in development for coding, data, planning and more. See what the checks cover and their current limits.

Code checks
Review short code examples.
Data analysis
Get the answer from the supplied records.
Source grounding
Notice conflicting or missing evidence.
Constraint planning
Keep the plan inside the actual limits.
Instruction fidelity
Follow the brief when distractions change.
Confidence checks
Follow rules for uncertain answers.

IntelliBench is still in development. No verified scores yet.

Explore IntelliBench

Read the raw numbers

Every speed result is public JSON, with the full method in /llms.txt. No key, no account. The response on the right is what the endpoint returned when this page was built.

# Latest SpeedBench batch, as JSON
curl -s https://benchalert.com/api/speed
Response · excerpt[ .JSON ]
{
  "method": {
    "version": 19,
    "unit": "ms",
    "payload_bytes": 2048
  },
  "batches": [
    {
      "sequence": 366,
      "status": "failed",
      "finished_at": "2026-09-24T18:02:51.169Z",
      "models": [
        {
          "label": "GPT-6 Astra",
          "effort": "high",
          "status": "completed",
          "summary": {
            "result_ms": 13314.35
          }
        },
        {
          "label": "GPT-6 Sol",
          "effort": "high",
          "status": "completed",
          "summary": {
            "result_ms": 8887.0757
          }
        },
        {
          "label": "GPT-5.6 Terra",
          "effort": "high",
          "status": "completed",
          "summary": {
            "result_ms": 7785.9166
          }
        }
      ]
    }
  ]
}
// … 6 more models and every attempt in the full response

Questions

Anything else? Read the method notes.

What is BenchAlert?

BenchAlert is an independent lab that tests the AI coding tools people pay for. We run the tests on our own paid plans and selected APIs, on our own lab machine, and publish the results openly.

Does it cost anything?

No. BenchAlert is 100% free. We don’t charge users, and there is nothing to sign up for or install.

How is speed measured?

Each model copies the same fixed 2,048-byte block of text at high effort. We show the first attempt that passes our checks, out of up to three tries. The clock covers the whole wait, including app startup and shutdown. It is a copying test, not a measure of general coding skill.

What happens when a test fails?

Failed attempts stay on the record, but their time does not count toward the result. If a whole run fails or never finishes, the board keeps the last verified results and shows their real date.

Does visiting the site run any AI models?

No. The site only shows results. Visiting, clicking or refreshing a page never starts a model call. Tests run on the lab’s own schedule.

Can I use the data?

Yes. Results are public JSON at /api/speed and /api/today, with method notes at /llms.txt and an RSS feed at /feed.xml. No key or account needed. Quote freely, and credit BenchAlert with the method, batch and date.

Is BenchAlert affiliated with Anthropic, OpenAI or xAI?

No. BenchAlert is an independent lab. It is not affiliated with, endorsed by, or speaking for Anthropic, OpenAI or xAI. The numbers are our own measurements, not official vendor figures.

See the numbers for yourself

Open the live boards, compare designs side by side, or read the raw JSON. 100% free. We don’t charge users.

Compare designsRead the blog