The smartest model is often unnecessary for an automation. A smaller or cheaper model may extract event details, classify messages, or draft routine summaries just as reliably. The useful question is: which model completes this task well enough at the lowest total cost?
This guide is for people choosing models through an API or automation platform. Staff using a normal ChatGPT, Claude, or Gemini subscription usually do not control the underlying API model or hosting provider.
Start with the intelligence-cost graph
Open the Artificial Analysis model comparison and find Intelligence Index vs. Cost per Task under “Intelligence Index Comparisons.” Models toward the upper-left score higher while costing less on the benchmark. Turn on the Pareto line to see models that are not simultaneously beaten on both measures.
The graph is a shortlist, not a purchasing decision. Its Intelligence Index combines several evaluations, while your automation may depend on one narrow ability such as structured data extraction, tool use, coding, or long-document analysis. A small difference in the index may have no effect on your work.
Cost per task is more informative than token price alone. A cheap model can become expensive when it produces long answers, needs repeated attempts, or makes an agent take extra steps. A pricier model can cost less when it succeeds on the first attempt.
A dated example
This snapshot was checked on August 27, 2026. It gives five reference points from very cheap to frontier-level, not a permanent ranking. Prices and benchmark results change; follow the linked live pages before making a decision.
| Model and setting | AA Intelligence Index | AA cost per benchmark task | Example OpenRouter ZDR provider | API input / output per 1M tokens |
|---|---|---|---|---|
| GPT-5.6 Luna (high) | 47 | $0.02 | Azure | $0.20 / $1.20 |
| GPT-5.6 Luna (max) | 52 | $0.05 | Azure | $0.20 / $1.20 |
| DeepSeek V4 Flash 0731 (max) | 52 | $0.11 | DeepInfra | $0.08 / $0.18 |
| GLM-5.2 (max) | 53 | $0.44 | DeepInfra | $0.75 / $2.40 |
| GPT-5.6 Sol (max) | 61 | $1.01 | Azure | $5.00 / $30.00 |
The Artificial Analysis columns estimate the cost of running its benchmark tasks. The final column is provider token pricing, which measures something different. DeepSeek, for example, had the same index score as Luna max in this snapshot and a higher estimated benchmark-task cost, but its example endpoint charged less per token. Together the columns show why both model behavior and provider price belong in the comparison.
The table also uses specific zero-data-retention endpoints listed by OpenRouter. Privacy classifications can change by endpoint. Check the current provider page and your account routing settings rather than assuming every host for a model follows the same policy.
Compare the provider as well as the model
After finding two to five candidates, use OpenRouter’s model pages or the providers’ own documentation to check:
- input, output, reasoning, and cache prices;
- support for tools, structured output, and the context length your workflow needs;
- recent speed, uptime, and any differences between provider implementations;
- where requests are processed, how long content is retained, and whether it may be used for training; and
- whether your organization needs a data processing agreement or other contractual commitments.
The company that created a model and the company serving it can be different. Open-weight models are often available from several hosts, each with different prices and data practices.
If your AI assistant can browse the web, you can also ask it to check the current OpenRouter prices and provider terms for you. Require direct links and a date checked; the prompt below provides a starting point.
Test the work you will actually send
Match the test to the downside of a mistake:
- Quick start: If a stronger model fits the projected monthly budget, start there. Watch it complete 5–10 varied examples, then try a cheaper model after the workflow is stable.
- Cost-sensitive workflow: Compare a low-cost default with a stronger fallback on a small sample that covers the main types of work.
- High-volume, unattended, or consequential workflow: Use 20–50 representative tasks, or more when failures would be hard to catch.
Remove or replace personal and confidential information. Define what counts as a pass before running the models, then record:
- pass rate and serious errors;
- how often a person must correct the result;
- retries or extra agent steps;
- total input, output, and reasoning cost; and
- speed or reliability problems that would affect the workflow.
Before running the workflow unattended, add basic error handling: reject malformed output, cap retries and spending, log failures, and send uncertain cases to a person or a stronger fallback model.
If errors are easy to catch and cheap to fix, choose the cheapest model that clears the quality threshold. If nuance or edge cases matter and the stronger model fits the budget, using it as the default can be a reasonable safety margin. Review the results again when a provider or model version changes.
Ask AI to prepare the shortlist
An AI assistant can gather the current facts, but require live sources and check the final recommendation yourself.
Help me shortlist AI models for an API or automation.
Today’s date: [DATE]
Workload: [DESCRIBE THE TASK, VOLUME, INPUTS, AND REQUIRED OUTPUT]
Required capabilities: [TOOLS, STRUCTURED OUTPUT, CONTEXT LENGTH, ETC.]
Minimum acceptable quality: [DESCRIBE A PASS]
Data sensitivity and required provider terms: [DESCRIBE]
Speed or reliability requirements: [DESCRIBE]
Use current primary sources. Start with Artificial Analysis’s Intelligence Index vs. Cost per Task chart, then verify model and provider details on OpenRouter and the provider’s own documentation.
Return:
1. A table of 3–5 candidates with benchmark score, cost per task, current input/output/cache pricing, provider, tool support, speed, retention, and training policy.
2. A clear distinction between the model creator, API host, and routing service.
3. Direct links and the date checked for every changing claim.
4. Any unknown or conflicting information. Do not infer a privacy promise from a model’s country of origin or license.
5. A recommended two-model test: one low-cost default and one stronger fallback.
6. A test plan proportional to risk, starting with 5–10 varied tasks and expanding for high-volume, unattended, or consequential work. Include total cost per successful result and basic error handling.Benchmarks narrow the search. Your own pass rate, review time, and total cost decide whether the model belongs in the workflow.
