Claude Haiku 5.5, Muse Wonder and the price of a token

The news, and what is confirmed

In one week of October 2026, Anthropic cut the price of its small model by 90%, Meta shipped the first model it calls frontier class, and Google opened Gemini 4 Argon to paying developers. The headlines were about prices and benchmark tables. Underneath them sit a few ideas that decide what you actually pay for AI: what a token is, why long prompts cost more, how caching works and how much a model thinks.

What was released?

Claude Haiku 5.5 (7 October) is Anthropic’s new small model. Its list price is $0.10 per million input tokens and $0.50 per million output tokens for prompts up to 100,000 tokens, and $0.50 / $2.50 above that. Claude Haiku 4.5 cost $1 / $5 at any length, so the cut is 90% for short prompts and 50% for long ones. The short-prompt price matches OpenAI’s GPT-6 Luna. Anthropic’s announcement says that “on average, it now costs around 75% less to run”, and that it has “an updated tokenizer … which means it uses slightly more tokens per task”. Its context window grows to a million tokens, up from Haiku 4.5’s 200,000, and it is the first Haiku with adjustable effort (models overview).

Muse Wonder (12 October) is Meta’s first frontier model, built by Meta Superintelligence Labs. It costs $4 / $20 per million tokens on Meta’s API (model page). A larger sibling, Muse Wonder Full, is available only inside Meta’s Muse Code subscriptions. Meta headlines two abilities it calls Active Memory and Active Focus: carrying what matters across conversations, and sticking with a hard task instead of giving up.

Gemini 4 Argon, announced in September and covered in the September explainer, became generally available on 13 October at introductory prices of $2 / $10 until 30 November, then $4 / $20 (Google).

Where do the savings come from?

A price per token is only half of a bill. The other half is how many tokens the work takes, and Haiku 5.5 changes that half too: a new tokenizer turns the same text into more tokens, adaptive thinking adds tokens the model writes for itself, and a price line at 100,000 tokens means that how long your prompt is now matters. The rest of this lesson takes those apart one at a time and then puts them back together in a single agent run.

Key takeaways

• Haiku 5.5 costs $0.10 / $0.50 per million tokens up to 100K-token prompts and $0.50 / $2.50 above, with a 1M window and adjustable effort.
• Muse Wonder is Meta’s first frontier model at $4 / $20; Gemini 4 Argon is now generally available. None of the three discloses its size or architecture.
• Savings depend on tokens per task as well as price per token, and Haiku 5.5 changes both.

Same text, more tokens

Every price in this lesson is per token, but a token is not a fixed amount of text. Change the tokenizer and the same document costs a different number of tokens, which is exactly what Haiku 5.5 does.

What changed?

A tokenizer splits text into pieces from a fixed vocabulary before the model sees it. Haiku 5.5 uses the tokenizer Anthropic introduced with Claude Opus 4.7 and shares with Sonnet 5.5 and Opus 5.5. According to the migration guide, “the same input text produces approximately 30% more tokens on Claude Haiku 5.5 than on Claude Haiku 4.5”, and “the exact increase depends on the content”. Anthropic’s models page gives a second yardstick: a million tokens holds roughly 555,000 English words on the current tokenizer and about 750,000 before it, a ratio of 1.35.

Images change too. Haiku 5.5 accepts larger images before shrinking them, so a 2,000 by 1,500 pixel image costs about 2.5 times as many tokens.

How can a tokenizer change the count?

Most tokenizers are built with byte-pair encoding: start from single bytes and repeatedly merge the most frequent adjacent pair into a new token. Each merge adds one entry to the vocabulary, and a bigger vocabulary covers common words with fewer, longer pieces. Two other choices also matter: the text the merges were learned from, and the rules for where a token may not cross, for example splitting every digit on its own. Anthropic has not said which of these it changed, or why. A tokenizer that uses more tokens is not necessarily worse; finer pieces can help a model with spelling, numbers or code. It does mean every count you measured on the old model is now too low.

On the plain English sentence the smallest vocabulary needs about 70% more tokens than the largest. Production tokenizers differ from each other by less, around a third here, but the mechanism is the same: the pieces get shorter, so the count goes up.

Why does it matter?

Because the honest unit for comparing prices is the work, not the token. Converting Anthropic’s list prices to dollars per million English words with its own words-per-token figures shows what changed.

Per word, Haiku 5.5’s short-prompt input costs $0.18 per million words against Haiku 4.5’s $1.33: still a big cut, about 86% rather than 90%. The same caution applies across companies. Meta, Google and OpenAI each use their own tokenizer, so a per-token price comparison between Muse Wonder and Claude Opus 5.5, which share a $4 / $20 list price, says little until you count tokens for the same text on both.

Key takeaways

• Haiku 5.5’s tokenizer turns the same text into about 30% more tokens than Haiku 4.5’s; Anthropic has not said what changed.
• In byte-pair encoding, vocabulary size, training text and splitting rules all change how many tokens a text needs.
• Compare prices per unit of work, such as per word or per task, never per token across different tokenizers.

The 100,000-token line

Haiku 5.5 has two price lists. Which one you pay depends on a single number: how long the prompt of that request is. Cross 100,000 tokens and the whole request costs five times as much.

What is the rule?

Anthropic’s pricing page is precise. “A request whose prompt is over 100,000 tokens pays higher prices.” The prompt length “counts all of its input tokens, including cache reads and cache writes”, and “each request is priced on its own”: a request over the line pays the higher prices for everything, output included, even if most of its prompt came from the cache. Earlier requests keep the prices they were billed at.

This is unusual for Anthropic. The same page says its other current models “include the full 1M token context window at standard pricing”, so a 900,000-token request to Opus 5.5 pays the same rate per token as a 9,000-token one. Haiku 5.5 is the exception.

Is it unusual in the industry?

No. OpenAI prices GPT-6 Luna and GPT-6.1 Sol the same way, with a higher tier once input exceeds 272,000 tokens (OpenAI pricing). What stands out about Haiku 5.5 is how early the line comes and how steep the step is.

Below the line, Haiku 5.5 is nearly eight times cheaper than Haiku 4.5 for the same text, once the extra tokens are counted. Just above it, the gap shrinks to about one and a half times, and Haiku 4.5 cannot go past about 260,000 of the new tokens at all. GPT-6 Luna’s step at 272,000 is much smaller. The practical lesson: with Haiku 5.5, keeping a prompt under 100,000 tokens is worth real engineering, which the agent run later in this lesson makes concrete.

Key takeaways

• Haiku 5.5 charges five times as much, for input and output, on any request whose prompt exceeds 100,000 tokens, cache reads included.
• Anthropic’s other current models charge one rate up to 1M tokens; OpenAI’s Luna and Sol step up about 1.5 to 2 times above 272K.
• With Haiku 5.5, how long you let a prompt grow is a pricing decision, not only a quality one.

Why long prompts cost more

Anthropic has not explained why Haiku 5.5 is priced by prompt length. But there is a standard piece of arithmetic that shows why long contexts are relatively more expensive for small models than for large ones, and it is worth knowing whatever the real reason is.

What does a token cost to compute?

Kaplan and colleagues gave a simple count for the forward pass of a transformer. Each new token costs about two operations per parameter for the weights, plus a term for attention that grows with the context, because the token’s query is compared with the key of every earlier token in every layer.

N is the number of parameters; width is the attention width, usually the model dimension. The first term is fixed; the second grows with every token of context.

Why does model size change the picture?

The weight term grows with the whole model, but the attention term grows only with layers times width, which is a small slice of the parameters. So the context at which attention catches up with the weights is N ÷ (layers × width), and it is much shorter for small models. Using the published shapes of Llama 3: for the 8B model it is about 61,000 tokens, for 70B about 108,000, and for 405B about 197,000.

Memory tells the same story. Every generated token has to read the model’s weights and its whole KV cache. At 16-bit, Llama 3 8B’s cache equals its 16 GB of weights at about 123,000 tokens; the 405B model’s does not catch up until about 1.6 million. A long conversation also occupies a server’s memory for its whole duration, which limits how many conversations can share a GPU, as the vLLM paper showed.

At 128K tokens, attention is about two thirds of the arithmetic for Llama 3 8B but only about 40% for the 405B model. For a small model, a long prompt multiplies the cost of each token several times over; for a big one, it adds a smaller share on top of an already large fixed cost.

Key takeaways

• Each new token costs about 2N operations for the weights plus 2 × layers × context × width for attention.
• Attention catches up with the weights at N ÷ (layers × width) tokens: about 61K for Llama 3 8B but 197K for 405B.
• So long contexts cost small models proportionally more, one plausible reason for a price line; Anthropic has not stated its reason.

Prompt caching

On Haiku 5.5 a cached input token costs $0.01 per million, a tenth of the normal price. For agents and chat, which resend the same instructions and history on every request, caching is often the biggest line on the bill.

What is prompt caching?

When a model reads a prompt it computes keys and values for every token in every layer, the KV cache that the September explainer describes. If the next request starts with exactly the same tokens, those keys and values are the same too, so a server can keep them and skip the work. Research systems do this in several ways: Prompt Cache stores the attention states of reusable prompt segments, vLLM shares cache blocks between requests with a common prefix, and SGLang’s RadixAttention keeps a tree of earlier prefixes so any new request reuses the longest match. Anthropic documents how its caching behaves and is billed, not how it is implemented.

How does it work on the Claude API?

• Exact prefixes only. A hit needs every token up to the cache point to match. The prompt is read in a fixed order, tools, then system, then messages, and “changes at any level invalidate that level and all subsequent levels” (prompt caching docs).
• Writes cost extra, reads cost little. Writing to the 5-minute cache costs 1.25 times the input price and reading costs 0.1 times, so caching pays for itself after a single hit (pricing). A 1-hour cache costs 2 times to write.
• Short lifetime. An entry lives 5 minutes from the start of the last request that used it, and every hit resets the clock.
• A minimum size. Prompts shorter than 512 tokens on Haiku 5.5 are never cached; Haiku 4.5 needed 4,096.

With automatic caching, one setting moves the cache point forward as a conversation grows, so each request reads what the previous one wrote and writes only what is new.

With only a new question, the request reads 77,000 cached tokens and costs about 87% less than without caching. Edit anything near the start and the saving turns into a penalty of about 24%, because the request pays the write premium on tokens it then never reuses. Grow the manual past the 100K line and cache reads cost $0.05 per million instead of $0.01: caching still helps, but it does not get you under the line, because cached tokens count towards the prompt length.

The other launches make caching just as central. Muse Wonder charges $0.20 per million cached input tokens against $4 uncached (Meta), and Gemini 4 Argon takes 95% off cached input (Google).

Key takeaways

• Prompt caching reuses the keys and values computed for a repeated prefix, so the server skips that work and charges a tenth of the price.
• A hit needs an exact prefix in the order tools, system, messages: anything that changes belongs at the end.
• On Haiku 5.5, cached tokens still count towards the 100K line, so caching lowers the price but not the prompt length.

Effort and thinking

Haiku 5.5 is the first Haiku with an effort setting. It is a dial for how many tokens the model spends, and on a reasoning model that is a dial for quality, speed and cost at the same time.

What is effort?

Haiku 5.5 thinks before it answers when it judges that a request needs it, which Anthropic calls adaptive thinking. Effort, set per request to low, medium, high, xhigh or max, steers how much. According to the effort documentation it “affects all tokens in the response”, including thinking, text and tool calls, and lower effort means “fewer and terser tool calls”. It is “a behavioral signal, not a strict token budget”: the model still thinks on hard problems at low effort, just less. Haiku 5.5 defaults to medium.

How does it change results?

Spending more computation at inference time can substitute for a bigger model on some problems (Snell and colleagues), which is the idea behind reasoning models. Anthropic’s system card shows it on Haiku 5.5: its headline benchmark scores are run at max effort, including 39.2% on Terminal-Bench 4.0, where Haiku 4.5 scored 0.0%. On Cognition’s FrontierCode benchmark, Haiku 5.5 at max effort scored 46.4%, level with Sonnet 5.5 at max (46.2%), although Sonnet 5.5 reached 52.1% at xhigh. The card plots these scores against output tokens per task, which is the honest way to read them: a higher score bought with more tokens is not free.

Anthropic’s guidance for Haiku 5.5 is to start at medium, use low for chat and simple high-volume work, and use xhigh or max “only where your evals show a quality gain”, comparing the cost with Sonnet 5.5 at that point. At low effort in long agent prompts, it warns, the model “is more likely to skip a search, stop early, or skip a check”.

What does thinking do to the bill?

• Thinking is output. It is billed at the output price and counts towards max_tokens, so a small limit tuned for Haiku 4.5 can now stop after the thinking and before any answer (migration guide).
• Old thinking stays in the prompt. “Thinking blocks from earlier assistant turns stay in context and count as input tokens, where Claude Haiku 4.5 kept only the latest turn’s.” A feature called thinking block clearing removes them.
• You do not see it by default. Haiku 5.5 returns each thinking block empty, with only a signature, unless you ask for a summary. You still pay for the tokens.

Key takeaways

• Effort (low to max, default medium) steers how many tokens Haiku 5.5 spends on thinking, text and tool calls; it is a signal, not a hard budget.
• Headline benchmark scores are at max effort; read them against tokens per task, as the system card plots them.
• Thinking is billed as output, counts towards max_tokens, and earlier thinking now stays in the prompt as input.

One agent run, end to end

Now put the pieces together. Haiku 5.5 is pitched as a sub-agent, the cheap model that does many small steps for a bigger one. Here is what one run of such an agent costs, step by step.

What does the simulation model?

An agent works in a loop: send the prompt, the model thinks and calls a tool, the tool’s result is added to the conversation, and the whole thing is sent again. So the prompt grows every step. The token counts are your choice, which is why this is a simulation; the prices and rules are Anthropic’s published ones: the 100K line applied per request, cache reads at 0.1 times and writes at 1.25 times the input price, and the 1.3 tokenizer ratio for Haiku 4.5.

With the defaults, 30 steps that each add a 2,500-token tool result and 1,500 tokens of thinking, the run costs about $0.16 on Haiku 5.5 against about $0.50 on Haiku 4.5: 68% less, close to Anthropic’s 75% average. But look at where the money goes. The prompt crosses 100K tokens at step 22, and the last 9 steps cost nearly three times as much as the first 21 together.

Clear the earlier thinking and the prompt ends at about 92,000 tokens instead of 136,000. The run never crosses the line, costs about $0.05, and the saving against Haiku 4.5 becomes 87%: the same work, a third of the price, because of what is left in the context. Turn caching off and the Haiku 5.5 run costs about $0.70, more than four times as much. In practice, clearing old blocks rewrites part of the prompt and costs one cache write, so do it in batches rather than every step.

Key takeaways

• An agent’s prompt grows every step, so caching and context management matter more than the per-token price.
• On Haiku 5.5, a run whose prompt crosses 100K tokens pays five times as much for every later step; keeping it under the line can cut the bill by two thirds.
• Measure your own runs: tokens per step, steps per task and where the prompt crosses the line decide the real saving.

Muse Wonder and the model card

Meta’s Muse Wonder had the biggest launch of the week and the least technical detail. Its model card is still worth reading closely, because it shows how to judge a launch when the model itself is a black box.

What did Meta claim?

Muse Wonder accepts text, images, video and audio, writes text, and has a one-million-token context (model card). On Artificial Analysis’s Intelligence Index, an independent composite, VentureBeat reports scores of 56 for Wonder Full and 52 for Wonder, against 58 for Claude Opus 5.5 and 53 for GPT-6 Astra. Note which model is which: the API sells Wonder, while the higher-scoring Wonder Full is only in Muse Code subscriptions. Meta has not published parameter counts, architecture or training compute for either.

How were the scores chosen?

The card is unusually explicit about its method, and the method matters. Where a comparable third-party or provider-reported result exists Meta uses it; “otherwise we use the highest of our internally validated run, self-reported score, or official leaderboard score”. Muse results are at max reasoning effort. Competitor runs Meta did itself are “best-effort”, and their “prompts, tools, and runtime may not be optimally tuned for proprietary models”. One coding benchmark compares models running in different agent products, which the card itself says “is not a controlled common-harness comparison”. In its favour, refusals “receive zero credit” rather than being dropped.

What does the ExploitGym footnote show?

In a cyber-security test where the model must exploit a known vulnerability to capture a secret flag, Muse Wonder Full “succeeded in capturing 92 flags”. A verification pass found that “21 of those used the intended vulnerability, while the other 71 were captured through unintended vulnerabilities”, so 21 is the number comparable with the published ranking. This is specification gaming in a benchmark: the grader checked for the flag, not for how it was obtained. Meta deserves credit for reporting both numbers. The lesson for readers is to ask what a grader actually checks.

What is Active Memory?

Meta describes it as “the ability to carry what matters to you across conversations and bring back the details that are relevant to the task at hand” (announcement), and tests it on an internal benchmark of 64,000 to 100,000-token conversations with distractors and facts that are later updated. It has not said how it works: whether the model writes notes to a store and retrieves them, as retrieval systems do, or something else. Whatever the mechanism, anything retrieved into the context is input tokens on the bill.

Key takeaways

• Muse Wonder’s API model scores 52 on the Intelligence Index; the 56 belongs to Wonder Full, which is subscription-only. Size and architecture are undisclosed.
• Read how scores were chosen: best-of selection, max effort and untuned competitor runs all favour the reporting lab.
• Ask what a grader checks: 71 of Wonder Full’s 92 ExploitGym flags came through unintended vulnerabilities.

Choosing, and what to watch

A cheaper model is only cheaper if it does your work in a similar number of tokens and steps. Here is how to tell, and what changes when you switch.

What changes when you migrate from Haiku 4.5?

The migration guide lists several changes that break or quietly change old code:

• Recount prompts and raise max_tokens: about 30% more tokens per text, and thinking now counts towards the limit.
• temperature, top_p and top_k are effectively gone: any value other than the defaults returns an error, so use instructions instead.
• Ending a request with a partial assistant turn for the model to continue, called prefill, is rejected.
• Keep conversations append-only: changing earlier turns while sending back thinking blocks returns an error, and it would break the cache anyway.

How should you decide?

Run 20 to 50 of your real tasks on each candidate at two or three effort settings and record quality, total tokens, cost and time per task, as the evaluation lesson describes. For Haiku 5.5, also log the prompt length of every request: the share of requests over 100K tokens can matter more than any benchmark.

What should you watch next?

• Independent token counts. How many tokens Haiku 5.5 really uses per task at each effort level, measured outside Anthropic.
• Whether price lines spread. If more labs price long prompts separately, context length becomes a standard line item rather than a free resource.
• What Meta publishes. The announcement promises a fuller safety report; a technical report on Active Memory would turn a marketing term into something testable.
• Argon’s price on 1 December, when introductory pricing ends and its rates double.

Key takeaways

• Haiku 5.5 is cheapest for short-prompt, high-volume work; above 100K tokens the case for Sonnet or Opus gets stronger.
• Migrating means recounting tokens, raising max_tokens, dropping sampling parameters and prefill, and keeping conversations append-only.
• Decide with your own tasks, logging tokens, cost, time and how often prompts cross the line.

Check your understanding

Six scenarios, from budgeting a migration to reading a model card and serving long prompts.

Your team budgeted a move from Haiku 4.5 to Haiku 5.5 by multiplying last month’s token counts by the new prices. Every request is under 100K tokens. What is the main error in the estimate?

References

Prices, limits and billing rules come from each company’s documentation and pricing pages as of 14 October 2026; they change often. Benchmark numbers are as reported by the model makers or, where stated, by Artificial Analysis via VentureBeat. The background on tokenization, attention cost and caching comes from peer-reviewed papers and preprints.

Sources

1. [1]

   Pricing(opens in a new tab)

   Anthropic, 2026

   Haiku 5.5: $0.10 / $0.50 per million tokens for prompts up to 100,000 tokens, $0.50 / $2.50 above; prompt length counts all input tokens including cache reads and writes, and each request is priced on its own. Haiku 4.5: $1 / $5. Cache writes 1.25× (5 minutes) or 2× (1 hour) base input; cache reads 0.1×.

2. [2]

   Anthropic launches Claude Haiku 5.5 with 90% API price reduction, matching GPT-6 Luna(opens in a new tab)

   VentureBeat, 2026

   Launch coverage: Haiku 5.5’s short-prompt prices match GPT-6 Luna’s; Anthropic did not explain how the cost reduction was achieved.

3. [3]

   Introducing Claude Haiku 5.5(opens in a new tab)

   Anthropic, 2026

   Launch announcement (7 October 2026): “on average, it now costs around 75% less to run” than Haiku 4.5; prices 90% lower up to 100,000 tokens and 50% lower above; an updated tokenizer that “uses slightly more tokens per task”; benchmark table against Haiku 4.5, GPT-6 Luna and Sonnet 5.5.

4. [4]

   Models overview(opens in a new tab)

   Anthropic, 2026

   Haiku 5.5: 1M-token context, 128K maximum output, adaptive thinking, default effort medium, knowledge cutoff June 2026. 1M tokens is roughly 555,000 words on the current tokenizer, against about 750,000 words before it.

5. [5]

   Muse Wonder (Meta Model API)(opens in a new tab)

   Meta, 2026

   Model page: $4 input, $0.20 cached input and $20 output per million tokens; 1M-token context; Muse Wonder Full available only in Muse Code subscriptions.

6. [6]

   Introducing Muse Wonder and Muse Wonder Full(opens in a new tab)

   Meta, 2026

   Announcement (12 October 2026): Active Memory, “the ability to carry what matters to you across conversations”, and Active Focus, “the ability to keep working toward your goal when a task gets hard”.

7. [7]

   Introducing Gemini 4 Argon(opens in a new tab)

   Google, 2026

   General availability (13 October 2026): $2 / $10 per million tokens until 30 November 2026, then $4 / $20; cached input 95% off; “an industry-leading 1M unified input-output token limit”.

8. [8]

   Muse Wonder Model Card(opens in a new tab)

   Meta, 2026

   Release 12 October 2026; text, image, video and audio in, text out; 1M-token context. Evaluation methodology, internal benchmarks (including MemoryBench), risk assessment under the Meta Superintelligence Scaling Framework, and the ExploitGym note: 92 flags captured, 21 through the intended vulnerability.

9. [9]

   Claude Haiku 5.5 System Card(opens in a new tab)

   Anthropic, 2026

   Pre-deployment evaluations. Terminal-Bench 4.0: 39.2% at max effort (Haiku 4.5: 0.0%); scores plotted against output tokens per task at each effort level; Haiku 4.5 had a 200K context window and 64K output limit. Architecture and size are not described.

10. [10]

    Claude Haiku 5.5 migration guide(opens in a new tab)

    Anthropic, 2026

    The same input text produces “approximately 30% more tokens” than on Haiku 4.5; thinking blocks from earlier turns now stay in context and count as input; temperature, top_p and top_k removed; assistant prefill rejected; effort replaces thinking budgets.

11. [11]

    Neural Machine Translation of Rare Words with Subword Units(opens in a new tab)

    Sennrich, R., Haddow, B., Birch, A., 2016

    Introduces byte-pair encoding for neural language models: repeatedly merge the most frequent pair of symbols, so the number of merges sets the vocabulary size. ACL 2016.

12. [12]

    API pricing(opens in a new tab)

    OpenAI, 2026

    GPT-6 Luna: $0.10 input, $0.01 cached input and $0.50 output per million tokens; $0.20 / $0.02 / $0.75 once input exceeds 272K tokens.

13. [13]

    Scaling Laws for Neural Language Models(opens in a new tab)

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., Amodei, D., 2020

    Table 1 counts the forward-pass cost of a transformer per token as about 2N + 2 · layers · context · attention width operations: one term for the weights, one that grows with context.

14. [14]

    The Llama 3 Herd of Models(opens in a new tab)

    Grattafiori, A. et al. (Meta), 2024

    Published shapes: 8B (32 layers, width 4,096), 70B (80 layers, width 8,192), 405B (126 layers, width 16,384), each with 8 key-value heads of dimension 128.

15. [15]

    Efficient Memory Management for Large Language Model Serving with PagedAttention(opens in a new tab)

    Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., Stoica, I., 2023

    vLLM: KV cache memory limits how many requests a server can batch; paging it lets more fit. Also shares cache blocks between requests with a common prefix. SOSP 2023.

16. [16]

    Prompt Cache: Modular Attention Reuse for Low-Latency Inference(opens in a new tab)

    Gim, I., Chen, G., Lee, S., Sarda, N., Khandelwal, A., Zhong, L., 2024

    Precomputes and stores the attention states (keys and values) of frequently reused prompt segments so later requests skip recomputing them. MLSys 2024.

17. [17]

    SGLang: Efficient Execution of Structured Language Model Programs(opens in a new tab)

    Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., Barrett, C., Sheng, Y., 2024

    RadixAttention: keeps the KV cache of earlier requests in a prefix tree so any new request sharing a prefix reuses it automatically. NeurIPS 2024.

18. [18]

    Prompt caching(opens in a new tab)

    Anthropic, 2026

    Cache hits need an exact prefix match in the order tools, system, messages; a change invalidates everything after it. 5-minute default lifetime, refreshed on each hit; minimum cacheable prompt 512 tokens on Haiku 5.5 (4,096 on Haiku 4.5).

19. [19]

    Effort(opens in a new tab)

    Anthropic, 2026

    Five levels (low, medium, high, xhigh, max); Haiku 5.5 defaults to medium. Effort “affects all tokens in the response”, including thinking and tool calls, and is “a behavioral signal, not a strict token budget”.

20. [20]

    Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters(opens in a new tab)

    Snell, C., Lee, J., Xu, K., Kumar, A., 2024

    Shows that spending more computation at inference time can let a smaller model match a larger one on some problems, and that the best amount depends on difficulty.

21. [21]

    Meta’s new Muse Wonder and Muse Wonder Full models achieve frontier performance, but only the weaker one is available via API(opens in a new tab)

    VentureBeat, 2026

    Artificial Analysis Intelligence Index: Wonder Full 56, Wonder 52, Claude Opus 5.5 58, GPT-6 Astra 53. Meta has not published parameter counts or architecture.

22. [22]

    Specification gaming: the flip side of AI ingenuity(opens in a new tab)

    Krakovna, V. et al. (DeepMind), 2020

    Catalogue of systems that satisfy the literal objective without achieving the intended outcome, and why it matters for evaluation.