There are three documented ways to pay less per token on Amazon Bedrock, they save different amounts, and two of them cannot be used together. The combination that looks best on paper, batch plus prompt caching, is explicitly unsupported.
How does Bedrock pricing work?
On-demand inference bills per million input tokens and per million output tokens, at a rate published per model on the Amazon Bedrock pricing page. Read the rates there rather than from any article: they change per model release, and a stale token price is worse than no price. What does not change weekly is the structure of the discounts, which is what this post is about.
| Lever | Documented saving | What you give up |
|---|---|---|
| Batch inference | "50% lower price compared to on-demand inference pricing" | Synchronous responses, tool calling, structured output, prompt caching |
| Prompt caching | "Cache read input tokens will be 75% less than on-demand input token price" | Nothing structural, but cache writes can cost more than uncached input |
| Provisioned Throughput | Hourly rate per model unit, lower with a 1-month or 6-month commitment | Elasticity. You pay for the units whether or not you use them |
When is batch inference worth it?
The discount is the largest single lever available and the pricing page states it flatly: Bedrock "offers select foundation models (FMs) from leading AI providers ... for batch inference at a 50% lower price compared to on-demand inference pricing". Note "select": it is not every model, and the pricing page carries the model list.
The batch inference documentation describes the shape of the job. You write prompts into files, upload them to S3, submit one request, and collect output files from S3 when it finishes. That is a different program, not a flag.
Three limitations decide most cases, all documented on that page:
-
No tool calling and no structured output. "Batch inference
does not support tool calling (function calling) or structured output
(
response_format). Each record in the input JSONL file is processed independently without multi-turn interaction." That rules out every agentic workload, which in our experience is most of what a FinOps product actually runs. - No provisioned models. "Batch inference isn't supported for provisioned models." The two commitment mechanisms do not stack.
- No prompt caching. Covered below, and it is the one that changes the arithmetic.
Where batch does fit: classification passes, bulk summarisation, embedding generation, evaluation runs, anything you would happily run overnight. If the work is a queue rather than a conversation, it is halfway to being a batch job already.
Two operational notes from the same documentation. Quotas are separate from on-demand ones and live in the Bedrock endpoints and quotas reference, so a large job can queue behind its own account limits. And you do not have to poll for completion: AWS supports EventBridge notifications on batch job state changes, which is the difference between a scheduled pipeline and a script that sleeps.
How much does prompt caching save?
The pricing page states it as a fixed ratio: "Cache read input tokens will be 75% less than on-demand input token price." The prompt caching documentation explains the mechanism as skipping recomputation: "By adding portions of your context to a cache, the model can use the cache to skip recomputation of inputs, allowing Bedrock to share in the compute savings and lower your response latencies."
The same page also states the cost that competitors' summaries tend to omit: "Depending on the model, tokens written to cache may be charged at a rate that is higher than that of uncached input tokens." Writing the cache can cost more than not caching. Caching a large prefix used twice may lose money. Caching one used two hundred times does not.
The break-even depends on reuse count, and reuse is bounded by the TTL: "The cache has a Time To Live (TTL), which resets with each successful cache hit ... If no cache hits occur within the TTL window, your cache expires. Many models support a 5-minute TTL." A prefix reused every four minutes stays warm indefinitely. One reused hourly is rewritten every time, at the higher rate, and saves nothing.
The silent failure mode
Cache checkpoints have a minimum size that varies by model, and missing it does not raise an error. From the documentation: "You can only create a cache checkpoint if your total prompt prefix meets the minimum number of tokens ... If you try to add a cache checkpoint before meeting the minimum number of tokens, your inference will still succeed, but your prefix will not be cached."
So a checkpoint set below the minimum produces a working application with no caching and no signal that caching is off. The minimum is per model and the spread is wide. AWS's examples include "Claude 3.7 Sonnet and Claude Sonnet 4.5 require at least 1,024 tokens per cache checkpoint, while Claude Opus 4.5, Claude Opus 4.6, and Claude Haiku 4.5 require at least 4,096 tokens per cache checkpoint", and the model table lists 512 for several current models.
Two consequences worth designing around. Changing model changes the minimum, so a model swap can silently disable caching that was working. And the only way to confirm caching is on is to read the cache token counts back from the API response, not to observe that the call succeeded. Check the usage fields, and check them again after any prompt or model change.
Where you set the checkpoint depends on the API. Caching works through the Converse and ConverseStream APIs, through InvokeModel, and through Bedrock Prompt management, where it is a checkbox rather than a request field. The APIs give the finer control: AWS notes they "provide you with the most flexibility and granular control over the prompt cache."
Caching and cross-Region inference
These interact, and not in your favour. The documentation notes that cross-Region inference "automatically selects the optimal AWS Region within your geography to serve your inference request", then adds: "At times of high demand, these optimizations may lead to increased cache writes." Cache writes are the expensive half. Under load, the mechanism that improves availability degrades the caching discount.
Why can't you use batch and prompt caching together?
Because AWS says so, in one sentence on the prompt caching page: "Prompt caching is only supported for on-demand inference endpoints. It is not supported with the batch inference API."
That forces a real choice on any workload with a large shared prefix, which is most retrieval-augmented and document-analysis work. A 50% cut on everything against a 75% cut on the cached portion of input only.
| Workload shape | Choose | Why |
|---|---|---|
| Large shared prefix, many queries against it, latency matters | Prompt caching | 75% off the reused input, and the responses stay synchronous |
| Every prompt different, no deadline, no tools | Batch | 50% off input and output, and there is nothing to cache anyway |
| Agentic loop with tool calls | Prompt caching | Batch does not support tool calling at all, so the choice is made for you |
| Steady, predictable, high-volume traffic on one model | Provisioned Throughput | Commitment pricing, though it excludes batch entirely |
We run an agentic FinOps assistant on Bedrock, so the third row is the one we live in: batch is unavailable to us at any price because the loop calls tools. That is worth saying out loud, because the "50% off with batch" advice is everywhere and it is inapplicable to a large share of the workloads reading it.
There is one more option that reads like a fourth lever and is not: the OpenAI-compatible Batch API Bedrock also exposes. It is a different interface to the same batch mechanism, so it carries the same discount and the same restrictions, including no caching and no tool calling.
When does Provisioned Throughput beat on-demand?
Provisioned Throughput is bought in model units billed per hour, and the pricing page publishes three columns per model: "Price per hour per model unit with no commitment", "for 1-month commitment", and "for 6-month commitment". Longer commitment, lower hourly rate, and the usual consequence: the units bill whether or not traffic arrives.
It is a capacity decision rather than a discount decision, and it is the only option for some custom models. The pricing page notes that "For Full Rank fine-tuned models, Provisioned Throughput (PT) is the available option", while parameter-efficient fine-tunes can choose either. Where you have that choice, the arithmetic is the same one as any commitment: measure sustained utilisation first, because unused model units are the AI equivalent of an over-provisioned Reserved Instance.
How do you attribute the remaining spend?
Cutting the rate is half the job. The other half is knowing which team spent
it, and CloudWatch cannot tell you: ModelId is the only dimension
on its Bedrock metrics. The two mechanisms that can are IAM principal cost
allocation and application inference profiles, set out in
Amazon Bedrock cost per team.
Once the data is in the CUR, the columns behave like any other service, with the same choice of cost basis described in which CUR cost column you should be summing. Provisioned Throughput commitments amortize; on-demand tokens do not. And because a bad prompt change can multiply token spend overnight with no infrastructure change to blame, anomaly detection rather than a budget is the alarm that catches it, subject to the detection lag documented there.