Most foundation models are paid for by the token, so the length of what you send and what the model sends back drives both the bill and the wait. This lesson explains how token-based pricing works, why input and output are priced separately, and the levers a team can pull to lower cost and speed up responses.
How you're charged
With on-demand use of a model in Amazon Bedrock, you pay for the tokens each request uses, with no upfront commitment. Every request has two sides:
- Input tokens: everything sent to the model, including the system instructions, the user's question, any retrieved documents and earlier conversation turns.
- Output tokens: everything the model generates in reply.
Input and output are counted and billed separately, at rates set per model. For most models an output token costs more than an input token, because generating text takes more work than reading it. Rates vary widely between models, so a small model can be many times cheaper per token than a large one.
What drives cost and latency
Cost is roughly the number of tokens times the rate, summed across every request. That means hidden tokens add up: a long system prompt or a stack of retrieved documents is billed again on every single request, and in a chat, each new turn usually resends the conversation so far.
Latency follows a similar pattern. The model reads the input quickly, but it writes output one token at a time, so long responses take noticeably longer to finish. Asking for a 50-word answer instead of a 500-word one cuts both the output bill and the waiting time.
Ways to lower token costs
| Lever | Effect |
|---|---|
| Shorter, focused prompts | Fewer input tokens on every call |
| Retrieve only relevant chunks | Fewer input tokens than pasting whole documents |
| Limit response length | Fewer output tokens and faster replies |
| Choose a smaller model for simple tasks | Lower rate per token |
| Prompt caching | Repeated prompt prefixes are read from cache at a lower rate and with lower latency |
| Batch inference | Lower price for supported models when results aren't needed right away |
Prompt caching suits workloads that send the same long content, such as a large document or a fixed set of instructions, with many different questions. Bedrock also offers a token-counting operation so teams can estimate a prompt's size before sending it.
Beyond on-demand
On-demand pricing fits variable or unpredictable traffic. For large, steady workloads, Amazon Bedrock also offers Provisioned Throughput, where you pay by the hour for reserved model capacity instead of per token. The trade-offs between these options are covered later in this domain.
On the exam
If a question asks why a GenAI bill is higher than expected, look for long prompts repeated on every call, whole documents pasted in, long outputs, or an oversized model. If it asks how to cut cost without changing the model, the usual answers are shorter prompts, sending only relevant context, capping output length, prompt caching for repeated content, and batch inference for work that can wait.