Once a model is chosen, applications need a way to send it requests and get answers back. There are two main ways to do that: call a managed API service, where someone else runs the model, or host the model yourself behind your own API. The exam tests the trade-offs between the two: effort, control and how you pay.
What "Using a Model in Production" Means
In production, an application, such as a website, a mobile app or a back-office system, sends input to the model and receives a prediction or generated content. This is inference. The model sits behind an API (application programming interface), a defined way for software to send requests and get responses. The question is who runs the machines behind that API, and who looks after them. The answer affects how fast you can launch, how much control you keep, and how the bill is calculated.
Managed API Services
With a managed API service, the provider hosts and operates the model. You send a request and get a response; you never see a server. Amazon Bedrock works this way for foundation models. Its on-demand option charges for what you use: for text models, that's measured in tokens (small chunks of text). The provider handles scaling up when traffic spikes, patching, and keeping the model available. The trade-off is less control: you use the models and settings the service offers.
Self-Hosted APIs
With a self-hosted API, you deploy the model onto infrastructure you choose and pay for, such as an Amazon SageMaker AI endpoint or Amazon EC2 instances. You pick the model, including an open-source or custom one, its exact version, the instance types, and how it sits in your network. You usually pay for the instances while they run, whether or not requests arrive, and you take on more of the work of sizing, scaling and updating. SageMaker AI reduces that work compared with raw EC2, but you still make the capacity decisions.
Choosing Between Them
| Managed API (e.g. Amazon Bedrock) | Self-hosted (e.g. SageMaker AI endpoint, EC2) | |
|---|---|---|
| Setup effort | Lowest | Higher |
| Control over model and infrastructure | Limited to what's offered | High |
| Typical cost model | Pay per request or per token | Pay for running instances |
| Who scales and maintains it | The provider | Mostly you |
| Can run your own custom model | Only where the service supports it | Yes |
On the Exam
Watch for the constraint in the scenario. "Fastest to launch", "no infrastructure to manage" or "unpredictable, low volume" point to a managed API. "Must run our own custom or open-source model", "full control of the environment" or "steady, very high volume on dedicated capacity" point to self-hosting. A common trap is assuming self-hosting is always cheaper. At low or spiky volume, paying for idle instances usually costs more than paying per request. Many real systems mix both: a managed API for general tasks and a self-hosted endpoint for one specialised model.