Before a generative AI model can read your text, the text is broken into tokens. Before it can search your documents, they are cut into chunks and turned into embeddings, which are lists of numbers called vectors. This lesson explains all four ideas and how they work together, because almost every exam question about retrieval, cost or search builds on them.
Tokens
A token is the unit of text a language model actually reads and writes. It is often a whole short word, but it can also be part of a longer word, a punctuation mark or a space. "Unbelievable" might be split into several tokens, while "cat" is one. Each model has its own way of splitting text, called its tokenizer, so the same sentence can be a different number of tokens on different models.
Tokens matter for three practical reasons. A model has a maximum number of tokens it can handle in one request, called its context window. Models are usually priced per token. And the more tokens a model has to read and write, the longer it takes to respond. Amazon Bedrock even offers a way to count a prompt's tokens before sending it, so you can estimate cost.
Chunking
Chunking means splitting a long document into smaller pieces before storing it for search. A 200-page manual is far too big to search as one block, and too big to hand to a model in full. Cut into chunks of a few hundred tokens, each piece can be matched to a question on its own, and only the relevant pieces are passed to the model.
Chunk size is a trade-off. Small chunks match a question precisely but may lose the surrounding context. Large chunks keep context but blur the match. Amazon Bedrock Knowledge Bases offers several strategies: fixed-size chunks with overlap, a default of about 300 tokens that respects sentence boundaries, hierarchical chunking (small child chunks for matching, larger parent chunks for context), semantic chunking that splits where the meaning changes, or no chunking at all.
Embeddings and vectors
An embedding is a numerical representation of a piece of content, such as a sentence, an image or a product, produced by an embedding model. The output is a vector: a long list of numbers that places the content as a point in a space with many dimensions. Content with similar meaning ends up close together in that space. "How do I reset my password?" and "I forgot my login details" use different words, but their vectors land near each other.
On AWS, Amazon Bedrock provides embedding models such as Amazon Titan Text Embeddings, which turn text into vectors for search, similarity comparison and clustering.
Similarity search
Once documents are chunked and embedded, the vectors are stored in a vector database. When a user asks a question, the question is embedded with the same model, and the database returns the chunks whose vectors are closest to it. This is called semantic search or similarity search. It finds matches by meaning rather than by exact keywords, which is the foundation of Retrieval Augmented Generation (RAG), covered in Domain 3.
| Step | What happens |
|---|---|
| Chunk | Split documents into smaller pieces |
| Embed | Turn each chunk into a vector |
| Store | Save the vectors in a vector database |
| Search | Embed the question and find the nearest chunks |
On the exam
If a scenario says keyword search misses results that use different wording, the answer involves embeddings and vector search. If it says answers lack context or cut off mid-thought, look at chunk size or the chunking strategy. And remember that the question and the documents must be embedded with the same model, or their vectors can't be compared.