Fine-Tuning Methods

Fine-tuning isn't one technique but a family of them. Each one changes a pre-trained model in a different way, and each needs a different kind of data. This lesson covers instruction tuning, domain adaptation, transfer learning and continued pre-training, and shows how to tell from a scenario…

10 cards

Study this deck

Fine-Tuning Methods

Transfer learning is the big idea

Transfer learning means reusing a model trained on one job as the starting point for a related job, instead of training from nothing. Every fine-tuning method in this lesson is a form of it. A foundation model already understands language in general, so it needs far less data, time and money to learn a narrower job than a brand-new model would. In practice, fine-tuning is how transfer learning gets done.

Instruction tuning teaches a model to follow requests

Instruction tuning trains a model on labeled examples written as instructions, each paired with the response you want. One example might be "Summarize this complaint in two sentences" plus an ideal two-sentence summary. In SageMaker JumpStart, this data is a file of prompt and response pairs. It makes a model better at following directions and at specific tasks, and it's the usual way to turn a raw text-completion model into a helpful assistant. Because every example needs a hand-written ideal answer, the data takes real effort to build.

Domain adaptation teaches a field's language

Domain adaptation fine-tuning trains a model on text from one field so it handles that field's jargon and style, for example legal contracts or financial filings. In SageMaker JumpStart, its training data is plain domain text, with no prompt and response pairs. Continued pre-training, covered in the previous lesson, works on the same principle, at a much larger scale. Both use unlabeled text and add knowledge rather than task skills.

Reinforcement fine-tuning learns from scores

Amazon Bedrock also offers reinforcement fine-tuning. Instead of labeled answers, you provide prompts and a reward function: a grader, written as code or run by another model, that scores each response. The model tries several answers per prompt and learns to favor high-scoring ones. It suits tasks where a good answer is easy to check but hard to write in advance, such as code that must pass tests.

Method	Data	Best when you want
Instruction tuning	Labeled prompt and response pairs	Better task behavior and instruction following
Domain adaptation	Unlabeled domain text	Comfort with a field's vocabulary
Continued pre-training	Large volumes of unlabeled domain text	Deep new domain knowledge
Reinforcement fine-tuning	Prompts plus a reward function	Answers that score well against a clear check

On the exam

Look at the data first. If the scenario has example questions with ideal answers, the answer is instruction tuning or supervised fine-tuning. If it has a pile of documents with no answers, it's domain adaptation or continued pre-training. If the question asks for the general principle of reusing a trained model, it's transfer learning. Reinforcement fine-tuning fits when you can score answers automatically but writing ideal answers would be impractical.

Quiz me on this

An e-commerce company has 5,000 past customer emails, each paired with the reply its best agent wrote. It wants a model that drafts replies in the same helpful style. Which fine-tuning method fits this data?

Domain adaptation fine-tuning on the email text alone
✓ Instruction tuning with the email and reply pairs
Continued pre-training on the replies alone
Training a new model from scratch on the emails

A team adapts a pre-trained language model to classify support tickets instead of training a new model from the beginning. Which general machine learning idea is it using?

Reinforcement learning from human feedback
Unsupervised clustering
Model distillation
✓ Transfer learning

A law firm's model answers general questions well but stumbles over legal terms in contracts. The firm has thousands of past contracts but no question-and-answer examples. Which approach targets this gap with the data it has?

✓ Domain adaptation fine-tuning on the contracts
Instruction tuning on the contracts
Reinforcement fine-tuning with a reward function
Distillation into a smaller student model

A software company wants a model that writes SQL queries for its reporting tool. It can automatically run each generated query and check whether the result is correct, but writing thousands of ideal queries by hand would be impractical. Which Amazon Bedrock customization method suits this?

Supervised fine-tuning on hand-written queries
Continued pre-training on database manuals
✓ Reinforcement fine-tuning with an auto-grader
Distillation using database manuals as prompts

A hospital ran continued pre-training on millions of unlabeled clinical notes. The model now knows medical terms well, but when staff ask it to "list the medications in this note", it just continues the text instead of answering. What should the team do next?

Repeat continued pre-training with more clinical notes
Raise the temperature so the model answers more directly
✓ Instruction-tune it on labeled request-answer pairs
Switch to domain adaptation on the same unlabeled notes

Transfer learning

Reusing a model trained on one job as the starting point for a related job. Fine-tuning is how it's usually done.

Instruction tuning

Fine-tuning on labeled prompt and response pairs phrased as instructions, so the model follows requests better.

Domain adaptation fine-tuning

Fine-tuning on plain, unlabeled text from one field so the model handles its jargon and style.

Reinforcement fine-tuning (Amazon Bedrock)

Customization that uses prompts plus a reward function that scores responses, instead of labeled answers.

Labeled vs unlabeled fine-tuning data

Raw, unlabeled text (continued pre-training) adds domain knowledge. Labeled examples (supervised fine-tuning) then teach the model to use it for tasks.