Clinic 05

Freeze Or Fine-Tune?

You have a pretrained model and a small labeled dataset. The question is not whether transfer helps. The question is how much of the model to unfreeze.

Back To Clinics Open Transfer Topic Open The Full Track

Situation

Small Data, Big Model

Full fine-tuning scored highest on train. But the validation gap is growing. Frozen features scored lower but the gap is tight.

Your Job

Choose The Transfer Strategy

Pick the strategy you would ship, name the risk of the rejected option, and say what data budget would change your mind.

Bad Habit To Avoid

Fine-Tune Everything By Default

If you always unfreeze because the training score is better, you are ignoring the generalization signal.

Situation

You are classifying product images into 12 categories using a pretrained ResNet-50 backbone. Your labeled dataset has 1,200 images (100 per class).

The packet says:

• three strategies were compared: frozen backbone + linear head, partial fine-tune (last block), full fine-tune
• all three beat the train-from-scratch baseline
• the full fine-tune has the best training accuracy but the worst validation gap

Artifact Packet

Read this packet before you decide:

strategy	train accuracy	val accuracy	train-val gap	epochs to best val	params trained
frozen_linear	0.841	0.823	0.018	8	24,588
partial_finetune	0.907	0.862	0.045	14	2,385,932
full_finetune	0.968	0.831	0.137	22	23,508,044
train_from_scratch	0.743	0.594	0.149	30	23,508,044

Per-class validation accuracy for the weakest three classes:

class	frozen_linear	partial_finetune	full_finetune
hardware_mount	0.75	0.80	0.70
cable_adapter	0.70	0.85	0.75
power_supply	0.80	0.90	0.65

Decision Prompt

Write the note before you open the reveal.

Your note should answer:

1. Which strategy would you deploy right now?
2. What is the main risk of full fine-tuning here?
3. What dataset size would make you reconsider full fine-tuning?
4. What evidence from the per-class table supports your decision?

Keep the note short. Four to six sentences is enough.

Strong Reasoning Looks Like

• it connects the small dataset to overfitting risk when too many parameters are freed
• it uses the per-class table to show full fine-tuning is worst on the hardest classes
• it picks partial fine-tune for the best val accuracy with a manageable gap
• it names a concrete data threshold (e.g., 5-10x more data) before reconsidering full fine-tuning

Common Wrong Moves

• choosing full fine-tune because it has the highest training accuracy
• choosing frozen features because it has the smallest gap, ignoring that partial is materially better
• ignoring the per-class breakdown entirely
• saying "fine-tune is always better with pretrained models"

Run The Clinic In Browser

Validate Your Decision In Browser

Reference Reveal

Open only after you write the note

The reference choice is: - `selected_strategy = partial_finetune` - `decision = deploy, monitor per-class metrics` Why: - partial fine-tuning has the best validation accuracy (0.862) with a moderate gap (0.045) - full fine-tuning memorizes training data: its gap is 0.137 on only 1,200 images - the per-class table confirms: full fine-tuning collapses on the hardest classes while partial fine-tuning is strongest there - frozen features are safe but leave accuracy on the table when partial unfreezing is available - with 10-50x more labeled data, full fine-tuning becomes viable because the model has enough signal to justify the parameter count The practical lesson: the right transfer strategy depends on the ratio of data to parameters, not on training accuracy alone.

What To Do Next

After this clinic:

1. open Transfer and Fine-Tuning
2. run the matching transfer example
3. use ResNet, BERT, and Fine-Tuning for the full representation-choice workflow