How do you test for catastrophic forgetting after fine-tuning?
A higher overall score can hide the loss of familiar capabilities. Use a paired retention test to decide whether a fine-tuned model is ready.
model selection
1 post with the tag “model selection”