Demystifying Forgetting in Language Model Fine-Tuning with Statistical Analysis of Example Associations
Language models (LMs) are known to suffer from forgetting of previously learned examples
when fine-tuned, breaking stability of deployed LM systems. Despite efforts on mitigating …
when fine-tuned, breaking stability of deployed LM systems. Despite efforts on mitigating …
Demystifying Language Model Forgetting with Low-Rank Example Associations
Large Language models (LLMs) suffer from forgetting of upstream data when fine-tuned.
Despite efforts on mitigating forgetting, few have investigated whether, and how forgotten …
Despite efforts on mitigating forgetting, few have investigated whether, and how forgotten …