Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs
Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs
About this item
Full title
Author / Creator
Publisher
Ithaca: Cornell University Library, arXiv.org
Journal title
Language
English
Formats
Publication information
Publisher
Ithaca: Cornell University Library, arXiv.org
Subjects
More information
Scope and Contents
Contents
Large language models can memorize and repeat their training data, causing privacy and copyright risks. To mitigate memorization, we introduce a subtle modification to the next-token training objective that we call the goldfish loss. During training, randomly sampled subsets of tokens are excluded from the loss computation. These dropped tokens are...
Alternative Titles
Full title
Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs
Authors, Artists and Contributors
Identifiers
Primary Identifiers
Record Identifier
TN_cdi_proquest_journals_3068911528
Permalink
https://devfeature-collection.sl.nsw.gov.au/record/TN_cdi_proquest_journals_3068911528
Other Identifiers
E-ISSN
2331-8422
