Predictability and Surprise in Large Generative Models
Predictability and Surprise in Large Generative Models
About this item
Full title
Author / Creator
Ganguli, Deep , Hernandez, Danny , Lovitt, Liane , DasSarma, Nova , Henighan, Tom , Jones, Andy , Nicholas, Joseph , Jackson Kernion , Mann, Ben , Askell, Amanda , Bai, Yuntao , Chen, Anna , Conerly, Tom , Drain, Dawn , Nelson Elhage , Sheer El Showk , t, Stanislav , Hatfield-Dodds, Zac , Johnston, Scott , Kravec, Shauna , Nanda, Neel , Ndousse, Kamal , Olsson, Catherine , Amodei, Daniela , Amodei, Dario , Brown, Tom , Kaplan, Jared , McCandlish, Sam , Olah, Chris and Clark, Jack
Publisher
Ithaca: Cornell University Library, arXiv.org
Journal title
Language
English
Formats
Publication information
Publisher
Ithaca: Cornell University Library, arXiv.org
Subjects
More information
Scope and Contents
Contents
Large-scale pre-training has recently emerged as a technique for creating capable, general purpose, generative models such as GPT-3, Megatron-Turing NLG, Gopher, and many others. In this paper, we highlight a counterintuitive property of such models and discuss the policy implications of this property. Namely, these generative models have an unusua...
Alternative Titles
Full title
Predictability and Surprise in Large Generative Models
Authors, Artists and Contributors
Author / Creator
Hernandez, Danny
Lovitt, Liane
DasSarma, Nova
Henighan, Tom
Jones, Andy
Nicholas, Joseph
Jackson Kernion
Mann, Ben
Askell, Amanda
Bai, Yuntao
Chen, Anna
Conerly, Tom
Drain, Dawn
Nelson Elhage
Sheer El Showk
t, Stanislav
Hatfield-Dodds, Zac
Johnston, Scott
Kravec, Shauna
Nanda, Neel
Ndousse, Kamal
Olsson, Catherine
Amodei, Daniela
Amodei, Dario
Brown, Tom
Kaplan, Jared
McCandlish, Sam
Olah, Chris
Clark, Jack
Identifiers
Primary Identifiers
Record Identifier
TN_cdi_proquest_journals_2629520415
Permalink
https://devfeature-collection.sl.nsw.gov.au/record/TN_cdi_proquest_journals_2629520415
Other Identifiers
E-ISSN
2331-8422
DOI
10.48550/arxiv.2202.07785
