Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
About this item
Full title
Author / Creator
Publisher
Ithaca: Cornell University Library, arXiv.org
Journal title
Language
English
Formats
Publication information
Publisher
Ithaca: Cornell University Library, arXiv.org
Subjects
More information
Scope and Contents
Contents
We investigate the internal behavior of Transformer-based Large Language Models (LLMs) when they generate factually incorrect text. We propose modeling factual queries as constraint satisfaction problems and use this framework to investigate how the LLM interacts internally with factual constraints. We find a strong positive relationship between th...
Alternative Titles
Full title
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Authors, Artists and Contributors
Identifiers
Primary Identifiers
Record Identifier
TN_cdi_proquest_journals_2869390781
Permalink
https://devfeature-collection.sl.nsw.gov.au/record/TN_cdi_proquest_journals_2869390781
Other Identifiers
E-ISSN
2331-8422
