Towards end-to-end reference-free summarization evaluation via negative sampling
Date
2022-08
Authors
Luo, Ge
Major Professor
Advisor
Bao, Sheng
Li, Qi
Zhang, Wensheng
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Altmetrics
Abstract
Evaluating machine-generated summaries without a human-written reference summary has been a need for a long time. In this paper, we present a proof-of-concept study to a summary evaluation approach without the presence of reference summaries. By negative sampling, massive data in existing summarization datasets are transformed for training by pairing documents with corrupted reference summaries. Two learning schemes are explored: weakly supervised learning with explicit number labels and preference learning with inexplicit labels. Extensive experiments on several datasets show that our approaches can produce scores highly correlated with human ratings.
Series Number
Journal Issue
Is Version Of
Versions
Series
Academic or Administrative Unit
Type
thesis