Cardon, Rémi
[UCL]
Bibal, Adrien
[UCL]
Souza Wilkens, Rodrigo
[UCL]
Alfter, David
[UCL]
Norré, Magali
[UCL]
Müller, Adeline
[UCL]
Watrin, Patrick
[UCL]
François, Thomas
[UCL]
Evaluating automatic text simplification (ATS) systems is a difficult task that is either performed by automatic metrics or user-based evaluations. However, from a linguistic point-of-view, it is not always clear on what bases these evaluations operate. In this paper, we propose annotations of the ASSET corpus that can be used to shed more light on ATS evaluation. In addition to contributing with this resource, we show how it can be used to analyze SARI’s behavior and to re-evaluate existing ATS systems. We present our insights as a step to improve ATS evaluation protocols in the future.


Bibliographic reference |
Cardon, Rémi ; Bibal, Adrien ; Souza Wilkens, Rodrigo ; Alfter, David ; Norré, Magali ; et. al. Linguistic Corpus Annotation for Automatic Text Simplification Evaluation.EMNLP 2022 (Abu Dhabi, United Arab Emirates, du 07/12/2022 au 11/12/2022). In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics : Abu Dhabi, United Arab Emirates2022, p. 1842–1866 |
Permanent URL |
http://hdl.handle.net/2078.1/268658 |