Evaluation of the morphological quality of machine translation outputs. The automatically generated test suites in English can be translated into different target languages (Czech, Latvian, German, French). The output is then analyzed and provides three types of information:
- Adequacy: has the morphological information been well conveyed from the source?
- Fluency: do we have local agreement?
- Consistency: how well is the system confident in its prediction?
The approach is described in the following paper:
Franck Burlot and François Yvon. 2017.
Evaluating the morphological competence of machine translation systems. In Proceedings of the Second
Conference on Machine Translation (WMT’17). Association for Computational Linguistics,
Copenhagen, Denmark.
pdf
bibtex
Test suite
English source sentences: download
Sentence tags: download
Running the evaluation
Evaluation scripts: follow the readme
Latvian dictionary: download
Test suite
English source sentences: download
Sentence tags: download
Running the evaluation
Evaluation scripts: follow the readme
French dictionary: download