Automatic Classification, Visualization and Analysis of Errors in Machine Translation

Abstract

Although the quality of machine translation (MT) has improved in recent years, machine translated documents still contain errors. MT quality is often evaluated using a single numeric score. However, this may not adequately characterise the system. We provide an error visualizer, which shows differences between corresponding lines of two translations. In addition to insertions, deletions and substitutions, our system also shows transpositions. We also provide an error analyzer which gives statistics of each type of error in the document. In addition, it shows errors in context: the words commonly adjacent to each error, and also the adjacent parts of speech (POS). This feature - unique to our system - allows the identification of the context in which errors occur, so they can be rectified easily. The system was evaluated by three MT system developers, who identified useful features and provided feedback which was used to improve the system.