TY - GEN

T1 - Assessment of Amharic-English Literary Translations Quality by Ryabko-Savina Method

AU - Lulu, Yeshewas Getachew

N1 - Conference code: 17

PY - 2025/12/18

Y1 - 2025/12/18

N2 - Translation quality assessment constitutes a central challenge in computational linguistics. This study investigates the application of the Ryabko–Savina (RS) method for evaluating translation accuracy. Traditional evaluation approaches, which rely on stylistic metrics and machine learning, are often influenced by text length and predefined linguistic features. To address these limitations, the RS method offers the possibility to utilize compression algorithms for translation quality analysis. The study examines the unconscious stylistic influence of translators by comparing multiple translations of the same literary works and applies the Ryabko–Savina method to differentiate between original Amharic texts, human-translated Amharic-to-English texts, and machine-translated outputs. Experiments were conducted using six original Amharic novels for authorship style analysis and well-known translated works produced by human and machine translators for translation quality assessment. Based on Cramer’s coefficient across multiple experiments, the Prediction by Partial Matching (PPM) algorithm exhibited the highest stability and was employed for subsequent analyses. The Ryabko–Savina method achieved high classification accuracy, with Cramer’s coefficients of 0.89 for Amharic authorship identification, 0.762 and 1.00 for human-translated texts, 0.91 for machine-translated Amharic-to-English texts, and 0.53 for English–Amharic machine translation tasks. Overall, the findings demonstrate that the Ryabko–Savina method provides a robust, language-independent framework for translation quality assessment, particularly advantageous for low-resource languages such as Amharic.

AB - Translation quality assessment constitutes a central challenge in computational linguistics. This study investigates the application of the Ryabko–Savina (RS) method for evaluating translation accuracy. Traditional evaluation approaches, which rely on stylistic metrics and machine learning, are often influenced by text length and predefined linguistic features. To address these limitations, the RS method offers the possibility to utilize compression algorithms for translation quality analysis. The study examines the unconscious stylistic influence of translators by comparing multiple translations of the same literary works and applies the Ryabko–Savina method to differentiate between original Amharic texts, human-translated Amharic-to-English texts, and machine-translated outputs. Experiments were conducted using six original Amharic novels for authorship style analysis and well-known translated works produced by human and machine translators for translation quality assessment. Based on Cramer’s coefficient across multiple experiments, the Prediction by Partial Matching (PPM) algorithm exhibited the highest stability and was employed for subsequent analyses. The Ryabko–Savina method achieved high classification accuracy, with Cramer’s coefficients of 0.89 for Amharic authorship identification, 0.762 and 1.00 for human-translated texts, 0.91 for machine-translated Amharic-to-English texts, and 0.53 for English–Amharic machine translation tasks. Overall, the findings demonstrate that the Ryabko–Savina method provides a robust, language-independent framework for translation quality assessment, particularly advantageous for low-resource languages such as Amharic.

KW - Measurement

KW - Translation

KW - Accuracy

KW - Data compression

KW - Linguistics

KW - Stability analysis

KW - Quality assessment

KW - Machine translation

KW - Thermal stability

KW - Information theory

KW - Ryabko-Savina-method

KW - Data-compression

KW - Amharic text translation

KW - Linguistic analysis

KW - Translation quality assessment

KW - Information Theory

UR - https://www.scopus.com/pages/publications/105031780784

UR - https://www.mendeley.com/catalogue/fa04d814-d002-3d32-ac9e-e1be7712f7c9/

U2 - 10.1109/apeie66761.2025.11289293

DO - 10.1109/apeie66761.2025.11289293

M3 - Conference contribution

SN - 979-8-3315-5917-5

SP - 1

EP - 6

BT - 2025 IEEE XVII International Scientific and Technical Conference on Actual Problems of Electronic Instrument Engineering (APEIE)

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2025 IEEE XVII International Scientific and Technical Conference on Actual Problems of Electronic Instrument Engineering

Y2 - 14 November 2025 through 16 November 2025

ER -