Missing the human touch? A computational stylometric analysis of GPT-4 translations of online Chinese literature

Abstract

Existing research suggests that machine translations of literary texts remain unsatisfactory. Such quality assessment often relies on automated metrics and subjective human ratings, with little attention to the stylistic features of machine translation. Current understanding is limited regarding the extent to which AI may transform the literary translation landscape, with implications for other critical domains for translation such as the creative industries more broadly. This pioneering study investigates the stylistic features of AI translations, specifically examining GPT-4's performance against human translations of Chinese online literature. Our computational stylometry analysis reveals that GPT-4 translations closely mirror human translations in lexical, syntactic and content features. In addition to showing the relevance of stylometry for analysing the features of AI translation, the study provides critical insights into the implications of AI for literary translation in the posthuman tradition, where the line between machine and human translation becomes increasingly blurry.