Validating an L2 academic group oral assessment: Insights from a spoken learner corpus

Abstract

This study determines the fine-grained bottom-up linguistic features involved in successful second language (L2) English academic group oral tutorial discussion through the use of a spoken learner corpus comprised of over 20 hours of L2 production. Student performances were graded by teacher-raters using a can-do rating scale which assessed students' ability to participate in a group oral discussion. The performances were transcribed and annotated for linguistic features of group discussion described in the literature such as L2 errors, a range of interactive and interpersonal metadiscourse features, and a range of temporal, prosodic, lexical and syntactic markers (or 'fluencemes') of (dis)fluency. The results of the corpus study suggest that frequent use of metadiscourse is the primary indicator of raters' positive evaluation of student performance in L2 academic tutorial discussion, alongside frequent use of discourse markers, filled pauses and a high speech rate per minute as fluencemes. Most L2 error types as well as syntactic fluencemes did not particularly feature in raters' positive (or negative) evaluations. The detailed cross-sectional data afforded by this corpus analysis serves as quantitative evidence of the linguistic features accompanying each grade awarded across the rating scale, and contributing to the construct validity of the assessment.