Assessment literacy of university English teachers in China : a mixed methods study

Abstract

Teacher assessment literacy (AL), generally understood as teacher competencies regarding the assessment of student learning, has been increasingly recognized as an essential component of teacher professionalism. While recent studies show its general inadequacy across educational settings and geographical contexts, teacher AL is still an underexplored area of research, especially among university English teachers in China who teach the largest group of adult English learners in the world. To address this gap, this mixed methods study explored ‘what’, ‘how’ and ‘why’ issues of teacher AL through both a quantitative survey test with a large sample of university English teachers (N=891) and qualitative case studies with three carefully-selected teachers (Linda, Rosa, and May, pseudonyms) working in three universities of different tiers in China. The quantitative study investigated the existing AL level and impact of teachers’ demographic characteristics on their AL performance by using the University English Teacher Assessment Literacy Questionnaire (UETALQ). After 891 responses were subjected to three-parameter logistic item response theory and confirmatory factor analysis, a one-factor model of 10 items with good fit was obtained. Results reveal a basic level of AL with limited influence from demographic characteristics. The case studies explored in a nuanced way how AL is enacted in practice. Drawing upon semester-long classroom observations, teacher interviews and relevant documents, the findings show distinctly stronger or weaker AL dimensions among the three teachers: May was skillful at handling feedback processes yet unconfident in grading; Rosa innovatively utilized technology in assessments yet struggled with quantifying student performance; May had great strategies of assessing class performance and quizzes, but was committed to preparing students for the tests. Across these cases, commonalities include the multi-dimensionality, situatedness and dynamics of teacher AL in practice, with significant influences from the teachers’ institutional contexts and individual conceptions of assessment. Discussions are centered on valid measures of teacher AL, causes and solutions for a basic AL level, prohibitive and facilitative conditions for teacher AL, teacher feedback literacy, and grading competency. These discussions are then synthesized into a conceptual framework of teacher assessment literacy in practice (TALiP). The TALiP framework connects various mediating factors of AL, reconceptualizes the concept from a static knowledge base to an iterative and dynamic system in which teachers make compromises within competing tensions, and posits teacher learning about assessment as the impetus and their identity construction as assessors as the ultimate goal of teacher AL development. The main significance of the study lies in the theoretical contribution of the TALiP framework to AL scholarship. It also advances our current knowledge of AL with original insights about teacher conceptions of assessment and institutional contexts as salient personal and contextual influential factors, teacher feedback literacy, and teacher grading competency. Implications for policy, professional development, and teacher practice are summarized. Limitations of this study and future research directions are then discussed, followed by the concluding remarks of this doctoral thesis project.