Designing speaking assessment questions in a First-Year Undergraduate EAP Course: Testing validity, managing knowledge

Abstract

Emerging alongside Hong Kong’s curriculum reform has been recognition within the ELT profession that highly diverse forms of academic literacies are required of students in the university environment (e.g. Biber, Conrad, Reppen, Byrd and Helt, 2002). One result of this trend has been a broadening of academic literacies that are taught and assessed in entry-level undergraduate EAP courses. A specific example of this is The University of Hong Kong’s Core University English course for first-year undergraduates, which assesses, amongst other skills, a spectrum of competencies for participating in academic discussions. Effective testing of multi-faceted discussion skills can be a challenge, with test question design being constrained by a range of factors imposed by the assessment context. This paper outlines the process through which developers of academic discussion assessment questions at the University of Hong Kong’s Centre for Applied English Studies have attempted to capture knowledge and experience gained through engaging in the development process. It is posited that enhanced rigour in the assessment development cycle can bring benefits that include increased confidence in the validity of assessment questions; a double-loop learning cycle (Mitri, 2003) that can facilitate positive washback into curriculum and assessment development; improved clarity of decision making for assessment developers; and a criteria-based tool that helps both to validate the content of designed test questions (Alderson, Clapham & Wall, 1995; Bachman & Palmer, 1996), and to capture the ongoing refinement of design expertise gained by the assessment developers. References Alderson, J. C., Clapham, C. M., & Wall, D. (1995). Language test construction and evaluation. Cambridge: Cambridge University Press. Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests (Vol. 1). Oxford: Oxford University Press. Biber, D., Conrad, S., Reppen, R., Byrd, P., & Helt, M. (2002). Speaking and writing in the university: A multidimensional comparison. TESOL Quarterly, 36(1), 9-48. Mitri, M. (2003). A knowledge management framework for curriculum assessment. The Journal of Computer Information Systems, 43(4), 15-24.