Diagnostic English language testing : is it useful to L2 learners in Hong Kong higher diploma?

Abstract

 Having taught the core English language courses, in which I included CEFR grammar content, at the Hong Kong Baptist University Higher Diploma in Creative Film Production programme for the past three years, I have noticed students lacked improvement regarding their grammar proficiency. To investigate possible causes, this case study used diagnostic language assessment to provide insights into the receptive grammar knowledge of Hong Kong higher diploma students. Additionally, CEFR benchmarked grammar items were used to gain insight into the appropriateness of their difficulty level in this study’s context. Furthermore, it sought to inform on how diagnostic language testing was experienced by the participants, and how they perceived the immediate usefulness of reporting diagnostic data. A total of 72 participants completed an online diagnostic grammar test developed based on common grammar errors identified from literature using CEFR benchmarked items. From this sample, 71 completed the diagnostic feedback questionnaire whereas 67 completed the test experience questionnaire. Rasch measurement was used to establish item difficulty and participant ability, revealing student performance on the test, and allowing for the isolation of most and least challenging CEFR grammar items. A consequent distractor analysis provided qualitative insights into how these items performed. The Likert scale data underwent parametric analysis to investigate test experience and immediate usefulness of diagnostic feedback perceptions. The findings suggest that participants positively experienced the diagnostic test, and that all categories of diagnostic feedback were perceived positively with task-based elements being the most positive and self-based elements the least. Results further suggest Hong Kong higher diploma students perform as expected on common grammar errors, and that some CEFR items may be perceived as more or less difficult as what they were benchmarked at. Further research which involves a larger sample with a range of grammatical abilities while also expanding the scope of grammar items tested can generate more useful and generalisable information. Additional further research adapting a exploratory factor analysis and correlation study could give additional insights to the diagnostic feedback perceptions discussed in this dissertation.