Understanding how ELLs are assessed in CBI: The interplay of cognitive and linguistic demands

Abstract

Content-based instruction (CBI) or content and language integrated learning (CLIL) programmes have become increasingly popular around the globe. These programmes are characterized by using a second/foreign language (L2) (very often English) as the language of instruction in content subjects (e.g., science, history). One of the underlying objectives of CBI is to allow students to simultaneously learn both content and the L2, since it is asserted that content subjects provide authentic communicative contexts for L2 acquisition. Such a two-fold goal, however, has not been adequately addressed in terms of assessment, which is of vital importance considering the issues of fairness and validity when students are assessed of their content knowledge through their L2. This study, therefore, seeks to examine the interplay between cognitive and linguistic demands of CBI assessment in order to understand what students are assessed of, as well as how the demands might influence students’ performance. Using a gridded and progressive framework, 387 questions in the Biology papers in a high-stakes public examination in Hong Kong taken by senior secondary school students (n=69679) were analyzed. It was found that cognitively, questions requiring ‘application’ and ‘analysis’ of knowledge predominated. These questions were represented by a relatively even split of questions demanding no production (e.g. multiple-choice questions) or sentence production (e.g. short questions). Remarkably, these two types of questions revealed differential effects of cognitive demands on performance. For multiple-choice questions, different cognitive demands (such as ‘recall’ vs ‘analysis’) did not influence performance. For questions requiring sentence production, students’ performance in ‘analysis’ questions were significantly better than ‘application’ questions, which were in turn better than ‘recall’ questions. Taken together, linguistic demands seemed to mediate the effect of increasing cognitive demands on students’ performance. Implications will be presented with special reference to scrutinizing and enhancing the validity of CBI assessment.