Conceptualising and developing evaluative judgement of holistic competencies : a focus on intercultural competence in engineering education

Abstract

Evaluative judgement, generally understood as the capability to assess the quality of one’s own and others’ work, prepares students for their lifelong learning beyond higher education. While its importance has been captured in the literature for some decades, evaluative judgement remains a poorly theorised and understood concept, especially in regard to judging holistic competencies (HC). Therefore, the overarching goal of this thesis is to advance the conceptual understanding of HC evaluative judgement, i.e., to conceptualise evaluative judgement in the context of judging HC. This overarching goal is achieved by synthesising three empirical studies under two phases. Phase One is labelled as the “generalising phase” as it explores the theoretical tenets underlying HC evaluative judgement without distinguishing different HC. This phase includes one study (i.e., Study 1) which engages 14 international experts in three rounds of Delphi inquiries. As a result, Study 1 generates a tentative framework with five dimensions and 20 components to conceptualise HC evaluative judgement. Phase Two is labelled as the “validating phase” as it contextualises, validates and nuances the tentative framework. It focuses on one example of HC – intercultural competence – among engineering students in China. There are two studies under this phase. Study 2 is an instrument development and validation study using a sample of 815 Chinese engineering students. Sequential exploratory and confirmatory factor analyses produce a six-factor 27-item instrument to measure engineering students’ perceived evaluative judgement of intercultural competence. Study 3 explores the process of evaluative judgement by engaging 20 engineering students to evaluate the intercultural competence of themselves and of other engineers in task-based interviews. In reference to the thesis’ overarching goal, Study 2 provides statistical evidence to validate and strengthen the tentative conceptual framework, whereas Study 3 contributes qualitative data to examining whether the framework can be empirically supported by students’ judgement process. Synthesising findings across the three studies, the thesis discusses the possibility to conceptualise HC evaluative judgement as a process through which students negotiate multiple interrelated dimensions to judge their HC development and that of others. A process-oriented framework has been proposed with six validated dimensions (i.e., “Capability to judge”, “Action towards improvement”, “Attitude towards development”, “Knowledge of HC”, “Identity as assessor” and “Awareness of bias”), alongside 21 components which provide specific descriptors under each dimension. Features of this framework (i.e., multidimensional, interrelated, negotiated) have also been discussed. The thesis concludes with its theoretical contributions, empirical insights, and practical implications. Theoretically, the thesis broadens the theoretical toolkit to understand the concept of evaluative judgement, and in particular, the concept of HC evaluative judgement. The process-oriented multidimensional framework is the first one to weave many relevant variables into a coherent whole to illustrate how HC evaluative judgement may be negotiated and developed. Empirically, the thesis offers original insights into Chinese engineering students’ evaluative judgement of intercultural competence, which adds to the limited literature on this topic in engineering education. Practically, the proposed conceptualisation brings new pedagogical insights into how students’ HC evaluative judgement can be developed in the curriculum.