Psychometric assessment of ICILS test items on Hong Kong and Korean students: a Rasch analysis

Abstract

The IEA International Computer and Information Literacy Study (ICILS) 2013 is the first international comparative study aims to examine the outcome of Grade 8 students’ computer and information literacy (CIL) across 19 countries. The measurement of CIL focuses on students’ ability to use computer within and outside of school in the information age. Students are asked to complete two test modules, out of a total of four, which are conducted exclusively on computers. The present study reports psychometric properties of the test modules based on the results garnered from the Field Trial which was in preparation for the main study. A total sample of 804 students from Hong Kong and South Korea participated in the Field Trial. Rasch-anchored analysis was employed to analyze the data. The items for the four test modules demonstrated adequate psychometric properties – they target the students from Hong Kong and Korea well. However, significant Differential Item Functioning (DIF) was detected in some items from the 4 test modules. Future work on analysis on items reporting significant DIF will be carried out.