File Download
There are no files associated with this item.
Supplementary
-
Citations:
- Appears in Collections:
Conference Paper: Evaluating the performance of the uncorrected and corrected mean reliability alpha for range restriction in meta-analysis: A Monte Carlo study
Title | Evaluating the performance of the uncorrected and corrected mean reliability alpha for range restriction in meta-analysis: A Monte Carlo study |
---|---|
Authors | |
Issue Date | 2013 |
Publisher | The British Psychological Society (BPS). |
Citation | The 2013 Annual Meeting of British Psychological Society, Harrogate, North Yorkshire, England, 9-11 April 2013 How to Cite? |
Abstract | Objectives: The procedure for conducting a meta-analysis of the reliability alpha has recently been proposed by Rodriguez and Maeda (2006). Since then, a large number of studies have used this procedure (e.g. Warne, 2011), so as to provide a mean reliability estimate in a research domain. Despite its recent use, this estimate is usually smaller than its true value, when the sample (e.g. college, clinical) in each single study is subject to range restriction. This study, therefore, seeks to evaluate the performance of the uncorrected and bias-corrected mean alphas based on a Monte Carlo study.
Design: Six factors were evaluated: sample size, selection ratio, selection procedure, true alpha, and numbers of items and studies.
Methods: Simulated examinees’ scores were generated through a Monte Carlo study – a widely used strategy for evaluating the robustness of a data-analytic method across replications. These examinees would then undergo a selection process; this produced a restricted sample in each singe study. Repeating this process for multiple studies in a research domain, the uncorrected and bias-adjusted mean alphas were estimated. Consequently, their performances were evaluated systematically across the simulation conditions.
Results: The results showed that the uncorrected mean alpha was undesirable, with a maximum bias of 99.7 per cent below its true value. By contrast, the bias-corrected mean alpha was generally accurate, with biases ranging from –9.4 per cent to 7.3 per cent.
Conclusions: The bias-corrected mean alpha was accurate, thereby providing a trustworthy method for reliability evaluation. This will likely change the conceptions of many mean alphas reported in recent meta-analyses. |
Description | Oral Presentations no. 315 Fulltext of the abstract in: http://abstracts.bps.org.uk/index.cfm?&ResultsType=Abstracts&ResultSet_ID=9923&FormDisplayMode=view&frmShowSelected=true&localAction=details |
Persistent Identifier | http://hdl.handle.net/10722/187663 |
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Li, JCH | en_US |
dc.date.accessioned | 2013-08-21T07:09:08Z | - |
dc.date.available | 2013-08-21T07:09:08Z | - |
dc.date.issued | 2013 | en_US |
dc.identifier.citation | The 2013 Annual Meeting of British Psychological Society, Harrogate, North Yorkshire, England, 9-11 April 2013 | en_US |
dc.identifier.uri | http://hdl.handle.net/10722/187663 | - |
dc.description | Oral Presentations no. 315 | - |
dc.description | Fulltext of the abstract in: http://abstracts.bps.org.uk/index.cfm?&ResultsType=Abstracts&ResultSet_ID=9923&FormDisplayMode=view&frmShowSelected=true&localAction=details | - |
dc.description.abstract | Objectives: The procedure for conducting a meta-analysis of the reliability alpha has recently been proposed by Rodriguez and Maeda (2006). Since then, a large number of studies have used this procedure (e.g. Warne, 2011), so as to provide a mean reliability estimate in a research domain. Despite its recent use, this estimate is usually smaller than its true value, when the sample (e.g. college, clinical) in each single study is subject to range restriction. This study, therefore, seeks to evaluate the performance of the uncorrected and bias-corrected mean alphas based on a Monte Carlo study. Design: Six factors were evaluated: sample size, selection ratio, selection procedure, true alpha, and numbers of items and studies. Methods: Simulated examinees’ scores were generated through a Monte Carlo study – a widely used strategy for evaluating the robustness of a data-analytic method across replications. These examinees would then undergo a selection process; this produced a restricted sample in each singe study. Repeating this process for multiple studies in a research domain, the uncorrected and bias-adjusted mean alphas were estimated. Consequently, their performances were evaluated systematically across the simulation conditions. Results: The results showed that the uncorrected mean alpha was undesirable, with a maximum bias of 99.7 per cent below its true value. By contrast, the bias-corrected mean alpha was generally accurate, with biases ranging from –9.4 per cent to 7.3 per cent. Conclusions: The bias-corrected mean alpha was accurate, thereby providing a trustworthy method for reliability evaluation. This will likely change the conceptions of many mean alphas reported in recent meta-analyses. | - |
dc.language | eng | en_US |
dc.publisher | The British Psychological Society (BPS). | - |
dc.relation.ispartof | Annual Meeting of the British Psychological Society | en_US |
dc.title | Evaluating the performance of the uncorrected and corrected mean reliability alpha for range restriction in meta-analysis: A Monte Carlo study | en_US |
dc.type | Conference_Paper | en_US |
dc.identifier.email | Li, JCH: lchjohn@hku.hk | en_US |
dc.identifier.authority | Li, JCH=rp01709 | en_US |
dc.identifier.hkuros | 217908 | en_US |
dc.publisher.place | England | - |