Validity and Reliability Assessments for Evaluating Oral Hygiene Apps

Abstract

Objectives: There is an increase in the development of ‘Apps’ (applications for smartphones and computers) to promote oral health. However, there is a lack of standardized approaches to the assessment of the validity and reliability of such Apps. This study aimed to assess the validity and reliability of a standardized approach to evaluate an Oral Hygiene App. Methods: As part of a clinical trial, preschool children (282) were asked to perform the task of tooth brushing in their usual way and their performance rated independently on a standardized 13-step assessment. Time spent brushing was also recorded by stopwatch. A clinical oral examination of oral hygiene status was conducted masked of tooth brushing performance using the Silness-Löe plaque index (PI). Primary caregiver (PCG) independently (and prior to other assessments) provided reports of children tooth brushing behaviour. Results: Assessments were completed on all children. Performance in tooth brushing (13-steps) was significantly associated with PCG reports of frequency of tooth brushing behaviour (P<0.001) and time spent tooth brushing (p<0.01). Recoded time spent brushing was significantly associated with PCG reports of time spent tooth brushing (p<0.001). This established concurrent validity. Performance in tooth brushing (13-steps) was significantly correlated with time spent tooth brushing (r 0.72, P<0.001). Performance in tooth brushing (13-steps) was significantly correlated with PI scores (r 0.67, P<0.001). This established construct validity. The Cronbach alpha value of the 13-step performance of tooth brushing assessment was 0.74, confirming internal reliability. The Intraclass correlation coefficient value of repeat assessments among 60 participants (~20% of sample) was 0.83, confirming test-retest reliability. Conclusions: The standardized approach to assessing an App for oral hygiene was valid and reliable. This has implications for evaluating oral hygiene Apps in a standardized way to facilitate the evidence base of Apps for oral health.