| dc.description.abstract | Within the assessment literature, there is an ongoing conversation pertaining to observed device-type mean score differences for unproctored internet-based tests (UIT), specifically on cognitive assessments. However, it remains unclear whether observed differences are the result of an aspect of the UIT device used to complete the assessments or some other aspect of the testing situation. Arthur, Keiser, and Doverspike' (2018) literature review concluded that mean score differences have been consistently observed in operational settings for cognitive ability UITs. However, these differences have not been obtained in laboratory, non-operational studies. The present study examines one of the explanations that have been advanced to account for this discrepancy, specifically, the self-selection hypothesis. This explanation posits that cognitive ability differences between those who choose to complete high-stakes cognitively demanding assessments on smartphones (a poor decision or choice) versus those who use devices more conducive to effective performance (e.g., laptops, desktops) may account for the observed differences in that in lab studies participants are randomly assigned to devices whereas in operational studies, participants self-select the device to use when taking the assessment. Using a sample of 488 participants, a 2-wave study was conducted in which participants completed a series of cognitive and noncognitive assessments on a device of their choosing at Time 1 and via paper-and-pencil at Time 2. Evidence of a self-selection effect would be indicated by participants who elected to complete the cognitive ability UIT via a high cognitively demanding device (e.g., smartphones or phablets) scoring lower at both Time 1 and Time 2 than participants who elected to complete the cognitive ability UIT via a low cognitively demanding device (e.g., tablets, desktops and laptops). Contrary to what was expected, the obtained pattern of results was not supportive of a device self-selection effect. Implications and limitations are discussed. | |
