| dc.description.abstract | It is purported that taking a should-do situational judgment test (SJT) engenders a more cognitive/knowledge-based orientation; in contrast to a would-do SJT which engenders a more personality/behavioral-based orientation. Although this effect is widely regarded as received doctrine, a detailed review of the literature indicates there have not been any direct tests of this effect using a single word manipulation with a manipulation check. Therefore, despite previous research on the influence of response instruction sets on the construct-related validity of SJTs, further examination of this issue is warranted.
Consequently, this study examines the SJT "would-do"/"should-do" response instruction effect. The present study accomplishes a number of objectives. First, it examines the recall of the focal words and the effectiveness of an intervention (i.e., increasing the saliency of the focal word) to increase recall, and by inference, processing. Second, it seeks to replicate the posited response instruction effect using this single word manipulation with a manipulation check. Third, it further extends the literature by examining how the construct assessed influences the instruction set effect.
Results indicated that participants failed to accurately recall the focal word (i.e., “would” or “should”) in the response instructions, even with a modified stronger degree of saliency (i.e., bold/underline). Furthermore, the pattern of results currently purported in the literature regarding the effect of response instructions on the construct-related validity of SJTs was not replicated with a single word manipulation. Finally, the construct assessed did not influence the instruction set effect or lack thereof.
However, the findings suggest that asking participants to recall the focal word after the first SJT item provides a priming effect and thus, can serve as an intervention to increase the likelihood that test-takers process the focal word. Furthermore, given the present study’s failure to replicate the instruction set effect with a single word manipulation, the importance that can be placed on the instruction set’s influence on the construct-related validity of SJTs is limited. Finally, a preliminary trend of a repetition effect was found such that the more SJT items a participant was exposed to before responding to a manipulation check, the higher the recall rate for that particular manipulation check. | en |
