Jurnal Internasional Ilmu Komp
Sage Publication : Social Science Computer Review
Cognitive interviewing is a well-established method for evaluating and improving a questionnaire
prior to fielding. However, its present implementation brings with it some challenges, notably in
terms of small sample sizes or the possibility of interviewer effects. In this study, the authors test
web surveys through nonprobability online panels as a supplemental means to implement cognitive
interviewing techniques. The overall goal is to tackle the above-mentioned challenges. The focus in
this article is on methodological features that pave the way for an eventual successful implementation
of category-selection probing in web surveys. The study reports on the results of 1,023
respondents from Germany. In order to identify implementation features that lead to a high number
of meaningful answers, the authors explore the effects of (1) different panels, (2) different
probing variants, and (3) different numbers of preceding probes on answer quality. The overall
results suggest that category-selection probing can indeed be implemented in web surveys. Using
data from two panels—a community panel where members can actively get involved, for example,
by creating their own polls, and a ‘‘conventional’’ panel where answering surveys is the members’
only activity—the authors find that high community involvement does not increase the likelihood
to answer probes or produce longer statements. Testing three probing variants that differ in
wording and provided context, the authors find that presenting the context of the probe (i.e., the
probed item and the respondent’s answer) produces a higher number of meaningful answers.
Finally, the likelihood to answer a probe decreases with the number of preceding probes.
However, the word count of those who eventually answer the probes slightly increases with an
increasing number of probes.
Tidak tersedia versi lain