MTurk Participants Have Substantially Lower Evaluative Subjective Well-Being Than Other Survey Participants

MTurk Participants Have Substantially Lower Evaluative Subjective Well-Being Than Other Survey Participants

Arthur Stone, Marta Walentynowicz, Stefan Schneider, Doerte Junghaenel and Cheng K. Wen

Journal paper

Computers in Human Behavior

2019

Amazon's MTurk platform has become a popular site for obtaining relatively inexpensive and convenient adult samples for use in behavioral research. Concerns have been raised about selection issues, because MTurk workers chose to participate in the platform and select the tasks they perform (of many offered to them). Prior studies have documented demographic and psychological differences with national samples. In this paper we studied evaluative subjective well-being (the Cantril Ladder) in an MTurk sample, a national Internet panel sample, and a national telephone survey conducted by Gallup-Sharecare. A surprising finding was that MTurk participants' Ladder scores were substantial lower than the other two samples. Analyses controlling for six demographic differences among the samples only slightly reduced the mean differences. However, patterns of demographic—well-being associations were similar within the samples. To corroborate these results, we conducted a secondary analysis on another three samples, one MTurk sample and two Internet panel samples. The same group differences in Ladder scores were observed. These findings add to the growing literature documenting the characteristics of MTurk samples and we discuss the implications for future research with such samples.