Unlocking Insights into Prolific: Research Implications, Participant Behavior and Motivations

Abstract

Online participant-recruitment platforms, such as Prolific, have become popular platforms to conduct experiments in economics and other social sciences, offering researchers convenient access to diverse participant pools. While Prolific is valued for its cost effectiveness, data quality, and ease of recruiting a diverse set of participants, less is known about how researcher decisions, particularly regarding compensation and study duration, affect participant selection and, in return, research findings and replicability. We study this by running a carefully designed experiment on Prolific, finding strong correlations between participants' hourly reservation wages and their economic and socioeconomic attributes. This suggests that compensations shape study sample, potentially introducing biases at lower compensation rates. Through sensitivity analyses, we provide further guidance on mitigating these effects. Additionally, we study perceptions about general participant behaviors and motivations. Our findings reveal that participants generally approach studies honestly and diligently. Overall, our results offer valuable guidance for improving study design and enhancing the reliability and generalizability of research conducted on Prolific.