Publication Bias Analysis in the Meta-Analysis of Psychological Research: An Extension with Empirical and Simulation Results
Over the past 30 years, meta-analysis has become the methodological tool de rigueur for quantitatively summarizing a wide array of literature domains within the social sciences and other broad disciplines (e.g., medicine, education, business, linguistics). In doing so, there is a longstanding concern for publication bias in meta-analysis, for instance, the fact that small or non-significant effect sizes may not reach the published literature, or the fact that certain laboratory settings may produce larger effect sizes than others. The proposed work imports modern-day sophisticated methods typically employed for detecting publication bias in the medical literature (i.e., funnel plots, regression tests for publication bias). The work extends these methods to meta-analysis in psychology via simulation and an empirical meta analysis/reanalysis, both of which first account for study characteristics that might covary with effect sizes, then plotting and analyzing the residual effects (i.e., residuals from WLS random effects regression). This is a meaningful extension of present-day publication bias analyses that will yield more meaningful information regarding (a) the influence of study effect sizes on averaged meta-analytic results and (b) inferences regarding the nature and magnitude of publication bias.