Potsdam Mind Research Repository

The Potsdam Mind Research Repository (PMR²) provides access to peer-reviewed publications along with data and scripts for analyses and figures reported in them. We refer to these units as "paper packages."  We hope to achieve the following goals:

• Document data and analyses used in our publications in a public forum.
• Invite readers (a) to reproduce our analyses/figures, (b) to try out and possibly publish alternative analyses, or (c) to adopt our scripts for their own data.
• Receive feedback about our scripts, both about necessary corrections of errors and more elegant alternative code.

Here are a few proposals about how we plan to manage and grow the site for the start-up time:

• We will add new scripts to a paper package if they correct our analyses or if they provide new results through an alternative analysis. Indeed, alternative analyses may lead to new publications and we will be glad to include these as new paper packages.
• We invite colleagues to submit paper packages if they fit the theme of our research topics. In general, our expectation is that other groups may follow suit, organizing content according to their own research program.
• Eventually, we may provide comment fields for paper packages, at least for some of them, or open a Blog to facilitate exchange about the paper packages. For now, we ask that such questions are directed to the author's email.

We will explore whether this site can serve as a repository for experimental results that were not published because they did not turn out as expected, assuming that there were no technical or other obvious reasons for the failure of the experiment. Making such data available in the context of research that did yield the desired results may inspire others to take a new look. Perhaps this way we (slightly) reduce  the problem associated with the well-known bias for publications with positive results.

With "R²" in the acronym PMR² and with the term  "paper package" we give credit to "The R Project for Statistical Computing"  (CRAN; http://www.r-project.org/ ). We model this site on the collaborative spirit of CRAN. They have served as our prime source of inspiration of how transparency and progress can be implemented as Open Science. With this site we hope to import this collaborative spirit of openness, sharing, and support into a few of the many areas of mind and brain research.

Moreover, most of the analyses scripts and dataframes are based on R. We make use of a very large number of the opportunities afforded by many contributors to this computing environment.  As representatives of all the contributors to R, we single out the authors of two packages that almost all our scripts rely on. We acknowledge the contribution of Douglas Bates and Martin Maechler, two members of the R Core Group and the authors of the lme4 package (and Matrix --among others). We rely on their work for statistical inferences about our results on a daily basis. For graphics we rely primarily on Hadley Wickham's ggplot2 package which sets new standards for ease of displaying experimental results.

Update (October 2011): We recently became aware of several, some of them long-standing, initiatives in the context of Reproducible Research. It appears that there is a momentum building in favor of this perspective on a new culture in science. Here are some very informative links on this topic:

- NSF (2011). Changing the conduct of science in the information age ( http://www.nsf.gov/pubs/2011/oise11003/)

- AAAS (2011). The digitization of science: Reproducibility and interdisciplinary knowledge transfer (http://www.stanford.edu/~vcs/AAAS2011/ )

Update (July 2014): We are using two services of RStudio for further dissemination of our reproducible research:

- RPubs, based on R Markdown;  commented R scripts for publications (e.g., http://rpubs.com/Reinhold/17313) or tutorial R scripts (e.g., http://rpubs.com/Reinhold/22193)

- Shiny;  e.g., a tutorial for shrinkage in linear mixed models: https://pmr2.shinyapps.io/Shrinkage/

Our research and the construction of this site have been funded by a European Collaborative Research Project (ESF 05_ECRP-FP006, 2006-2009) and a German Research Foundation Research Group (DFG FOR868, since 2008).

{jcomments off}

Matuschek, et al. (2017). Balancing Type I Error and Power in Linear Mixed Models

Abstract: Linear mixed-effects models have increasingly replaced mixed-model analyses of variance for statistical inference in factorial psycholinguistic experiments. The advantages of LMMs over ANOVAs, however, come at a cost: Setting up an LMM is not as straightforward as running an ANOVA. One simple option, when numerically possible, is to fit the full variance-covariance structure of random effects (the maximal model; Barr et al., 2013), presumably to keep Type I error down to the nominal $\alpha$ in the presence of random effects. Although it is true that fitting a model with only random intercepts may lead to higher Type I error, fitting a maximal model also has a cost: it can lead to a significant loss of power. We demonstrate this with simulations and suggest that for typical psychological and psycholinguistic data, models with a random effect structure that is supported by the data have optimal Type I error and power properties.

Also available at arXiv.

To download, use right-click or CTRL-click and then "save as..."

MatuschekPreprint2015.zip	[ ]	[ ]	18307 Kb	16/02/2017 15:02
manuscript.pdf	[ ]	[ ]	1945 Kb	28/07/2016 14:56