Potsdam Mind Research Repository

Reproducible Research

  • Increase font size
  • Default font size
  • Decrease font size
Potsdam Mind Research Repository (PMR2)

The Potsdam Mind Research Repository


The Potsdam Mind Research Repository (PMR2) provides access to peer-reviewed publications along with data and scripts for analyses and figures reported in them. We refer to these units as "paper packages."  We hope to achieve the following goals:

  • Document data and analyses used in our publications in a public forum.
  • Invite readers (a) to reproduce our analyses/figures, (b) to try out and possibly publish alternative analyses, or (c) to adopt our scripts for their own data.
  • Receive feedback about our scripts, both about necessary corrections of errors and more elegant alternative code.

Here are a few proposals about how we plan to manage and grow the site for the start-up time:

  • We will add new scripts to a paper package if they correct our analyses or if they provide new results through an alternative analysis. Indeed, alternative analyses may lead to new publications and we will be glad to include these as new paper packages.
  • We invite colleagues to submit paper packages if they fit the theme of our research topics. In general, our expectation is that other groups may follow suit, organizing content according to their own research program.
  • Eventually, we may provide comment fields for paper packages, at least for some of them, or open a Blog to facilitate exchange about the paper packages. For now, we ask that such questions are directed to the author's email.

We will explore whether this site can serve as a repository for experimental results that were not published because they did not turn out as expected, assuming that there were no technical or other obvious reasons for the failure of the experiment. Making such data available in the context of research that did yield the desired results may inspire others to take a new look. Perhaps this way we (slightly) reduce  the problem associated with the well-known bias for publications with positive results.

With "R2" in the acronym PMR2 and with the term  "paper package" we give credit to "The R Project for Statistical Computing"  (CRAN; http://www.r-project.org/ ). We model this site on the collaborative spirit of CRAN. They have served as our prime source of inspiration of how transparency and progress can be implemented as Open Science. With this site we hope to import this collaborative spirit of openness, sharing, and support into a few of the many areas of mind and brain research.

Moreover, most of the analyses scripts and dataframes are based on R. We make use of a very large number of the opportunities afforded by many contributors to this computing environment.  As representatives of all the contributors to R, we single out the authors of two packages that almost all our scripts rely on. We acknowledge the contribution of Douglas Bates and Martin Maechler, two members of the R Core Group and the authors of the lme4 package (and Matrix --among others). We rely on their work for statistical inferences about our results on a daily basis. For graphics we rely primarily on Hadley Wickham's ggplot2 package which sets new standards for ease of displaying experimental results.

Update (October 2011): We recently became aware of several, some of them long-standing, initiatives in the context of Reproducible Research. It appears that there is a momentum building in favor of this perspective on a new culture in science. Here are some very informative links on this topic:

- NSF (2011). Changing the conduct of science in the information age ( http://www.nsf.gov/pubs/2011/oise11003/)

- AAAS (2011). The digitization of science: Reproducibility and interdisciplinary knowledge transfer (http://www.stanford.edu/~vcs/AAAS2011/ )

Update (July 2014): We are using two services of RStudio for further dissemination of our reproducible research:

- RPubs, based on R Markdown;  commented R scripts for publications (e.g., http://rpubs.com/Reinhold/17313) or tutorial R scripts (e.g., http://rpubs.com/Reinhold/22193)

- Shiny;  e.g., a tutorial for shrinkage in linear mixed models: https://pmr2.shinyapps.io/Shrinkage/

Our research and the construction of this site have been funded by a European Collaborative Research Project (ESF 05_ECRP-FP006, 2006-2009) and a German Research Foundation Research Group (DFG FOR868, since 2008).

Last Updated on Friday, 25 July 2014 10:57

Kliegl, R. (2014). Linear Mixed Models Workshop: Analyses of Experiments with R


Tutorial scripts from linear-mixed-models workshop

The scripts represent the core of seminars and workshops on multivariate statistics for the analyses of experimental data. They have benefited from input by students at the University of Potsdam and participants of workshops at various occasions. Publications serving also as LMM tutorials are filed under "Other Experiments and Analyses" at this website. Feedback and suggestions for improvement are welcome. They are "evolving" along with my understanding of the matter, hopefully, and as new versions of software (mostly lme4, ggplot2, plyr packages in R) become available.

We have also started to use two services of RStudio for dissemination of our reproducible research:

RPubs, based on R Markdown;  commented R scripts for publications (e.g., http://rpubs.com/Reinhold/17313) or tutorial R scripts (e.g.,http://rpubs.com/Reinhold/22193)

Shiny;  e.g., a tutorial for shrinkage in linear mixed models: https://pmr2.shinyapps.io/Shrinkage/

Version history

2014: Update from LMM workshops at the Center for Research on Brain, Language, and Music, McGill University, Montreal (March 2014), at the Institute of Psychology at Chinese Academy of Sciences, Beijing (May 2014), and at the Institute of Linguistics, Academia Sinica, Taipei (August 2014).

2013: Update from LMM workshops at the European Conference on Eye Movements, Lund (August 2013), and at the Institute for Behavior Genetics, University of Colorado, Boulder (August 2013).

2012: Update from an LMM workshop at the MARCS, University of Western Sydney (June 2012) and an Introduction to R at the IPU Advanced Research Training Seminar (ARTS), Stellenbosch (July 2012).

2011: Update from LMM workshop at the Department of Psychology, University of Victoria (April 2011).

2010-Aug-10: Start of Potsdam Mind Research Repository w/ material from workshops at the REMICS winter school, Zakopane (February 2009), at the ERASMUS spring school, Blaubeuren (March 2009), at the Sino-German-Center, Beijing (September 2009), at the Department of Cognitive Science, San José, Costa Rica (March 2010), and at the Institute of Cognitive Science, National Cheng Kung University, Tainan (May 2010).


To download, use right-click or CTRL-click and then "save as..."
Download this file (LMM_TutorialScripts.v4.zip)LMM_TutorialScripts.v4.zip[ ][01]10139 Kb20/08/2014 22:04
Download this file (remef.v0.6.10.R)remef.v0.6.10.R[The remef function for R][99]11 Kb05/02/2015 00:01
Last Updated on Sunday, 24 August 2014 01:06

Hohenstein, Matuschek, & Kliegl (2016, PBR). Linked linear mixed models: A joint analysis of fixation locations and fixation durations in natural reading.


Hohenstein, S., Matuschek, H., & Kliegl, R. (2016). Psychonomic Bulletin & Review.

Abstract: The complexity of eye-movement control during reading allows measurement of many dependent variables, the most prominent ones being fixation durations and their locations in words. In current practice, either variable may serve as dependent variable or covariate for the other in linear mixed models (LMMs) featuring also psycholinguistic covariates of word recognition and sentence comprehension. Rather than analyzing fixation location and duration with separate LMMs, we propose linking the two according to their sequential dependency. Specifically, we include predicted fixation location (estimated in the first LMM from psycholinguistic covariates) and its associated residual fixation location as covariates in the second, fixation-duration LMM. This linked LMM affords a distinction between direct and indirect effects (mediated through fixation location) of psycholinguistic covariates on fixation durations. Results confirm the robustness of distributed processing in the perceptual span. They also offer a resolution of the paradox of the inverted optimal viewing position (IOVP) effect (i.e., longer fixation durations in the center than at the beginning and end of words) although the opposite (i.e., an OVP effect) is predicted from default assumptions of psycholinguistic processing efficiency: The IOVP effect in fixation durations is due to the residual fixation-location covariate, presumably driven primarily by saccadic error, and the OVP effect (at least the left part of it) is uncovered with the predicted fixation-location covariate, capturing the indirect effects of psycholinguistic covariates. We expect that linked LMMs will be useful for the analysis of other dynamically related multiple outcomes, a conundrum of most psychonomic research.

doi: 10.3758/s13423-016-1138-y

Published 2016, in Psychonomic Bulletin & Review (Open access)

To download, use right-click or CTRL-click and then "save as..."
Access this URL (https://link.springer.com/content/pdf/10.3758%2Fs13423-016-1138-y.pdf)Hohenstein_PBR.pdf[manuscript][01]0 Kb15/03/2017 15:56
Access this URL (https://static-content.springer.com/esm/art%3A10.3758%2Fs13423-016-1138-y/MediaObjects/13423_2016_1138_MOESM1_ESM.pdf)Hohenstein_PBR_supplement.pdf[supplement][02]254 Kb15/03/2017 16:01
Download this file (PSCr.rds)PSCr.rds[data][03]1437 Kb15/03/2017 16:03
Download this file (linkedLMM_PSC.Rnw)linkedLMM_PSC.Rnw[tex file with R code][04]82 Kb15/03/2017 16:07
Download this file (references.bib)references.bib[references][05]36 Kb15/03/2017 16:08
Download this file (supplement.zip)supplement.zip[files for supplementary material/simulations][06]370 Kb15/03/2017 16:10
Last Updated on Wednesday, 15 March 2017 15:57

Risse & Kliegl (2014, JEP:HPP): Dissociating Preview Validity and Preview Difficulty in Parafoveal Processing of Word n+1 during Reading


Risse, S., & Kliegl, R. (2014).  Dissociating Preview Validity and Preview Difficulty in Parafoveal Processing of Word n+1 during Reading. Journal of Experimental Psychology: Human Perception and Performance, 40 (2), 653-668.

Abstract. Many studies have shown that previewing the next word n + 1 during reading leads to substantial processing benefit (e.g., shorter word viewing times) when this word is eventually fixated. However, evidence of such preprocessing in fixations on the preceding word n when in fact the information about the preview is acquired is far less consistent. A recent study suggested that such effects may be delayed into fixations on the next word n + 1 (Risse & Kliegl, 2012). To investigate the time course of parafoveal information-acquisition on the control of eye movements during reading, we conducted 2 gaze-contingent display-change experiments and orthogonally manipulated the processing difficulty (i.e., word fre- quency) of an n + 1  preview word and its validity relative to the target word. Preview difficulty did not affect fixation durations on the pretarget word n but on the target word n + 1 . In fact, the delayed preview-difficulty effect was almost of the same size as the preview benefit associated with the n + 1  preview validity. Based on additional results from quantile-regression analyses on the time course of the 2 preview effects, we discuss consequences as to the integration of foveal and parafoveal information and potential implications for computational models of eye guidance in reading.


DOI: 10.1037/a0034997

To download, use right-click or CTRL-click and then "save as..."
Download this file (Risse&Kliegl.JEPHPP.inpress.pdf)Risse&Kliegl.JEPHPP.inpress.pdf[ ][0]466 Kb24/11/2014 20:50
Download this file (RisseKliegl.JEPHPP.2014.zip)RisseKliegl.JEPHPP.2014.zip[ ][1]1061 Kb24/11/2014 20:51
Last Updated on Monday, 24 November 2014 20:55

Hohenstein & Kliegl (2014, JEP:LMC). Semantic preview benefit during reading


Hohenstein, S., & Kliegl., R. (2014). Semantic preview benefit during reading. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 166-190.

Abstract: Word features in parafoveal vision influence eye movements during reading. The question of whether readers extract semantic information from parafoveal words was studied in 3 experiments by using a gaze-contingent display change technique. Subjects read German sentences containing 1 of several preview words that were replaced by a target word during the saccade to the preview (boundary paradigm). In the 1st experiment the preview word was semantically related or unrelated to the target. Fixation durations on the target were shorter for semantically related than unrelated previews, consistent with a semantic preview benefit. In the 2nd experiment, half the sentences were presented following the rules of German spelling (i.e., previews and targets were printed with an initial capital letter), and the other half were presented completely in lowercase. A semantic preview benefit was obtained under both conditions. In the 3rd experiment, we introduced 2 further preview conditions, an identical word and a pronounceable nonword, while also manipulating the text contrast. Whereas the contrast had negligible effects, fixation durations on the target were reliably different for all 4 types of preview. Semantic preview benefits were greater for pretarget fixations closer to the boundary (large preview space) and, although not as consistently, for long pretarget fixation durations (long preview time). The results constrain theoretical proposals about eye movement control in reading.

doi: 10.1037/a0033670

To download, use right-click or CTRL-click and then "save as..."
Download this file (Hohenstein_Kliegl_2014.pdf)Hohenstein_Kliegl_2014.pdf[Article][00]972 Kb22/01/2014 18:20
Download this file (Exp1.R)Exp1.R[script][01]6 Kb19/12/2013 13:36
Download this file (Exp1.rds)Exp1.rds[ ][02]40 Kb19/12/2013 13:47
Download this file (Exp1_skip.rds)Exp1_skip.rds[ ][03]5 Kb19/12/2013 13:46
Download this file (Exp2.R)Exp2.R[script][11]10 Kb19/12/2013 13:46
Download this file (Exp2.rds)Exp2.rds[ ][12]58 Kb19/12/2013 13:45
Download this file (Exp2_skip.rds)Exp2_skip.rds[ ][13]6 Kb19/12/2013 13:47
Download this file (Exp3.R)Exp3.R[script][21]7 Kb19/12/2013 13:47
Download this file (Exp3.rds)Exp3.rds[ ][22]149 Kb19/12/2013 13:47
Download this file (Exp3_skip.rds)Exp3_skip.rds[ ][23]10 Kb19/12/2013 13:48
Download this file (meta_analysis.R)meta_analysis.R[script][31]5 Kb19/12/2013 13:48
Download this file (remef.v0.6.10.R)remef.v0.6.10.R[Additional function][90]11 Kb19/12/2013 14:57
Last Updated on Thursday, 30 April 2015 12:17

Balancing Type I Error and Power in Linear Mixed Models


Balancing Type I Error and Power in Linear Mixed Models

Hannes Matuschek, Reinhold Kliegl, Shravan Vasishth, Harald Baayen, Douglas Bates

Abstract: Linear mixed-effects models have increasingly replaced mixed-model analyses of variance for statistical inference in factorial psycholinguistic experiments. The advantages of LMMs over ANOVAs, however, come at a cost: Setting up an LMM is not as straightforward as running an ANOVA. One simple option, when numerically possible, is to fit the full variance-covariance structure of random effects (the maximal model; Barr et al., 2013), presumably to keep Type I error down to the nominal $\alpha$ in the presence of random effects. Although it is true that fitting a model with only random intercepts may lead to higher Type I error, fitting a maximal model also has a cost: it can lead to a significant loss of power. We demonstrate this with simulations and suggest that for typical psychological and psycholinguistic data, models with a random effect structure that is supported by the data have optimal Type I error and power properties.

Also available at arXiv.

To download, use right-click or CTRL-click and then "save as..."
Download this file (MatuschekPreprint2015.zip)MatuschekPreprint2015.zip[ ][ ]18307 Kb16/02/2017 15:02
Download this file (manuscript.pdf)manuscript.pdf[ ][ ]1945 Kb28/07/2016 14:56
Last Updated on Thursday, 28 July 2016 15:00

Copyright © 2017 Potsdam Mind Research Repository (PMR2). All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.