What I believe today:
- Supposed structure of the data can be used to decide whether you include a random effect. (By analogy, one would use a repeated measures ANOVA for repeated measures designs even if the between-subject variation was minimal.)
- If you want to make a test, then use the log likelihood ratio test (LRT), not MCMC-based methods as none of the MCMC samples will have zero variance.
- A significant p from LRT means you should definitely leave the random effect in.
- A non statistically significant p might mean you can leave it out.
- Though you can’t really trust the loglikelihood ratio to have a chi-square distribution; you’d be better off using numerical methods.