Trend target: 20, Cluster target: 11
Lineups consist of \(M\) plots (usually 20) which are evaluated by \(K\) individuals.
We can examine visual statistics (plots) and conduct tests using participant plot evaluations.
If most participants select a single plot, we conclude there's a visually significant difference.
Andrew Gelman proposed a less-formalized method of posterior predictive model checking in a [JCGS discussion article](http://www.stat.columbia.edu/~gelman/research/published/p755.pdf) in 2004
We reject \(H_0\) if the calculated p-value is less than 0.05
vinferenceR package calculates p-values accounting for this scenario (using simulation)
V3reference distribution are used for comparison purposes in this study
This model accounts for all panels in a lineup
The marginal distributions of the Multinomial model reduce to Beta-Binomials, each representing one panel of the lineup.
We will mostly work with the beta-binomial/marginal models
Allow the evaluation data to dominate any prior beliefs
We strongly believe all plots are equal
But perhaps some plots are more equal than other plots?
We compare Model 1 to Model 2 using Bayes Factors
V3p-value for each scenario