Calculate p-value for a visual test — calc_p

This function calculate the p-value for a visual test.

Usage

calc_p_value(
  detect,
  n_sel,
  n_plot = 20,
  n_sim = 50000,
  dist = "dirichlet",
  alpha = 1,
  cache_env = NULL,
  method = "linear_approx",
  replace_0 = TRUE
)

Arguments

detect: Integer. A vector of boolean values indicating whether the subject detect the actual data plot.
n_sel: Integer. A vector of the number of selections.
n_plot: Integer. Number of plots in the lineup.
n_sim: Integer. Number of simulations draws.
dist: Character. Name of the distribution used for the attractiveness simulation. One of "uniform" and "dirichlet".
alpha: Numeric. A single parameter value used by the Dirichlet distribution.
cache_env: Environment. A provided environment for caching.
method: Character. Method of p-value calculation. "linear_approx" for linear approximation and "simulate" for simulation.
replace_0: Boolean. Whether to treat evaluations with number of selections equal to 0 as selecting all the plots.

Value

A numeric value representing the p-value.

Details

If method == "simulate", the null distribution is simulated by using function sim_dist() and the p-value is the probability of the number of detections greater or equal to the observed value.

If method == "linear_approx", the null distribution is calculated using function exact_dist() and the p-value is the probability of the detections greater or equal to the weighted observed value. The weighted observed detections is calculated by incrementing 1/n_sel for a detection, and 1/n_plot for zero selections. Since the null distribution is discrete, and weighted observed detections is a real value, linear approximation is used for the decimal part.

It is encouraged to provide a cache environment to boost up the performance when this function needs to be reused and method == "simulate". The cache environment will remember the result corresponding to the combinations of n_eval and n_sim.

There are two attractiveness distribution available, one is uniform distribution, another is Dirichlet distribution. Uniform distribution ensures the marginal distribution of the probability of every plot being selected is uniform. When \(\alpha = 1\), Dirichlet distribution ensures the probability of every plot being selected is evenly distributed in a standard \(M - 1\) simplex.

Examples

calc_p_value(c(1, 1, 0), c(1, 1, 2), alpha = 0.5)
#> [1] 0.01761364
calc_p_value(1, 1, 1)
#> [1] 1