Conduct a auto visual inference check with a computer vision model

This function conducts a visual inference check with a computer vision model. The result will be stored in self$check_result.

Usage

AUTO_VI$check(
  null_draws = 100L,
  boot_draws = 100L,
  fitted_model = self$fitted_model,
  keras_model = self$keras_model,
  null_method = self$null_method,
  p_value_type = "quantile",
  data = self$get_data(),
  node_index = self$node_index,
  keep_data = FALSE,
  keep_plot = FALSE,
  extract_feature_from_layer = NULL
)

Arguments

null_draws

Integer. Number of simulation draws for AUTO_VI$null_vss().

boot_draws

Integer. Number of simulation draws for AUTO_VI$boot_vss().

fitted_model

Model. A model object, e.g. lm.

keras_model

Keras model. A trained computer vision model.

null_method

Function. A method to simulate residuals from the null hypothesis distribution. For lm, the recommended method is residual rotation AUTO_VI$rotate_resid().

p_value_type

Character. Either "quantile" or "lineup". See also AUTO_VI$p_value().

data

Data frame. The data used to fit the model. See also AUTO_VI$get_data().

node_index

Integer. An index indicating which node of the output layer contains the visual signal strength. This is particularly useful when the keras model has more than one output nodes.

keep_data

Boolean. Whether to keep the simulated data.

keep_plot

Boolean. Whether to keep the simulated plots.

extract_feature_from_layer

Character/Integer. A layer name or an integer layer index for extracting features from a layer.

Value

Return the object itself.

Examples

keras_model <- try(get_keras_model("vss_phn_32"))
if (!inherits(keras_model, "try-error")) {
  myvi <- auto_vi(lm(dist ~ speed, data = cars), keras_model)

  myvi$check()
  myvi
}
#> ✔ Generate null data.
#> ✔ Generate null plots.
#> Computing auxiliary inputs ■■■■■■■■■■■                       34% | ETA:  2s
#> Computing auxiliary inputs ■■■■■■■■■■■■■■                    43% | ETA:  2s
#> Computing auxiliary inputs ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100% | ETA:  0s
#> ✔ Compute auxilary inputs.
#> Saving images ■■■■■■■■■■■■                      36% | ETA:  2s
#> Saving images ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100% | ETA:  0s
#> ✔ Predict visual signal strength for 100 images.
#> ✔ Generate bootstrapped data.
#> ✔ Generate bootstrapped plots.
#> Warning: shull: complete hull invisible, data scaling error, will retry with some jitter
#> Warning: triangle collapsed!
#> Warning: three points coincide or are collinear!
#> Warning: Problem in area computation (Returns NA)
#> Computing auxiliary inputs ■■■■■■■■■■■■■■                    42% | ETA:  1s
#> Warning: triangle collapsed!
#> Warning: three points coincide or are collinear!
#> Warning: Problem in area computation (Returns NA)
#> Warning: shull: complete hull invisible, data scaling error, will retry with some jitter
#> Warning: shull: complete hull invisible, data scaling error, will retry with some jitter
#> Warning: Problem in area computation (Returns NA)
#> Computing auxiliary inputs ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100% | ETA:  0s
#> ✔ Compute auxilary inputs.
#> Saving images ■■■■■■■■■■                        31% | ETA:  2s
#> Saving images ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100% | ETA:  0s
#> ✔ Predict visual signal strength for 100 images.
#> ✔ Predict visual signal strength for 1 image.
#> 
#> ── <AUTO_VI object>
#> Status:
#>  - Fitted model: lm
#>  - Keras model: (None, 32, 32, 3) + (None, 5) -> (None, 1)
#>     - Output node index: 1
#>  - Result:
#>     - Observed visual signal strength: 3.162 (p-value = 0)
#>     - Null visual signal strength: [100 draws]
#>        - Mean: 1.4
#>        - Quantiles: 
#>           ╔══════════════════════════════════════════╗
#>           ║  25%   50%   75%   80%   90%   95%   99% ║
#>           ║1.006 1.331 1.794 1.843 2.115 2.223 2.611 ║
#>           ╚══════════════════════════════════════════╝
#>     - Bootstrapped visual signal strength: [100 draws]
#>        - Mean: 2.638 (p-value = 0.01)
#>        - Quantiles: 
#>           ╔══════════════════════════════════════════╗
#>           ║  25%   50%   75%   80%   90%   95%   99% ║
#>           ║2.201 2.753 3.183 3.249 3.371 3.485 3.586 ║
#>           ╚══════════════════════════════════════════╝
#>     - Likelihood ratio: 0.603 (boot) / 0.008473 (null) = 71.16