Article,
On the probability of a causal inference is robust for internal validity
(2019)cite arxiv:1906.08726Comment: 33 pages , 4 figures and 1 table.
Abstract
The internal validity of observational study is often subject to debate. In
this study, we define the counterfactuals as the unobserved sample and intend
to quantify its relationship with the null hypothesis statistical testing
(NHST). We propose the probability of a causal inference is robust for internal
validity, i.e., the PIV, as a robustness index of causal inference. Formally,
the PIV is the probability of rejecting the null hypothesis again based on both
the observed sample and the counterfactuals, provided the same null hypothesis
has already been rejected based on the observed sample. Under either
frequentist or Bayesian framework, one can bound the PIV of an inference based
on his bounded belief about the counterfactuals, which is often needed when the
unconfoundedness assumption is dubious. The PIV is equivalent to statistical
power when the NHST is thought to be based on both the observed sample and the
counterfactuals. We summarize the process of evaluating internal validity with
the PIV into an eight-step procedure and illustrate it with an empirical
example (i.e., Hong and Raudenbush (2005)).
