A statistic with demonstrated insensitivity to unmeasured bias for 2 × 2 × S tables in observational studies

Are weak associations between a treatment and a binary outcome always sensitive to small unmeasured biases in observational studies? This possibility is often discussed in epidemiology. The familiar Mantel‐Haenszel test for a 2×2×S$$ 2\times 2\times S $$ contingency table exaggerates sensitivity to unmeasured biases when the population odds ratios vary among the S$$ S $$ strata. A statistic built from several components, here from the S$$ S $$ strata, is said to have demonstrated insensitivity to bias if it uses only those components that provide indications of insensitivity to bias. Briefly, such a statistic is a d$$ d $$‐statistic. There are 2S−1$$ {2}^S-1 $$ candidate statistics with S$$ S $$ strata, and a d$$ d $$‐statistic considers them all. To have level α$$ \alpha $$, a test based on a d$$ d $$‐statistic must pay a price for its double use of the data, but as the sample size increases, that price becomes small, while the gain may be large. The price is paid by conditioning on the limited information used to identify components that are insensitive to a bias of specified magnitude, basing the test result on the information that remains after conditioning. In large samples, the d$$ d $$‐statistic achieves the largest possible design sensitivity, so it does not exaggerate sensitivity to unmeasured bias. A simulation verifies that the large sample result has traction in samples of practical size. A study of sunlight as a cause of cataract is used to illustrate issues and methods. Several extensions of the method are discussed. An R package dstat2x2xk implements the method.