################################################################################### Hypothesis Testing ################################################################################### .. note:: * [wiki] `A/B testing `_ * [paper] `Top Challenges from the first Practical Online Controlled Experiments Summit `_ * Section 5: Heterogeniety in Treatmnt Effect * Section 3: Long-term effects * Section 4: Overall evaluation criterion metric * The Example of Computational Advertising Bing: `Counterfactual Reasoning and Learning Systems `_ * `Estimation and Inference of Heterogeneous Treatment Effects using Random Forests `_ * [chrisstucchio.com] `Bayesian Bandits - optimizing click throughs with statistics `_ * `Amazon papers `_ .. seealso:: * [nathanieldake.com] `Bayesian A/B Testing `_ * `Bandit Algorithms `_ * `Misunderstandings between experimentalists and observationalists about causal inference `_ * useless [interviewquery.com] `Statistics & A/B Testing Interview Questions `_ *********************************************************************************** Classical Inference *********************************************************************************** Generic Setup =================================================================================== Likelihood Ratio Test (LRT) ----------------------------------------------------------------------------------- Specific Setup =================================================================================== Mean Difference : Wald Test ----------------------------------------------------------------------------------- Multinomial Parameter Value : Chi-Squared Test ----------------------------------------------------------------------------------- Two Sample Test : Permutation Test ----------------------------------------------------------------------------------- *********************************************************************************** Bayesian Inference ***********************************************************************************