Python class to perform AB test analysis
Python class to perform AB test analysis
Overview
This repo contains a Python class to perform an A/B/C… test analysis with proportion-based metrics (including posthoc test). In practice, the class is can be used along with any appropriate RDBMS retrieval tool (e.g. google.cloud.bigquery module for BigQuery) so that, together, they result in an end-to-end analysis process, i.e. from querying the experiment data stored originally in SQL to arriving at the complete analysis results.
The ABTest Class
The class is named ABTest. It is written on top of several well-known libraries (numpy, pandas, scipy, and statsmodels). The class’ main functionality is to consume an experiment results data frame (experiment_df), metric information (nominator_metric, denominator_metric), and meta-information about the platform being experimented (platform) to perform two layers of statistical tests.
First,