Code Monkey home page Code Monkey logo

msprt's Introduction

Python mSPRT Package:

This package provides a Python implementation for calculating the Mixture Sequential Probability Ratio Test (mSPRT).

mSPRT is a statistical hypothesis test that can be used to decide if a observed data supports one of two hypotheses, based on a sequence of independent and identically distributed observations.

Main functionalities:

  1. Calculating mixture variance

$$ \tau^2 = \sigma^2 \frac{\Phi(-b)}{\frac{1}{b}\phi(b)-\Phi(-b)} $$

  1. Calculating test statistic for normal distribution

$$ \tilde{\Lambda}_n = \sqrt{\frac{2\sigma^2}{V_n + n\tau^2}}\exp\left(\frac{n^2\tau^2(\bar{Y}_n - \bar{X}_n-\theta_0)^2}{4\sigma^2(2\sigma^2+n\tau^2)}\right). $$

  1. Calculating test statistic for Bernoulli distribution

$$ \tilde{\Lambda}_n = \sqrt{\frac{V_n}{V_n + n\tau^2}}\exp{\left(\frac{n^2\tau^2(\bar{Y}_n - \bar{X}_n-\theta_0)^2}{2V_n(V_n+n\tau^2)}\right)} $$

Installation:

The mSPRT package can be easily installed using pip:

pip install msprt

Pre-requisite

Python >=3.10;<3.13

Dependencies:

The mSPRT package depends on the following Python libraries:

  • Numpy
  • Scipy
  • Matplotlib

These dependencies can also be easily installed using pip:

pip install numpy scipy matplotlib

How to Use:

First, import the mSPRT package:

from msprt import msprt

Then, prepare the two sample lists that you want to compare.

x = [0.1, 0.2, 0.3, 0.5, 0.7, 0.9]
y = [0.2, 0.1, 0.4, 0.6, 0.7, 0.8]

Next, call the msprt object with observation lists, along with the parameters for the mSPRT test, such as the alpha and the theta values (by default it assumes you are using a normal distribution and alpha is set to 0.05).

result = msprt(x=x, y=y, sigma=1.0)

If you want to use a Bernoulli distribution, specify it as such:

result = msprt(x=x, y=y, theta=0.5, distribution='bernoulli')

To plot the results, use the plot method:

result.plot()

For detailed information about each parameter, please refer to the comments in the source code.

Contact:

If you find any problems with the implementation, you can leave the ticket on Github.

mSPRT GitHub Page

License:

This project is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation. See the LICENSE file for more information.

References (real heroes)

  1. Johari, R., Pekelis, L., & Walsh, D. J. (2019). Always Valid Inference: Bringing Sequential Analysis to A/B Testing. arXiv:1512.04922 [math.ST]. Link to the paper
  2. The R and C++ implementations of the paper are available in the GitHub repository maintained by Erik Stenberg: GitHub Repository.

msprt's People

Contributors

ovidijusku avatar

Stargazers

 avatar Timofey Tkachenko avatar

Watchers

 avatar

msprt's Issues

Theta parameter tuning

As I understand theta is a parameter that explains the difference between the mathematical expectations of the two samples in our hypotheses, for example:

$$H_0: \theta = 0; \theta = \mu_1 - \mu_0 = 0$$ $$H_1: \theta \ne 0; \theta = \mu_1 - \mu_0 \ne 0$$

As I understand we can test another hypothesis with different theta:

$$H_0: \theta = \tau; \theta = \mu_1 - \mu_0 = \tau$$ $$H_1: \theta \ne \tau; \theta = \mu_1 - \mu_0 \ne \tau$$

How correct is my understanding?

What sigma do I need to use?

I have run the AB test and I'm logging the outcomes at every step, then I have two samples, control, and treatment, what sigma I should pick in my case (conversion)?

I tried (sample_A.std() + sample_B.std())/2 but maybe you have some reccomendations

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.