Statwing Statistical Testing Overview | Statwing

Statwing Statistical Testing Overview

Statwing selects statistical tests with the goal of making statistical testing intuitive and error-free.

This page describes overarching themes of Statwing’s approach, and the following describe specific decisions for specific tests:

 

Assumptions

Whenever possible, Statwing typically defaults to tests that have fewer assumptions. For example, independent samples t-tests can be calculated in several ways, depending on whether equally sized samples or variances are assumed. Statwing runs the test with the least assumptions.

In addition, Statwing intelligently mitigates violations of the assumptions of statistical tests. For example, t-tests on relatively small samples require normally distributed data to be accurate. Outliers or non-normal distributions create misleading results. Every datapoint of

[1, 2, 3, 3, 4, 4, 5, 5, 5, 6, 6, 7, 7, 8, 9, 10]

is lower than every datapoint in

[11, 12, 13, 13, 14, 14, 15, 15, 15, 16, 16, 17, 17, 18, 19, 2000]

but an independent samples t-test on those groups does not yield a statistically significant difference because the outlier 2000 violates t-test assumptions. Statwing notices the outlier and recommends aranked t-test instead, which does yield a very clear difference between the groups (here’s that example in Statwing—click “Advanced” to see the ranked vs. unranked results).

 

Rank Transformations

Statwing frequently uses the rank transform method for running nonparametric tests when violations of parametric test assumptions are detected. Statwing’s rank transformation replaces values with their rank ordering—for example
[86, 95, 40] is transformed to [2, 3, 1]
—then runs the typical parametric test on the transformed data. Tied values are given the average rank of the tied values, so
[11, 35, 35, 52] becomes [1, 2.5, 2.5, 4].
Most commonly encountered in the difference between Pearson and Spearman correlations, rank-transformed tests are robust to non-normal distributions and outliers, and are conceptually simpler than using slightly more common nonparametric tests like Kruskal-Wallis or Wilcoxon-Mann-Whitney (Conover and Iman, 1981Zimmerman, 2012).

 

Documentation

In the interest of transparency, Statwing publishes all pertinent technical decisions (and their motivations) in this documentation for every Statwing statistical test (see navigation bar to the right). Please contact us if you have questions or feedback about this documentation.