README.md (10255B)
1 <!-- 2 3 @license Apache-2.0 4 5 Copyright (c) 2018 The Stdlib Authors. 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18 19 --> 20 21 # incrmgrubbs 22 23 > Moving [Grubbs' test][grubbs-test] for outliers. 24 25 <section class="intro"> 26 27 [Grubbs' test][grubbs-test] (also known as the **maximum normalized residual test** or **extreme studentized deviate test**) is a statistical test used to detect outliers in a univariate dataset assumed to come from a normally distributed population. [Grubbs' test][grubbs-test] is defined for the hypothesis: 28 29 - **H_0**: the dataset does **not** contain outliers. 30 - **H_1**: the dataset contains **exactly** one outlier. 31 32 For a window of size `W`, the [Grubbs' test][grubbs-test] statistic for a two-sided alternative hypothesis is defined as 33 34 <!-- <equation class="equation" label="eq:grubbs_test_statistic" align="center" raw="G = \frac{\max_{i=0,\ldots,W-1} |Y_i - \bar{Y}|}{s}" alt="Grubbs' test statistic."> --> 35 36 <div class="equation" align="center" data-raw-text="G = \frac{\max_{i=0,\ldots,W-1} |Y_i - \bar{Y}|}{s}" data-equation="eq:grubbs_test_statistic"> 37 <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@d157db87026794d6fab579039be09a8916cad4e0/lib/node_modules/@stdlib/stats/incr/mgrubbs/docs/img/equation_grubbs_test_statistic.svg" alt="Grubbs' test statistic."> 38 <br> 39 </div> 40 41 <!-- </equation> --> 42 43 where `s` is the sample standard deviation. The [Grubbs test][grubbs-test] statistic is thus the largest absolute deviation from the sample mean in units of the sample standard deviation. 44 45 The [Grubbs' test][grubbs-test] statistic for the alternative hypothesis that the minimum value is an outlier is defined as 46 47 <!-- <equation class="equation" label="eq:grubbs_test_statistic_min" align="center" raw="G = \frac{\bar{Y} - Y_{\textrm{min}}}{s}" alt="Grubbs' test statistic for testing whether the minimum value is an outlier."> --> 48 49 <div class="equation" align="center" data-raw-text="G = \frac{\bar{Y} - Y_{\textrm{min}}}{s}" data-equation="eq:grubbs_test_statistic_min"> 50 <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@d157db87026794d6fab579039be09a8916cad4e0/lib/node_modules/@stdlib/stats/incr/mgrubbs/docs/img/equation_grubbs_test_statistic_min.svg" alt="Grubbs' test statistic for testing whether the minimum value is an outlier."> 51 <br> 52 </div> 53 54 <!-- </equation> --> 55 56 The [Grubbs' test][grubbs-test] statistic for the alternative hypothesis that the maximum value is an outlier is defined as 57 58 <!-- <equation class="equation" label="eq:grubbs_test_statistic_max" align="center" raw="G = \frac{Y_{\textrm{max}} - \bar{Y}}{s}" alt="Grubbs' test statistic for testing whether the maximum value is an outlier."> --> 59 60 <div class="equation" align="center" data-raw-text="G = \frac{Y_{\textrm{max}} - \bar{Y}}{s}" data-equation="eq:grubbs_test_statistic_max"> 61 <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@d157db87026794d6fab579039be09a8916cad4e0/lib/node_modules/@stdlib/stats/incr/mgrubbs/docs/img/equation_grubbs_test_statistic_max.svg" alt="Grubbs' test statistic for testing whether the maximum value is an outlier."> 62 <br> 63 </div> 64 65 <!-- </equation> --> 66 67 For a two-sided test, the hypothesis that a dataset does **not** contain an outlier is rejected at significance level α if 68 69 <!-- <equation class="equation" label="eq:grubbs_test_two_sided" align="center" raw="G > \frac{W-1}{\sqrt{W}} \sqrt{\frac{t^2_{\alpha/(2W),W-2}}{W - 2 + t^2_{\alpha/(2W),W-2}}}" alt="Two-sided Grubbs' test."> --> 70 71 <div class="equation" align="center" data-raw-text="G > \frac{W-1}{\sqrt{W}} \sqrt{\frac{t^2_{\alpha/(2W),W-2}}{W - 2 + t^2_{\alpha/(2W),W-2}}}" data-equation="eq:grubbs_test_two_sided"> 72 <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@d157db87026794d6fab579039be09a8916cad4e0/lib/node_modules/@stdlib/stats/incr/mgrubbs/docs/img/equation_grubbs_test_two_sided.svg" alt="Two-sided Grubbs' test."> 73 <br> 74 </div> 75 76 <!-- </equation> --> 77 78 where `t` denotes the upper critical value of the _t_-distribution with `W-2` degrees of freedom and a significance level of `α/(2W)`. 79 80 For a one-sided test, the hypothesis that a dataset does **not** contain an outlier is rejected at significance level α if 81 82 <!-- <equation class="equation" label="eq:grubbs_test_one_sided" align="center" raw="G > \frac{W-1}{\sqrt{W}} \sqrt{\frac{t^2_{\alpha/W,W-2}}{W - 2 + t^2_{\alpha/W,W-2}}}" alt="One-sided Grubbs' test."> --> 83 84 <div class="equation" align="center" data-raw-text="G > \frac{W-1}{\sqrt{W}} \sqrt{\frac{t^2_{\alpha/W,W-2}}{W - 2 + t^2_{\alpha/W,W-2}}}" data-equation="eq:grubbs_test_one_sided"> 85 <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@d157db87026794d6fab579039be09a8916cad4e0/lib/node_modules/@stdlib/stats/incr/mgrubbs/docs/img/equation_grubbs_test_one_sided.svg" alt="One-sided Grubbs' test."> 86 <br> 87 </div> 88 89 <!-- </equation> --> 90 91 where `t` denotes the upper critical value of the _t_-distribution with `W-2` degrees of freedom and a significance level of `α/W`. 92 93 </section> 94 95 <!-- /.intro --> 96 97 <section class="usage"> 98 99 ## Usage 100 101 ```javascript 102 var incrmgrubbs = require( '@stdlib/stats/incr/mgrubbs' ); 103 ``` 104 105 #### incrmgrubbs( window\[, options] ) 106 107 Returns an accumulator `function` which incrementally performs [Grubbs' test][grubbs-test] for outliers. The `window` parameter defines the number of values over which to perform [Grubbs' test][grubbs-test]. 108 109 ```javascript 110 var accumulator = incrmgrubbs( 20 ); 111 ``` 112 113 The function accepts the following `options`: 114 115 - **alpha**: significance level. Default: `0.05`. 116 117 - **alternative**: alternative hypothesis. The option may be one of the following values: 118 119 - `'two-sided'`: test whether the minimum or maximum value is an outlier. 120 - `'min'`: test whether the minimum value is an outlier. 121 - `'max'`: test whether the maximum value is an outlier. 122 123 Default: `'two-sided'`. 124 125 #### accumulator( \[x] ) 126 127 If provided an input value `x`, the accumulator function returns updated test results. If not provided an input value `x`, the accumulator function returns the current test results. 128 129 ```javascript 130 var rnorm = require( '@stdlib/random/base/normal' ); 131 132 var accumulator = incrmgrubbs( 3 ); 133 134 var results = accumulator( rnorm( 10.0, 5.0 ) ); 135 // returns null 136 137 results = accumulator( rnorm( 10.0, 5.0 ) ); 138 // returns null 139 140 results = accumulator( rnorm( 10.0, 5.0 ) ); 141 // returns <Object> 142 143 results = accumulator(); 144 // returns <Object> 145 ``` 146 147 The accumulator function returns an `object` having the following fields: 148 149 - **rejected**: boolean indicating whether the null hypothesis should be rejected. 150 - **alpha**: significance level. 151 - **criticalValue**: critical value. 152 - **statistic**: test statistic. 153 - **df**: degrees of freedom. 154 - **mean**: sample mean. 155 - **sd**: corrected sample standard deviation. 156 - **min**: minimum value. 157 - **max**: maximum value. 158 - **alt**: alternative hypothesis. 159 - **method**: method name. 160 - **print**: method for pretty-printing test output. 161 162 The `print` method accepts the following options: 163 164 - **digits**: number of digits after the decimal point. Default: `4`. 165 - **decision**: `boolean` indicating whether to print the test decision. Default: `true`. 166 167 </section> 168 169 <!-- /.usage --> 170 171 <section class="notes"> 172 173 ## Notes 174 175 - [Grubbs' test][grubbs-test] **assumes** that data is normally distributed. Accordingly, one should first **verify** that the data can be _reasonably_ approximated by a normal distribution before applying the [Grubbs' test][grubbs-test]. 176 - The minimum `window` size is `3`. In general, the larger the `window`, the more robust outlier detection will be. However, larger windows entail increased memory consumption. 177 - Until `window` values have been provided, the accumulator returns `null`. 178 - Input values are **not** type checked. If provided `NaN` or a value which, when used in computations, results in `NaN`, the accumulated test statistic is `NaN` for **at least** `W-1` future invocations. If non-numeric inputs are possible, you are advised to type check and handle accordingly **before** passing the value to the accumulator function. 179 180 </section> 181 182 <!-- /.notes --> 183 184 <section class="examples"> 185 186 ## Examples 187 188 <!-- eslint no-undef: "error" --> 189 190 ```javascript 191 var sensorData = require( '@stdlib/datasets/suthaharan-single-hop-sensor-network' ); 192 var incrmgrubbs = require( '@stdlib/stats/incr/mgrubbs' ); 193 194 var data; 195 var opts; 196 var acc; 197 var N; 198 var r; 199 var i; 200 201 // Get a test dataset: 202 data = sensorData(); 203 N = 0; 204 for ( i = 0; i < data.length; i++ ) { 205 if ( data[ i ].mote_id === 1 ) { 206 N += 1; 207 data[ i ] = data[ i ].temperature; 208 } 209 } 210 data.length = N; 211 212 // Create a new accumulator which analyzes the last 5 minutes of data: 213 opts = { 214 'alternative': 'two-sided' 215 }; 216 acc = incrmgrubbs( 60, opts ); 217 218 // Update the accumulator: 219 for ( i = 0; i < data.length; i++ ) { 220 r = acc( data[ i ] ); 221 if ( r && r.rejected ) { 222 console.log( 'Index: %d', i ); 223 console.log( '' ); 224 console.log( r.print() ); 225 } 226 } 227 ``` 228 229 </section> 230 231 <!-- /.examples --> 232 233 <section class="references"> 234 235 * * * 236 237 ## References 238 239 - Grubbs, Frank E. 1950. "Sample Criteria for Testing Outlying Observations." _The Annals of Mathematical Statistics_ 21 (1). The Institute of Mathematical Statistics: 27–58. doi:[10.1214/aoms/1177729885][@grubbs:1950a]. 240 - Grubbs, Frank E. 1969. "Procedures for Detecting Outlying Observations in Samples." _Technometrics_ 11 (1). Taylor & Francis: 1–21. doi:[10.1080/00401706.1969.10490657][@grubbs:1969a]. 241 242 </section> 243 244 <!-- /.references --> 245 246 <section class="links"> 247 248 [grubbs-test]: https://en.wikipedia.org/wiki/Grubbs%27_test_for_outliers 249 250 [@grubbs:1950a]: https://doi.org/10.1214/aoms/1177729885 251 252 [@grubbs:1969a]: https://doi.org/10.1080/00401706.1969.10490657 253 254 </section> 255 256 <!-- /.links -->