time-to-botec

Benchmark sampling in different programming languages
Log | Files | Refs | README

README.md (11052B)


      1 <!--
      2 
      3 @license Apache-2.0
      4 
      5 Copyright (c) 2020 The Stdlib Authors.
      6 
      7 Licensed under the Apache License, Version 2.0 (the "License");
      8 you may not use this file except in compliance with the License.
      9 You may obtain a copy of the License at
     10 
     11    http://www.apache.org/licenses/LICENSE-2.0
     12 
     13 Unless required by applicable law or agreed to in writing, software
     14 distributed under the License is distributed on an "AS IS" BASIS,
     15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     16 See the License for the specific language governing permissions and
     17 limitations under the License.
     18 
     19 -->
     20 
     21 # stdevch
     22 
     23 > Calculate the [standard deviation][standard-deviation] of a strided array using a one-pass trial mean algorithm.
     24 
     25 <section class="intro">
     26 
     27 The population [standard deviation][standard-deviation] of a finite size population of size `N` is given by
     28 
     29 <!-- <equation class="equation" label="eq:population_standard_deviation" align="center" raw="\sigma = \sqrt{\frac{1}{N} \sum_{i=0}^{N-1} (x_i - \mu)^2}" alt="Equation for the population standard deviation."> -->
     30 
     31 <div class="equation" align="center" data-raw-text="\sigma = \sqrt{\frac{1}{N} \sum_{i=0}^{N-1} (x_i - \mu)^2}" data-equation="eq:population_standard_deviation">
     32     <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@10c5eae557369a7da1da27a5f12a45b56e592834/lib/node_modules/@stdlib/stats/base/stdevch/docs/img/equation_population_standard_deviation.svg" alt="Equation for the population standard deviation.">
     33     <br>
     34 </div>
     35 
     36 <!-- </equation> -->
     37 
     38 where the population mean is given by
     39 
     40 <!-- <equation class="equation" label="eq:population_mean" align="center" raw="\mu = \frac{1}{N} \sum_{i=0}^{N-1} x_i" alt="Equation for the population mean."> -->
     41 
     42 <div class="equation" align="center" data-raw-text="\mu = \frac{1}{N} \sum_{i=0}^{N-1} x_i" data-equation="eq:population_mean">
     43     <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@10c5eae557369a7da1da27a5f12a45b56e592834/lib/node_modules/@stdlib/stats/base/stdevch/docs/img/equation_population_mean.svg" alt="Equation for the population mean.">
     44     <br>
     45 </div>
     46 
     47 <!-- </equation> -->
     48 
     49 Often in the analysis of data, the true population [standard deviation][standard-deviation] is not known _a priori_ and must be estimated from a sample drawn from the population distribution. If one attempts to use the formula for the population [standard deviation][standard-deviation], the result is biased and yields an **uncorrected sample standard deviation**. To compute a **corrected sample standard deviation** for a sample of size `n`,
     50 
     51 <!-- <equation class="equation" label="eq:corrected_sample_standard_deviation" align="center" raw="s = \sqrt{\frac{1}{n-1} \sum_{i=0}^{n-1} (x_i - \bar{x})^2}" alt="Equation for computing a corrected sample standard deviation."> -->
     52 
     53 <div class="equation" align="center" data-raw-text="s = \sqrt{\frac{1}{n-1} \sum_{i=0}^{n-1} (x_i - \bar{x})^2}" data-equation="eq:corrected_sample_standard_deviation">
     54     <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@10c5eae557369a7da1da27a5f12a45b56e592834/lib/node_modules/@stdlib/stats/base/stdevch/docs/img/equation_corrected_sample_standard_deviation.svg" alt="Equation for computing a corrected sample standard deviation.">
     55     <br>
     56 </div>
     57 
     58 <!-- </equation> -->
     59 
     60 where the sample mean is given by
     61 
     62 <!-- <equation class="equation" label="eq:sample_mean" align="center" raw="\bar{x} = \frac{1}{n} \sum_{i=0}^{n-1} x_i" alt="Equation for the sample mean."> -->
     63 
     64 <div class="equation" align="center" data-raw-text="\bar{x} = \frac{1}{n} \sum_{i=0}^{n-1} x_i" data-equation="eq:sample_mean">
     65     <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@10c5eae557369a7da1da27a5f12a45b56e592834/lib/node_modules/@stdlib/stats/base/stdevch/docs/img/equation_sample_mean.svg" alt="Equation for the sample mean.">
     66     <br>
     67 </div>
     68 
     69 <!-- </equation> -->
     70 
     71 The use of the term `n-1` is commonly referred to as Bessel's correction. Note, however, that applying Bessel's correction can increase the mean squared error between the sample standard deviation and population standard deviation. Depending on the characteristics of the population distribution, other correction factors (e.g., `n-1.5`, `n+1`, etc) can yield better estimators.
     72 
     73 </section>
     74 
     75 <!-- /.intro -->
     76 
     77 <section class="usage">
     78 
     79 ## Usage
     80 
     81 ```javascript
     82 var stdevch = require( '@stdlib/stats/base/stdevch' );
     83 ```
     84 
     85 #### stdevch( N, correction, x, stride )
     86 
     87 Computes the [standard deviation][standard-deviation] of a strided array `x` using a one-pass trial mean algorithm.
     88 
     89 ```javascript
     90 var x = [ 1.0, -2.0, 2.0 ];
     91 
     92 var v = stdevch( x.length, 1, x, 1 );
     93 // returns ~2.0817
     94 ```
     95 
     96 The function has the following parameters:
     97 
     98 -   **N**: number of indexed elements.
     99 -   **correction**: degrees of freedom adjustment. Setting this parameter to a value other than `0` has the effect of adjusting the divisor during the calculation of the [standard deviation][standard-deviation] according to `N-c` where `c` corresponds to the provided degrees of freedom adjustment. When computing the [standard deviation][standard-deviation] of a population, setting this parameter to `0` is the standard choice (i.e., the provided array contains data constituting an entire population). When computing the corrected sample [standard deviation][standard-deviation], setting this parameter to `1` is the standard choice (i.e., the provided array contains data sampled from a larger population; this is commonly referred to as Bessel's correction).
    100 -   **x**: input [`Array`][mdn-array] or [`typed array`][mdn-typed-array].
    101 -   **stride**: index increment for `x`.
    102 
    103 The `N` and `stride` parameters determine which elements in `x` are accessed at runtime. For example, to compute the [standard deviation][standard-deviation] of every other element in `x`,
    104 
    105 ```javascript
    106 var floor = require( '@stdlib/math/base/special/floor' );
    107 
    108 var x = [ 1.0, 2.0, 2.0, -7.0, -2.0, 3.0, 4.0, 2.0 ];
    109 var N = floor( x.length / 2 );
    110 
    111 var v = stdevch( N, 1, x, 2 );
    112 // returns 2.5
    113 ```
    114 
    115 Note that indexing is relative to the first index. To introduce an offset, use [`typed array`][mdn-typed-array] views.
    116 
    117 <!-- eslint-disable stdlib/capitalized-comments -->
    118 
    119 ```javascript
    120 var Float64Array = require( '@stdlib/array/float64' );
    121 var floor = require( '@stdlib/math/base/special/floor' );
    122 
    123 var x0 = new Float64Array( [ 2.0, 1.0, 2.0, -2.0, -2.0, 2.0, 3.0, 4.0 ] );
    124 var x1 = new Float64Array( x0.buffer, x0.BYTES_PER_ELEMENT*1 ); // start at 2nd element
    125 
    126 var N = floor( x0.length / 2 );
    127 
    128 var v = stdevch( N, 1, x1, 2 );
    129 // returns 2.5
    130 ```
    131 
    132 #### stdevch.ndarray( N, correction, x, stride, offset )
    133 
    134 Computes the [standard deviation][standard-deviation] of a strided array using a one-pass trial mean algorithm and alternative indexing semantics.
    135 
    136 ```javascript
    137 var x = [ 1.0, -2.0, 2.0 ];
    138 
    139 var v = stdevch.ndarray( x.length, 1, x, 1, 0 );
    140 // returns ~2.0817
    141 ```
    142 
    143 The function has the following additional parameters:
    144 
    145 -   **offset**: starting index for `x`.
    146 
    147 While [`typed array`][mdn-typed-array] views mandate a view offset based on the underlying `buffer`, the `offset` parameter supports indexing semantics based on a starting index. For example, to calculate the [standard deviation][standard-deviation] for every other value in `x` starting from the second value
    148 
    149 ```javascript
    150 var floor = require( '@stdlib/math/base/special/floor' );
    151 
    152 var x = [ 2.0, 1.0, 2.0, -2.0, -2.0, 2.0, 3.0, 4.0 ];
    153 var N = floor( x.length / 2 );
    154 
    155 var v = stdevch.ndarray( N, 1, x, 2, 1 );
    156 // returns 2.5
    157 ```
    158 
    159 </section>
    160 
    161 <!-- /.usage -->
    162 
    163 <section class="notes">
    164 
    165 ## Notes
    166 
    167 -   If `N <= 0`, both functions return `NaN`.
    168 -   If `N - c` is less than or equal to `0` (where `c` corresponds to the provided degrees of freedom adjustment), both functions return `NaN`.
    169 -   The underlying algorithm is a specialized case of Neely's two-pass algorithm. As the standard deviation is invariant with respect to changes in the location parameter, the underlying algorithm uses the first strided array element as a trial mean to shift subsequent data values and thus mitigate catastrophic cancellation. Accordingly, the algorithm's accuracy is best when data is **unordered** (i.e., the data is **not** sorted in either ascending or descending order such that the first value is an "extreme" value).
    170 -   Depending on the environment, the typed versions ([`dstdevch`][@stdlib/stats/base/dstdevch], [`sstdevch`][@stdlib/stats/base/sstdevch], etc.) are likely to be significantly more performant.
    171 
    172 </section>
    173 
    174 <!-- /.notes -->
    175 
    176 <section class="examples">
    177 
    178 ## Examples
    179 
    180 <!-- eslint no-undef: "error" -->
    181 
    182 ```javascript
    183 var randu = require( '@stdlib/random/base/randu' );
    184 var round = require( '@stdlib/math/base/special/round' );
    185 var Float64Array = require( '@stdlib/array/float64' );
    186 var stdevch = require( '@stdlib/stats/base/stdevch' );
    187 
    188 var x;
    189 var i;
    190 
    191 x = new Float64Array( 10 );
    192 for ( i = 0; i < x.length; i++ ) {
    193     x[ i ] = round( (randu()*100.0) - 50.0 );
    194 }
    195 console.log( x );
    196 
    197 var v = stdevch( x.length, 1, x, 1 );
    198 console.log( v );
    199 ```
    200 
    201 </section>
    202 
    203 <!-- /.examples -->
    204 
    205 * * *
    206 
    207 <section class="references">
    208 
    209 ## References
    210 
    211 -   Neely, Peter M. 1966. "Comparison of Several Algorithms for Computation of Means, Standard Deviations and Correlation Coefficients." _Communications of the ACM_ 9 (7). Association for Computing Machinery: 496–99. doi:[10.1145/365719.365958][@neely:1966a].
    212 -   Ling, Robert F. 1974. "Comparison of Several Algorithms for Computing Sample Means and Variances." _Journal of the American Statistical Association_ 69 (348). American Statistical Association, Taylor & Francis, Ltd.: 859–66. doi:[10.2307/2286154][@ling:1974a].
    213 -   Chan, Tony F., Gene H. Golub, and Randall J. LeVeque. 1983. "Algorithms for Computing the Sample Variance: Analysis and Recommendations." _The American Statistician_ 37 (3). American Statistical Association, Taylor & Francis, Ltd.: 242–47. doi:[10.1080/00031305.1983.10483115][@chan:1983a].
    214 -   Schubert, Erich, and Michael Gertz. 2018. "Numerically Stable Parallel Computation of (Co-)Variance." In _Proceedings of the 30th International Conference on Scientific and Statistical Database Management_. New York, NY, USA: Association for Computing Machinery. doi:[10.1145/3221269.3223036][@schubert:2018a].
    215 
    216 </section>
    217 
    218 <!-- /.references -->
    219 
    220 <section class="links">
    221 
    222 [standard-deviation]: https://en.wikipedia.org/wiki/Standard_deviation
    223 
    224 [mdn-array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array
    225 
    226 [mdn-typed-array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray
    227 
    228 [@stdlib/stats/base/dstdevch]: https://www.npmjs.com/package/@stdlib/stats/tree/main/base/dstdevch
    229 
    230 [@stdlib/stats/base/sstdevch]: https://www.npmjs.com/package/@stdlib/stats/tree/main/base/sstdevch
    231 
    232 [@neely:1966a]: https://doi.org/10.1145/365719.365958
    233 
    234 [@ling:1974a]: https://doi.org/10.2307/2286154
    235 
    236 [@chan:1983a]: https://doi.org/10.1080/00031305.1983.10483115
    237 
    238 [@schubert:2018a]: https://doi.org/10.1145/3221269.3223036
    239 
    240 </section>
    241 
    242 <!-- /.links -->