time-to-botec

Benchmark sampling in different programming languages
Log | Files | Refs | README

README.md (9604B)


      1 <!--
      2 
      3 @license Apache-2.0
      4 
      5 Copyright (c) 2020 The Stdlib Authors.
      6 
      7 Licensed under the Apache License, Version 2.0 (the "License");
      8 you may not use this file except in compliance with the License.
      9 You may obtain a copy of the License at
     10 
     11    http://www.apache.org/licenses/LICENSE-2.0
     12 
     13 Unless required by applicable law or agreed to in writing, software
     14 distributed under the License is distributed on an "AS IS" BASIS,
     15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     16 See the License for the specific language governing permissions and
     17 limitations under the License.
     18 
     19 -->
     20 
     21 # nanvariancewd
     22 
     23 > Calculate the [variance][variance] of a strided array ignoring `NaN` values and using Welford's algorithm.
     24 
     25 <section class="intro">
     26 
     27 The population [variance][variance] of a finite size population of size `N` is given by
     28 
     29 <!-- <equation class="equation" label="eq:population_variance" align="center" raw="\sigma^2 = \frac{1}{N} \sum_{i=0}^{N-1} (x_i - \mu)^2" alt="Equation for the population variance."> -->
     30 
     31 <div class="equation" align="center" data-raw-text="\sigma^2 = \frac{1}{N} \sum_{i=0}^{N-1} (x_i - \mu)^2" data-equation="eq:population_variance">
     32     <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@e50971c12e3bc534fe464a6f19ba362bb168c8e4/lib/node_modules/@stdlib/stats/base/nanvariancewd/docs/img/equation_population_variance.svg" alt="Equation for the population variance.">
     33     <br>
     34 </div>
     35 
     36 <!-- </equation> -->
     37 
     38 where the population mean is given by
     39 
     40 <!-- <equation class="equation" label="eq:population_mean" align="center" raw="\mu = \frac{1}{N} \sum_{i=0}^{N-1} x_i" alt="Equation for the population mean."> -->
     41 
     42 <div class="equation" align="center" data-raw-text="\mu = \frac{1}{N} \sum_{i=0}^{N-1} x_i" data-equation="eq:population_mean">
     43     <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@e50971c12e3bc534fe464a6f19ba362bb168c8e4/lib/node_modules/@stdlib/stats/base/nanvariancewd/docs/img/equation_population_mean.svg" alt="Equation for the population mean.">
     44     <br>
     45 </div>
     46 
     47 <!-- </equation> -->
     48 
     49 Often in the analysis of data, the true population [variance][variance] is not known _a priori_ and must be estimated from a sample drawn from the population distribution. If one attempts to use the formula for the population [variance][variance], the result is biased and yields a **biased sample variance**. To compute an **unbiased sample variance** for a sample of size `n`,
     50 
     51 <!-- <equation class="equation" label="eq:unbiased_sample_variance" align="center" raw="s^2 = \frac{1}{n-1} \sum_{i=0}^{n-1} (x_i - \bar{x})^2" alt="Equation for computing an unbiased sample variance."> -->
     52 
     53 <div class="equation" align="center" data-raw-text="s^2 = \frac{1}{n-1} \sum_{i=0}^{n-1} (x_i - \bar{x})^2" data-equation="eq:unbiased_sample_variance">
     54     <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@e50971c12e3bc534fe464a6f19ba362bb168c8e4/lib/node_modules/@stdlib/stats/base/nanvariancewd/docs/img/equation_unbiased_sample_variance.svg" alt="Equation for computing an unbiased sample variance.">
     55     <br>
     56 </div>
     57 
     58 <!-- </equation> -->
     59 
     60 where the sample mean is given by
     61 
     62 <!-- <equation class="equation" label="eq:sample_mean" align="center" raw="\bar{x} = \frac{1}{n} \sum_{i=0}^{n-1} x_i" alt="Equation for the sample mean."> -->
     63 
     64 <div class="equation" align="center" data-raw-text="\bar{x} = \frac{1}{n} \sum_{i=0}^{n-1} x_i" data-equation="eq:sample_mean">
     65     <img src="https://cdn.jsdelivr.net/gh/stdlib-js/stdlib@e50971c12e3bc534fe464a6f19ba362bb168c8e4/lib/node_modules/@stdlib/stats/base/nanvariancewd/docs/img/equation_sample_mean.svg" alt="Equation for the sample mean.">
     66     <br>
     67 </div>
     68 
     69 <!-- </equation> -->
     70 
     71 The use of the term `n-1` is commonly referred to as Bessel's correction. Note, however, that applying Bessel's correction can increase the mean squared error between the sample variance and population variance. Depending on the characteristics of the population distribution, other correction factors (e.g., `n-1.5`, `n+1`, etc) can yield better estimators.
     72 
     73 </section>
     74 
     75 <!-- /.intro -->
     76 
     77 <section class="usage">
     78 
     79 ## Usage
     80 
     81 ```javascript
     82 var nanvariancewd = require( '@stdlib/stats/base/nanvariancewd' );
     83 ```
     84 
     85 #### nanvariancewd( N, correction, x, stride )
     86 
     87 Computes the [variance][variance] of a strided array `x` ignoring `NaN` values and using Welford's algorithm.
     88 
     89 ```javascript
     90 var x = [ 1.0, -2.0, NaN, 2.0 ];
     91 
     92 var v = nanvariancewd( x.length, 1, x, 1 );
     93 // returns ~4.3333
     94 ```
     95 
     96 The function has the following parameters:
     97 
     98 -   **N**: number of indexed elements.
     99 -   **correction**: degrees of freedom adjustment. Setting this parameter to a value other than `0` has the effect of adjusting the divisor during the calculation of the [variance][variance] according to `n-c` where `c` corresponds to the provided degrees of freedom adjustment and `n` corresponds to the number of non-`NaN` indexed elements. When computing the [variance][variance] of a population, setting this parameter to `0` is the standard choice (i.e., the provided array contains data constituting an entire population). When computing the unbiased sample [variance][variance], setting this parameter to `1` is the standard choice (i.e., the provided array contains data sampled from a larger population; this is commonly referred to as Bessel's correction).
    100 -   **x**: input [`Array`][mdn-array] or [`typed array`][mdn-typed-array].
    101 -   **stride**: index increment for `x`.
    102 
    103 The `N` and `stride` parameters determine which elements in `x` are accessed at runtime. For example, to compute the [variance][variance] of every other element in `x`,
    104 
    105 ```javascript
    106 var floor = require( '@stdlib/math/base/special/floor' );
    107 
    108 var x = [ 1.0, 2.0, 2.0, -7.0, -2.0, 3.0, 4.0, 2.0, NaN ];
    109 var N = floor( x.length / 2 );
    110 
    111 var v = nanvariancewd( N, 1, x, 2 );
    112 // returns 6.25
    113 ```
    114 
    115 Note that indexing is relative to the first index. To introduce an offset, use [`typed array`][mdn-typed-array] views.
    116 
    117 <!-- eslint-disable stdlib/capitalized-comments -->
    118 
    119 ```javascript
    120 var Float64Array = require( '@stdlib/array/float64' );
    121 var floor = require( '@stdlib/math/base/special/floor' );
    122 
    123 var x0 = new Float64Array( [ 2.0, 1.0, 2.0, -2.0, -2.0, 2.0, 3.0, 4.0, NaN ] );
    124 var x1 = new Float64Array( x0.buffer, x0.BYTES_PER_ELEMENT*1 ); // start at 2nd element
    125 
    126 var N = floor( x0.length / 2 );
    127 
    128 var v = nanvariancewd( N, 1, x1, 2 );
    129 // returns 6.25
    130 ```
    131 
    132 #### nanvariancewd.ndarray( N, correction, x, stride, offset )
    133 
    134 Computes the [variance][variance] of a strided array ignoring `NaN` values and using Welford's algorithm and alternative indexing semantics.
    135 
    136 ```javascript
    137 var x = [ 1.0, -2.0, NaN, 2.0 ];
    138 
    139 var v = nanvariancewd.ndarray( x.length, 1, x, 1, 0 );
    140 // returns ~4.33333
    141 ```
    142 
    143 The function has the following additional parameters:
    144 
    145 -   **offset**: starting index for `x`.
    146 
    147 While [`typed array`][mdn-typed-array] views mandate a view offset based on the underlying `buffer`, the `offset` parameter supports indexing semantics based on a starting index. For example, to calculate the [variance][variance] for every other value in `x` starting from the second value
    148 
    149 ```javascript
    150 var floor = require( '@stdlib/math/base/special/floor' );
    151 
    152 var x = [ 2.0, 1.0, 2.0, -2.0, -2.0, 2.0, 3.0, 4.0 ];
    153 var N = floor( x.length / 2 );
    154 
    155 var v = nanvariancewd.ndarray( N, 1, x, 2, 1 );
    156 // returns 6.25
    157 ```
    158 
    159 </section>
    160 
    161 <!-- /.usage -->
    162 
    163 <section class="notes">
    164 
    165 ## Notes
    166 
    167 -   If `N <= 0`, both functions return `NaN`.
    168 -   If `n - c` is less than or equal to `0` (where `c` corresponds to the provided degrees of freedom adjustment and `n` corresponds to the number of non-`NaN` indexed elements), both functions return `NaN`.
    169 -   Depending on the environment, the typed versions ([`dnanvariancewd`][@stdlib/stats/base/dnanvariancewd], [`snanvariancewd`][@stdlib/stats/base/snanvariancewd], etc.) are likely to be significantly more performant.
    170 
    171 </section>
    172 
    173 <!-- /.notes -->
    174 
    175 <section class="examples">
    176 
    177 ## Examples
    178 
    179 <!-- eslint no-undef: "error" -->
    180 
    181 ```javascript
    182 var randu = require( '@stdlib/random/base/randu' );
    183 var round = require( '@stdlib/math/base/special/round' );
    184 var Float64Array = require( '@stdlib/array/float64' );
    185 var nanvariancewd = require( '@stdlib/stats/base/nanvariancewd' );
    186 
    187 var x;
    188 var i;
    189 
    190 x = new Float64Array( 10 );
    191 for ( i = 0; i < x.length; i++ ) {
    192     x[ i ] = round( (randu()*100.0) - 50.0 );
    193 }
    194 console.log( x );
    195 
    196 var v = nanvariancewd( x.length, 1, x, 1 );
    197 console.log( v );
    198 ```
    199 
    200 </section>
    201 
    202 <!-- /.examples -->
    203 
    204 * * *
    205 
    206 <section class="references">
    207 
    208 ## References
    209 
    210 -   Welford, B. P. 1962. "Note on a Method for Calculating Corrected Sums of Squares and Products." _Technometrics_ 4 (3). Taylor & Francis: 419–20. doi:[10.1080/00401706.1962.10490022][@welford:1962a].
    211 -   van Reeken, A. J. 1968. "Letters to the Editor: Dealing with Neely's Algorithms." _Communications of the ACM_ 11 (3): 149–50. doi:[10.1145/362929.362961][@vanreeken:1968a].
    212 
    213 </section>
    214 
    215 <!-- /.references -->
    216 
    217 <section class="links">
    218 
    219 [variance]: https://en.wikipedia.org/wiki/Variance
    220 
    221 [mdn-array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array
    222 
    223 [mdn-typed-array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray
    224 
    225 [@stdlib/stats/base/dnanvariancewd]: https://www.npmjs.com/package/@stdlib/stats/tree/main/base/dnanvariancewd
    226 
    227 [@stdlib/stats/base/snanvariancewd]: https://www.npmjs.com/package/@stdlib/stats/tree/main/base/snanvariancewd
    228 
    229 [@welford:1962a]: https://doi.org/10.1080/00401706.1962.10490022
    230 
    231 [@vanreeken:1968a]: https://doi.org/10.1145/362929.362961
    232 
    233 </section>
    234 
    235 <!-- /.links -->