This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
make t/op/rand.t fail less often
authorDavid Mitchell <davem@iabyn.com>
Tue, 22 Mar 2011 00:26:33 +0000 (00:26 +0000)
committerDavid Mitchell <davem@iabyn.com>
Tue, 22 Mar 2011 00:26:33 +0000 (00:26 +0000)
commit81b6403efb14bcb2ad83ce521ef54aa7e65a197c
tree98ea07aa6a963b8fabca9e306bdb2410d5cdc5af
parentec529ca2776325d42c8b36b52e472cdc1bf4ca4f
make t/op/rand.t fail less often

test 5 grabs many random numbers, and analyses the distribution
of 0 and 1 bits in the returned values.

It currently fails if the standard deviation is > 3.3.
This means it will fail on roughly 0.1% of runs even for a good system.

Given that an individual smoke report may contain 60+ runs, and given the
number of smokers running, this means that we see false negatives
essentially every day.

Increase the SD threshold to 4.0, which means that it will now fail every
0.006% runs; i.e. go from 1 in 1,000 false positives to one in 17,000
(approx).

(Note that just increasing the number of iterations in the run doesn't
change the 0.1% run failure rate.)
t/op/rand.t