benchmark js 的结果是什么意思?
What do the results from benchmark js mean?
我正在使用 Node 的 Benchmark JS 版本,但找不到有关如何读取结果的任何信息。
首先,是否有一个地方详细说明了您可以从 Benchmark JS 中提取的所有数据?
其次,我目前在我的控制台中得到以下结果:
Test x 2,276,094 ops/sec ±0.84% (190 runs sampled)
所有这些信息是什么意思?
测试:我测试的名字,我知道那个
x 2,276,094 ops/sec:我假设这是代码理论上可以在一秒钟内 运行 的平均次数?
±0.84%:不知道
190 运行s sampled: 多少次benchmark运行代码得到结果?
您的问题可能不是重复的,但有趣的是 the answer to it is as GolezTroi points out in 。
如果该问题被删除(极不可能),这里是 John-David Dalton 的答案全文:
I wrote Benchmark.js, which jsPerf uses.
"ops/sec
" stands for operations per second.
That is how many times a test is projected to execute in a second.
A test is repeatedly executed until it reaches the minimum time needed to get a percentage uncertainty for the measurement of less than or equal to 1%
. The number of iterations will vary depending on the resolution of the environment’s timer and how many times a test can execute in the minimum run time. We collect completed test runs for 5
seconds (configurable), or at least 5
runs (also configurable), and then perform statistical analysis on the sample. So, a test may be repeated 100,000
times in 50 ms
(the minimum run time for most environments), and then repeated 100
times more (5
seconds). A larger sample size (in this example, 100
), leads to a smaller margin of error.
We base the decision of which test is faster on more than just ops/sec by also accounting for margin of error. For example, a test with a lower ops/sec but higher margin of error may be statistically indistinguishable from a test with higher ops/sec and lower margin of error.
We used a welch t-test, similar to what SunSpider uses, but switched to an unpaired 2-sample t-test for equal variance (the variance is extremely small) because the welch t-test had problems comparing lower ops/sec and higher ops/sec with small variances which caused the degrees of freedom to be computed as less than 1
. We also add a 5.5%
allowance on tests with similar ops/sec because real world testing showed that identical tests can swing ~5%
from test to re-test. T-tests are used to check that differences between tests are statistically significant.
我正在使用 Node 的 Benchmark JS 版本,但找不到有关如何读取结果的任何信息。
首先,是否有一个地方详细说明了您可以从 Benchmark JS 中提取的所有数据?
其次,我目前在我的控制台中得到以下结果:
Test x 2,276,094 ops/sec ±0.84% (190 runs sampled)
所有这些信息是什么意思?
测试:我测试的名字,我知道那个
x 2,276,094 ops/sec:我假设这是代码理论上可以在一秒钟内 运行 的平均次数?
±0.84%:不知道
190 运行s sampled: 多少次benchmark运行代码得到结果?
您的问题可能不是重复的,但有趣的是 the answer to it is as GolezTroi points out in
如果该问题被删除(极不可能),这里是 John-David Dalton 的答案全文:
I wrote Benchmark.js, which jsPerf uses.
"
ops/sec
" stands for operations per second. That is how many times a test is projected to execute in a second.A test is repeatedly executed until it reaches the minimum time needed to get a percentage uncertainty for the measurement of less than or equal to
1%
. The number of iterations will vary depending on the resolution of the environment’s timer and how many times a test can execute in the minimum run time. We collect completed test runs for5
seconds (configurable), or at least5
runs (also configurable), and then perform statistical analysis on the sample. So, a test may be repeated100,000
times in50 ms
(the minimum run time for most environments), and then repeated100
times more (5
seconds). A larger sample size (in this example,100
), leads to a smaller margin of error.We base the decision of which test is faster on more than just ops/sec by also accounting for margin of error. For example, a test with a lower ops/sec but higher margin of error may be statistically indistinguishable from a test with higher ops/sec and lower margin of error.
We used a welch t-test, similar to what SunSpider uses, but switched to an unpaired 2-sample t-test for equal variance (the variance is extremely small) because the welch t-test had problems comparing lower ops/sec and higher ops/sec with small variances which caused the degrees of freedom to be computed as less than
1
. We also add a5.5%
allowance on tests with similar ops/sec because real world testing showed that identical tests can swing ~5%
from test to re-test. T-tests are used to check that differences between tests are statistically significant.