Octane currently does not include a test that would stress 32-bit floating 
point math using Math.fround. This is very important for lots of asm.js 
content, particularly games. The Massive Box2D benchmark is a good example:

https://github.com/kripken/Massive/blob/master/box2d/box2d_f32_2.js

It would be good to add something to fill this gap also in Octane.

What steps will reproduce the problem?
1. build both v8 shell binaries for your system
2. run octane with the shell version - "shell run.js"

What is the expected output? What do you see instead?
Expected (which I see using d8)...
Richards: 12465
DeltaBlue: 14943
Crypto: 16186
RayTrace: 15450
EarleyBoyer: 23748
RegExp: 3490
Splay: 5101
NavierStokes: 18642
PdfJS: 12069
Mandreel: 13183
Gameboy: 13953
CodeLoad: 15532
Box2D: 13448
----
Score (version 8): 12437

Actual result...
run.js:38: Error executing file
load('pdfjs.js');
^

What version of the product are you using? On what operating system?
v1 I think, but it was the trunk version from SVN today.  svn info says 
revision 22.  This is on an Xubuntu 12.10 x64 machine.  The shell version was 
3.16.3.1, but it also happens on 3.15.11.5.  And the shell fails in a 3.14 and 
3.13 version I have...but I don't have d8 for those handy.

Please provide any additional information below.
We've run v8 like this using the "shell" version and it worked fine.  We didn't 
plan to debug anything so the simpler executable seemed like a better choice.

It seems that mandreel benchmark is totally Mandreel-generated code based on a 
c++ one. So does anyone know where is the original c++ source code? Is it 
open-source? Thanks

What steps will reproduce the problem?
1. Run the latest Chrome (35.0.1853.2) / Windows 7 64bit
2. Run http://octane-benchmark.googlecode.com/svn/latest/index.html
3. Typescript score will be "NaN", hence total result NaN

After reloading, the NaN result disappears.

Command Line    "C:\Users\Mathias\AppData\Local\Google\Chrome 
SxS\Application\chrome.exe" --enable-high-resolution-time --flag-switches-begin 
--enable-accelerated-filters --enable-autologin --enable-direct-write 
--enable-experimental-canvas-features 
--enable-experimental-web-platform-features --enable-http2-draft-04 
--js-flags=--harmony --enable-nacl --enable-quic --enable-spdy4a2 
--enable-experimental-extension-apis --flag-switches-end

gbemu.js, in GameBoyContext, creates a "fake" image-data buffer:

    result.data = new Uint8Array(w * h);

This is wrong though, it should be:

    result.data = new Uint8Array(w * h * 4);

In GameBoyCore.prototype.dispatchDraw, 75% of the writes to that array are 
curently out-of-bounds. This doesn't make the benchmark very representative of 
the original emulator.

The bug:
In Code-Load's `cacheBust` function the `replacement` value repeats every so 
often so JS engines hit their cache/optimizations:
https://code.google.com/p/octane-benchmark/source/browse/trunk/code-load.js#1491

To address, you can add `+ seed;` to `replacement += ((salt + i * 7) % 
36).toString(36)`
.. making it `replacement += ((salt + i * 7) % 36).toString(36) + seed;`

You'll notice once you fix the issue with `replacement` in `cacheBust` that 
browsers' scores drop a bit for code-load because they cant leverage their 
internal caches or other opts (it's busting the caches/opts better).

We run octane for our chip validation.
we run using automation, results are generated in log, as our log buffer is too 
low, results are overwritten by different logs. 
Can you guys make results as ui elements.
we are using uiautomator dump in android to dump results UI and parse the 
results.
or provide any other way to parse results.

To reproduce, increase the iteration count in code-load.js (say to 10000/1000):

# d8 run-code-load.js --trace_gc
[5650]        7 ms: Scavenge 2.1 (38.5) -> 2.1 (38.5) MB, 0.7 ms [allocation 
failure].
[5650]        9 ms: Scavenge 2.1 (38.5) -> 2.1 (39.5) MB, 0.7 ms [allocation 
failure].
[5650]       24 ms: Scavenge 3.8 (40.5) -> 3.4 (40.5) MB, 0.3 ms [allocation 
failure].
[5650]       44 ms: Scavenge 5.7 (42.5) -> 4.8 (42.5) MB, 0.6 ms [allocation 
failure].
[5650]       62 ms: Scavenge 7.0 (42.5) -> 6.2 (43.5) MB, 0.6 ms [allocation 
failure].
[5650]       82 ms: Scavenge 8.5 (44.5) -> 7.6 (44.5) MB, 0.9 ms [allocation 
failure].
[5650]      128 ms: Scavenge 13.0 (47.5) -> 10.9 (48.5) MB, 1.0 ms (+ 1.5 ms in 
23 steps since last GC) [allocation failure].
[5650]      168 ms: Scavenge 15.3 (50.5) -> 13.7 (52.5) MB, 1.3 ms (+ 3.8 ms in 
42 steps since last GC) [allocation failure].
[5650]      210 ms: Scavenge 18.3 (52.5) -> 16.6 (54.5) MB, 1.9 ms (+ 3.1 ms in 
44 steps since last GC) [allocation failure].
[5650] Speed up marking because of heap size increase
[5650] Marking speed increased to 3
[5650]      248 ms: Mark-sweep 20.2 (55.5) -> 18.6 (57.5) MB, 5.4 ms (+ 11.1 ms 
in 144 steps since start of marking, biggest step 0.6 ms) [GC interrupt] [GC in 
old space requested].
[5650]      335 ms: Scavenge 29.8 (61.5) -> 25.6 (62.5) MB, 1.9 ms [allocation 
failure].
[5650]      401 ms: Scavenge 33.8 (65.5) -> 30.6 (67.5) MB, 2.5 ms [allocation 
failure].
[5650]      476 ms: Scavenge 39.9 (70.5) -> 36.4 (73.5) MB, 3.4 ms [allocation 
failure].
[5650]      643 ms: Scavenge 57.7 (81.5) -> 49.5 (83.5) MB, 3.8 ms [allocation 
failure].
[5650]      776 ms: Scavenge 66.1 (90.5) -> 59.7 (94.5) MB, 5.1 ms [allocation 
failure].
[5650]      922 ms: Scavenge 78.3 (102.5) -> 71.2 (106.5) MB, 4.6 ms 
[allocation failure].
[5650]     1083 ms: Scavenge 89.0 (114.5) -> 82.2 (118.5) MB, 5.3 ms (+ 12.9 ms 
in 171 steps since last GC) [allocation failure].
[5650]     1247 ms: Scavenge 100.1 (126.5) -> 93.2 (130.5) MB, 5.3 ms (+ 13.8 
ms in 173 steps since last GC) [allocation failure].
[5650]     1416 ms: Scavenge 111.9 (137.5) -> 105.0 (142.5) MB, 5.3 ms (+ 18.5 
ms in 173 steps since last GC) [allocation failure].
[5650]     1580 ms: Scavenge 122.9 (148.5) -> 116.0 (152.5) MB, 5.3 ms (+ 14.7 
ms in 173 steps since last GC) [allocation failure].
[5650] Speed up marking because marker was not keeping up
[5650] Marking speed increased to 3
[5650] Speed up marking because marker was not keeping up
[5650] Marking speed increased to 6
[5650]     1690 ms: Mark-sweep 123.5 (156.5) -> 115.4 (158.5) MB, 24.9 ms (+ 
84.2 ms in 762 steps since start of marking, biggest step 2.8 ms) [GC 
interrupt] [GC in old space requested].
[5650]     1859 ms: Scavenge 137.1 (167.5) -> 128.5 (168.5) MB, 4.0 ms 
[allocation failure].
[5650]     1990 ms: Scavenge 145.0 (174.5) -> 138.6 (180.5) MB, 5.4 ms 
[allocation failure].
[5650]     2137 ms: Scavenge 157.7 (187.5) -> 150.5 (191.5) MB, 5.0 ms 
[allocation failure].
[5650]     2278 ms: Scavenge 168.2 (198.5) -> 161.4 (202.5) MB, 5.3 ms 
[allocation failure].
[5650]     2421 ms: Scavenge 179.6 (210.5) -> 172.6 (214.5) MB, 5.2 ms 
[allocation failure].
[5650]     2563 ms: Scavenge 190.5 (221.5) -> 183.6 (226.5) MB, 5.4 ms 
[allocation failure].
[5650]     2706 ms: Scavenge 201.6 (231.5) -> 194.7 (235.5) MB, 5.4 ms 
[allocation failure].
[5650]     2848 ms: Scavenge 212.6 (243.5) -> 205.7 (247.5) MB, 5.5 ms 
[allocation failure].
[5650]     2991 ms: Scavenge 223.7 (255.5) -> 216.8 (259.5) MB, 5.6 ms 
[allocation failure].
[5650]     3133 ms: Scavenge 234.7 (266.5) -> 227.8 (270.5) MB, 5.6 ms 
[allocation failure].
[5650]     3276 ms: Scavenge 245.8 (276.5) -> 238.8 (281.5) MB, 5.7 ms 
[allocation failure].
[5650]     3418 ms: Scavenge 256.8 (287.5) -> 249.9 (293.5) MB, 5.8 ms 
[allocation failure].
[5650]     3561 ms: Scavenge 267.8 (299.5) -> 260.9 (303.5) MB, 5.9 ms 
[allocation failure].
[5650]     3704 ms: Scavenge 278.8 (311.5) -> 271.9 (315.5) MB, 5.8 ms 
[allocation failure].
[5650]     3846 ms: Scavenge 289.9 (322.5) -> 282.9 (326.5) MB, 5.9 ms 
[allocation failure].
[5650]     3988 ms: Scavenge 300.9 (332.5) -> 294.0 (337.5) MB, 6.0 ms 
[allocation failure].
[5650]     4131 ms: Scavenge 311.9 (344.5) -> 305.0 (349.5) MB, 6.0 ms 
[allocation failure].
[5650]     4274 ms: Scavenge 323.5 (356.5) -> 316.6 (360.5) MB, 6.0 ms 
[allocation failure].
[5650]     4416 ms: Scavenge 334.5 (367.5) -> 327.6 (371.5) MB, 6.1 ms 
[allocation failure].
[5650]     4558 ms: Scavenge 345.6 (378.5) -> 338.6 (382.5) MB, 6.2 ms 
[allocation failure].
[5650]     4701 ms: Scavenge 356.6 (390.5) -> 349.7 (394.5) MB, 6.2 ms 
[allocation failure].
[5650]     4844 ms: Scavenge 367.6 (400.5) -> 360.7 (405.5) MB, 6.4 ms 
[allocation failure].
[5650]     4986 ms: Scavenge 378.7 (412.5) -> 371.7 (416.5) MB, 6.4 ms 
[allocation failure].
--- end of first benchmark ---
[5650]     5197 ms: Scavenge 390.3 (423.5) -> 384.9 (427.5) MB, 7.9 ms 
[allocation failure].
[5650]     5377 ms: Scavenge 400.7 (434.5) -> 395.9 (439.5) MB, 8.9 ms 
[allocation failure].
[5650]     5574 ms: Scavenge 413.0 (446.5) -> 407.8 (451.5) MB, 8.9 ms 
[allocation failure].
[5650]     5765 ms: Scavenge 424.3 (457.5) -> 419.4 (463.5) MB, 9.2 ms 
[allocation failure].
[5650]     5952 ms: Scavenge 436.0 (469.5) -> 430.8 (475.5) MB, 9.2 ms 
[allocation failure].
[5650]     6140 ms: Scavenge 447.5 (480.5) -> 442.4 (486.5) MB, 9.4 ms 
[allocation failure].
[5650]     6330 ms: Scavenge 459.0 (493.5) -> 453.9 (499.5) MB, 9.4 ms 
[allocation failure].
[5650]     6522 ms: Scavenge 471.1 (504.5) -> 466.1 (511.5) MB, 10.2 ms (+ 2.3 
ms in 11 steps since last GC) [allocation failure].
[5650]     6735 ms: Scavenge 482.7 (517.5) -> 477.6 (523.5) MB, 10.3 ms (+ 14.0 
ms in 134 steps since last GC) [allocation failure].
[5650]     6950 ms: Scavenge 494.2 (529.5) -> 489.0 (536.5) MB, 10.5 ms (+ 15.0 
ms in 138 steps since last GC) [allocation failure].
[5650] Speed up marking because marker was not keeping up
[5650] Marking speed increased to 3
[5650]     7193 ms: Scavenge 505.7 (542.5) -> 500.6 (548.5) MB, 10.6 ms (+ 38.9 
ms in 133 steps since last GC) [allocation failure].
[5650]     7437 ms: Scavenge 517.1 (553.5) -> 512.0 (560.5) MB, 10.7 ms (+ 40.9 
ms in 136 steps since last GC) [allocation failure].
[5650]     7684 ms: Scavenge 528.6 (567.5) -> 523.5 (572.5) MB, 10.8 ms (+ 41.5 
ms in 138 steps since last GC) [allocation failure].
[5650]     7938 ms: Scavenge 540.1 (578.5) -> 534.9 (585.5) MB, 11.2 ms (+ 46.8 
ms in 139 steps since last GC) [allocation failure].
[5650]     8192 ms: Scavenge 551.6 (591.5) -> 546.5 (597.5) MB, 11.4 ms (+ 46.1 
ms in 138 steps since last GC) [allocation failure].
[5650] Speed up marking after 1024 steps
[5650] Marking speed increased to 6
[5650]     8540 ms: Mark-sweep 563.0 (603.5) -> 467.5 (603.5) MB, 75.1 ms (+ 
320.7 ms in 1104 steps since start of marking, biggest step 1.6 ms) [GC 
interrupt] [GC in old space requested].
[5650]     8780 ms: Scavenge 488.0 (606.5) -> 482.2 (609.5) MB, 9.4 ms 
[allocation failure].
[5650]     8952 ms: Scavenge 496.4 (611.5) -> 491.8 (614.5) MB, 10.6 ms 
[allocation failure].
[5650]     9169 ms: Scavenge 509.8 (617.5) -> 504.4 (619.5) MB, 10.2 ms 
[allocation failure].
[5650]     9357 ms: Scavenge 520.3 (622.5) -> 515.4 (625.5) MB, 10.9 ms 
[allocation failure].
[5650]     9564 ms: Scavenge 532.5 (628.5) -> 527.4 (630.5) MB, 10.7 ms 
[allocation failure].
[5650]     9763 ms: Scavenge 543.8 (632.5) -> 538.8 (636.5) MB, 11.3 ms 
[allocation failure].
[5650]     9961 ms: Scavenge 555.5 (640.5) -> 550.4 (645.5) MB, 11.0 ms 
[allocation failure].
[5650]    10159 ms: Scavenge 567.0 (648.5) -> 561.9 (653.5) MB, 11.1 ms 
[allocation failure].
[5650]    10357 ms: Scavenge 578.4 (657.5) -> 573.4 (662.5) MB, 11.4 ms 
[allocation failure].
[5650]    10555 ms: Scavenge 589.9 (666.5) -> 584.8 (671.5) MB, 11.3 ms 
[allocation failure].
[5650]    10756 ms: Scavenge 601.4 (675.5) -> 596.3 (680.5) MB, 11.5 ms 
[allocation failure].
[5650]    10956 ms: Scavenge 612.8 (685.5) -> 607.8 (690.5) MB, 11.8 ms 
[allocation failure].
[5650]    11155 ms: Scavenge 624.3 (697.5) -> 619.2 (703.5) MB, 11.9 ms 
[allocation failure].
[5650]    11353 ms: Scavenge 635.8 (708.5) -> 630.7 (714.5) MB, 12.1 ms 
[allocation failure].
[5650]    11553 ms: Scavenge 647.2 (720.5) -> 642.1 (726.5) MB, 12.2 ms 
[allocation failure].
[5650]    11752 ms: Scavenge 658.7 (732.5) -> 653.6 (738.5) MB, 12.4 ms 
[allocation failure].
[5650]    11954 ms: Scavenge 673.2 (747.5) -> 668.1 (753.5) MB, 12.5 ms 
[allocation failure].
[5650]    12155 ms: Scavenge 684.6 (758.5) -> 679.5 (764.5) MB, 12.9 ms 
[allocation failure].
[5650]    12368 ms: Scavenge 696.1 (770.5) -> 691.0 (776.5) MB, 14.0 ms 
[allocation failure].
[5650]    12575 ms: Scavenge 707.5 (782.5) -> 702.4 (787.5) MB, 13.1 ms 
[allocation failure].
[5650]    12781 ms: Scavenge 719.0 (794.5) -> 713.9 (800.5) MB, 13.5 ms 
[allocation failure].
[5650]    12994 ms: Scavenge 730.5 (805.5) -> 725.4 (811.5) MB, 14.1 ms 
[allocation failure].
[5650]    13230 ms: Scavenge 741.9 (817.5) -> 736.8 (823.5) MB, 14.6 ms 
[allocation failure].
[5650]    13442 ms: Scavenge 753.4 (829.5) -> 748.3 (835.5) MB, 13.9 ms 
[allocation failure].
[5650]    13647 ms: Scavenge 764.8 (841.5) -> 759.7 (847.5) MB, 14.0 ms 
[allocation failure].
[5650]    13853 ms: Scavenge 776.3 (852.5) -> 771.2 (858.5) MB, 14.1 ms 
[allocation failure].
[5650]    14062 ms: Scavenge 787.8 (864.5) -> 782.6 (870.5) MB, 14.0 ms 
[allocation failure].
[5650]    14269 ms: Scavenge 799.2 (876.5) -> 794.1 (881.5) MB, 14.5 ms 
[allocation failure].
[5650]    14473 ms: Scavenge 810.7 (887.5) -> 805.6 (893.5) MB, 14.4 ms 
[allocation failure].
[5650]    14680 ms: Scavenge 822.1 (899.5) -> 817.0 (905.5) MB, 14.6 ms 
[allocation failure].
[5650]    14883 ms: Scavenge 833.6 (910.5) -> 828.5 (916.5) MB, 14.7 ms 
[allocation failure].
[5650]    15085 ms: Scavenge 845.0 (923.5) -> 839.9 (929.5) MB, 14.8 ms 
[allocation failure].
CodeLoad: 20689

Looks like garbage accumulates that cannot be freed (not even after benchmark 
finishes, it seems). Expected: no leak.
Admittedly, benchmark score doesn't really suffer from this on v8, as long as 
enough memory is available.

What version of the product are you using? v8 3.28.38, octane 2.0

I found something odd with the Splay benchmark. It create lots and lots of 
objects with properties, `array` and `string`, that are never accessed/modified 
whose `array` value is also fixed. I think it would better reflect real world 
scenarios if the object was created with an `array` property that was not a 
fixed value, in other words be more like `string`, and whose properties were 
modified in some way.

The snippet I'm referring to is:

    return {
      array  : [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ],
      string : 'String for key ' + tag + ' in leaf node'
    };

A basic tweak would be something like:

    return {
      array  : [ 0, 1, 2, 3, 4, 5, 6, 7, 8, depth ],
      string : 'String for key ' + tag + ' in leaf node'
    };

This doesn't cover the modification/access angle but you get the idea.

There is a memory leak when running pdfjs for extended periods of time. The 
problem is that the "canvas_logs" array keeps growing with each run. It should 
probably be reset in the tearDownPdfJS function.

I think a JSIL benchmark would be a good candidate for inclusion since it tends 
to stress various parts of a browser, GC & JS runtime in realistic ways. In 
particular:

http://www.playescapegoat.com/?replayURI=/stages2and3.replay&fastReplay&disableS
ound

Is a fully automatic benchmark based on a recording of real gameplay from a 
moderately large game compiled to JS. Timing data is dumped to the log upon 
completion, and it would probably be possible to run it entirely headless by 
faking out the rendering backend and DOM interaction.

In base.js there is `var performance = performance || {};`. In IE and Safari 
the `var performance` is initialized as `undefined` which overwrites the 
built-in `performance` object and causes them to use the `Date.now` fallback 
which can cost them on the benchmark. A way to avoid this is to do a simple 
`typeof` check. `if (typeof performance == 'undefined') performance = {};`.

What steps will reproduce the problem?
1.nook hd+ (possibly nook HD also) 
2.load chrome
3.run octane


What is the expected output? What do you see instead?
Page crashes every time at zlib bench

What version of the product are you using? On what operating system?
Nook hd+ (customized Android 4.04)

Please provide any additional information below.
Does not crash on Dolphin browser. Untested on Firefox.
I have tried clearing site settings and cache with the same result. 

Although the seed might take values between 0 and 0xffff ffff, the function 
returns values between 0 and 0xfff ffff / 0x1000 0000: 

base.js
l 130: return (seed & 0xfffffff) / 0x10000000;

To fully exploit the range of all possible numbers on 32-bit signed, it should 
do instead:

l 130: return (seed & 0x7fffffff) / 0x80000000;

I guess the original intend was to remove the sign bit and compute the ratio 
with a number immediately greater than the maximum positive value on 32-bit 
signed (0x80000000).

It means that the current random number generator won't produce some values 
between 0xfff ffff (7 'f') / 0x1000 0000 and 1.

I believe it is not such a big deal for current benchmarks and I can't think of 
a case where that would matter, but maybe it could be an issue someday?

What steps will reproduce the problem?
1. Modify line 9373 of gbemu-part2.js so that the setting for "Scale the canvas 
in JS, or let the browser scale the canvas?" is "true".
2. Run the benchmark.

What is the expected output? What do you see instead?
The benchmark page shows "Error" for GB Emulator and the Octane score is marked 
as incomplete. On the console, you get:
Gameboy: HTML5 Canvas support required: this.drawContextOffscreen.getImageData 
is not a functionfile: gbemu-part2.js, line: 5031
Gameboy: Error

What version of the product are you using? On what operating system?
Firefox 29

Please provide any additional information below.
The root cause of the problem is that lines 9753 and 9754 read undefined 
values. With the default settings of gbemu, these values are ignored and 
replaced by default values in lines 5005 and 5006 of gbemu-part2.js. With the 
changed settings, however, the values propagate to line 77 of gbemu-part1.js 
and cause an exception, because NaN is passed to Uint8Array. Even worse, the 
exception is caught in line 5030 of gbemu-part2.js and the error handling code 
calls an undefined function. This call leads to another exception, which is not 
caught and causes the benchmark to crash.

What steps will reproduce the problem?
1. Obtain V8 port to PowerPC https://github.com/andrewlow/v8ppc
2. Compile V8 on PowerPC running big endian
3. Run octane benchmark using V8 on a big endian system

What is the expected output? What do you see instead?

Ideally we'd like the Octane benchmark to apply to all systems capable of 
running JS. Unfortunately with TypedArrays it is possible to create endian 
specific code in JS, this often breaks when switching endian.

What version of the product are you using? On what operating system?

Latest version of octane: 
svn checkout http://octane-benchmark.googlecode.com/svn/trunk/ 
octane-benchmark-read-only

PowerPC in big endian mode / Linux (RHEL).

Please provide any additional information below.

There are only a couple of tests that fail:
mandreel
zlib

The mandreel tests bail with: "Mandreel: Alert called with argument: Check sum 
mismatch: expected ???, actual 6152943" - this is because of the use of typed 
arrays for heap emulation AND the assumption that you can mix and match 32bit 
accesses and 8bit accesses (classic endian breaking code).

The mandreel code is difficult to propose a patch as it's been run through a 
C++ -> JS tool. The correct 'fix' may be to have the tool deal with big endian 
systems. It may be possible to tweak the mixed 8/16 bit accesses with some 
offset adjustments via a manual patch.

The zlib test is more descriptive with it's failure:
"Assertion failed: Typed arrays 2 must be run on a little-endian system:
Error
    at Error (<anonymous>)
    at F (eval at InitializeZlibBenchmark (zlib-data.js:65:37), <anonymous>:1:3255)
    at z (eval at InitializeZlibBenchmark (zlib-data.js:65:37), <anonymous>:1:3312)
    at eval (eval at InitializeZlibBenchmark (zlib-data.js:65:37), <anonymous>:1:6179)
    at eval (native)
    at InitializeZlibBenchmark (zlib-data.js:65:37)
    at Benchmark.runZlib [as run] (zlib.js:37:5)
    at Measure (base.js:307:17)
    at BenchmarkSuite.RunSingleBenchmark (base.js:325:5)
    at RunNextBenchmark (base.js:369:20)
zlib: Assertion: Assertion failed: Typed arrays 2 must be run on a 
little-endian system"

The zlib problem may be easier to address as it's clear the code is checking 
for it. I've not looked into it yet.

Ultimately it would be good to have Octane work across all JS capable platforms 
(both endians) as I believe it to be a good choice for benchmarking JS (and 
specifically V8) performance.

A policy statement from the team maintaining Octane on the support of big 
endian (or not) would also be welcomed.

Emscripten is a LLVM to JS compiler, thus it's CPU-intensive and seems to be a 
good benchmark. Why don't you include some tests from 
https://github.com/kripken/emscripten/tree/master/tests into Octane?

I'm not sure why I get so low score in this test 20035 points while running the 
latest trunk version of Chromium 31.0.1627.0 build 222244, my friend got 24296 
points.

We both have exactly the same browser with default settings. And I did a clean 
install and still get so low score.

Check attachments.

I noticed a bunch of core code in the latest github version is different than 
what is used in Octane trunk. Specifically things like the APU core (A more 
efficient version has landed since your copy of it) and XAudioJS library are 
out of date.

Some unrelated nitpicks about GB-EMU:
I'm also surprised that the Octane modification to make sure no audio is 
generated was done within the audio library, rather than by changing the value 
stored at index 0 of the settings array to false. The GBC core will still 
emulate the APU, but simply not use the audio lib, which is already "dummy 
emulated" by Octane's modifications anyhow. I'm also surprised Octane is 
passing a full dummy canvas object in rather than having the emulator output to 
a fully hidden canvas. Lastly, I noticed that the bench is said to require 
typed arrays, while the official copy of GameBoy-Online explicitly does not 
require typed array support. The code was designed around legacy arrays with 
typed arrays as a "bonus," which means things like 8-bit math are bitwise AND 
guarded manually in the code (Which actually is faster than requiring the typed 
arrays to implicitly perform the overflow wrap, as some browser engines slow 
down a lot on performing the implied overflows).

What steps will reproduce the problem?
1. Go to https://developers.google.com/octane/benchmark
2. Scroll down to 'Based on Box2DWeb, the popular 2D physics engine [...]'
3. Click on Box2DWeb ( http://code.google.com/p/box2dweb/%22 )

What is the expected output? What do you see instead?
  Link contains an extra «"» at the end.

The Regexp benchmark has an out-of-bounds error that results in checking 
whether "undefined" matches some regular expression. The problem is that the 
arrays s45 and s46 have a different length, but they are always used together, 
assuming that their lengths are the same. As a result, line 77 passes 
"undefined" to RegExp.exec(), which results in an array that contains the 
string "undefined" multiple times.

The problem doesn't crash the benchmark but it's doing a nonsense computation. 
To fix the problem, just change the number of variants in one of the two arrays 
s45 and s46, so that they have the same length.

Attached is a patch with a fix. 

What steps will reproduce the problem?
Go to octane-benchmark.googlecode.com/svn/latest/index.html on an android 
browser (2.3.7/Nexus One, 3.2/GalaxyTab10.1)

What is the expected output? What do you see instead?
Expected: page with button "Start Octane"
Displayed: redirection to 
octane-benchmark.googlecode.com/svn/latest/[object%20Object] => 404

What version of the product are you using? On what operating system?
Android devices: 2.3.7/Nexus One, 3.2/GalaxyTab10.1

What steps will reproduce the problem?

---
1. Load octane URL  in IE9.0 (current version of IE at time of writing)

http://octane-benchmark.googlecode.com/svn/latest/index.html

2. Click "Start Octane"
3. Nothing happens


What is the expected output? What do you see instead?

---
Expected is suite should run, using commonly used backwards compatibility shims 
used by web developers for their own applications.
What is seen is nothing happens


What version of the product are you using? On what operating system?

---
IE9.0 fully patched on Windows 7 fully patched.


Please provide any additional information below.

---

Developers write apps today using typed arrays targetting IE9, IE8 and 
downlevel Android clients. The apps run JUST FINE. 

Perhaps they are slow, but there is no reliable way to determine this without a 
benchmark which also runs on those platforms.

Adding a polyfill shim for TypedArrays will turn this into a realistic 
benchmark for those scenarios, and will mirror good development practice 
actually in use at this time.

1964js is a Nintendo 64 emulator that converts MIPS code to Javascript and then 
uses it to emulate games. While the emulator does use WebGL for displaying 
graphics, it does not use it for the CPU. This benchmark would be CPU intensive 
especially later on, when the core is more complete. 

Need to add versioning to the score "Octane V1.0 Score: xxxx"
If there is a change, addition, or update, this will help 
reviewers/benchmarking articles maintain consistency.

You say “The final score is the geometric mean of the single scores, where a 
failed tests is counted as zero”.  Mathematically, a failure and thus 0 
geomean'd with other non-zero scores would be indeterminant. Need to be 
specific how a failed test impacts the score.

r27 added skipBenchmarks argument to BenchmarkSuite.RunSuites, but run.js does 
not supply the new argument.

What steps will reproduce the problem?
1. Go here: http://octane-benchmark.googlecode.com/svn/latest/index.html
2. Click the "start the browser anew" link

What is the expected output? What do you see instead?
EXPECTED: I don't know, some documentation?
ACTUAL: 
https://login.corp.google.com/saml_idp?KeyID=w1n&SAMLRequest=fVLJTsMwEL0j8Q%2BW7
0maAFJlNakKCFGJJWoDB26OO00dvASP08Lf46YgygGOfn5%2By3gm03etyBYcSmtymsYjSsAIu5Kmyel
TdRON6bQ4PZkg16pjs95vzALeekBPwkuDbLjIae8MsxwlMsM1IPOCLWf3dyyLR6xz1lthFSXz65xurOx
Eo9tGK27WLW%2B72rzWTaONgUZb3cm2rWvbUvL8HSvbx5oj9jA36LnxARqlWTQaR1lapRcsO2dn4xdKy
i%2BnS2kODf6LVR9IyG6rqozKx2U1CGzlCtxDYOe0sbZREAur9%2FYlR5TbAK%2B5QqBkhgjOh4BX1mC
vwS3BbaWAp8VdaOl9hyxJdrtd%2FCOT8OT4IJAWw2TZUM4djfT%2F6PzbmhY%2FepPkSKr4%2BrF9kfl
1aZUUH2SmlN1dOeA%2BtPCuDyVurNPc%2F%2B2WxumAyFW0HqisN9iBkGsJK0qS4uD6ezXCwnwC&Rela
yState=https%3A%2F%2Fwww.google.com%2Fa%2Fgoogle.com%2FServiceLogin%3Fservice%3D
ah%26passive%3Dtrue%26continue%3Dhttps%253A%252F%252Fappengine.google.com%252F_a
h%252Fconflogin%253Fcontinue%253Dhttps%253A%252F%252Fdevsite.googleplex.com%252F
octane%252Fbenchmark%26ltmpl%3Dga%26shdf%3DCioLEgZhaG5hbWUaHkdvb2dsZSBEZXZTaXRlI
ENvbnRlbnQgU3RhZ2luZwwSAmFoIhREgbLGiVjn1ZyrDrrP2Ib_y9sRuygBMhQ95d_ziXdfY_4mUtZsq
Ch-cp1J0A

What version of the product are you using? On what operating system?
MacOS 10.7.4, Chrome 21.0.1180.81 beta

Please provide any additional information below.

I suspect you won't be able to reproduce this if you're actually logged in to 
corp.google.com.