farof / heroprotocoljs Goto Github PK

Right now everything goes in the output directory.

Right now, to use this library one passes in the filename and archive to this library.

Thus the library does two things, extracts AND parses.

I feel additional .parse()-only methods should be created in the event the files and header were already extracted from the archive (perhaps a faster, compiled method) via another process.

I propose two convenience methods:

• parseHeader(buffer) will parse out and return the header object
• parseFile(buffer, int version) will parse a file based on the version supplied.

If there's no objection, I'll start on this.

I'd like to add a simple query tool that returns values of a certain property in a given file across multiple replays. Probably nothing that replaces a real parsing and analysis tool but a quick way to view some data that would be useful to build the reference.

Hi @jnovack, @Farof
From first view I cant find the way to match players from replay to battlenet account. m_toon.m_id is not a bnet account id, but something else. And the only data I can really use is m_name, but it is not unique.
here is sample of user data i can see

     "m_name": "Bazar",
     "m_toon": {
       "m_region": 1,
       "m_programId": "Hero",
       "m_realm": 1,
       "m_id": 4287138
     },
     "m_race": " ",
     "m_color": {
       "m_a": 255,
       "m_r": 255,
       "m_g": 0,
       "m_b": 0
     },
     "m_control": 2,
     "m_teamId": 1,
     "m_handicap": 0,
     "m_observe": 0,
     "m_result": 2,
     "m_workingSetSlotId": 7,
     "m_hero": "Abathur"
   }

Can you advise, where can I find bnet account Id or full battletag, to match replay players with bnet accounts?

Thank you!

The direct port is ugly by JavaScript standards by moments, see about remedying to that with performance in mind.

Grunt could help improve the code quality and prevent mistakes.
#8 does not work on both my Windows and OS X, grunt lint works for a while and then gives me a memory error.

Would be nice to make it work. Low priority for me.

Writing an automated reference builder should be fairly easy using the protocol data structure and events description.

It should be able to track what was added or removed in each build and be backward compatible.

It could either output directly a reference or some intermediate files to be consumed by existing automatic documentation builder.

Hi There,

Is the README relavant? Can't seem to get it to work. Not getting any errors or output when I try and parse a replay. The original Python version does indeed work on the replay.

Cheers,

From #1.

Probably some process.cwd() that could be avoided messing things up.

Works fine when calling directly in Node from the CLI.

Replays from patch 15.6 (build 40087) and 15.7 (build 40322) both use the patch 15.5 (build 39951) protocol even though protocols for those two patches have been released.

Patch build is found in header#m_dataBuildNumand protocol used in header#m_version.m_baseBuild.

Need to figure out how to use that information sensibly.

how to specify outer folder ?
like that
node heroprotocol.js replayFile --details /tmp

need possibility to specify an absolute path if possible
thx

I'm failing on v4.2.6 with this command, whatever it is. I've never seen it before.

[jnovack@gyarados ~/Source/heroprotocoljs]$ node bin/extract.js replays/towers-of-doom.StormReplay /tmp/ -p

/Users/jnovack/Source/heroprotocoljs/heroprotocol.js:137
            [...archive.protocol[decoderMap[archiveFile]](archive.readFile(archiveFile))];
             ^^^

SyntaxError: Unexpected token ...
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:374:25)
    at Object.Module._extensions..js (module.js:417:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Module.require (module.js:354:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/Users/jnovack/Source/heroprotocoljs/bin/extract.js:13:22)
    at Module._compile (module.js:410:26)
    at Object.Module._extensions..js (module.js:417:10)

That's the biggest file of replays and 112 different event types. Probably looks worse than it is.

Identifying what areas would benefit from test coverage and writing them.

Make a pass everywhere and add comments where useful. Try to stick to a format readable by automatic documentation tools?

Rather than using git (which may not be available on every platform), use a postinstall script (in package.json to download the zip and work with extracted files.

Reference:

• https://github.com/electron-userland/electron-prebuilt/blob/master/install.js

Right now the tool overrides existing files. Add option to skip them. Would help and save time if a very long extraction is cancelled or crashes and you restart it.

Trickle exceptions down and handle them properly.

This is not an active bug, the decoders and protocols are pretty much 0 or 1 at the moment and those published work, but it would be nice for debugging purpose?

The current Readme is a mix of the original one and additions I made. It probably needs a bit of a cleanup.

What sections should it keep, add and remove to be both informative and non-cluttered?

https://github.com/Farof/heroprotocoljs/blob/master/lib/protocol32455.js#L400

I intentionally (read: unintentionally, due to my poor coding) corrupted the data and found a bug. This MAY be upstream as well.

The following line does NOT throw a decoder.CorruptedError as one would expect.

    if (typeid === null) throw new decoder.CorruptedError('eventid(' + eventid + ') at ' + decoder.toString());

expected output

Error: eventid(2) at buffer(6/4,[66]=71)
...stack trace...

actual output

/Users/jnovack/Source/heroprotocoljs/index.js:115
            for (let event of archive.protocol[decoderMap[archiveFile]](archive.readFile(archiveFile))) {
                                                                       ^

TypeError: decoder.CorruptedError is not a function
    at _decode_event_stream (/Users/jnovack/Source/heroprotocoljs/lib/protocol32524.js:401:32)
    at next (native)
    at Object.exports.decodeReplayGameEvents (/Users/jnovack/Source/heroprotocoljs/lib/protocol32524.js:425:21)
    at next (native)
    at Object.archive.getFile (/Users/jnovack/Source/heroprotocoljs/bin/storm.js:92:72)
    at Object.<anonymous> (/Users/jnovack/Source/heroprotocoljs/bin/storm.js:107:15)
    at Module._compile (module.js:410:26)
    at Object.Module._extensions..js (module.js:417:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)

m_timeUTC and m_timeLocalOffset are in FILETIME format.

FILETIME to Javascript Date is easy. return new Date(FILETIME / 10000 - 11644473600000)

130990020780514880 becomes Wed Feb 03 2016 14:41:18 GMT-0500 (EST).

$ node
> new Date(130990020780514880 / 10000 - 11644473600000)
Wed Feb 03 2016 14:41:18 GMT-0500 (EST)

Unfortunately, the extract from the heroparser.js yields a strange number because JSON.parse() does not like HUGE numbers.

$ node bin/heroprotocol.js replay.StormReplay -d | grep m_time
 m_timeLocalOffset: -637575432,
 m_timeUTC: -1009418524,

They should either be stored as strings, OR converted to date objects (Microsoft's recommendation).

The initial commit was an almost straightforward port of the original library and departed a bit from it now.

Should the main heroprotocol.js stay as close as possible to the python original in terms of usage and functionalities or expand on it? The parsing part should certainly by made as fast as possible, so that means trimming and optimization on what I currently did.

The current replay.js abstraction is not documented and more a proof of concept and way to keep track of what useful data is known as opposed to the full obscure reference at this point. Though I think it would be great to have people working on their own abstraction and API to access information, I believe there is a place for one in this project, for the moment and until/if a popular one emerges at least.

Any ideas on what a nice and practical abstraction could look like?

Still in the spirit that here is as good a place as any until something better emerges, I think this project needs tools. I already made a bulk replay extraction one. I'd like to make ones that extract all possible values for a particular property (think internal name for mounts, skins, talents, etc…), an automated reference builder and probably a query tool.

See #4 for discussion on the reference.

Or maybe I should immediately make something like "stormreplay", "stormreplay-docs" and "stormreplay-tools" separate projects and link them here? Wouldn't it fragment contribution too much? Maybe not just now but later if the project picks up?

I would like to add code quality and unit testing but I have no real experience with it so any experienced contribution welcomed.

Did I miss anything in this discussion of the project direction?

Right now the extraction tool extract everything. It can take time, especially for the big replay.game.events file. Need to add options to only extract certain files, similar to the heroprotocols.js ones.

As noted in the reference files the _gameLoop value is always null. this is not the case with the original python lib. Its impossible to track the time events occurred without this