Comments (3)
While trying to create a snippet i noticed i was commit 4049a8a. With latest HEAD it's blazing fast as well 🎉
from tink_json.
I assume the original data is confidential, but is there any chance you could make a snippet to produce dummy data that has the same performance issue?
from tink_json.
Btw, it was somehow related to string parsing. This was my snippet:
import sys.io.File;
typedef Activity = {
final ID: Int;
final AccountID: Int;
final GameID: Int;
final TimeStamp: String;
final Bet: Int;
final Win: Int;
final CurrencyID: Int;
final StartBalance: Int;
final EndBalance: Int;
final Bonus: Null<String>;
final RoundId: Int;
final Denomination: Int;
final MathID: Int;
final JsonString: String;
final Rounds: Int;
final QuotaLevel: Int;
final CurrencyQuality: Int;
}
class Main {
public static function main() {
final outp: Array<Activity> = [for (i in 0...25000) {
ID: 0,
AccountID: 0,
GameID: 0,
TimeStamp: "2020-07-17 00:00:00",
Bet: 0,
Win: i,
CurrencyID: 0,
StartBalance: 0,
EndBalance: 0,
Bonus: null,
RoundId: 0,
Denomination: 0,
MathID: 0,
// JsonString: "",
JsonString: "{\"gametable\":[[4,3,3],[2,2,2],[1,1,1],[3,3,7],[2,2,2]],\"reelstrip_id\":1,\"win\":0,\"wins\":[]}",
Rounds: 0,
QuotaLevel: 0,
CurrencyQuality: 0,
}];
trace('writing activities');
File.saveContent('activities1.json', haxe.Json.stringify(outp, null, '\t'));
trace('activities written');
trace('parsing activities');
final astart = haxe.Timer.stamp();
final inp: Array<Activity> = tink.Json.parse(File.getContent('activities1.json'));
final aend = haxe.Timer.stamp();
trace('${inp.length} activities parsed in: ${aend - astart}');
final wins = Lambda.fold(inp, (act, sum) -> sum + act.Win, 0);
trace('wins: $wins');
}
}
Running this resulted in:
npx haxe -lib hxnodejs -lib tink_json -cp src -dce full -D analyzer-optimize -js main.js -main Main && node main.js
src/Main.hx:48: writing activities
src/Main.hx:50: activities written
src/Main.hx:52: parsing activities
src/Main.hx:56: 25000 activities parsed in: 0.29508491200977005
src/Main.hx:58: wins: 312487500
And when using the empty JsonString
:
npx haxe -lib hxnodejs -lib tink_json -cp src -dce full -D analyzer-optimize -js main.js -main Main && node main.js
src/Main.hx:48: writing activities
src/Main.hx:50: activities written
src/Main.hx:52: parsing activities
src/Main.hx:56: 25000 activities parsed in: 6.694707926013507
src/Main.hx:58: wins: 312487500
And it got exponentially worse the more entries i tested with.
from tink_json.
Related Issues (20)
- Writer tracks `__first` in anonymous objects even when there are no optional fields.
- from/to + representation HOT 1
- Field parsing should use some sort of state machine.
- Do something about nulls in writer. HOT 5
- Add validationless mode.
- Support child class / interface implementations
- Allow final class fields to be serialized? HOT 2
- Support guards in @:jsonParse
- haxe 4.2 nightly build bug. HOT 1
- Dynamic parser references and type parameters HOT 2
- Support GADT?
- State machine bug?
- parse() broken for Haxe 4 HOT 2
- Allow open enum abstracts
- GenReader fails when enum abstract is inited with enum abstract values
- `DynamicAccess<T>` does not seem to honor `@:jsonParse` defined on the concrete type substituted for `T`
- Allow parsing final @:structInit classes.
- Roundtrip of an enum doesn't work if one of the enum parameters is a haxe.ds.Option HOT 4
- tink_json cannot stringify cpp.CastCharStar
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tink_json.