advantageous / boon Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
Ever seen issues with stackOverflow error and DateTime in json parsing?
Boon/QBit fail in a horrible way if you use a JODA DateTime.
Exception in thread "WebSocketProtocolEncoder-4" java.lang.StackOverflowError
at io.advantageous.boon.json.serializers.impl.FieldSerializerUseAnnotationsImpl.serializeField(FieldSerializerUseAnnotationsImpl.java:118)
at io.advantageous.boon.json.serializers.impl.JsonSerializerImpl.serializeField(JsonSerializerImpl.java:181)
at io.advantageous.boon.json.serializers.impl.InstanceSerializerImpl.serializeSubtypeInstance(InstanceSerializerImpl.java:84)
at io.advantageous.boon.json.serializers.impl.JsonSerializerImpl.serializeSubtypeInstance(JsonSerializerImpl.java:240)
at io.advantageous.boon.json.serializers.impl.FieldSerializerUseAnnotationsImpl.serializeField(FieldSerializerUseAnnotationsImpl.java:374)
at io.advantageous.boon.json.serializers.impl.JsonSerializerImpl.serializeField(JsonSerializerImpl.java:181)
at io.advantageous.boon.json.serializers.impl.InstanceSerializerImpl.serializeSubtypeInstance(InstanceSerializerImpl.java:84)
Conversions not handling boolean Value correctly
does not call toBoolean on Value
Hi,
It appears that the source/target version is not specified in the pom.xml file, and a dependency on Java 8 byte code version is introduced by the machine that creates the jar files (that find their way to Maven Central).
This caused some issues for me at runtime, and thought you might want to know that your library might be more useful to others in Maven Central if you specifically target java 7 (or 6?) in the pom file.
Hi
@Override
public TypeType type() {
return TypeType.NUMBER;
}
Is this intentional ? NumberValue reads in a Float/Double.
However, the type will always return NUMBER instead of field type
.
TLDR: Json parsing is failing if long values has same digit count with Long.MAX_VALUE
.
Details:
I'm using gatling for performance testing, and it's using boon under the hood. When I tried to parse a specific json I got an interesting exception:
io.advantageous.boon.core.Exceptions$SoftenedException: not a long
at io.advantageous.boon.core.Exceptions.die(Exceptions.java:97)
at io.advantageous.boon.core.value.NumberValue.longValue(NumberValue.java:231)
at io.advantageous.boon.core.value.NumberValue.doToValue(NumberValue.java:130)
at io.advantageous.boon.core.value.NumberValue.toValue(NumberValue.java:104)
at io.advantageous.boon.core.value.LazyValueMap.buildMap(LazyValueMap.java:226)
at io.advantageous.boon.core.value.LazyValueMap.entrySet(LazyValueMap.java:214)
...
I had nothing to do with any long values, and this code was working fine with other jsons.
Funny thing is, it was sometimes successful with no changes whatsoever. So I dig
As the exception implies, for parsing Long values, the following method is used under NumberValue.java
:
@Override
public long longValue () {
if( CharScanner.isLong(buffer, startIndex, endIndex - startIndex)){
return CharScanner.parseLong(buffer, startIndex, endIndex);
} else {
return Exceptions.die(int.class, "not a long");
}
}
This is where I get the exception from.
Under CharScanner.java
isLong
method is comparing given value to Long.MAX_VALUE
by their length:
public static boolean isLong( char[] digitChars, int offset, int len ) {
String cmpStr = digitChars[offset]=='-' ? MIN_LONG_STR_NO_SIGN : MAX_LONG_STR;
int cmpLen = cmpStr.length();
if ( len > cmpLen ) return false;
if (verifyValueFitsInNumber(digitChars, offset, len, cmpStr, cmpLen)) return false;
if (digitChars[offset]=='-') {
return isDigits(digitChars, ++offset, --len);
} else {
return isDigits(digitChars, offset, len);
}
}
Length of Long.MAX_VALUE
(9223372036854775807) is 19.
This comparison is causing Json parsing(probably it has some other impacts too) to fail.
Later I realized successfull cases was because, in these scenarios, long field in my json was returning a smaller number. Mystery solved.
I didn't investigated that much but I believe a similar problem is present for integer as well.
Sorry for not coming with a fix, I'm a bit busy atm. But I thought the sooner I raise this the better
Note:
I guess when throwing this exception we should use long.class
instead.
return Exceptions.die(int.class, "not a long");
It is due to proxy produced by spring, solution can be cast to proxy class and get target class.
Advised advised = (Advised) proxy;
Class<?> cls = advised.getTargetSource().getTargetClass();
Have no idea how to workaround this in boon.
http://docs.spring.io/spring/docs/4.2.x/javadoc-api/org/springframework/aop/support/AopUtils.html#getTargetClass
Exception in thread "main" MESSAGE: name preFiltered setter public abstract void org.springframework.aop.framework.Advised.setPreFiltered(boolean) getter public abstract boolean org.springframework.aop.framework.Advised.isPreFiltered()
CAUSE io.advantageous.boon.core.Exceptions$SoftenedException :: Unable to extract annotation for property preFiltered of class interface org.springframework.aop.framework.Advised useRead false
io.advantageous.boon.core.Exceptions$SoftenedException: name preFiltered setter public abstract void org.springframework.aop.framework.Advised.setPreFiltered(boolean) getter public abstract boolean org.springframework.aop.framework.Advised.isPreFiltered()
CAUSE io.advantageous.boon.core.Exceptions$SoftenedException :: Unable to extract annotation for property preFiltered of class interface org.springframework.aop.framework.Advised useRead false
CAUSE io.advantageous.boon.core.Exceptions$SoftenedException :: Required object assertion exception
at io.advantageous.boon.core.Exceptions.handle(Exceptions.java:164)
at io.advantageous.boon.core.reflection.fields.BaseField.(BaseField.java:313)
at io.advantageous.boon.core.reflection.fields.PropertyField.(PropertyField.java:46)
at io.advantageous.boon.core.reflection.Reflection.getPropertyFieldAccessors(Reflection.java:625)
at io.advantageous.boon.core.reflection.ClassMeta.(ClassMeta.java:180)
at io.advantageous.boon.core.reflection.ClassMeta.classMeta(ClassMeta.java:273)
at io.advantageous.qbit.service.impl.BoonServiceMethodCallHandler.init(BoonServiceMethodCallHandler.java:603)
at io.advantageous.qbit.service.impl.BaseServiceQueueImpl.(BaseServiceQueueImpl.java:170)
at io.advantageous.qbit.service.impl.ServiceQueueImpl.(ServiceQueueImpl.java:50)
at io.advantageous.qbit.service.ServiceBuilder.build(ServiceBuilder.java:423)
at io.advantageous.qbit.service.ServiceBuilder.buildAndStart(ServiceBuilder.java:444)
at io.advantageous.qbit.service.impl.ServiceBundleImpl.addServiceObject(ServiceBundleImpl.java:312)
at io.advantageous.qbit.service.impl.ServiceBundleImpl.addService(ServiceBundleImpl.java:249)
at io.advantageous.qbit.server.ServiceEndpointServerImpl.initServices(ServiceEndpointServerImpl.java:365)
at io.advantageous.qbit.server.EndpointServerBuilder.build(EndpointServerBuilder.java:619)
at com.mm.base.Main.main(Main.java:24)
Caused by: io.advantageous.boon.core.Exceptions$SoftenedException: Unable to extract annotation for property preFiltered of class interface org.springframework.aop.framework.Advised useRead false
https://github.com/alexey-kanashevich/qbit-jwt-auth
Run Main.
Was here:
https://github.com/advantageous/qbit/issues/488#issuecomment-151407258
add support for LocalDate
Anil 1:48 PM
QBit.factory().createJsonMapper().toJson(object) .. will add [,,,,,123,,13]
JsonFactory.toJson(object) will add [null, null, null, null, 123, null, 13]
how do i get the same behavior as second one using QBit.factory jsonmapper
Rick Hightower 1:50 PM
ok
let me write a unit test and try to reproduce this
Anil 1:50 PM
ok. awesome!
Rick Hightower 1:54 PM
ok JsonFactory uses the lax parser but QBit use the strict one and the bug seems to be in the strict parser.
@Test
public void badSerializer() throws Exception {
String json = QBit.factory().createJsonMapper().toJson(Lists.list(1, 2, null, 3));
puts(json);
json = JsonFactory.toJson(Lists.list(1, 2, null, 3));
puts(json);
}
Show less
[1,2,,3]
[1,2,null,3]
Anil 1:56 PM
hmm. I don't have the control if use Callback<MyObject> then in the endpoint.
Then need to do Callback<HttpTextResponse>
Any other way to tell QBit factory use the lax one?
Rick Hightower 1:59 PM
ok
it has to do with JsonSimpleSerializerImpl vs BasicObjectSerializerImpl
here is SimpleSerializer
public final void serializeCollection( Collection<?> collection, CharBuf builder ) {
if ( collection.size () == 0 ) {
builder.addChars ( EMPTY_LIST_CHARS );
return;
}
builder.addChar( '[' );
for ( Object o : collection ) {
if (o == null) {
builder.addNull();
} else {
serializeObject(o, builder);
}
builder.addChar ( ',' );
}
builder.removeLastChar ();
builder.addChar( ']' );
}
Show more
Rick Hightower 2:04 PM
here is the wrong one
@Override
public final void serializeCollection ( JsonSerializerInternal serializer, Collection<?> collection, CharBuf builder ) {
if ( collection.size () == 0 ) {
builder.addChars ( EMPTY_LIST_CHARS );
return;
}
builder.addChar( '[' );
for ( Object o : collection ) {
serializer.serializeObject ( o, builder );
builder.addChar ( ',' );
}
builder.removeLastChar ();
builder.addChar( ']' );
}
Show more
Anil 2:05 PM
hmm.. looking at it.
Rick Hightower 2:06 PM
it is missing the addNull
if (o == null) {
builder.addNull();
} else {
serializeObject(o, builder);
}
builder.addChar ( ',' );
Show more
Anil 2:06 PM
I see
ok its in CollectionSerializerImpl
Anil 2:12 PM
Any workaround that you can think of?
like using our own custom CollectionSerializer?
Rick Hightower 2:13 PM
I will get a new release to you with a fix in the pub repo in a few hours.. and snapshot fix in a few minutes.
Hi,
While trying to find something that outperforms GSON, I found Boon and some pretty promising benchmarks:
https://github.com/bura/json-benchmarks
I'm currently working on a web service that takes in alot of small (up to 100kb) JSON strings and needs to decode those to a POJO.
So right now with GSON I hit 55k QPS in my test environment and when I just switch from GSON to Boon, I only hit about 45k QPS.
I can't help but think i'm doing something wrong.
MyObject object = JsonFactory.fromJson(content, MyObject.class);
// MyObject object = gson.fromJson(content, MyObject.class);
I noticed the benchmark I linked up there uses the JsonFastParser
which only deserializes to an Object unlike Jackson and GSON that deserializes to the actual Type which may be the reason why it outperforms them so much.
Am I missing something here to make it faster ?
Thanks
https://github.com/advantageous/qbit/issues/631strict mode
I have an endpoint which takes in a JAVA POJO in a POST request.
Java POJO has Integer member, say fieldInt.
If I create JSON, with say fieldInt : “some string” instead of fieldInt : 1, I get a POJO with incorrect values for the member “fieldInt".
I was wondering, if there a setting like strict json mapping to help this fix
I am able to reproduce this:
Service method:
@RequestMapping(value = "/echoEmployee", method = RequestMethod.POST)
public Employee echoEmployee(Employee employee) {
return employee;
}
...
@Test
public void echoEmployeeBadString() {
final HttpTextResponse httpResponse = httpServerSimulator.postBodyPlain("/es/echoEmployee",
"{\"id\":\"a\",\"name\":\"Rick\"}");
assertEquals(200, httpResponse.code());
assertEquals("{\"id\":49,\"name\":\"Rick\"}", httpResponse.body());
}
It seems that we are trying to convert 'a' into an int using ASCII table conversation which would be to subtract 48 from a (decimal ACII 97) which gives 49.
If you pass a "1", you get a 1 as 1 is 49 ASCII decimal - 48 (the offset) is 1.
The Boon mapping is loose so this actually works (on purpose):
@Test
public void echoEmployeeStringToInt() {
final HttpTextResponse httpResponse = httpServerSimulator.postBodyPlain("/es/echoEmployee",
"{\"id\":\"1\",\"name\":\"Rick\"}");
assertEquals(200, httpResponse.code());
assertEquals("{\"id\":1,\"name\":\"Rick\"}", httpResponse.body());
}
Which is fine, but the "a" case should not yield 49 so this is probably a fix that needs to be done in Boon not QBit.
boonproject/boon#355
boonproject/boon#356
This was reported and patched as part of Boon.
I know that the JSON spec doesn't guarantee attributes order, but in practice, browsers do.
And so does Jackson.
LazyMap
indeed stores internally in a LinkedHashMap
, so order is preserved.
But LazyValueMap
uses a regular HashMap
.
Would you consider either aligning, or add an option? I can provide a PR as soon as #11 gets merged (would conflict otherwise).
The following invalid Json numbers are parsed as numbers by Boon:
034
parsed as 34
, instead is invalid JSON, some other implementations parse it as an octal 28
that is what javascript would do. Same for -034
. See http://stackoverflow.com/a/27361596/1536382
0.
, invalid for JSON, parsed as 0
like javascript would do, note that .0
is treated as an error (also invalid for JSON while for js is also ok), so for consistency also 0.
should be, IMHO. If there is a way of doing relaxed-parsing may be both should be allowed in that case.
See also http://deron.meranda.us/python/comparing_json_modules/numbers#t5-3 (today unavailable, cached)
See same issue for genson owlike/genson#111
I have following code:
package boonbug;
import io.advantageous.boon.json.*;
import java.io.*;
public class BoonBug {
public static void main(String[] args) throws IOException {
FileWriter pw = new FileWriter("test");
JsonFactory.create().writeValue(pw, "blah-blah");
pw.close();
}
}
It produces the test file that has "blah-blah" padded with NULLs. So the file size is 4kb.
This becomes problematic when I use it in a servlet returning JSON, Chrome/FF don't like it and report JSON parsing error.
Is it a bug? Am I doing something wrong?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.