ameyag / protobuf-java-format Goto Github PK
View Code? Open in Web Editor NEWAutomatically exported from code.google.com/p/protobuf-java-format
License: BSD 3-Clause "New" or "Revised" License
Automatically exported from code.google.com/p/protobuf-java-format
License: BSD 3-Clause "New" or "Revised" License
XmlFormat requires a unit test
Original issue reported on code.google.com by eliran.bivas
on 7 Jul 2009 at 2:44
You have the JSON - would be interesting to be able to generate the IDL/schema
and service bindings for Avro support.
Original issue reported on code.google.com by [email protected]
on 12 Jul 2013 at 3:34
I see that you have taken away all of my permissions to be able to contribute
to the project without providing any notice or explanation. Could you please
fix the permissions so that I could continue to contribute!
Thanks,
Alex
Original issue reported on code.google.com by [email protected]
on 24 Oct 2010 at 3:13
I see that you have taken away all of my permissions to be able to contribute
to the project without providing any notice or explanation. Could you please
fix the permissions so that I could continue to contribute!
Thanks,
Alex
Original issue reported on code.google.com by [email protected]
on 24 Oct 2010 at 3:13
What steps will reproduce the problem?
1.include the current 1.2 module as a build dependency for a java 1.5 jar
2.compile
What is the expected output?
no complaints and a working build
What do you see instead?
/WorkSpace/p4/modules/serve/main/data-transform/src/main/java/com/comp/data/tran
sform/protocolbuffer/ProtocolBufferToJsonTransformer.java:[5,-1] cannot access
com.googlecode.protobuf.format.JsonFormat
bad class file:
/media/WorkSpace/repo/com/googlecode/protobuf-java-format/protobuf-java-format/1
.2/protobuf-java-format-1.2.jar(com/googlecode/protobuf/format/JsonFormat.class)
class file has wrong version 50.0, should be 49.0
What version of the product are you using? On what operating system?
1.2, Linux, Java 1.5
Please provide any additional information below.
I was able to fix this by rebuilding the jar with the following changes to the
pom.xml file. No code changes were required.
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
</plugins>
Original issue reported on code.google.com by [email protected]
on 27 Jun 2011 at 4:05
What steps will reproduce the problem?
simple code example
import com.google.protobuf.ByteString;
import com.google.protobuf.Message;
import com.googlecode.protobuf.format.JsonFormat;
import com.viaden.network.yoga.core.api.YogaCoreDomain;
/**
* protobuf message
* <p/>
* {@code message TestObject { required bytes data = 1; }}
*/
public class JacksonTest {
private static final char BELL_CHARACTER = '\u0007';
public static void main(String[] args) throws Exception {
final YogaCoreDomain.TestObject.Builder testObject = YogaCoreDomain.TestObject.newBuilder();
testObject.setData(ByteString.copyFrom(new byte[]{BELL_CHARACTER}));
final Message.Builder builder = YogaCoreDomain.TestObject.getDefaultInstance().newBuilderForType();
final String result = JsonFormat.printToString(builder.mergeFrom(testObject.build()).build());
}
}
What is the expected output? What do you see instead?
expected:
{"data":"\u0007"}
actual
{"data":"\a"}
What version of the product are you using? On what operating system?
1.2
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 22 Jan 2013 at 12:23
JSON document:
{
"username":"検索jan5検索[email protected]"
,"password":"12341234"
,"display_name":"1231212"
,"country":"US"
}
parses username field as: [“jan5”[email protected]]
Original issue reported on code.google.com by [email protected]
on 27 Apr 2011 at 10:19
What steps will reproduce the problem?
1. A string field in a message contains characters like à, ò, è, é (etc)
2. Look at the output of String jsonFormat =
jsonFormat.printToString(someProto);
What is the expected output? What do you see instead?
I want to see the original characters, not \xxx octal sequences.
What version of the product are you using? On what operating system?
protobuf-format-java-1.1.jar
Please provide any additional information below.
I tried to modify the method escapeBytes in JsonFormat.java but I can't see
my original characters.
Original issue reported on code.google.com by [email protected]
on 3 Mar 2010 at 2:03
What steps will reproduce the problem?
1. create an GzipOutputstream
2. print to that stream
3. print again, it fails since the stream is closed.
What is the expected output? What do you see instead?
Please use labels and text to provide additional information.
Original issue reported on code.google.com by [email protected]
on 20 Jun 2011 at 8:33
Please publish the JAR as Maven artifact (ideally in the central repository
but a Google Code one would do)
Original issue reported on code.google.com by [email protected]
on 11 Dec 2009 at 9:03
1. Define any message with a bytes field.
2. Create an instance of that message with that field set to a value that
contains bytes 0x07 or 0x0b
3. Call JsonFormat.printToString. The generated JSON will contain a string
value for this field with the tokens \a and \v in this value. This is invalid
JSON according to the JSON spec (http://www.json.org/) because 'a' and 'v' are
not amongst the characters that are allowed to follow a reverse solidus. No
standards-compliant JSON parser (e.g. Jackson) or validator (e.g.
www.jsonlint.org) will parse this JSON.
These escapes are explicitly added by the method JsonFormat.escapeBytes(). It's
unclear why \a and \v need special handling rather than just being turned into
unicode characters.
Environment:
* JsonFormat 1.1 (though the head of the current source (r. 34) currently has
this issue as well)
* OS X 10.6.4
* Java:
** java version "1.6.0_20"
** Java(TM) SE Runtime Environment (build 1.6.0_20-b02-279-10M3065)
** Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01-279, mixed mode)
* Jackson 1.5.4 (though any other standards-compliant parser will fail to parse
this as well, e.g. www.jsonlint.org)
Original issue reported on code.google.com by [email protected]
on 31 Jul 2010 at 7:22
What steps will reproduce the problem?
1. download pjfeoi.tgz
2. extract it (tar -xzvf pjfeoi.tgz)
3. cd pjf_empty_object
4. make clean && make && make run
What is the expected output? What do you see instead?
expected:
textJson: {"text":"foo"}
textPdu: {"text": "foo"}
authAckJson: {"authAck":{}}
authAckPdu: {"authAck": {}}
actual:
textJson: {"text":"foo"}
textPdu: {"text": "foo"}
authAckJson: {"authAck":{}}
com.googlecode.protobuf.format.JsonFormat$ParseException: 1:13: Expected
identifier. -}
at com.googlecode.protobuf.format.JsonFormat$Tokenizer.parseException(JsonFormat.java:765)
at com.googlecode.protobuf.format.JsonFormat$Tokenizer.consumeIdentifier(JsonFormat.java:575)
at com.googlecode.protobuf.format.JsonFormat.handleMissingField(JsonFormat.java:970)
at com.googlecode.protobuf.format.JsonFormat.mergeField(JsonFormat.java:936)
at com.googlecode.protobuf.format.JsonFormat.merge(JsonFormat.java:870)
at com.googlecode.protobuf.format.JsonFormat.merge(JsonFormat.java:818)
at com.chrisdew.Main.main(Main.java:26)
authAckPdu: {}
What version of the product are you using? On what operating system?
$ ls libs/protobuf*
libs/protobuf-2.5.0.jar libs/protobuf-java-format-1.2.jar
$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.2 LTS"
$ uname -a
Linux chris-work 3.2.0-45-generic #70-Ubuntu SMP Wed May 29 20:12:06 UTC 2013
x86_64 x86_64 x86_64 GNU/Linux
Please provide any additional information below.
pjfeoi.proto:
package com.chrisdew.pjfeoi;
message Pdu {
optional string text = 4;
optional AuthAck auth_ack = 5;
}
message AuthAck {
}
Main.java:
package com.chrisdew;
import com.google.protobuf.UninitializedMessageException;
import com.googlecode.protobuf.format.JsonFormat;
import com.chrisdew.pjfeoi.Pjfeoi;
public class Main {
public static void main(String[] args) {
String textJson = "{\"text\":\"foo\"}";
System.out.println("textJson: " + textJson);
Pjfeoi.Pdu.Builder builderForText = Pjfeoi.Pdu.newBuilder();
try {
JsonFormat.merge(textJson, builderForText);
} catch (JsonFormat.ParseException e) {
e.printStackTrace();
}
Pjfeoi.Pdu textPdu = builderForText.build();
System.out.println("textPdu: " + JsonFormat.printToString(textPdu));
String authAckJson = "{\"authAck\":{}}";
System.out.println("authAckJson: " + authAckJson);
Pjfeoi.Pdu.Builder builderForAuthAck = Pjfeoi.Pdu.newBuilder();
try {
JsonFormat.merge(authAckJson, builderForAuthAck);
} catch (JsonFormat.ParseException e) {
e.printStackTrace();
}
Pjfeoi.Pdu authAckPdu = builderForAuthAck.build();
System.out.println("authAckPdu: " + JsonFormat.printToString(authAckPdu));
}
}
Original issue reported on code.google.com by [email protected]
on 7 Aug 2013 at 8:51
Attachments:
Problem:
JsonFormat Line ~1281:
case 'u':
// UTF8 escape
code =
(16 * 3 * digitValue(input.charAt(i+1))) +
(16 * 2 * digitValue(input.charAt(i+2))) +
(16 * digitValue(input.charAt(i+3))) +
digitValue(input.charAt(i+4));
i = i+4;
result[pos++] = (byte) code;
break;
Correct (not multiplied but by the Power of 16):
case 'u':
// UTF8 escape
code =
(16 * 16 * 16 * digitValue(input.charAt(i+1))) +
(16 * 16 * digitValue(input.charAt(i+2))) +
(16 * digitValue(input.charAt(i+3))) +
digitValue(input.charAt(i+4));
i = i+4;
result[pos++] = (byte) code;
break;
I attached a version with handels the Umlaute correct.
What steps will reproduce the problem?
1. Build a Message containing a String with Umlaute (äöüÄÖÜäàéè)
2. JsonFormat.print(req.build(), sb);
3. JsonFormat.merge(sb,resp);
What is the expected output? What do you see instead?
The String in the resp Message isn't correct.
I hope this helps to improve this usefull Util.
Original issue reported on code.google.com by [email protected]
on 23 Apr 2011 at 5:23
Attachments:
Create a common interface for all format types so that they are easily
injectable & switchable using various frameworks (like spring). Some kind of
factory would be useful as well.
Original issue reported on code.google.com by [email protected]
on 11 Jul 2011 at 1:26
When field like (which was generated using XMLFomat.printToString):
<email>[email protected]</email>
is parsed, tokenizer detects these 2 tokens:
'juraj@michalak' and 'com'
The error output is:
Caused by: com.googlecode.protobuf.format.XmlFormat$ParseException: 1:71:
Expected ">".
at com.googlecode.protobuf.format.XmlFormat$Tokenizer.parseException(XmlFormat.java:668)
at com.googlecode.protobuf.format.XmlFormat$Tokenizer.consume(XmlFormat.java:467)
at com.googlecode.protobuf.format.XmlFormat.consumeClosingElement(XmlFormat.java:791)
at com.googlecode.protobuf.format.XmlFormat.mergeField(XmlFormat.java:875)
at com.googlecode.protobuf.format.XmlFormat.handleObject(XmlFormat.java:993)
at com.googlecode.protobuf.format.XmlFormat.handleValue(XmlFormat.java:886)
at com.googlecode.protobuf.format.XmlFormat.mergeField(XmlFormat.java:866)
at com.googlecode.protobuf.format.XmlFormat.merge(XmlFormat.java:774)
at com.googlecode.protobuf.format.XmlFormat.merge(XmlFormat.java:722)
at com.protobufexample.ListPeople.main(ListPeople.java:64)
Correct would be only one token:
'[email protected]'
protobuf-java-format 1.2
Original issue reported on code.google.com by [email protected]
on 23 Nov 2011 at 10:39
JsonFormat performs a build instead of buildPartial. As such if required field
is missing in base object then it successfully perform merge, however if some
nested message's required field is missing, merge causes exception. Changing
build to buildPartial at all the 2 places inside this file should work.
Original issue reported on code.google.com by [email protected]
on 25 Mar 2012 at 7:54
JsonFormat require a unit test
Original issue reported on code.google.com by eliran.bivas
on 7 Jul 2009 at 2:43
What steps will reproduce the problem?
1. define message with a field with default value.
2. populate a builder of that message, but don't set that field.
3. observe that json doesn't contain default value.
What is the expected output? What do you see instead?
should contain that field, but with default value set.
What version of the product are you using? On what operating system?
version 1.2, on a mac.
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 25 Oct 2012 at 2:57
What steps will reproduce the problem?
1.Use JavaPropsFormat.printToString(myAddressBookProtoObj) to get a properties
string
2.use this string to generate a proto object
What is the expected output? What do you see instead?
addressBookObject should be like this:
addressbook:{
person {
name: "jack"
id: 1
email: "[email protected]"
phone {
number: "1234"
type: HOME
}
phone {
number: "5678"
type: WORK
}
}
}
However, there is an exception occured:
Exception in thread "main" java.lang.NullPointerException
at com.google.protobuf.GeneratedMessage$FieldAccessorTable.access$100(Unknown Source)
at com.google.protobuf.GeneratedMessage$Builder.newBuilderForField(Unknown Source)
at com.googlecode.protobuf.format.JavaPropsFormat.mergeField(JavaPropsFormat.java:937)
at com.googlecode.protobuf.format.JavaPropsFormat.merge(JavaPropsFormat.java:850)
at com.googlecode.protobuf.format.JavaPropsFormat.merge(JavaPropsFormat.java:795)
at com.example.tutorial.ProtoTest.print(ProtoTest.java:160)
at com.example.tutorial.ProtoTest.main(ProtoTest.java:19)
What version of the product are you using? On what operating system?
1.2
Please provide any additional information below.
There is same issue for XmlFormat and JsonFormat.
It seems that JavaPropsFormat can printToString successfully and the properties
string is as follows:
person[0].name="jack"
person[0].id=1
person[0].email="[email protected]"
person[0].phone[0].number="1234"
person[0].phone[0].type=HOME
person[0].phone[1].number="5678"
person[0].phone[1].type=WORK
Original issue reported on code.google.com by [email protected]
on 28 Feb 2012 at 2:39
What steps will reproduce the problem?
1. Create a collection of Message instances
2. Use JsonFormat to serialize them
What is the expected output? What do you see instead?
The expected output is an array of Messages:
[ {"name":"msg1"}, {"name":"msg2"}, {"name":"msg3"} ]
Since there is no support for serializing a collection of Message
instances, the calling code needs to "manually" open and close the array
brakets.
Ideally, the calling code would call a method with the following signature:
JsonFormat.print(Iterable<Message> messages, Appendable a);
What version of the product are you using? On what operating system?
1.1
Please provide any additional information below.
Or is the pattern to create a Message type that has a repeatable field of
other Message types?
message Foo {
required string name = 1;
}
message Foos {
repeatable Foo foos = 1;
}
Original issue reported on code.google.com by [email protected]
on 13 May 2010 at 9:38
Here is a patch to add support for the smile
(http://wiki.fasterxml.com/SmileFormatSpec) format using the jackson processor.
It closely matches the jsonformat, but adds a few more tests & works for
extensions.
Please let me know if you would prefer to get this patch another way.
thanks
Original issue reported on code.google.com by [email protected]
on 6 Apr 2011 at 6:33
Attachments:
What steps will reproduce the problem?
1. Produce any protobuf that contains a date like 11 Mar 2009 as a value
2. Convert it to XML
3. XMLFormat will throw an exception
What is the expected output? What do you see instead?
I expect to see a date, without any errors
What version of the product are you using? On what operating system?
1.1, java 1.6.0, Windows 7
Please provide any additional information below.
The regular expression (TOKEN) will group a number by itself, before
grouping a date like 11 March 2009 together. Subsequently, the tokenizer
will look for an end tag and not find one.
amoffetATgmailDOTcom
Original issue reported on code.google.com by [email protected]
on 4 Mar 2010 at 10:12
What steps will reproduce the problem?
1. Create any protobuf where a String that consists only of spaces is valid
2. Transform it to xml using this tool
3. The field will be incorrect in the XML
What is the expected output? What do you see instead?
I see the first characters of the closing tag - I expect to see the spaces.
What version of the product are you using? On what operating system?
1.1, Java 1.6.0_18, Windows 7
Please provide any additional information below.
Elements are allowed to have significant spaces in their text. The method
nextToken(), however, will eliminate any spaces following a valid token -
including the closing bracket of an opening element tag. So, if I have
something like this:
<AnElementWithSpaces> </AnElementWithSpaces>
The tokenizer will correctly recognize the closing bracket of
<AnElementWithSpaces>, but will remove the following (significant) spaces.
The software should remove insignificant spaces and retain the others. A
quick look at the code suggests that the only time this is necessary is in
the mergeField() method, where it first consumes the opening bracket and
then consumes the identifier. Consuming the identifier should not delete
the significant spaces.
amoffetATgmailDOTcom
Original issue reported on code.google.com by [email protected]
on 4 Mar 2010 at 7:38
What steps will reproduce the problem?
1.Create a POJO class with an user defined class's object as instance member
2.Instantiate this pojo object with the member object's value as null
3.Convert this POJO to json string and invoke JsonFormat.merge()
What is the expected output? What do you see instead?
Json string should be properly mapped to its corresponding ProtoBuf object. But
Getting exception
Exception in thread "main" com.googlecode.protobuf.format.JsonFormat$ParseException: 1:16: Expected "{".
at com.googlecode.protobuf.format.JsonFormat$Tokenizer.parseException(JsonFormat.java:765)
at com.googlecode.protobuf.format.JsonFormat$Tokenizer.consume(JsonFormat.java:530)
at com.googlecode.protobuf.format.JsonFormat.handleObject(JsonFormat.java:1128)
at com.googlecode.protobuf.format.JsonFormat.handleValue(JsonFormat.java:1004)
at com.googlecode.protobuf.format.JsonFormat.mergeField(JsonFormat.java:952)
at com.googlecode.protobuf.format.JsonFormat.merge(JsonFormat.java:870)
at com.googlecode.protobuf.format.JsonFormat.merge(JsonFormat.java:818)
at com.ms.JSONToPB.main(JSONToPB.java:48)
What version of the product are you using? On what operating system?
protobuf-java-format - 1.2. OS - Windows 7
Please provide any additional information below.
main function I used
--------------------
public static void main(String[] args)(){
Source sObj = new Source();
sObj.setMemberClass(null);
SourceProto.Source.Builder builder = SourceProto.Source.newBuilder();
ObjectMapper objMapper = new ObjectMapper();
String jsonString = objMapper.writeValueAsString(sObj);
JsonFormat.merge(jsonString, builder);
SourceProto.Source obj = builder.build();
}
Source.class
------------
public class Source{
private MemberClass memberClass;
public MemberClass getMemberClass() {
return memberClass;
}
public void setMemberClass(MemberClass memberClass) {
this.memberClass = memberClass;
}
}
MemberClass.class
------------------
public class MemberClass {
private int num;
public int getNum() {
return num;
}
public void setNum(int num) {
this.num = num;
}
}
If we keep null value for primitive type, it is parsed properly. For
non-primitive types, error is thrown
I went through the code and modified the JsonFormat.java at line number : 1123.
I added the following code
if("null".equals(tokenizer.currentToken)){
tokenizer.consume("null");
return null;
}
With this null check, issue is resolved. Please check whether this is the
proper resolution
Original issue reported on code.google.com by [email protected]
on 14 Jan 2015 at 5:25
What steps will reproduce the problem?
1. prepare this json {"name":"Aamore</1"}
2. merge with JsonFormat.merge(json, aProtoObj);
3.com.googlecode.protobuf.format.JsonFormat$ParseException: 1:9: Invalid escape
sequence: '\/'
at com.googlecode.protobuf.format.JsonFormat$Tokenizer.parseException(JsonFormat.java:765)
What version of the product are you using? On what operating system?
protobuf-java-format 1.2 on mac osx
slash can be optionally escaped, here is not handled
Original issue reported on code.google.com by [email protected]
on 9 Jan 2012 at 2:06
What steps will reproduce the problem?
1. Call XmlFormat.merge(Readable input, Message.Builder builder) with an XML
string
What is the expected output? What do you see instead?
Expect XML to be parsed. Instead I get an error about a missing "{" from the
JsonFormat class.
What version of the product are you using? On what operating system?
1.1
Please provide any additional information below.
It seems that the code in XmlFormat is calling the following instead of just
calling its own merge method:
JsonFormat.merge(input, ExtensionRegistry.getEmptyRegistry(), builder);
Original issue reported on code.google.com by [email protected]
on 20 Dec 2010 at 9:27
1. set a String field with an escaped char
2. view the page in html and the chars appears escaped instead of actual char
What is the expected output? What do you see instead?
normal character, let the browser handle the encoding
Original issue reported on code.google.com by eliran.bivas
on 14 Jul 2009 at 11:11
HtmlFormat requires a unit test
Original issue reported on code.google.com by eliran.bivas
on 7 Jul 2009 at 2:44
To reporoduce convert to XML data containing special symbols e.g. '<' or '&'
Expected: @amp; @lt; etc.
Got: &, <
To fix: add precessing for special symbols in method
XMLFormat.escapeBytes(ByteString input)
Original issue reported on code.google.com by [email protected]
on 18 Jan 2010 at 8:10
Hi,
I was wondering why a json parser like jackson was not used for json
coding/decoding. I wrote a simple codec for json which uses jackson as the
underlying framework and found that the performance for json serialization
almost doubled. The implementation is quite basic and lacks supports for
unknownfieldsets.
Would appreciate a feedback since I just started out with protobuf.
Original issue reported on code.google.com by [email protected]
on 1 Dec 2010 at 5:07
What steps will reproduce the problem?
1. Create a valid XML file representing a protobuf object. You could call
XmlFormat.print on a protobuf object, for instance.
2. Add a valid XML comment to the file
3. Call XmlFormat.merge with that file and you'll get an exception
What is the expected output? What do you see instead?
I'd expect it to create the protobuf object. Instead it causes an exception of
the form:
com.google.protobuf.XmlFormat$ParseException: 4:4: Expected identifier. -!
at com.google.protobuf.XmlFormat$Tokenizer.parseException(XmlFormat.java:662)
What version of the product are you using? On what operating system?
1.1 on Mac OS X.
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 10 Mar 2011 at 9:54
What steps will reproduce the problem?
1. Serialize a object with bytes "P\001"
2. Try to deserialize it (XML)
3. Cannot deserialize
A little fix for that (bytes are still readable, plus serialize by default in
hex):
--- XmlFormat.java 2010-06-03 17:06:18.000000000 +0100
+++ XmlFormat.java 2010-10-14 15:46:09.792528214 +0100
@@ -39,6 +39,7 @@
import com.google.protobuf.Descriptors.EnumValueDescriptor;
import com.google.protobuf.Descriptors.FieldDescriptor;
+import com.google.protobuf.util.*;
/**
* Provide ascii text parsing and formatting support for proto2 instances. The implementation
@@ -361,10 +362,11 @@
private static final Pattern WHITESPACE =
Pattern.compile("(\\s|(#.*$))++", Pattern.MULTILINE);
private static final Pattern TOKEN = Pattern.compile(
- "extension|" + "[a-zA-Z_\\s;@][0-9a-zA-Z_\\s;@+-]*+|" + // an
identifier with special handling for 'extension'
+ "extension|" + // special handling
for 'extension'
+ "[0-9a-zA-Z\\\\\\\"\\\']++|" + // a \000 byte
sequence for bytes handling
+ "[a-zA-Z_\\s;@][0-9a-zA-Z_\\s;@+-]*+|" + // an identifier
"[.]?[0-9+-][0-9a-zA-Z_.+-]*+|" + // a number
"</|" + // an '</' closing element marker
- "[\\\\0-9]++|" + // a \000 byte
sequence for bytes handling
"\"([^\"\n\\\\]|\\\\.)*+(\"|\\\\?$)|" + // a double-quoted string
"\'([^\'\n\\\\]|\\\\.)*+(\'|\\\\?$)", // a single-quoted string
Pattern.MULTILINE);
@@ -1041,13 +1043,11 @@
builder.append("\\\"");
break;
default:
- if (b >= 0x20) {
+ if ((b >= 0x30 && b <= 0x39) || (b >= 0x41 && b <= 0x5A)
|| (b >= 0x61 && b <= 0x7A)) {
builder.append((char) b);
} else {
- builder.append('\\');
- builder.append((char) ('0' + ((b >>> 6) & 3)));
- builder.append((char) ('0' + ((b >>> 3) & 7)));
- builder.append((char) ('0' + (b & 7)));
+ builder.append("\\x");
+ builder.append(HexUtils.getHexString(b, 2));
}
break;
}
Original issue reported on code.google.com by [email protected]
on 14 Oct 2010 at 2:47
What steps will reproduce the problem?
1. printToJson from a Message
2.
3.
What is the expected output? What do you see instead?
Expected for a message
Person {
optioanl string name = 1;
}
with the value name = test:
{ "name": "test" }
Actual:
{ "person": { "name": "test" }}
What version of the product are you using? On what operating system?
2.3.0 on osx
Please provide any additional information below.
This top-level wrapping isn't very json-esque but more like what you would see
in xml. It should be possible to disable.
Original issue reported on code.google.com by [email protected]
on 13 Jun 2012 at 9:02
What steps will reproduce the problem?
1. Create a proto and add unknown fields (using UnknownFieldSet.Builder) to it
with wire type fixed32 or fixed64
2. Serialize the proto to JSON using JsonFormat.printToString()
3. Inspect the output string
What is the expected output? What do you see instead?
The output will contain:
"9000": [0x00002328], "9001": [0x0000000000002329]
In this case, there is an unknown fixed32 field with tag 9000 and value 9000 (=
0x2328), and an unknown fixed64 field with tag 9001 and value 9001 (= 0x2329).
However, this output is not valid JSON. Numbers in JSON can only be represented
in decimal form.
If we attempt to parse this output using another JSON library (such as GSON),
it may fail outright due to the bad syntax, but even if it is lenient enough to
perform the conversion, there may not be a way to determine whether the value
in the input stream was "0x" + 8 digits or "0x" + 16 digits. In such a case,
there would be no way to determine which wire type was originally used to store
the value (fixed32 or fixed64), which means we can't reconstruct the proto in a
way that would allow us to reserialize it in binary form without loss.
There are important use cases for representing unknown fields losslessly. For
example a server may hand an object to a client that only knows about an older
version of the proto. Then the client may update some fields it knows about,
and hand the object back to the server. Fields of the proto that are unknown to
the client should not be disturbed. The ability to handle this kind of scenario
seamlessly, without the need to synchronously update all consumers of a proto
definition, is a key advantage of protocol buffers.
In effect, serialization of unknown fields is only lossless when they will be
read by another instance of protobuf-java-format. This negates the value of
using a standard format like JSON. Instead, the values should be encoded in
such a way that only standard JSON constructs are used, and the wire type is
preserved, for example:
"9000":[{"fixed32":9000},{"fixed64":9001}]
What version of the product are you using? On what operating system?
1.2.
Original issue reported on code.google.com by [email protected]
on 1 Aug 2013 at 11:54
Add ability to de-serialize Protobuf objects from XML source.
Original issue reported on code.google.com by [email protected]
on 5 Jul 2009 at 4:42
What steps will reproduce the problem?
1. Create a builder and set a field on an inner message to a non-default
value
2. Create the appropriate XML that you would use to populate the builder
above with. Leave the field changed above in it's default state - which
means that the field does not appear in the XML.
3. Invoke merge(xml, builder) with the two and notice that a merge does not
occur because the inner message overwrites the one on the builder.
Consequently, the value in the builder that you wished to "merge" with the
XML is gone.
What is the expected output? What do you see instead? I expect that the
values from the XML are merged into the builder - with xml values
overwriting those that are set in the builder. This behavior is consistent
with the mergefrom methods in PB. Instead, the XML is serialized into a PB
, destroying any prior PB state that you wanted merged in.
Admittedly, this may not be the intended functionality - but it would have
been nice to have named the method appropriately if it is not.
What version of the product are you using? On what operating system?
1.1, JDK 1.6, Windows
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 10 Mar 2010 at 5:35
What steps will reproduce the problem?
1. Create a JSON string with field Foo that has a double as a key
2. use JsonFormat.merge with a Builder that does not have Foo defined
What is the expected output? What do you see instead?
The call to JsonFormat.merge fails with a message like:
Parseexception: 1:14: Couldn't parse integer: For input string: "0.95"
What version of the product are you using? On what operating system?
1.2
Please provide any additional information below.
Defining Foo as a double in the protobuf is a workaround, but I think it would
be better if support for double was added to handleMissingField()
Original issue reported on code.google.com by [email protected]
on 12 Sep 2012 at 8:44
Unknown fields are printed in a single element and not repeated.
Original issue reported on code.google.com by eliran.bivas
on 7 Jul 2009 at 2:42
What steps will reproduce the problem?
1.create a simple protobuf message, message CharTest{required string a=1}
2.create a protobuf instance, with a=
3. CharTest m CharTest.newBuilder().setA("my.test").build();
String xmlText = XmlFormat.printToString(m);
UserAuthReq.Builder builder2 = UserAuthReq.newBuilder();
XmlFormat.merge(xmlText, builder2);
What is the expected output? What do you see instead?
It should run without exception.
Actual output:
com.google.protobuf.XmlFormat$ParseException: 1:33: Expected ">".
at com.google.protobuf.XmlFormat$Tokenizer.parseException(XmlFormat.java:662)
at com.google.protobuf.XmlFormat$Tokenizer.consume(XmlFormat.java:461)
at com.google.protobuf.XmlFormat.consumeClosingElement(XmlFormat.java:785)
at com.google.protobuf.XmlFormat.mergeField(XmlFormat.java:869)
at com.google.protobuf.XmlFormat.merge(XmlFormat.java:768)
at com.google.protobuf.XmlFormat.merge(XmlFormat.java:734)
at com.google.protobuf.XmlFormat.merge(XmlFormat.java:709)
What version of the product are you using? On what operating system?
protobuf-java-format 1.1, protobuf-2.3, Linux
This seems to be the way how to handle the nextToken. it doens't match the
Token pattern.
Original issue reported on code.google.com by [email protected]
on 6 Aug 2010 at 3:29
The wiki contains the following path to use as a maven repository:
http://protobuf-java-format.googlecode.com/svn/trunk/
The path to the protobuf-format-java artifact looks like this:
com/google/protobuf/protobuf-java-format/1.1.1/
But the artifact is not named protobuf-java-format, it's named
protobuf-format-java, which causes maven to be unable to resolve the pom and
jar correctly when added as a dependency as follows:
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-format-java</artifactId>
<version>1.1.1</version>
</dependency>
Original issue reported on code.google.com by [email protected]
on 22 Feb 2011 at 5:44
What steps will reproduce the problem?
1. Define any message with a string field, e.g.
message TestMessage {
required string test_string = 1;
}
2. Create an instance of this message with a string containing a single quote,
e.g.
TestMessage msg = TestMessage.newBuilder().setTestString( "a string with
'single quotes'").build()
String serialized = JsonFormat.printToString(msg);
3. The serialized string will look like the following:
{ "test_string":"a string with \'single quotes\'" }
But according to the JSON spec (http://www.json.org/), a single quote is not a
valid symbol to follow a reverse solidus (single quotes don't need to be
escaped at all in JSON). Trying to deserialize this through any
standards-compilant JSON parser (e.g. Jackson) will result an error related to
the fact that \' is not a valid character in JSON-encoded data.
What is the expected output? What do you see instead?
For syntactically-valid / standards-compliant JSON, I would expect to see
instead
{ "test_string":"a string with 'single quotes'" }
[note the lack of reverse solidus before the single quotes.]
What version of the product are you using? On what operating system?
OS X 10.6, library version 1.1, Jackson 1.5.4
Thanks.
Original issue reported on code.google.com by [email protected]
on 29 Jun 2010 at 4:01
What steps will reproduce the problem?
1. Create an extensible message
2. Create an extension
3. Instantiate both and initialize the extensible instance with an extension
instance
4. Serialize to JSON.
What is the expected output? What do you see instead?
Using this .proto descriptor:
message Foo {
required uint32 foo = 1;
extensions 100 to max;
}
message Bar {
required uint32 bar = 1;
extend Foo {
optional Bar bar_ext = 100;
}
}
Instantiate and serialize:
FooBar.Bar bar = FooBar.Bar.newBuilder().setBar(42).build();
JsonFormat.printToString(
FooBar.Foo.newBuilder()
.setFoo(4)
.setExtension(FooBar.Bar.barExt,
FooBar.Bar.newBuilder()
.setBar(42).build()
)
);
Expected:
{"foo": 4,"Bar.bar_ext": {"bar": 42}}
Actual:
{"foo": 4,["Bar.bar_ext"]: {"bar": 42}}
What version of the product are you using? On what operating system?
protobuf-java-format-1.1
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 6 Jul 2010 at 8:12
Attachments:
This may not be possible, but could the MessageLite interface be supported?
XmlFormat.print(Message, Appendable)
+ XmlFormat.print(MessageLite, Appendable)
Original issue reported on code.google.com by [email protected]
on 24 Jan 2011 at 6:00
What steps will reproduce the problem?
1. Snippet from my test program:
// javascript in browser gets byte string embedded in larger message, using ajax. if the
// data array has 0x07 or 0x0b, browser throws exception for the ajax thread. please see the
// standard ECMA-262, at "http://www.ecma-international.org/publications/standards/Ecma-262.htm",
// section 15.12.1.1, titled "The JSON Lexical Grammar". That section clearly states:
//
// JSONEscapeCharacter :: one of " / \ b f n r t
//
// There is no mention of "\a" or "\v" as legal escape shortcuts in a JSON string literal.
//
// snippet from .proto file ..
//
// required uint64 data_id = 1;
// required uint64 event_id = 2;
// optional bytes data = 3;
//
// note .. in C-language (or, at least, Microsoft C compilers), there are extra
// escapes allowed:
//
// \a ==> 0x07 "alert" (ascii BEL)
// \v ==> 0x0b "vertical tab" (ascii VT)
//
byte[] caseAbytes = { 0x06, 0x07, 0x08 }; // expect "\u0006\u0007\b", get "\u0006\a\b"
byte[] caseVbytes = { 0x0a, 0x0b, 0x0c }; // expect "\n\u0016\f", get "\n\v\f"
ByteString caseA = ByteString.copyFrom( caseAbytes );
ByteString caseV = ByteString.copyFrom( caseVbytes );
// The field "output_event_data" is a submessage of type EventData. We
// want to alter the field "data" in that submessage.
//
// at ..
//
DynamicMessage.Builder editorProto = parsedProto.newBuilder( parsedProto );
DynamicMessage.Builder editorOED = editorProto.newBuilderForField( fieldOED );
fieldOED_data_id = fieldOED.getMessageType().findFieldByName( "data_id" );
fieldOED_event_id = fieldOED.getMessageType().findFieldByName( "event_id" );
fieldOED_data = fieldOED.getMessageType().findFieldByName( "data" );
try
{
Message patchProto = null;
editorOED.setField( fieldOED_data_id, new Long(1) );
editorOED.setField( fieldOED_event_id, new Long(2) );
editorOED.setField( fieldOED_data, caseA );
patchProto = editorOED.build();
if (DUMP_CASE_A)
{
String formatCaseA = JsonFormat.printToString( patchProto );
System.out.println( "illegal escape, demo case \\a: " + formatCaseA );
System.out.println( " ** should have been: \\u0006\\u0007\\b **" );
}
editorOED = editorProto.newBuilderForField( fieldOED );
editorOED.setField( fieldOED_data_id, new Long(3) );
editorOED.setField( fieldOED_event_id, new Long(4) );
editorOED.setField( fieldOED_data, caseV );
patchProto = editorOED.build();
if (DUMP_CASE_V)
{
String formatCaseV = JsonFormat.printToString( patchProto );
System.out.println( "illegal escape, demo case \\v: " + formatCaseV );
System.out.println( " ** should have been: \\n\\u0011\\f **" );
}
}
catch (Exception e)
{
e.printStackTrace();
}
2. Wrap the snippet above in a compilable java source and run it.
The println() calls and comments document what should happen.
Impact if using ajax to feed ByteString to browser is that a browser such as
Firefox complains about "invalid escaped character".
Using 1.2 release on linux 64-bit Centos 6.2 build.
Please provide any additional information below:
You can see in the source at
"http://code.google.com/p/protobuf-java-format/source/browse/trunk/src/main/java
/com/googlecode/protobuf/format/JsonFormat.java", line 1061, method
escapeBytes() that someone ignored a strong hint about \a and \v and went ahead
to (improperly) escape them for JSON output.
Original issue reported on code.google.com by [email protected]
on 14 Sep 2012 at 4:42
I see that you have taken away all of my permissions to be able to contribute
to the project without providing any notice or explanation. Could you please
fix the permissions so that I could continue to contribute!
Thanks,
Alex
Original issue reported on code.google.com by [email protected]
on 24 Oct 2010 at 3:14
RE: Issue-35
Original issue reported on code.google.com by [email protected]
on 22 Aug 2011 at 5:22
BJF could be fixed to allow JSON field names in camelCase or sname_case.
What steps will reproduce the problem?
1. Make a .proto file with a snake_case field_name.
2. Generate JSON from protobuf using https://github.com/chrisdew/protobuf/
3. The JSON will have a camelCase fieldName
4. PJF will fail to parse the JSON.
e.g.
1. download pjf_empty_object.tgz
2. extract it (tar -xzvf pjf_empty_object.tgz)
3. cd pjf_empty_object
4. make clean && make && make run
What is the expected output? What do you see instead?
expected:
textJson: {"text":"foo"}
textPdu: {"text": "foo"}
authAckJson: {"authAck":{}}
authAckPdu: {"authAck": {}}
actual:
textJson: {"text":"foo"}
textPdu: {"text": "foo"}
authAckJson: {"authAck":{}}
com.googlecode.protobuf.format.JsonFormat$ParseException: 1:13: Expected
identifier. -}
at com.googlecode.protobuf.format.JsonFormat$Tokenizer.parseException(JsonFormat.java:765)
at com.googlecode.protobuf.format.JsonFormat$Tokenizer.consumeIdentifier(JsonFormat.java:575)
at com.googlecode.protobuf.format.JsonFormat.handleMissingField(JsonFormat.java:970)
at com.googlecode.protobuf.format.JsonFormat.mergeField(JsonFormat.java:936)
at com.googlecode.protobuf.format.JsonFormat.merge(JsonFormat.java:870)
at com.googlecode.protobuf.format.JsonFormat.merge(JsonFormat.java:818)
at com.chrisdew.Main.main(Main.java:26)
authAckPdu: {}
What version of the product are you using? On what operating system?
$ ls libs/protobuf*
libs/protobuf-2.5.0.jar libs/protobuf-java-format-1.2.jar
$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.2 LTS"
$ uname -a
Linux chris-work 3.2.0-45-generic #70-Ubuntu SMP Wed May 29 20:12:06 UTC 2013
x86_64 x86_64 x86_64 GNU/Linux
Please provide any additional information below.
pjfeoi.proto:
package com.chrisdew.pjfeoi;
message Pdu {
optional string text = 4;
optional AuthAck auth_ack = 5;
}
message AuthAck {
}
Main.java:
package com.chrisdew;
import com.google.protobuf.UninitializedMessageException;
import com.googlecode.protobuf.format.JsonFormat;
import com.chrisdew.pjfeoi.Pjfeoi;
public class Main {
public static void main(String[] args) {
String textJson = "{\"text\":\"foo\"}";
System.out.println("textJson: " + textJson);
Pjfeoi.Pdu.Builder builderForText = Pjfeoi.Pdu.newBuilder();
try {
JsonFormat.merge(textJson, builderForText);
} catch (JsonFormat.ParseException e) {
e.printStackTrace();
}
Pjfeoi.Pdu textPdu = builderForText.build();
System.out.println("textPdu: " + JsonFormat.printToString(textPdu));
String authAckJson = "{\"authAck\":{}}";
System.out.println("authAckJson: " + authAckJson);
Pjfeoi.Pdu.Builder builderForAuthAck = Pjfeoi.Pdu.newBuilder();
try {
JsonFormat.merge(authAckJson, builderForAuthAck);
} catch (JsonFormat.ParseException e) {
e.printStackTrace();
}
Pjfeoi.Pdu authAckPdu = builderForAuthAck.build();
System.out.println("authAckPdu: " + JsonFormat.printToString(authAckPdu));
}
}
P.S. this was previously
https://code.google.com/p/protobuf-java-format/issues/detail?id=48
Original issue reported on code.google.com by [email protected]
on 7 Aug 2013 at 10:46
Attachments:
What steps will reproduce the problem?
1. in .proto:
message BigData { required bytes d = 1; }
2. in .java:
BigData bd = BigData.newBuilder()
.setD(ByteString.copyFrom(new byte[1024]))
.build();
JsonFormat.merge(JsonFormat.printToString(bd), BigData.newBuilder());
What is the expected output? What do you see instead?
Saw StackOverflow exception:
Exception in thread "main" java.lang.StackOverflowError
at java.util.regex.Pattern$CharProperty.match(Unknown Source)
at java.util.regex.Pattern$Branch.match(Unknown Source)
at java.util.regex.Pattern$GroupHead.match(Unknown Source)
at java.util.regex.Pattern$Loop.match(Unknown Source)
at java.util.regex.Pattern$GroupTail.match(Unknown Source)
at java.util.regex.Pattern$BranchConn.match(Unknown Source)
at java.util.regex.Pattern$CharProperty.match(Unknown Source)
at java.util.regex.Pattern$Branch.match(Unknown Source)
at java.util.regex.Pattern$GroupHead.match(Unknown Source)
at java.util.regex.Pattern$Loop.match(Unknown Source)
at java.util.regex.Pattern$GroupTail.match(Unknown Source)
at java.util.regex.Pattern$BranchConn.match(Unknown Source)
......
What version of the product are you using? On what operating system?
protobuf-format-java-1.1 on Windows
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 19 Sep 2009 at 3:48
What steps will reproduce the problem?
1. test.proto
message TestItem{
required string name = 1;
}
2. Java code
Testproto.TestItem.Builder itemBuilder = Testproto.TestItem.newBuilder();
itemBuilder.setName("name with spaces");
String itemTxt = XmlFormat.printToString(itemBuilder.build());
itemBuilder = Testproto.TestItem.newBuilder();
XmlFormat.merge(itemTxt, itemBuilder);
System.out.println("success");
What is the expected output?
success
What do you see instead?
com.google.protobuf.XmlFormat$ParseException: 1:27: Expected ">".
What version of the product are you using? On what operating system?
protobuf 2.2.0
protobuf-java-format 1.1
OS: win32
Please provide any additional information below.
Tokenizer works not correctly.
Way of fix in attached patch.
Original issue reported on code.google.com by [email protected]
on 18 Dec 2009 at 8:55
Attachments:
What steps will reproduce the problem?
1. Try to convert a <object><string></string></object> into java instance,
will get a object.string equal to "</" instant of expected ""
Fix:
--- XmlFormat.java 2010-06-03 17:06:18.000000000 +0100
+++ XmlFormat.java 2010-10-14 17:52:51.500656265 +0100
@@ -392,6 +394,14 @@
return currentToken.length() == 0;
}
+ public String getCurrentToken() {
+ return currentToken;
+ }
+
+ public void setCurrentToken(String string) {
+ currentToken = string;
+ }
+
/**
* Advance to the next token.
*/
@@ -857,6 +867,10 @@
tokenizer.consume(">");
+ if(tokenizer.getCurrentToken().equals("</")) {
+ tokenizer.setCurrentToken("");
+ }
+
Object value = handleValue(tokenizer, extensionRegistry, builder, field, extension);
if (field.isRepeated()) {
PS: Also should allow for short tags e.g. <data/> not only <data></data> which
will be possible with this fix.
Original issue reported on code.google.com by [email protected]
on 14 Oct 2010 at 5:01
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.