ahausladen / jsondataobjects Goto Github PK

Hi, actually you can't do the following:

var
  v: variant;
begin
  v := jsonobject['my name']; 
end;

It would be nice to have a new property like:

v := jsonobject.V['my name'];

To return value to a variant with the datatype stored in json datavalue. Of course this should raise an exception if you are trying to cast an object or an array.

It would be useful to to have an overload of TObjectArray.Add procedure to add a variant value to an array using the vartype of the value to determine json datatype.

Actually I do this with 2 procedures, I don't want to change your source code because later is a madness to maintain updates.

With best regards
Christian

Can you add Marshalling support for record and arrays too ?
(like in https://github.com/onryldz/x-superobject)

Thank you

Hi Andy,

This library is not functioning correctly on iOS 64 bit targets (ARM64). All other targets seem to work fine. I have some sample code at the end of this post.

I have dug a bit and found this so far:

class procedure TJsonBaseObject.StrToJSONStr(const AppendMethod: TWriterAppendMethod; const S: string);
2266: //EndP := P + Length(S);
2267: EndP := P + PLongInt(@PByte(S)[-4])^;

Line 2267 is calculating huge EndP values in ARM64 and causing the while loops to crash. However, if I comment out line 2267 and use line 2266 then EndP correctly points to the line end.

Next:

procedure TJsonOutputWriter.TJsonStringBuilder.DoneConvertToString(var S: string);
6803: Pointer(S) := P; // keep the RefCnt=1

In ARM64 after line 6803 executes S is not updated and still points to the blank text it was initialized to earlier in this method. Line 6803 seems to have no effect when investigated in the debugger.

Sample code:

procedure TForm1.Button1Click(Sender: TObject);
var
jo: TJSONObject;
begin
jo := TJSONObject.Create;

jo.I['ACCOUNTID'] := 225;
jo.B['BOOLVAL'] := False;
jo.S['VERSION'] := '1.0';
jo.S['LOCATION'] := 'RVK';
jo.S['USERID'] := 'S.TESTER';
jo.S['GARBAGE#####'] := 'Ooga\booga';
jo.S['SUBTOCALL'] := 'G2.WEB.CONNECT.SUB';
jo.D['DT'] := Now;
jo.S['ACTION'] := 'CONNECT';

Memo1.Text := jo.ToJSON(true);
// Crash if line 2267 is active, no crash if line 2266 is active.
// When no crash then this always returns an empty string.

jo.Free;
end;

Following code rises Exception:

procedure TForm1.Button1Click(Sender: TObject);
var
jso: TJsonObject;
begin
jso := TJsonObject.Parse('{"ml_model": "J250'#$A'IMEI"}') as TJsonObject;
jso.Free;
end;

Project Project1.exe raised exception class EJsonParserException with message 'Invalid character in string (1, 19)'.

I checked that string complies with the JSON string specification.

Hi!

I have a 100Mb Json file, it takes a while to read. It is possible to show a progress state?

Thanks.

XE3
JsonDataObjects.pas(2117) Error: E2003 Undeclared identifier: 'TryStrToUInt64'

I'm working on a package manager, (currently called DPM, might change), I create a pull request to add dpm support for jsondataobjects - it just adds the a dspec file which defines what goes in the package.

Can I ask that you also add "dpmpackage" as a topic to the repo (manage topics under the repo name). That will make it possible to find it when searching for dpm packages.

FWIW, I'm using jsondataobjects in DPM itself.

/*1.json
{"receiver":{"type":"single","id":"lisi"},"sender":"zhangsan","msgtype":"text","text":{"content":"Holiday Request For Pony(http://xxxxx)"}}
*/
TJsonObject *Json=new TJsonObject();
Json->LoadFromFile(L"1.json");
ShowMessage(Json->O[L"text"]->S[L"content"]);
delete Json;

[ilink32 Error] Error: Unresolved external '__fastcall Jsondataobjects::TJsonObject::GetObjectW(System::UnicodeString)' referenced from UNIT1.OBJ

Compile environment : Win10 64bit & C++Builder 10.2.3 Community Edition

In addition, please add {$HPPEMIT ' #pragma link "Jsondataobjects"} to the source code for ease of use of CB users, thank you.

I have to serialize and transmit data structures like this:

TFoo = class
  Html: WideString;
  FileContent: UTF8String;
end;

e.g. The producer may be Internet Explorer or IDE which limits the type/encoding.

It seems that I have to cast them to/from string in both sides. Since the content is pretty large, it would be great if the object model could support these two types natively. It should be also more efficient as I use UTF8-encoding for transmission.

From my benchmark JsonDataObjects is the fastest lib for read write json in delphi, It's awersome! but is poor in documentation.

I'm searching for examples for writing large json (over 50000 nodes, less or more 90 MB) with the best performance.

Eg. is better writing the objects with the full path:

var
  Obj: TJsonObject;
  i: integer;
begin
  Obj := TJsonObject.Create;
  for i := 0 to 50000 do begin
     Obj['foo']['bar']['baz'].A['array'].Add(i);
  end;
  Obj.Free;
end;

or write by caching the parent, eg.:

var
  Obj: TJsonObject;
  aArray:  TJsonArray;
  i: integer;
begin
  Obj := TJsonObject.Create;
  aArray := Obj['foo']['bar']['baz'].A['array'];
  for i := 0 to 50000 do begin
     aArray.Add(i);
  end;
  Obj.Free;
end;

other trick and tips are welcome!

I'm using JsonDataObjects to serialize DTO to send to Java REST Server, but the Java serializer does not recognize Names with UpperCaseStartingCamelCase, it must be lowercaseStartingCamelCase, and I cannot change the property names in Delphi Objects, I have to follow the Delphi Naming convention.

To workaround this problem, I changed TJsonObject.FromSimpleObject, to convert property name

if StartNamesLowerCase then
  PropName := LowerCase(UTF8ToString(PropList[Index].Name).Chars[0]) + UTF8ToString(PropList[Index].Name).Substring(1)
else
  PropName := UTF8ToString(PropList[Index].Name);

Is possible to have this feature built in?

Is the current master branch the beta 1.0 release?
For XE7 I get an compile error: [dcc32 Error] JsonDataObjects.pas(3894): E2036 Variable required
oops forget it my mistake, sorry

Sometimes values are encoded in Base64. After parsing file there is a need to decode such value. It would be nice if JsonDataObjects library provided function to decode the value.

Parsing seems to fail for all Int64 values x where MaxInt < x <= 2*MaxInt + 1. Other values work fine. The problem seems to be that FLook.HI = 0 and therefore FLook.I is used instead of FLook.L. Unfortunately I don't know how to fix it so I can't provide you with a pull request.

var
  lInput: String;
  lInt64: Int64;
  lJsonObject: TJsonObject;
begin
  lInt64 := 2 * Int64(MaxInt);
  lInput := Format('{ "num": %d }', [lInt64]);

  lJsonObject := TJsonObject.Parse(lInput) as TJsonObject;
  try
    Writeln(lInput); // { "num": 4294967294 }
    Writeln(lInt64); // 4294967294
    Writeln(lJsonObject.L['num']); // -2
  finally
    lJsonObject.Free;
  end;
end;

This is Delphi XE5.

Delphi Rio 10.3.2
TJsonObject.FromSimpleObject use GetPropList. But GetPropList works with only published.
Directive {$M+} doesn't work.
Help says GetPropList only works for components or component type.

When I use JsonDataObjects,pas (in Delphi XE) like this:

procedure doParseUTF8();
var
aJson: TJsonObject;
aJsonStr: string;
aUTF8Str: UTF8String;
begin
aJsonStr := '{"123":{"abc":{"efg":"hij"}}}';
aUTF8Str := UTF8Encode(aJsonStr);

aJson := TJsonObject.Create;
try
aJson.FromUtf8JSON(aUTF8Str);
ShowMessage(aJson.ToJSON);
finally
aJson.Free;
end;
end;

I get message '{"12":{"ab":{"ef":"hi"}}}'.
The Key and The Value all have been cut.

this Error also in parse from UTF8 file.

Hi, there is a bug when using .path to set values.
I can it reproduce with following code:

procedure TForm2.Button2Click(Sender: TObject);
var
  doc: TJsonObject;
  v: variant;
begin
  doc := TJsonObject.Create;

  doc.Path['ferrcod'] := 2;
  doc.Path['ferrmsg'] := 'Debe haber un usuario';

  Memo1.Lines.Text := doc.ToJSON(false);
end;

Instead of returning:

{
    "ferrcod": 2
    "ferrmsg": "Debe haber un usuario"
}

It returns:

{
    "ferrcod": "Debe haber un usuario"
}

I'm trying the find the problem but I still not found.

Looked through the unit tests, looked through the source code. Not seeing anything for TArray support ... this is my time looking at the code, so I may be missing something. Is it supported?

Amazing work on issue #18! I get a JSON string now in ARM64!

But something is wrong with the string reference count I think. When the JSON string that gets returned goes out of scope an exception is raised. Consider this code:

procedure TForm1.Button1Click(Sender: TObject);
var
jo: TJSONObject;
s: string;
begin
jo := TJSONObject.Create;
jo.S['USERID'] := 'S.TESTER';
s := jo.ToJSON(true); // Get a value now :)
s := ''; // Exception :(
jo.Free;
end;

Exception is also raised if I do not set the string to empty and the method ends (string goes out of scope). This is where the exception is:

function System._UStrClr()
24641: FreeMem(P);

Can you add UInt64 support?

Some real time web-services like Periscope use Net Time Protocol data format. It's UInt64 (explain).

Example:

{
    "message": {
        "channel": "blablabla",
        "ntpForBroadcasterFrame": 14567816253328083,
        "ntpForLiveFrame": 14567816253328083,
        "participant_index": 1,
        "signature": "blablabla",
        "signer_str_ntpForBroadcasterFrame": "14567816253328083",
        "signer_str_ntpForLiveFrame": "14567816253328083"
    },
    "channel": "blablabla"
}

Wrong test code:

var
  s1, s2: string;
  json: TJsonObject;
begin
  s1 := '{"long_val":15744383709429629494,"long_str":"15744383709429629494"}';
  json := TJsonObject.Parse(s1) as TJsonObject;
  s2 := json.ToJSON;
  Assert(s1 = s2);
end;

Sometimes the date given to JsonDataObjects is in UTC already, for example when generated by functions such as EncodeDate(). When a local time offset is then subtracted internally by JsonDataObjects you can end up with a day earlier and some hours offset.

Maybe another property additionally to D[], such as DUtc[] would work, so you can intentionally set some dates as UTC and some as local time/date.

TJsonSerializationConfig.UseUtcTime is not enough, since TzSpecificLocalTimeToSystemTime() will be called regardless of that setting.

Thanks for the very well designed interface, btw!

Any plans for JsonPath support - I'd be willing to add the support if you haven't already got something planned?

Hi Andreas,

I suggest to change this line:
https://github.com/ahausladen/JsonDataObjects/blob/master/Source/JsonDataObjects.pas#L2571

From:

Result := string(FValue.S) = 'true';

to:

Result := BoolToStr(string(FValue.S));

... this would allow to use "True" or "TRUE" also inside the JSON file...

Hi, when you parse a Json string, null values are stored as jdtObject instead of jdtNone.

Can be reproduced with following code:

procedure TForm2.Button2Click(Sender: TObject);
var
  doc: TJsonObject;
begin
  doc := TJsonObject.Create;
  try
    doc.FromJSON('{"ferrcod": 2, "ferrmsg": null, "ferrval": "pepe"}');
    if doc['ferrmsg'].Typ <> jdtNone then
      raise Exception.Create('Why???');
  finally
    doc.Free;
  end;
end;

This doesn't happen when values are assigned one by one:

procedure TForm2.Button2Click(Sender: TObject);
var
  doc: TJsonObject;
begin
  doc := TJsonObject.Create;
  try
    doc['ferrcod'] := 2;
    doc['ferrmsg'] := null;
    doc['ferrval'] := "pepe";

    if doc['ferrmsg'].Typ = jdtNone then
      raise Exception.Create('Now its ok');
  finally
    doc.Free;
  end;
end;

add feature, configure json serialization engine with option ignore fields with empty value

For instance https://github.com/bonecode/TBCEditor/issues/303 uses the objects, but when there is an error in the JSON, the exception is very generic, having a column/line number would help figure out what's wrong.

Minimally this would be for raw JSON syntax errors (like an unbalanced array), and ideally at the DOM level, so that the application that uses the DOM can provide meaningful error (like "field whatever line xxx should be a string")

Hi,
Thank you for your great work. Do you have any plan to implement "Object schema validation", similar to https://github.com/hapijs/joi

Thank you

In TestJsonDataObjects.pas, in procedure TestTJsonBaseObject.TestVariant, there's a call to CompareFloatRelative().

I have searched the Net and all my code repo (of about 20 years) and I couldn't find it.

It turns out it exists in XE10 (and maybe in other versions above XE2, I didn't check) and since it is MPL licensed, I am copying the relevant code below --in case anyone else needs it.

BTW, these are overloaded.

function CompareFloatRelative(expected, actual: Extended) : Boolean;
begin
  Result := CompareFloatRelative(expected, actual, 0.0000001); //This is a 99.99999% accuracy
end;

function CompareFloatRelative(expected, actual: Extended; RelativeError : Extended) : Boolean;
begin
  Result := False;
  if expected = actual then  := True
  else if abs((actual - expected) / expected) < RelativeError then Result := True;
end;

now if we try to cast a null value to any type of data an exception is raised:

var
  doc: TJsonObject;
  v: variant;
begin
  doc := TJsonObject.Create;
  try
    doc['nullvalue'] := null;

    // any of this raise an exception
    v := doc['nullvalue'].Value;
    v := doc['nullvalue'].IntValue;
    v := doc['nullvalue'].LongValue;
    v := doc['nullvalue'].FloatValue;
    v := doc['nullvalue'].DateTimeValue;
    v := doc['nullvalue'].BoolValue;

  finally
    doc.Free;
  end;
end;

Why don't we cast nulls like if it is jdtNone?

Also DataSet fields cast nulls to required datatypes:

  // if DataField.IsNull
  v := DataField.AsString;  // this return empty string
  v := DataField.AsInteger;  // this return 0
  ...

Also VarToString function:

  v := VarToString(null);   // this return empty string

I would like to implement this but I like to ask you first.

Regards
Christian

When I clone '{ "foo": [ "bar", {}, null, true, false, { "keaa": "value" } ] }', my result at "ClonedObj.ToJSON(True)" is '{"fo":["ba",{},null,true,false,{"ke":"valu"}]}'.

Analyzing, can you see that keys are resulting without the last char.

I'm using the example shown in the Readme .

Would be great to have Linux support for JsonDataObjects. In 10.2 Tokyo, ASM sections are not supported on Linux.

[DCC Error] JsonDataObjects.pas(1386): E2003 Undeclared identifier: 'UtcDateTime'

Hello,
The SaveFromStream works, but LoadFromStream raise an exception.

Right now, there are several JSON implementations available. Whats the difference of this among others? Is it speed, scalability or a low footprint?

• JSON delphi library uLkJSON.pas
• Integrated System.Json
• ISuperObject implementations hgourvest/superobject or onryldz/x-superobject

TStringIntern.Init contains the following code:

FCount := 0;
FCapacity := 16;
GetMem(FStrings, FCapacity * SizeOf(FStrings[0]));
FCapacity := 17;
GetMem(FBuckets, FCapacity * SizeOf(FBuckets[0]));
FillChar(FBuckets[0], FCapacity * SizeOf(FBuckets[0]), $FF);

This sets the length of FStrings to 1 less than FBuckets. In TStringIntern.InternAdd, TStringInterm.Grow is only triggered when the size reaches FCapacity (now 17):

if FCount = FCapacity then
Grow;
Index := FCount;
Inc(FCount);

Bucket := @FBuckets[(AHash and $7FFFFFFF) mod FCapacity];
with FStrings[Index] do
begin
Next := Bucket^;
Hash := AHash;
Pointer(Name) := Pointer(S);
Inc(PLongint(@PByte(Name)[-8])^);
end;

This means that when FCount reaches 16, writes are performed to FStrings[16], which is beyond the memory allocated during Init.

There seems to be some support for auto ref counting but when i define AUTOREFCOUNT can't compile project in D2009

[DCC Error] JsonDataObjects.pas(2257): E2003 Undeclared identifier: 'FRefCount'
[DCC Error] JsonDataObjects.pas(2270): E2003 Undeclared identifier: 'FRefCount'
[DCC Error] JsonDataObjects.pas(2777): E2003 Undeclared identifier: '__ObjRelease'
[DCC Error] JsonDataObjects.pas(2783): E2003 Undeclared identifier: 'AtomicIncrement'
[DCC Error] JsonDataObjects.pas(2783): E2003 Undeclared identifier: 'FRefCount'
[DCC Error] JsonDataObjects.pas(4677): E2003 Undeclared identifier: 'FRefCount'
[DCC Error] JsonDataObjects.pas(4690): E2003 Undeclared identifier: 'FRefCount'

[DCC Error] JsonDataObjects.pas(1468): E2003 Undeclared identifier: 'TSystemTime'
[DCC Error] JsonDataObjects.pas(1470): E2003 Undeclared identifier: 'DateTimeToSystemTime'
[DCC Error] JsonDataObjects.pas(1472): E2029 ']' expected but identifier 'wYear' found

But for Windows 32 , above errors does not appear.

This Json:
{"Cards":[{"PAN":"456789","Status":51},{"PAN":"123456","Status":21},{"PAN":"123456","Status":21}]}
raised exception when loaded and parsed!

Check the position of semicolon in function parameters

Accidentally stumbled upon this:

Should that line not use JsonMemInfoMainBlockStart instead of JsonMemInfoBlockStart?

Sample json:

{
"promotions":[
{
"id":1,
"title":"Special offer",
"start_at":"2014-11-01 00:00:00",
"finish_at":"2014-11-05 00:00:00",
"promotion_type":1,
"promotion_value":5000,
"promotion_on_type":1,
"products":[
1,
2
],
"promotion_on_value":[
1,
2
],
"activate":2,
"created_at":"2011-09-22 01:15:35"
},
{
"id":2,
"title":"Winter sale",
"start_at":"2014-12-01 00:00:00",
"finish_at":"2014-12-30 00:00:00",
"promotion_type":2,
"promotion_value":25,
"promotion_on_type":1,
"products":[
1,
2
],
"promotion_on_value":[
1,
2
],
"activate":2,
"created_at":"2011-09-22 01:15:35"
}
]
}

It is often not necessary to create DOM (JsonObject) when serializing objects to a stream. It is more efficient to expose a public TJsonWriter to perform such task.

TJsonOutputWriter is used internally so far. Also, UTF8String should be considered in this part.

Hi,
The JsonDataObjects can't parse this json string:

[1,2,3,\"This is a test\",{\"a\":\"a\"}]

You can get that json string by executing this code in Firefox or Chrome:

JSON.stringify([1, 2, 3, "This is a test", {"a": "a"}]);

When trying to use JsonDataObjects we found that on one of ours computers it gives in XE7 (without Update 1) three E2036 Variable required errors. These are on lines 1965, 3755, 3760. This happens only on this one computer (not reproducible on others (XE7 Update 1 installed)) but we are unsure what causes this.

How to get each item(Object, not a single Value) from the ARRAY?
thks!

I'm interested in implement a jdtDateTime type to handle of DateTime values in the same way that Float values are handled.
This will let us to transparently handle DateTime values instead of manually convert or format them when needed.
This also save memory because it will store a double value instead of 24 chars.
Some examples of use could be:

procedure TForm2.Button2Click(Sender: TObject);
var
  doc: TJsonObject;
  variantvar: variant;
  datetimevar: TDateTime;
  stringvar: string;
begin
  doc := TJsonObject.Create;
  try

    // assign a datetime value and internally store as Double, with Typ = jdtDateTime
    doc['mydatetimevalue'] := now;
    doc.D['mydatetimevalue'] := now;

    // return value as TDateType
    variantvar := doc['mydatetimevalue'];
    variantvar := doc['mydatetimevalue'].DateTimeValue;
    variantvar := doc.D['mydatetimevalue'];

    // return value as TDateType
    datetimevar := doc['mydatetimevalue'];
    datetimevar := doc['mydatetimevalue'].DateTimeValue;
    datetimevar := doc.D['mydatetimevalue'];

    // cast to string to standard JsonFormat
    stringvar := doc['mydatetimevalue'];
    stringvar := doc['mydatetimevalue'].Value;

    // output dates as standard JSON format
    doc.ToJSON; // will output -> {"mydatetimevalue": "yyyy-mm-ddThh:mm:ss.zzzZ"}

  finally
    doc.Free;
  end;
end;

I think that this doesn't broke or change anything. But I want to hear your opinions before start to work.
If you agree I will do it in my fork and then I'll put a pull request.

Can Delphi 2007 be supported，Downward compatibility with lower versions!

[DCC Error] JsonDataObjects.pas(2004): E2089 Invalid typecast
[DCC Error] JsonDataObjects.pas(2009): E2089 Invalid typecast
[DCC Error] JsonDataObjects.pas(2052): E2089 Invalid typecast
[DCC Error] JsonDataObjects.pas(6793): E2037 Declaration of 'Realloc' differs from previous declaration
[DCC Error] JsonDataObjects.pas(6797): E2003 Undeclared identifier: 'Capacity'
[DCC Error] JsonDataObjects.pas(6800): E2003 Undeclared identifier: 'FBytes'
[DCC Error] JsonDataObjects.pas(6803): E2008 Incompatible types
[DCC Error] JsonDataObjects.pas(6816): E2008 Incompatible types
[DCC Error] JsonDataObjects.pas(899): E2065 Unsatisfied forward or external declaration: 'TJsonBytesStream.Realloc'

Does the code currently only work for 32-bit targets? (I was compiling it for iOS)

Hi, I had some problems compiling the TBCEditor control, because of a compilation error with XE7 (no updates installed).

See: https://github.com/bonecode/TBCEditor/issues/400

The fix was to remove "overload" from TJsonStringBuilder.AppendIntro (which doesn't have any overloads). Without that change, XE7 doesn't seem to understand that you try to pass a method pointer instead of executing it. It's probably a bug in the compiler, but the fix seems to be easy and I don't think it'll affect other versions.