gregsdennis / manatee.json Goto Github PK

IL generation is supported in .Net Core (since net standard 1.1). The TypeGenerator class needs to be updated to support the framework.

See branch feature/type-generation-in-core.

This occurs when downloading the schema from the online source, then deserializing and using it to validate some JSON.

The problem seems to stem from something in the JsonSchemaReference dereferencing logic and the root.

Serialization and deserialization needs better error reporting, as is implemented in the parsing process.

Arkadiusz Wasniewski (itmind on BitBucket) wrote:

When json schema has:

"type": {
  "type": "string",
  "enum": [
    "FeatureCollection"
  ]
}

then returns StringSchema but I think it should be EnumSchema because, like in example above, type is enum but elements of enum could be type of string

Arkadiusz Wasniewski (itmind on BitBucket) wrote:

When type value is array like:

 "street": {
    "description": "street",
    "type": [
      "string",
      "null"
    ]
  },

library returns "Manatee.Json.JsonValueIncorrectTypeException: Cannot access value of type Array as type String" when JsonSchemaFactory.Load is executed

Not sure what is causing this. Likely caused by one of the registry classes or some other static data.

Using the Template generator throws an exception when I try to serialize a class that has a nullable enum. The enum has values 1, 2, 3, 4.

Exception is occurring in EnumNameSerializer.Serialize. obj value is 0.

See attached log for details.

Originally created at 2016-02-17 19:11:04+00:00 (UTC) by Danoceline as a(n) major issue.

When assigning a null value from a variable, the null is converted to JsonValue.Null. But when assigning null directly, the (.Net) null value is used.

JsonValue json1 = null;  // this is actually null
string myVar = null;
JsonValue json2 = myVar;  // this is JsonValue.Null

These two statements should behave the same.

When type value is array like:

  "street": {
    "description": "street",
    "type": [
      "string",
      "null"
    ]
  },

library returns "Manatee.Json.JsonValueIncorrectTypeException: Cannot access value of type Array as type String" when JsonSchemaFactory.Load is executed

Originally created at 2016-04-01 13:54:19+00:00 (UTC) by itmind as a(n) major issue.

There is a json schema:

{
  "properties": {
    "id_person": {
      "type": "string"
    }
  }
}

JsonPath.Parse("$.properties.id_person") returns nothing. To workaround we have to use JsonPathWith.Name("properties").Name("id_person")

Originally created at 2016-04-04 07:59:22+00:00 (UTC) by itmind as a(n) trivial issue.

Currently, only index lists [0,1,3] or slices [1:7:2] are supported. The original JS implementation supports combining these within a single array operator:

$.key2[0,1::2]

I think this was an assumption when setting it up, thinking it operated similarly to the anyOf and allOf schema types.

This was discovered as part of #24.

This would be a class-level attribute which takes a JsonSchema. It would indicate to the deserializer that the JsonSchema needs to be validated before the JSON could be deserialized.

When an array of schema is provided, each item in the array is validated against the index-corresponding schema.

Found as part of #24.

When building a Path that contains a condition that uses a field, if the field is null at evaluation, an NRE is thrown.

Originally created at 2016-05-14 01:40:16+00:00 (UTC) by gregsdennis as a(n) major issue.

This would include support for things like BSON and comments.

Ideally, I'd like to update the parser system to allow for injection of sub-parsers from external code. This would mean that the additional parsers could live in a secondary library, leaving Manatee.Json to implement "pure" JSON.

Given the schema { "type" : "array" } the value "string" should fail, but doesn't.

Hi,

I have not found in the docs any reference to the Json-Schema official test suite. In order to be considered a full-fledged alternative to Json.Net schema this seems an unavoidable requirement.

Is it in your roadmap or this project is frozen now?
Best regards, and congrats for your work.

The JsonValue parsing logic has been updated for a while and no longer uses Manatee.StateMachine. This was done during the performance enhancements some time back.

However the reference remains as the JsonPath parsing logic still uses this mechanism. I'd like to update this logic to something similar to the JsonValue logic so that the reference can be completely removed.

I'll look at also addressing issues #10 and #11 during this refactor.

http://jsonselect.org/#overview

William Armstrong (Danoceline on Bitbucket) wrote:

Using the Template generator throws an exception when I try to serialize a class that has a nullable enum. The enum has values 1, 2, 3, 4.
Exception is occurring in EnumNameSerializer.Serialize. obj value is 0.

See attached log for details.
Exception.txt

Given a schema, create example JSON for use as a template.

Still need to build negative tests (invalid paths and such) around JsonPath.

Changing the test value in ValidateReturnsErrorOnInvalidUriFormat to test that also isn't a valid uri causes the test to fail.

Test after change:

public void ValidateReturnsErrorOnInvalidUriFormat()
{
	var schema = new JsonSchema {Type = JsonSchemaTypeDefinition.String, Format = StringFormat.Uri};
	var json = (JsonValue) "test";
	var results = schema.Validate(json);

	Assert.AreNotEqual(0, results.Errors.Count());
	Assert.AreEqual(false, results.Valid);
}

Instead (or along with) JsonValue.Null, perhaps the library could support a proper null .Net value to represent the JSON null.

When a schema that defines an id, that URI can be registered so that it's returned automatically. This should be fairly simple with the recent 5.0.0 refactor.

The following doesn't dereference properly.

{ "$ref" : "http://json-schema.org/draft-04/schema#/definitions/positiveInteger" }

This should download the draft-04 schema, then navigate into its definitions property to find positiveInteger.

If the input string is

{"hello":"worl

an IndexOutOfRangeException is thrown. This should be anticipated and translated to a JsonSyntaxException.

The build process is becoming a headache. I need to find a better way to support these frameworks.

This will eventually translate to Manatee.Trello as well.

See branch feature/multi-project for progress thus far.

Support for unity3d IL2CPP?

Hi, I've recently switched from Newtonsoft.Json to this package as I've found that you have got the best support for JSONPath expressions.

I have only a small problem in that if I apply a filter expression to a value that is not an array I get no results. This does work in the java and javascript implementations of JSONPath.

Using the standard book store model i'm trying to get the bicycle object.

This expression return the bicycle object fine:
$.store.bicycle

However with a filter expression it does not:
$.store.bicycle[?(1==1)]

if you apply the same filter expression to the book array its works:
$.store.book[?(1==1)]

I've created a .netfiddle to highlight my issue https://dotnetfiddle.net/LpISLu

The kind of things i'm trying to achieve is to only return the store if the price of the bicycle is less than 20
$[?($.store.bicycle.price < 20)]

Hi,

I'm getting an ArgumentNullException when using JsonSchemaFactory.Load(fileName).

Parameter name: source

Some of my stack trace:

       at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
       at Manatee.Json.Schema.JsonSchema.FromJson(JsonValue json, JsonSerializer serializer) in D:\temp\532b9b0\Manatee.Json\Schema\JsonSchema.cs:line 610
       at Manatee.Json.Schema.JsonSchemaFactory.FromJson(JsonValue json) in D:\temp\532b9b0\Manatee.Json\Schema\JsonSchemaFactory.cs:line 106
       at Manatee.Json.Schema.JsonSchema.FromJson(JsonValue json, JsonSerializer serializer) in D:\temp\532b9b0\Manatee.Json\Schema\JsonSchema.cs:line 624
       at Manatee.Json.Schema.JsonSchemaFactory.FromJson(JsonValue json) in D:\temp\532b9b0\Manatee.Json\Schema\JsonSchemaFactory.cs:line 106
       at Manatee.Json.Schema.JsonSchemaFactory.Load(String path) in D:\temp\532b9b0\Manatee.Json\Schema\JsonSchemaFactory.cs:line 78

I cannot show my schema due to it being a commercial product, but I'll see if I can write up similar a failing schema.

JsonSchema.Empty and JsonSchema.Draft04 should be read-only and throw exceptions when an attempt to alter them is made.

When id property contains localhost Manatee library returns error:

{
  "id": "http://localhost/json-schemas/address.json"
}

The problem is that regex is not good way to resolve Uri. I think library should use Uri.IsWellFormedUriString: Uri.IsWellFormedUriString("http://localhost/json-schemas/address.json", UriKind.Absolute)

Originally created at 2016-04-04 09:48:09+00:00 (UTC) by itmind as a(n) minor issue.

As an example

"some text\\"

should parse to a JsonValue with the string value of

some text

But it seems that the \" is taking precedence over the \\ causing the end of the string to not be found.

Currently, only property-friendly keys are supported. This would add support for keys like:

• First Name
• 2birds
• SaveMe!!

While not obviously useful, these are valid keys in JSON and should be supported.

http://www.trentrichardson.com/jsonsql/

Looks fairly straightforward.

May be a moot point since Manatee.Json is fully LINQ-compatible.

Consider following test:

[TestMethod]
[DeploymentItem(@"Files\IssueRefCore.json", "Files")]
[DeploymentItem(@"Files\IssueRefChild.json", "Files")]
public void UriReferenceSchemaTest()
{
	const string coreSchemaUri = "http://example.org/IssueRefCore.json";
	const string childSchemaUri = "http://example.org/IssueRefChild.json";

	string coreSchemaPath = System.IO.Path.Combine(TestContext.TestDeploymentDir, @"Files\IssueRefCore.json");
	string childSchemaPath = System.IO.Path.Combine(TestContext.TestDeploymentDir, @"Files\IssueRefChild.json");

	var coreSchemaText = string.Empty;
	var childSchemaText = string.Empty;

	using (TextReader reader = File.OpenText(coreSchemaPath))
	{
		coreSchemaText = reader.ReadToEnd();
	}

	using (TextReader reader = File.OpenText(childSchemaPath))
	{
		childSchemaText = reader.ReadToEnd();
	}

	var requestedUris = new List<string>();
	JsonSchemaOptions.Download = (uri) => 
	{
		requestedUris.Add(uri);
		switch(uri)
		{
			case coreSchemaUri:
				return coreSchemaText;
				
			case childSchemaUri:
				return childSchemaText;
		}
		return coreSchemaText;
	};
	var schema = JsonSchemaRegistry.Get(childSchemaUri);

	var testJson = new JsonObject();
	testJson["myProperty"] = "http://example.org/";
	   
	var result = schema.Validate(testJson);
	
	Assert.IsTrue(result.Valid);
	Assert.AreEqual(requestedUris[0], childSchemaUri);
	Assert.AreEqual(requestedUris[1], coreSchemaUri);
}

Where:

Files\IssueRefCore.json is:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "http://example.org/IssueRefCore#",
  "title": "Core Schema",
  "description": "",
  "definitions": {
	"myProperty": {
	  "type": "string",
	  "format": "uri"
	}
  }
}

and Files\IssueRefChild.json is:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "http://example.org/IssueRefChild#",
  "title": "Child Schema",
  "description": "",
  "properties": {
	"myProperty": { "$ref": "IssueRefCore.json#/definitions/myProperty" }
  },
   "required" : [ "myProperty" ]
}

this test fails.

"myProperty": { "$ref": "IssueRefCore.json#/definitions/myProperty" }

should download http://example.org/IssueRefCore.json and navigate to /definitions/myProperty but it tries to get http://example.org/IssueRefChild.json again instead.

It looks like JsonSchemaFactory.Load() it is trying to look up a level when searching for external schema files.
For example:

Two files, a.json and b.json are in the folder Game
If a.json references b.json like this: { "$ref": "b.json#" },

I get an error that the file cannot be found (and the path in the exception is pointing to Debug folder, not Debug/Game

I have to change it to:
{ "$ref": "Game/b.json#" },

I had been waiting for the project structure stuff to be completed before doing this, but that's done now, so I suppose I can do this.

This would be a property/field attribute that contains a JsonPath (probably as a string since only compile-time constants can be used in attributes) to the value in the data structure.

For instance, the given the object

{"first":"Greg","last":"Dennis","other data":{"location":"NZ"}}

you could model with the following attributed class

class Person
{
    [Path("$.first")]
    public string First { get; set; }
    [Path("$.last")]
    public string Last { get; set; }
    [Path("$.'other data'.location")]
    public string Location { get; set; }
}

The catch is that the path returns an array of values. There will likely have to be configuration around behavior when multiple values are returned (throw, take first, etc.).

This feature would enable serialization while parsing. This is extremely useful for very large JSON structures that may cause memory issues when the entire structure is loaded.

Suggested by Jesper on a StackOverflow question.

Originally created at 2015-12-26 22:41:48+00:00 (UTC) by gregsdennis as a(n) minor issue.

JsonPath should support returning a list of expanded paths to the results instead of just the list of results. This would apply to operators that can return more than one item, like wildcards and searches.

I don't know how I missed this.

Hi, I'm not sure if this a new bug, but I'm getting vague (or incorrect?) error messages with the latest release:

        var schema = JsonSchemaRegistry.Get(schemaPath);
        SchemaValidationResults results = null;
        results = _schema.Validate(@"{}"); // Or some other schema.
        foreach (SchemaValidationError error in results.Errors)
        {
            _logger.Error(error);
        }

Test schema:

{
  "$schema": "http://json-schema.org/draft-04/schema",
  "title": "JSON Schema for Universally Unique Identifiers (UUIDs).",
  "type": "object",
  "properties": {
  "uuid": {
  "description": "A universally unique identifier (UUID).",
  "type": "string",
  "pattern": "^([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})|[a-f0-9]{32}$
}

Result:

"Expected String; Actual String."

Edit:

Looks like it is only a problem when passing a string to Validate instead of JsonValue.Parse() (I may be wrong, though)

Edit 2:
Looks like it will also cause a valid schema to fail, unless you pass a JsonObject into Validate

JsonPath should support bracket notation for property names.

$['test']
$["test"]

Should add caching around the web client to prevent multiple downloads of schema from the same URL.

I ran some performance tests comparing Manatee.Json to Newtonsoft, and Manatee lost. Badly.

There are opportunities for making Manatee.Json (much) faster. This task is for finding and implementing those improvements.

As of publishing the parsing improvements:

• Newtonsoft beats Manatee in tests with 500+ iterations.
• Manatee prevails with less than 500 (singles are best).

It seems that Newtonsoft has some sort of caching mechanism that helps with rapid repeated serialization.

At 10K iterations, Manatee takes 10x the time of Newtonsoft. (This is down from ~50x.) The reverse is true for single iterations (Manatee takes 1/10 the time of Newtonsoft).

MyGet provides automated build/package/publish services. The aim is to auto-publish to private feed, then forward to Nuget.

Dependent upon work in task/multi-project branch.

William Armstrong (Danoceline on Bitbucket) wrote:

I didn't see any entries in the wiki on how to implement the IResolver with the Json Serializer. In particular I would like to be able to resolve my custom classes and still use the default resolver when type T is not my class.