Extend Filter function with cross-record boolean logic about gridify HOT 12 CLOSED

Hi,
I combined a few ideas and now in gridify v2.4.0 we have a newer way to manually create our queries using QueryBuilder class. the good news is QueryBuilder has a few APIs that supports condition evaluation.

var builder = new QueryBuilder<Level1>()
                 .AddCondition("Rank=1,Val1<5")
                 .AddCondition("Rank=2,Val1>5");

var evaluationResult = builder.Evaluate(array);

Using a single string

Now because we found a way to evaluate multiple conditions if you want to somehow use a single string you can write your own logic or separator.
e.g using +:

void AddConditionsToBuilder<T>(string conditions, string seprator, IQueryBuilder < T > builder) =>
       conditions.Split(seprator).Select(c => builder.AddCondition(c));

var conditions = "(Rank=1,Val1<5)+(Rank=2,Val1>5)";
var builder = new QueryBuilder<Level1>();
AddConditionsToBuilder(conditions,"+",builder);
var result = builder.Evaluate(array);

Using builder to get Filtering expression

I suggest instead of using the mapper and gridifyQuery from now on use the new QueryBuilder

var compiledExpression = builder.BuildFilteringExpression().Compile();

from gridify.

It doesn't work for filtering as all conditions cannot be met. I was thinking more a boolean condition evaluator, it's close in functionality but outcome is different.

Some of this can be evaluated in code but trying to do it just via the Filter is more desirable. Ability to apply logic to a data set and determine if all of the rules are true/false.

Example1:

Level1[] array = new Level1[3];
            array[0] = new Level1() { Rank = 1, Val1 = 4.3 };  // true
            array[1] = new Level1() { Rank = 2, Val1 = 5.2 };  // true
            array[2] = new Level1() { Rank = 3, Val1 = 7.5 };

Filter = "(Rank=1,Val1<5),(Rank=2,Val1>5)"
Logic = Condition1 AND Condition2 [where condition1 is chained logic (Rank=1,Val1<5)]
Result = true

Example2:

Level1[] array = new Level1[3];
            array[0] = new Level1() { Rank = 1, Val1 = 4.3 };  // true
            array[1] = new Level1() { Rank = 2, Val1 = 4.9 };  // false
            array[2] = new Level1() { Rank = 3, Val1 = 7.5 };

Filter = "(Rank=1,Val1<5),(Rank=2,Val1>5)"
Logic = Condition1 AND Condition2
Result = false

Example3:

Level1[] array = new Level1[3];
            array[0] = new Level1() { Rank = 1, Val1 = 4.3 };   // true
            array[1] = new Level1() { Rank = 2, Val1 = 4.9 };   // false
            array[2] = new Level1() { Rank = 3, Val1 = 7.5 };

Filter = "(Rank=1,Val1<5) | (Rank=2,Val1>5)"
Logic = Condition1 OR Condition2
Result = true

from gridify.

You understand correctly. This is a feature suggestion to make a rule builder rather than a filter (ie the filter/rule string may not return any records but still be valid). I believe there is quite an extensive use-case for this as I'm often asked for complex queries which cannot be done in a simple filter, the performance of Gridify makes could make this more achievable.

from gridify.

I think using the | (or) operator can solve this problem. "(Rank=1,Val1<5)|(Rank=2,Val1>5)"

if not, please provide your expected LINQ query for Example3.

from gridify.

It's not strictly OR since both values must be true. The nearest LINQ I can think of at the moment is

bool linqExpression = array.Count(x => (x.Rank == 1 && x.Val1 < 5) || (x.Rank == 2 && x.Val1 > 5)) == 2;

but really I am trying to evaluate if both
(x.Rank == 1 && x.Val1 < 5) == true AND (x.Rank == 2 && x.Val1 > 5) == true

Not sure if it's possible.

That's why I don't think it's a FilterExpression but a new feature such as LogicExpression where certain rules can be appended and determined if true/false.

from gridify.

according to your example if we use OR the problem can be solved easily:
e.g

	var query3 = new GridifyQuery() { Filter = "(Rank=1,Val1<5)  |  (Rank=2,Val1>5)" };
	var expression3 = query3.GetFilteringExpression(gridifyMapper).Compile();

	var HasAny = array.Any(expression3);   // get bool result from query
	var Count = array.Count(expression3);  // count the results

the thing is expressions are not only a filtering operation, you can use an expression to do whatever you want.

but really I am trying to evaluate if both
(x.Rank == 1 && x.Val1 < 5) == true AND (x.Rank == 2 && x.Val1 > 5) == true

also, as you said before (there cannot be any records that are ranked both 1 and 2) so we can not use AND between our two conditions here. I am confused what are you asking

from gridify.

So if I understood correctly you don't want to get any results back from your array. you just want to evaluate the whole collection to make sure certain conditions passed or not. the problem is unfortunately, this feature doesn't have any general use-case in most applications that's why I don't think adding this to gridify be a good idea.
another problem is this isn't really a single operation: (this is linq version of what you described)

var x = array.Any(a => a.Rank == 1 && a.Val1 < 5) && 
        array.Any(a => a.Rank == 2 && a.Val1 > 5);

gridify is fully compatible with ORMs and LINQ itself, having this feature breaks this compatibility.

I don't close this issue, maybe later someone or I found a solution to have this in a single Linq query.
thank you for your contribution.

from gridify.

This is brilliant! Thank you so much, this library is excellent.

Only have one question... :)

Is it possible to compile the QueryBuilder so you get the same performance as the FilteringExpression?

from gridify.

Thank you,

Is it possible to compile the QueryBuilder so you get the same performance as the FilteringExpression?

You still can compile any expression that returns from QueryBuilder but you can not compile the builder.

The output of these methods can be compiled since both returns an Expression.

• BuildFilteringExpression()

Also, feel free to give your feedback.
I've created a discussion about these new features:

#40
#1

from gridify.

Forget my last comment,
In v2.4.3 we have three special build methods for the compiling builder.

• BuildCompiledEvaluator
• BuildCompiled
• BuildWithPagingCompiled

from gridify.

I hope you don't mind my testing and feedback, you are doing great work here.

On the BuildCompiledEvaluator, I'm seeing quite a large difference in performance between doing the actual filter/eval after the expression has been compiled.

Example1:

var filterQuery = new GridifyQuery() { Filter = "SomeFilter" };
var compiledFilterExpression = filterQuery.GetFilteringExpression(filterMapper).Compile();

// --> Measure performance from here

var filter1 = arrayToFilter.Where(compiledFilterExpression);
var match = filter1.Any();

// --> Measure performance to here

Example 2:

var builder = new QueryBuilder<TestClass>()
            .AddCondition("SomeFilter");

var compiledEvaluator = builder.BuildCompiledEvaluator();


// --> Measure performance from here

var evaluate = compiledEvaluator(arrayToFilter);

// --> Measure performance to here

Example1: circa 50µs
Example2: circa 8000µs

I can't provide any data right now but may get some time next week.

from gridify.

I hope you don't mind my testing and feedback, you are doing great work here.

No, I love feedbacks :). just try to open another issue if it is a different topic or use discussions.

in the first example, you are filtering the data in the second one you evaluating conditions, these two are completely different operations. you can compare BuildCompiled with GetFilteringExpression these two are very much the same although BuildCompiled has extra ordering and paging checks. (eg. Benchmark)

update:
also, you were right I found a problem in CompiledEvaluator that will be fixed in the next version

from gridify.