ritterim / filter Goto Github PK

In src/Filter.Nest/SearchDescriptorExtensions.cs create a Filter<T> as a sibling to PostFilter<T>.

• Filter<T> can compute relevancy score for the results that will be returned (rather than all results returned prior to using PostFilter<T>).
• This may be useful with aggregation concerns as well.

Ref.

Currently, when a valid-result converting incoming string to IRange<T> happens, the binding context is set to the value of the RangeResult<T> and not its value—IRange<T>.

This leads to the ASP.NET Core framework reporting an error when taking RangeResult<T> and trying to make it IRange<T>.

{
    "message": "Validation Failed",
    "errors": [
        {
            "message": "The value provided was not valid",
            "messages": [
                "The value provided was not valid"
            ],
            "field": "..."
        }
    ]
}

Ref. https://github.com/elastic/elasticsearch-net

We need to add an extension which implements the same basic feature set for IEnumerable and IQueryable, but for SearchDescriptor<T> where T: class. Additionally, due to the nature of storing data in Elasticsearch which may be different than the representation of the incoming filter request, we also need to provide a mapping between properties. This will need to encompass things like completely different property names and case-sensitivity.

Currently, Range<T> requires a lower and upper-bound. What we want, however, are open-ended versions of both-bounds (i.e. (,1] or [1,)). This will require modifying IRange<T> and removing the covariance on the interface¹. I'm not too worried about the latter since it wasn't part of the original design requirement and added as an "unknown-use future" feature.

1: Invalid variance: The type parameter 'T' must be invariantly valid on 'RimDev.Filter.Range.Generic.IRange<T>.MaxValue'. 'T' is covariant.

This construct will be a single key/value in the querystring. It will be used in places where the client app. cannot provide a fully serialized filter-object using something like JSON.net.

Possible examples:

?<key>=<property1>:<value1>,<value2>;<property2>:<value1>
?<key>=<property1>:<value1>,<value2>|<property2>:<value1>

Similar to

If there is a text field in the data with values like:

1. Apples
2. Oranges
3. Apples and Oranges
4. Pears
5. Kiwi
6. Pears and Peaches
7. Tomatoes
8. Apples, Oranges, Pears and Peaches

• Passing in a string array of: { "PEARS", "peach" } should return 4, 6 and 8.
• Passing in an array of { "APPLes", "kiwi" } should return 1, 3, 5, 8.

It's possible that there needs to be a way to specify entire value matching vs substring matching. There may also need to be a way to specify case-sensitive vs case-insensitive.

My opinion is that case-insensitive and substring searches should be the default.

I was reading the ElasticSearch documentation and came upon this explanation of how bool clauses effect the relevancy score of hit in the result set.

GET /_search
{
  "query": {
    "bool": {
      "should": [
        { "match": { "title":  "War and Peace" }},
        { "match": { "author": "Leo Tolstoy"   }},
        { "bool":  {
          "should": [
            { "match": { "translator": "Constance Garnett" }},
            { "match": { "translator": "Louise Maude"      }}
          ]
        }}
      ]
    }
  }
}

Why did we put the translator clauses inside a separate bool query? All four match queries are should clauses, so why didn’t we just put the translator clauses at the same level as the title and author clauses?

The answer lies in how the score is calculated. The bool query runs each match query, adds their scores together, then multiplies by the number of matching clauses, and divides by the total number of clauses. Each clause at the same level has the same weight. In the preceding query, the bool query containing the translator clauses counts for one-third of the total score. If we had put the translator clauses at the same level as title and author, they would have reduced the contribution of the title and author clauses to one-quarter each.

https://www.elastic.co/guide/en/elasticsearch/guide/current/multi-query-strings.html

The approach taken in our implementation gives every filter property the same weight regardless of its actual logical weight.

That said, as long as we are using post filter in an "exact" match mentality, also known as Structured Search, then we should be ok.

Any reason you don't ToList() or ToArray() the intermediate property list at https://github.com/ritterim/filter/blob/master/src/Filter/Generic/IEnumerable%601Extensions.cs#L25

The expected behavior is to treat empty collections as faux-nullable for purposes of filtering.

Example of current behavior:

public class Person
{
    public char FavoriteLetter { get; set; }
}

public class PersonFilter
{
    public PersonFilter()
    {
        FavoriteLetter = new List<char>();
    }

    public IEnumerable<char> FavoriteLetter { get; set; }
}

var people = new List<Person>()
{
    new Person()
    {
        FavoriteLetter = 'a'
    },
    new Person()
    {
        FavoriteLetter = 'b'
    },
};

var filteredPeople = people.Filter(new PersonFilter());

// Zero instead of two.
filteredPeople.Count()

Right now, when searching for a PlanYear, you have to say:

PlanYear=[2015,2015]
or
PlanYear=[2015,2016]

From a usability standpoint on the API side, that is not ideal and it would be better if:

• Number ranges without inclusive/exclusive indicators should default to inclusive (i.e. 2015,2015 => [2015,2015])
• A hyphen should be supported instead of a comma, e.g. "2015-2015"
• Single numbers should automatically set the min/max to the same value, e.g. "2015"

@RIMBOT version auth
@RIMBOT health accounts