social-native / elastic-composer Goto Github PK

Created because of this
Type(s) for the inclusion option(s) should be declared and exported.

export type InclusionKind = 'include' | 'exclude';

I suspect that network faults during effect processing can lead to stale state data might be an issue. There needs to be some validation if this is in fact a problem.

If it is a problem, I think a few solutions exist:

• Add retry logic with some sort of back off to effect processing. Expose a hasNetworkFault attribute to indicate that the UI is out of date

or

• Add a concept of request and response filter state. (This is a large change and dangerous b/c test coverage is very poor!)
  • The ui is driven off the response state
  • Effect requests are driven off the request state
  • Submission of the effects successfully updates filter response states

Thanks to @dearsaturn for suggesting this.

There should be the ability to config the Composer with an API for a non-elasticsearch source that supports the Elastic Composer Suggestion API. Then a suggestion can be configured to use this interface. The request should be put on the side effect queue like everything else to ensure debouncing etc...

Example use case:
Use Google Knowledge Graph to get suggestions
Use a different ES index to search text/keywords whose corresponding IDs can then be used in the Filter API

This could be used for surfacing data visualizations about the filtered documents

Ideally, the next release will include (some or all of):

• api documentation on exported types is added
• documentation on Important Notes is added - relating to:
  • purpose of aggsEnabled
  • importance of setting range intervals and what happens when they are too small
  • how filters and suggestions affect one another
• field aliases take precedence over field names if they are specified
• adds a History API for rehydration, saving global filter and suggestion state, and moving forward and backwards in time.
• adds Order API
• adds third param to setFieldFilter to parity ES filter options

Created because of this.

The library seems to reverse the order of the response from elastic search when generating a list of fuzzy suggestions.

For example

const field = 'tags';
const results = composer.suggestions.fuzzy.fieldSearches[field];
// 'results' is something like this. 
// Notice how suggestion: 'food' is the last entry. 
// In the network request, it is the first entry.
// [
//     ...other results before,
//     {
//         suggestion: 'Food & Bev',
//         count: 2995
//     },
//     {
//         suggestion: 'food',
//         count: 3530
//     }
// ];

With a corresponding network response looking something like this:

{
  "took": 16,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 4063,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "tags__fuzzy_suggestion": {
      "doc_count_error_upper_bound": 138,
      "sum_other_doc_count": 11949,
      "buckets": [
        {
          "key": "food",
          "doc_count": 3530
        },
        {
          "key": "Food & Bev",
          "doc_count": 2995
        },
        // ... Other entries
      ]
    }
  }
}

The consequence is that the result must be "un" reversed by elastic composer users.

I believe it should be one or the other, right? It generates a warning on npm install.

When calculating active subFitlers it can be a pain to figure out what filter type has subFilters. There should be a standard type or a getter that each filter implements to handle subfilters

This library currently does not support nested object filters. Additionally, I believe the current abstractions are partially incompatible with nested objects. Below is a proposal to support it:

Introspection step:

• The object key should return a nestedObj type.

• For each key in the nested obj:

  { my_parent_key: { 
       nested_key_1: <some value>,
       nested_key_2: <some value>,
       nested_key_3: <some value>
  }
  
  

  The fields and their type would be:

  my_parent_key.nested_key_1 === 'nestedObj'
  my_parent_key.nested_key_1 === 'nestedObj'
  my_parent_key.nested_key_1 === 'nestedObj'
  

  Filter step:

  A new attribute is added to the nestedObj filter kind called nestedObjMap. This is a map of parent keys to the nested object. This allows you to tell that the 3 keys above are all part of the same nested object.

   {
      my_parent_key.nested_key_1: 
            { my_parent_key.nested_key_1: <type>  my_parent_key.nested_key_2: <type>,  my_parent_key.nested_key_3: <type> },
      my_parent_key.nested_key_2: 
            { my_parent_key.nested_key_1: <type>  my_parent_key.nested_key_2: <type>,  my_parent_key.nested_key_3: <type> },
      my_parent_key.nested_key_3: 
            { my_parent_key.nested_key_1: <type>  my_parent_key.nested_key_2: <type>,  my_parent_key.nested_key_3: <type> },
   }
   
  

  When a nested obj field filter is selected, the sibling fields are looked up in the nestedObjMap. This allows the filter to know which types are available on the nested object and which fields should be grouped together b/c they are part of the same nested object.

  From here, the logic is similar to a terms filter....i think.

In the above code, a new method was added to the mapping parser to support a specific Elasticsearch version. This pattern presents issues b/c it requires conditional logic to use the method (you have to know which version you want and which methods go with which version...). The usual solution would be to add a new mapping parser implementer (specific to a ES version) that conforms to the mapping parser interface.

@markrsocialnative @dearsaturn

Implementors of various interfaces should have typeguards. These should also be documented in the README

Initial thoughts are that it should should have methods:

• addToSort: (fieldName: string) => void
• incrementSortOrder: (fieldName: string) => void
• decrementSortOrder: (fieldName: string) => void
• order: { [fieldName: string]: number } // number is 1+
• removeFromSort: (fieldName: string) => void

Also, sort should be an object accessible from the manager

I'm not sure if this has caused any problems yet, but it raises some concerns. JSON.stringify processes keys in order of insertion (see rfc), so it's inherently unstable. A more reliable solution would be to compute a hash for each state and use that for comparison instead. There are also a few packages that implement a stable JSON.stringify but I don't know about quality/reliability.

Add a suggestion that can get the top N terms, with N as input

The history API should probably:

• be a discrete class that takes a manager object
• react to filter or config changes
• store filters and config in a serialized format
• reactions to changes should be throttled (500 ms?)
• have the methods:
  • forward
  • backwards
  • clearHistory
• have the attributes
  • history
  • locationInHistory
• have persistent storage via a client
  • the clients could be graphql based or local storage based

Changes to the existing code to support this are:

• addition of a serialize method to the manager
• addition of a rehydrate method to the manager (rehydrates a serialized object)