Comments (5)
mikegoatly
commented on June 7, 2024
2
Hi!
The = in a LIFTI query is used to restrict a search to a specific field. At the moment there's no syntax to escape that. If you don't need the full LIFTI query syntax, then this should work for you:
var index = new FullTextIndexBuilder<string>()
+ .WithSimpleQueryParser()
.WithObjectTokenization<MyModel>(
itemOptions => itemOptions
.WithKey(b => b.Title)
.WithField("Title", b => b.Title, tokenOptions => tokenOptions.CaseInsensitive().AccentInsensitive()))
.Build();
await index.AddRangeAsync(new List<MyModel>
{
new MyModel("1", "MyText=test3")
});
var results = index.Search("MyText=test3");
Console.WriteLine(results.Count());
public sealed record MyModel(string Key, string Title);I see what you were trying to do with the .IgnoreCharacters('=') part of the second example, but IgnoreCharacters actually strips a set of characters from the input as if they were never there, so that's definitely not what you want :)
from lifti.
stealthin
commented on June 7, 2024
1
Crystal clear! Thank you for your quick answer!
from lifti.
mikegoatly
commented on June 7, 2024
1
No problem - glad I could help. I just had another thought - you could have also worked around this using a manually constructed query:
var index = new FullTextIndexBuilder<string>()
.WithObjectTokenization<MyModel>(
itemOptions => itemOptions
.WithKey(b => b.Title)
.WithField("Title", b => b.Title, tokenOptions => tokenOptions.CaseInsensitive().AccentInsensitive()))
.Build();
await index.AddRangeAsync(new List<MyModel>
{
new MyModel("1", "MyText=test3")
});
+ var normalizedSearchText = index.GetTokenizerForField("Title").Normalize("MyText=test3");
+ var query = new Query(new ExactWordQueryPart(normalizedSearchText));
+ var results = index.Search(query);
Console.WriteLine(results.Count());That way you're bypassing the query parser completely and are being explicit about the fact you want the = in the word.
from lifti.
stealthin
commented on June 7, 2024
1
That makes sense! This is what I ended up doing 😄
from lifti.
stealthin
commented on June 7, 2024
If I test the following code:
var index = new FullTextIndexBuilder<string>()
.WithObjectTokenization<MyModel>(
itemOptions => itemOptions
.WithKey(b => b.Title)
.WithField("Title", b => b.Title, tokenOptions => tokenOptions.CaseInsensitive().AccentInsensitive().SplitOnPunctuation(false).IgnoreCharacters('=')))
.Build();
await index.AddRangeAsync(new List<MyModel>
{
new MyModel("1", "TEST=test3,TEST=othertest")
});
var tokenizer = index.GetTokenizerForField("Title");
var search = tokenizer.Normalize("TEST=test3");
var suggestions = GetSuggestions(search);
var results = index.Search(search);
Console.WriteLine(results.Count());
IEnumerable<string> GetSuggestions(string input)
{
using var navigator = index.CreateNavigator();
navigator.Process(input.AsSpan());
return navigator.EnumerateIndexedTokens().ToList();
}
public sealed record MyModel(string Key, string Title);
I get TESTTEST3,TESTOTHERTEST from suggestions, which is not what I'd like. It should be TEST=test3,TEST=othertest instead.
from lifti.
Related Issues (20)
- Remove dependency on System.Collections.Immutable HOT 2
- Suggestion: custom stemmers HOT 2
- Escaped characters in LIFTI query syntax HOT 1
- Q: is possible to fetch the whole document by Id? HOT 2
- Refresh documentation HOT 20
- Split IdPool and ItemStore HOT 1
- Consider switching to using ValueTask across the library HOT 1
- Operaterrors as a text HOT 3
- Standardize terminology
- Track source object type against a document's metadata
- Add a "not contains" query operator
- v6 documentation changes
- Create a standardised way of rehydrating an index from a serializer
- Write up implementing a custom serializer
- Apply field and document filtering when collecting results from IndexNavigator HOT 1
- Add README.md to nuget package
- Execution plans
- Consider dropping support for netstandard2
- How to nest queries? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lifti.