Comments (3)
We have also encountered a memory issue, causing replicas to slowly use up all configured memory amount that wasn't the case before with the regular redis server, without modules and no RediSearch indexing, so now we opted for a custom LUA Redis Function to do the aggregation, which calls KEYS prefix*, then loads the HashMaps with HGETALL, does the aggregation and returns an Array reply with Array items consisting of the tag and 4 counters.
The service performs at a stable 100ms response time, which is 2-3 times worse than it would be with FT.AGGREGATE, if it wouldn't slow down as max_doc_id increases.
I have looked at the BITFIELD type, which would be another way to implement this, but that would raise additional problems to solve around consistency, reading back data in pipelined workflows, in a use-case where everything changes every second.
from redisearch.
I had the same problem with the latest version(redis/redis-stack:7.2.0-v9):
Some simple JSON data (1K) was created with only 1 record.
Create an index (idx:Test) and then query the data through the index.
FT.SEARCH idx:Test *
(1) Queries are very fast at first.
(2) After trying to update about 1 million times, I started to stop updating the data and then query the data, which takes 200-400ms.
(3) I tried to delete the index and then re-create the index, which was to query the data again very quickly.
FT.INFO idx:Test
FT.DROPINDEX idx:Test
from redisearch.
We're using redis-stack server with only redisearch module enabled ver 20813. We observing similar behavior, where we have index with a lot of HSET
updates of indexed keys and constantly growing FT.AGGREGATE
execution time. We have ~300K keys that updated at average rate of ~800 hset/sec. Standalone Redis, no disk writes, Ubuntu 22.04 LTS server. After rebooting the Redis FT.AGGREGATE execution time is about 400 msec. It continuously growing and reaching 3000 msec after about 2 days of operation.
FT.CREATE metric-idx ON HASH PREFIX 1 metric:
SCHEMA pid TAG category1
TAG category2
TAG metric NUMERIC SORTABLE
cnt NUMERIC SORTABLE
upd NUMERIC SORTABLE
predictedGood NUMERIC SORTABLE
predictedBad NUMERIC SORTABLE
metric-idx * WITHCURSOR COUNT 1000 MAXIDLE 30000 GROUPBY 2 @category1 @category2
REDUCE MIN 1 @metric AS min_metric
REDUCE MAX 1 @metric AS max_metric
REDUCE SUM 1 @cnt AS count
REDUCE QUANTILE 2 @metric 0.5 AS median
REDUCE SUM 1 @predictedGood AS predictedGood
REDUCE SUM 1 @predictedBad AS predictedBad
from redisearch.
Related Issues (20)
- [BUG] Order of precedence not honored in APPLY functions with exponents HOT 1
- MOD-7133: [Feature Request] Add FT.ALIASGET command HOT 2
- L2 distance computation misunderstanding in documentation HOT 4
- [BUG] I can't run "make build" command successfully HOT 3
- Document Distributed Search (RSCoordinator) build/installation HOT 7
- Boost File Error when building 2.8.13 with Bullseye HOT 1
- [BUG] RediSearch HNSW indexing deadlock? HOT 2
- Facing build issue on PPC64LE architecture. HOT 3
- [BUG] APPLY substr function not using -1 count as documented - [MOD-6959]
- the results obtained after indexing are incomplete HOT 29
- Please Help Fix RSCoordinator So that Redis Search (RediSearch) Can Be Used Across Redis Cluster - module-oss.so initialization failed HOT 17
- Configuration of Custom Tokenizer HOT 1
- [BUG] Redis freezes and stops responding with 100% CPU Utilization while using redissearch with HNSW vector indexes HOT 2
- [Feature Request] int8 or float16 support for vector fields
- make cmd not working on linux system [RediSearch Installation] HOT 8
- [BUG] IDF change to 0 after writing the same text three times HOT 6
- [BUG] `disjoint` query for geoshapes yields incorrect results
- [BUG]The num_neighbours parameter of knn does not take effect HOT 1
- [BUG] High insert/delete, index with filter expression gradual slowdown HOT 4
- Filter negatively impacting scores HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from redisearch.