Comments (7)
Thanks for reporting this issue, @baaaaast !
I will try to debug this and get back to you.
from druid.
A couple days ago we upgraded to v30
. Started to notice this error too. It turned out there is increased number of failed tasks. Now thinking of rolling back to v28
, most because of kafka ingestion.
from druid.
A couple days ago we upgraded to
v30
. Started to notice this error too. It turned out there is increased number of failed tasks. Now thinking of rolling back tov28
, most because of kafka ingestion.
Did you try to set the propertydruid.indexer.tasklock.batchSegmentAllocation=false
to the overlord ?
from druid.
ruid.indexer.tasklock.batchSegmentAllocation=false
No we did not yet. Wondering how (much) it can hit the ingestion performance.
from druid.
I haven't seen any difference at all: no lag for millions of messages consumed and no extra load on the overlord side.
I just don't have any more failed tasks
from druid.
I haven't seen any difference at all: no lag for millions of messages consumed and no extra load on the overlord side. I just don't have any more failed tasks
Thanks, sure we'll try out and get back here. For the context, normally we ingest few millions messages per second.
from druid.
Setting druid.indexer.tasklock.batchSegmentAllocation=false
fixed the duplicate key
error.
Looking forward to the complete resolution of this github issue 👍
from druid.
Related Issues (20)
- Install on Apple M2 fails HOT 1
- `KafkaSupervisor` not entering idle mode when last message is sent inside a kafka transaction HOT 2
- Auto-compaction tasks are not submitted when single compact task runs for longtime HOT 1
- Coordinator is not calulating compaction taskSlots correctly HOT 2
- Ingestion using systemFields results in exception
- Provide/publish docker images based on debian:trixie? HOT 1
- useApproximateCountDistinct implicitly makes all aggregates using distinct unplannable HOT 5
- AVG function gives same result with or without distinct
- K8s MM-less Hadoop ingestion fails on K8sDiscoveryModule HOT 1
- Actively Reading tasks fails even though Pending completion tasks are a success HOT 2
- Planner optimization for comparison query HOT 6
- how to query data from deep storage through SQL HOT 2
- How to configure Transparent Application Continuity (TAC) with Oracle JDBC Thin and druid connection pool? HOT 1
- Non-coordinator services produce excessive logging from druid-basic-security extension HOT 1
- Metric taskSlot/idle/count doesn't exclude disabled workers HOT 1
- druid-deltalake-extensions support for StructType HOT 4
- Druid indexer tasks gets stuck in PUBLISHING state
- Druid Query Returning Different number of Results when Timezone with Period is used in Granularity HOT 1
- Datasources are fully available when no historical is present HOT 1
- 不支持add column if not exists语法,解析会报错 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from druid.