Comments (6)
@mshustov Cause Avro has only a few Primitive Types, but there's lots of data types in ClickHouse. If these types are ignored when using, exceptions will occur according to the current implementation.
It has several possible solutions, the first one is to mapping all types within ClickHouse data type. but i think it will cause lots of efforts. The second one is to point out which data types in ClickHouse is already implemented. The root cause of my exception is UINT64 which i'm using but it hasn't been covered, and it cannot pass the data schema validation.
Hope it works, the connector is helpful. thank you.
from clickhouse-kafka-connect.
I looked into the source code, and found the root cause is the type mismatch, Also in com.clickhouse.kafka.connect.sink.db.mapping.Column, it only includes only a few types in clickhouse.
If there's some avro type and clickhouse type mapping document, it will be helpful for the user.
from clickhouse-kafka-connect.
configuration issue, not a bug, maybe should deliver an enhancement
from clickhouse-kafka-connect.
configuration issue, not a bug, maybe should deliver an enhancement
@licongchao what would you suggest adding to the documentation to improve the user experience?
from clickhouse-kafka-connect.
The only types currently not supported by the Sink are bytes & struct as value. We introduce some implicit conversions within Dates see #63 issue [#57]. If you see any other need to implicit conversions, please open an issue, and we will address it.
from clickhouse-kafka-connect.
Given the code added to v0.0.10 by @hamsterready this should be resolved - if it continues to persist, please reopen and thanks!
from clickhouse-kafka-connect.
Related Issues (20)
- Make connect compatable with kafka plugin.discovery
- Release 1.0.12 Release Notes, 23.3 Minimum CH Version HOT 1
- Support parallel inserts HOT 7
- Unreasonable cap of tableRefreshInterval at 600ms HOT 2
- Fix cloud tests
- Avro record with an ENUM crashes task HOT 2
- Connect state table may not exist on all clickhouse node in cluster HOT 3
- Can't connect to CliclHouse from local minikube HOT 6
- Make async_insert=0 configurable HOT 1
- Add logging for cases when Kafka authentication for Consumer / Producer is wrong HOT 6
- Use the new V2 version of the java client to simplify serialization HOT 1
- Losing event in around million events HOT 20
- Table mapping always choose the default database. HOT 3
- no support for hebrew language HOT 2
- [BUG] Table with nested failing to load HOT 2
- fetch.max.wait.ms setting of kafka connect not working as expected with CH sink connector HOT 1
- DateTime64(9) are inserted incorrectly (near unix epoch) when using KafkaConnect TimestampConverter to convert from string to org.apache.kafka.connect.data.Time HOT 9
- ClassCastException due to invalid field value to column name mapping for Tuples
- Extend docs with built-in transformations HOT 1
- No documents? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clickhouse-kafka-connect.