Comments (5)
hi @darionyaphet, yes, Maxwell will create a database named maxwell
on each instance, to store schema snapshots as well as its replication position. Currently we don't have plans to implement any alternative storage for this state, although I haven't ruled out someday putting all the state in kafka itself.
The reasoning is that I wanted maxwell (a) recoverable -- any process of maxwell requires no local state, and so may pick up where another maxwell left off, and (b) low ops-overhead -- I didn't want to require more bits of architecture than necessary.
from maxwell.
Thank you for you relay @osheroff , I know what you mean keep schema metadata and replication point in reading MySQL instance would reduce another storage , it's very useful . Just a small question , our product environment or DBA maybe not allow create another database instance on each replication slave machine . I try to put this in a config to point where to storage metadata . Thank you for you great work and useful help :)
from maxwell.
I see, so you just want to be able to configure a different server to store schemas and positions in?
from maxwell.
I'm not sure . I wish to design a common metadata schema . Store in MySQL maybe not a goods idea for example MySQL has single point problem .
from maxwell.
ok, not quite sure what you're after, but you can check out https://github.com/zendesk/maxwell/blob/master/src/main/java/com/zendesk/maxwell/schema/Schema.java as a starting point -- that's the in-memory representation of the schema. I'll close this issue out, but feel free to re-open it if you have something more specific.
from maxwell.
Related Issues (20)
- Allow persistent variables in Javascript filters HOT 4
- User sessions
- Failed to deserialize data of EventHeaderV4 HOT 4
- NOT FOUND mysqlBaseListener class HOT 1
- Maxwell Sending final heartbeat: 1706848518690
- row_query_max_length - variable type error
- Request to fix CVEs found in Maxwell v1.41.0 HOT 1
- Need help using javascript filter
- maxwell uses utc to convert timestamp, but not converting datetime. How can we convert datetime by utc as well?
- maxwell 1.13.2 夜里突然无响应,binlog不能产生kafka消息了 HOT 1
- I use maxwell to synchronize MySQL data to Redis, but there is a delay of nearly 30 minutes. How can I improve Maxwell's performance HOT 3
- crash in alters
- parse error on WAIT=30
- Couldn't find chained schema HOT 2
- mysql-binlog-connector-java: Deadlock when connection try to disconnect
- Error: No matches for kafka version: 2.7.0
- Maxwell v1.41.2 which based on openjdk:11-jkd-slim is vulnerable and outdated
- Couldn't find table X in database Y HOT 4
- The key offset in the JSON document is 31315 and is too big for the binary form of the document
- "MASTER“ is removed in mysql8.4 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from maxwell.