Comments (6)
We're still feeling out RDS over here - we're in the process of migrating stuff over to AWS so we haven't put a lot of effort into testing things out. Orchestrator did see the node when I put it in to be discovered from its IP address, but had trouble tracing back the replication agreement, as the IP address the replica used to connect to the master isn't routable to things that aren't running in RDS.
I'll stand up an RDS instance again and try to get answers to your questions. I know that SHOW BINLOG EVENTS
only needs REPLICATION SLAVE
privs which I think might work, but I'll have to try it out.
Thanks for your feedback, all; I'll update this issue again when I have more concrete stuff for everyone to look at.
from orchestrator.
@shlomi-noach @cswingler While orchestrator isn't something high on my priority list, it is something we will be exploring in the future and we are running gh-ost
on RDS so we've got over these sorts of hurdles in the past. If you have any RDS issues or questions, I'm happy to be pinged and help out where I can.
from orchestrator.
I think that orchestrator has just not been used on RDS. Recognising any differences which are needed should not be hard, it would just then be an issue of adjusting behaviour where that's needed.
I have not used RDS and I'm not sure if Shlomi has, so probably providing information on errors you see in orchestrator when trying to talk to an RDS instance would help us see what could be done to make things work better. Changing the way to do a "change master" should be easy but you first need to recognise a server is an RDS server so this can be applied at the appropriate moment. I'm not sure how that's best determined.
Looking in the code around here: https://github.com/github/orchestrator/blob/master/go/inst/instance.go#L184 should indicate how this is done to distinguish Oracle MySQL from MariaDB or Percona Server. A similar entry would be needed for RDS.
from orchestrator.
Indeed I'm not using RDS
. However, I got some insights from gh-ost
on RDS: github/gh-ost#163
From that issue I learned that you can't just issue a STOP SLAVE
. And that you don't have SUPER
privileges. I'd need to first understand (and someone else will have to do the research, I'm afraid, as this isn't a priority for me) exactly what queries orchestrator
uses today are incompatible with RDS
. I suspect many of them, including CHANGE MASTER TO..
(do I have to pass credentials?), SHOW BINLOG EVENTS
and others.
For such queries I need to know what alternatives exist, in the form of specialized stored routines or external hooks.
Running external hooks is not trivial and would require faith in the user's implementation.
I likewise have little knowledge of the naming/resolve methods in RDS. I'd need more info.
from orchestrator.
I don't think the identification part can be done as for MariaDB/MySQL because RDS does not change @@version_comment. However, the datadir or the existence of one routine could be used to check this:
mysql> select @@datadir;
+----------------+
| @@datadir |
+----------------+
| /rdsdbdata/db/ |
+----------------+
1 row in set (0.16 sec)
mysql> select @@version_comment;
+------------------------------+
| @@version_comment |
+------------------------------+
| MySQL Community Server (GPL) |
+------------------------------+
1 row in set (0.16 sec)
mysql> select count(routine_name) from information_schema.routines where routine_name='rds_kill_query';
+---------------------+
| count(routine_name) |
+---------------------+
| 1 |
+---------------------+
1 row in set (0.17 sec)
from orchestrator.
RDS doesn't allow true root access, but it exposes some of the necessary functionality via stored procedures. E.g.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/mysql_rds_set_external_master.html
CALL mysql.rds_start_replication;
, CALL mysql.rds_stop_replication
CALL mysql.rds_set_external_master (
host_name
, host_port
, replication_user_name
, replication_user_password
, mysql_binary_log_file_name
, mysql_binary_log_file_location
, ssl_encryption
);
There are certain things that doable, like creating RDS read replica from another RDS instance w/o using RDS replication. Some other things aren't doable like specifying only a subset of tables to be replicated.
They've expanded the set of available operations over time, so it's possible if someone gets a list of needed operations, they'd expose them one day.
from orchestrator.
Related Issues (20)
- Project dead? HOT 3
- DeadIntermediateMaster auto Recovery HOT 1
- Failure from hooks are not reflected on UI HOT 1
- Automated recovery invalid HOT 5
- Should set old master's read_only to false under DeadCoMaster failure
- Report IntermediateMaster errors under CoMaster deployment
- Set the old primary host read_only=0 if failover/switch over failed.
- Whether ORC provides APIs that enable/disable recovery for specific clusters? HOT 1
- Using Azure AD Authentication
- How to set autocommit for orchestrtor detection thread
- How do you perform system tests? HOT 1
- Improve logging of problems during db connection
- Orchestrator able to display topology but still reports as failure
- After the Mysql domain name connection is configured。If the orchestrator topology displays both ip addresses and domain names of the same backup database, how can I fix this problem
- Discovery doesn't discover in extremely simple setup HOT 4
- Does orchestrator's website :3000 has login account?
- Does anyone know why updates stopped? HOT 2
- add windows version
- Ptrhon API Token
- How to login with username and password through API ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from orchestrator.