Code Monkey home page Code Monkey logo

ansible-proxysql's Introduction

proxysql


Currently this role is not compatible with ProxySQL >= v2.0.1.

This role is just compatible with Ansible 2.7.9 or higher due to a bug in the migration from MySQLdb to PyMySQL (#40123) in the MySQL module utils.

Release 0.3.0 is compatible with Ansible 2.7.0 or lower.


This role installs and configures ProxySQL - the high performance, high availability, protocol aware proxy for MySQL.

Since version 2.3.0 Ansible is providing an module to configure ProxySQL itself. This Ansible role is using this functionality but adding some (hopefully useful) features on top:

Please also take a look at the "Known issues or: Good to know" section in this document.

Requirements

This role requires Ansible 2.5.0 or higher.

You can simply use pip to install (and define) a stable version:

pip install ansible==2.7.9

All platform requirements are listed in the metadata file.

Install

ansible-galaxy install timorunge.proxysql

Role Variables

The variables that can be passed to this role. For all variables, take a look at defaults/main.yml.

# Enable / disable ProxySQL as a service.
# Type: Bool
proxysql_service_enabled: True

# Restart ProxySQL if static variables are changing. For a list of static
# variables take a look at `proxysql_non_dynamic_variables` in `vars/main.yml`.
# Type: Bool
proxysql_restart_on_static_variables_change: True

# Repository

# If both, `proxysql_use_official_repo` and `proxysql_use_percona_repo` are set
# to `False` the module will automatically download the defined
# `proxysql_version` as a package from Github and install the same.

# Use the official ProxySQL repository.
# Type: Bool
proxysql_use_official_repo: True

# Use the Percona repository.
# Type: Bool
proxysql_use_percona_repo: False

# Define the repository version for the percona repository.
# Type: Str
proxysql_percona_release: latest

# The ProxySQL version which should be installed if not using the ProxySQL
# repository.
# Type: Int
proxysql_version: 1.4.15

# Configuration

# The path where ProxySQL should save it's database and logs.
# Type: Str
proxysql_datadir: /var/lib/proxysql

# Define the proxysql.cnf template
# Type: Str
proxysql_proxysql_cnf_template: proxysql.cnf.j2

# Define the proxysql-admin.cnf template
# Type: Str
proxysql_proxysql_admin_cnf_template: proxysql-admin.cnf.j2

# The login variables for the configuration of ProxySQL itself. They are just
# used inside the `main.yml` file and here to simplify the configuration.
# Type: Str
proxysql_login_admin_host: 127.0.0.1
proxysql_login_admin_password: admin
proxysql_login_admin_port: 6032
proxysql_login_admin_user: admin

# Global variables
# `admin_variables` in `proxysql_global_variables_kv`: contains global
# variables that control the functionality of the admin interface.
# `admin_variables` are prefixed with `admin-`.
# `mysql_variables`: in `proxysql_global_variables_kv` contains global
# variables that control the functionality for handling the incoming
# MySQL traffic.
# `mysql_variables` are prefixed with `mysql-`.

# The variables should be either a string or an integer. You should mark
# a boolean value as a string, e.g. "True" or "False".

# For a full reference take a look at:
# https://github.com/sysown/proxysql/wiki/Global-variables

# Format:
# Type: Dict
# proxysql_global_variables:
#   load_to_runtime: "True"
#   save_to_disk: "True"
#   login_host: "{{ proxysql_login_admin_host }}"
#   login_password: "{{ proxysql_login_admin_password }}"
#   login_port: "{{ proxysql_login_admin_port }}"
#   login_user: "{{ proxysql_login_admin_user }}"
proxysql_global_variables:
  login_host: "{{ proxysql_login_admin_host }}"
  login_password: "{{ proxysql_login_admin_password }}"
  login_port: "{{ proxysql_login_admin_port }}"
  login_user: "{{ proxysql_login_admin_user }}"
# Format:
# Type: Dict
# proxysql_global_variables_kv:
#   key: value
# e.g.:
# proxysql_global_variables_kv:
#   admin-admin_credentials: "{{ proxysql_login_admin_user }}:{{ proxysql_login_admin_password }}"
#   admin-mysql_ifaces: "{{ proxysql_login_admin_host }}:{{ proxysql_login_admin_port }}"
#   mysql-interfaces: 0.0.0.0:6033
#   mysql-commands_stats: "True"
#   mysql-threads: 4
proxysql_global_variables_kv: {}

# Backend servers
# `proxysql_backend_servers`: contains rows for the mysql_servers table from
# the admin interface. Basically, these define the backend servers towards
# which the incoming MySQL traffic is routed.

# For a full reference take a look at:
# https://docs.ansible.com/ansible/latest/modules/proxysql_backend_servers_module.html
# Important: This module uses `hostgroup` (which is the correct name in the
# database) instead of `hostgroup_id` (which is the default in the Ansible
# module)!

# Format:
# Type: Dict
# proxysql_backend_servers:
#   mysql-srv1-hg1:
#     comment: mysql-srv1-hg1
#     hostgroup: 1
#     hostname: 172.16.77.101
#     login_host: "{{ proxysql_login_admin_host }}"
#     login_password: "{{ proxysql_login_admin_password }}"
#     login_port: "{{ proxysql_login_admin_port }}"
#     login_user: "{{ proxysql_login_admin_user }}"
#     max_connections: 1000
#     max_replication_lag: 0
#     status: ONLINE
#     weight: 1
#   mysql-srv1-hg2:
#     comment: mysql-srv1-hg2
#     hostgroup: 2
#     hostname: 172.16.77.101
#     login_host: "{{ proxysql_login_admin_host }}"
#     login_password: "{{ proxysql_login_admin_password }}"
#     login_port: "{{ proxysql_login_admin_port }}"
#     login_user: "{{ proxysql_login_admin_user }}"
#     max_connections: 1000
#     max_replication_lag: 0
#     status: ONLINE
#     weight: 1
proxysql_backend_servers: {}

# ProxySQL servers
# `proxysql_proxysql_servers`: contains rows for the proxysql_servers table
# from the admin interface. Basically, these define the ProxySQL servers
# which are used for clustering.

# For a full reference take a look at:
# `library/proxysql_proxysql_servers.py` since this is not a part of the
# official Ansible package.

# Format:
# Type: Dict
# proxysql_proxysql_servers:
#   proxysql-srv-1:
#     comment: proxysql-srv-1
#     hostname: 172.16.77.201
#     login_host: "{{ proxysql_login_admin_host }}"
#     login_password: "{{ proxysql_login_admin_password }}"
#     login_port: "{{ proxysql_login_admin_port }}"
#     login_user: "{{ proxysql_login_admin_user }}"
#     weight: 0
#   proxysql-srv-2:
#     comment: proxysql-srv-2
#     hostname: 172.16.77.202
#     login_host: "{{ proxysql_login_admin_host }}"
#     login_password: "{{ proxysql_login_admin_password }}"
#     login_port: "{{ proxysql_login_admin_port }}"
#     login_user: "{{ proxysql_login_admin_user }}"
#     weight: 0
proxysql_proxysql_servers: {}

# Replication hostgroups
# `proxysql_replication_hostgroups`: represent a pair of writer_hostgroup
# and reader_hostgroup. ProxySQL will monitor the value of read_only for all
# the servers in specified hostgroups, and based on the value of read_only
# will assign the server to the writer or reader hostgroups.

# For a full reference take a look at:
# https://docs.ansible.com/ansible/latest/modules/proxysql_replication_hostgroups_module.html

# Format:
# Type: Dict
# proxysql_replication_hostgroups:
#   Cluster:
#     comment: Cluster
#     login_host: "{{ proxysql_login_admin_host }}"
#     login_password: "{{ proxysql_login_admin_password }}"
#     login_port: "{{ proxysql_login_admin_port }}"
#     login_user: "{{ proxysql_login_admin_user }}"
#     reader_hostgroup: 2
#     writer_hostgroup: 1
proxysql_replication_hostgroups: {}

# Users
# `proxysql_mysql_users`: contains rows for the mysql_users table from the
# admin interface. Basically, these define the users which can connect to the
# proxy, and the users with which the proxy can connect to the backend servers.

# For a full reference take a look at:
# http://docs.ansible.com/ansible/latest/proxysql_mysql_users_module.html

# Format:
# Type: Dict
# proxysql_mysql_users:
#   user1:
#     active: 1
#     backend: 1
#     default_hostgroup: 1
#     fast_forward: 0
#     frontend: 1
#     login_host: "{{ proxysql_login_admin_host }}"
#     login_password: "{{ proxysql_login_admin_password }}"
#     login_port: "{{ proxysql_login_admin_port }}"
#     login_user: "{{ proxysql_login_admin_user }}"
#     max_connections: 10000
#     password: Passw0rd
#     transaction_persistent: 1
#     username: user1
#   user2:
#     active: 1
#     backend: 1
#     default_hostgroup: 2
#     fast_forward: 0
#     frontend: 1
#     login_host: "{{ proxysql_login_admin_host }}"
#     login_password: "{{ proxysql_login_admin_password }}"
#     login_port: "{{ proxysql_login_admin_port }}"
#     login_user: "{{ proxysql_login_admin_user }}"
#     max_connections: 1000
#     password: dr0wssaP
#     transaction_persistent: 1
#     username: user2
proxysql_mysql_users: {}

# Query rules
# `proxysql_query_rules` contains rows for the mysql_query_rules table from
# the admin interface. Basically, these define the rules used to classify and
# route the incoming MySQL traffic, according to various criteria (patterns
# matched, user used to run the query, etc.).

# For a full reference take a look at:
# http://docs.ansible.com/ansible/latest/proxysql_query_rules_module.html

# Format:
# Type: Dict
# proxysql_query_rules:
#   catchall:
#     active: 1
#     apply: 1
#     destination_hostgroup: 1
#     flagIN: 0
#     login_host: "{{ proxysql_login_admin_host }}"
#     login_password: "{{ proxysql_login_admin_password }}"
#     login_port: "{{ proxysql_login_admin_port }}"
#     login_user: "{{ proxysql_login_admin_user }}"
#     match_pattern: .*@.*
#     negate_match_pattern: 0
#     rule_id: 1
#   selectforupdate:
#     active: 1
#     apply: 1
#     destination_hostgroup: 1
#     flagIN: 0
#     login_host: "{{ proxysql_login_admin_host }}"
#     login_password: "{{ proxysql_login_admin_password }}"
#     login_port: "{{ proxysql_login_admin_port }}"
#     login_user: "{{ proxysql_login_admin_user }}"
#     match_pattern: ^SELECT.*FOR UPDATE
#     negate_match_pattern: 0
#     rule_id: 2
#   select:
#     active: 1
#     apply: 0
#     destination_hostgroup: 2
#     flagIN: 0
#     login_host: "{{ proxysql_login_admin_host }}"
#     login_password: "{{ proxysql_login_admin_password }}"
#     login_port: "{{ proxysql_login_admin_port }}"
#     login_user: "{{ proxysql_login_admin_user }}"
#     match_pattern: ^SELECT.*
#     negate_match_pattern: 0
#     rule_id: 3
proxysql_query_rules: {}

# Percona ProxySQL Admin
# The ProxySQL Admin (proxysql-admin) solution configures Percona
# XtraDB cluster nodes into ProxySQL.

# For a full reference take a look at:
# https://github.com/percona/proxysql-admin-tool

# Format:
# Type: Dict
# proxysql_percona_admin_tool:
#   PROXYSQL_DATADIR: "{{ proxysql_datadir }}"
#   PROXYSQL_USERNAME: "{{ proxysql_login_admin_user }}"
#   PROXYSQL_PASSWORD: "{{ proxysql_login_admin_password }}"
#   PROXYSQL_HOSTNAME: "{{ proxysql_login_admin_host }}"
#   PROXYSQL_PORT: "{{ proxysql_login_admin_port }}"
#   CLUSTER_USERNAME: admin
#   CLUSTER_PASSWORD: admin
#   CLUSTER_HOSTNAME: localhost
#   CLUSTER_PORT: 3306
#   MONITOR_USERNAME: monitor
#   MONITOR_PASSWORD: monit0r
#   CLUSTER_APP_USERNAME: proxysql_user
#   CLUSTER_APP_PASSWORD: passw0rd
#   WRITE_HOSTGROUP_ID: 10
#   READ_HOSTGROUP_ID: 11
#   MODE: singlewrite
proxysql_percona_admin_tool:
  PROXYSQL_DATADIR: "{{ proxysql_datadir }}"
  PROXYSQL_USERNAME: "{{ proxysql_login_admin_user }}"
  PROXYSQL_PASSWORD: "{{ proxysql_login_admin_password }}"
  PROXYSQL_HOSTNAME: "{{ proxysql_login_admin_host }}"
  PROXYSQL_PORT: "{{ proxysql_login_admin_port }}"

Examples

1) Full configuration example

Here you can see a full example of a configuration of ProxySQL. In this case the role will download the 1.4.15 package directly and not use the repository (proxysql_use_official_repo is set to False).

This is basically (with some small changes) the test.yml file which is used for testing.

- hosts: proxysql
  gather_facts: True
  vars:
    proxysql_version: 1.4.15
    proxysql_service_enabled: True
    proxysql_use_official_repo: True
    proxysql_login_admin_host: 127.0.0.1
    proxysql_login_admin_password: admin
    proxysql_login_admin_port: 6032
    proxysql_login_admin_user: admin
    proxysql_global_variables:
      login_host: "{{ proxysql_login_admin_host }}"
      login_password: "{{ proxysql_login_admin_password }}"
      login_port: "{{ proxysql_login_admin_port }}"
      login_user: "{{ proxysql_login_admin_user }}"
    proxysql_global_variables_kv:
      admin-admin_credentials: "{{ proxysql_login_admin_user }}:{{ proxysql_login_admin_password }}"
      admin-mysql_ifaces: "{{ proxysql_login_admin_host }}:{{ proxysql_login_admin_port }}"
      mysql-commands_stats: "True"
      mysql-connect_retries_on_failure: 10
      mysql-connect_timeout_server: 3000
      mysql-default_charset: utf8
      mysql-default_query_delay: 0
      mysql-default_query_timeout: 300000
      mysql-default_schema: information_schema
      mysql-default_sql_mode: >
                              STRICT_TRANS_TABLES,
                              ERROR_FOR_DIVISION_BY_ZERO,
                              NO_AUTO_CREATE_USER,
                              NO_ENGINE_SUBSTITUTION
      mysql-interfaces: 127.0.0.1:6033
      mysql-max_connections: 8192
      mysql-monitor_read_only_interval: 1500
      mysql-monitor_read_only_timeout: 500
      mysql-ping_timeout_server: 500
      mysql-poll_timeout: 2000
      mysql-query_retries_on_failure: 1
      mysql-sessions_sort: "True"
      mysql-threads: 4
    proxysql_backend_servers:
      mysql-srv1-hg1:
        comment: mysql-srv1-hg1
        hostgroup: 1
        hostname: 172.16.77.101
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        max_connections: 1000
        max_replication_lag: 0
        status: ONLINE
        weight: 1
      mysql-srv1-hg2:
        comment: mysql-srv1-hg2
        hostgroup: 2
        hostname: 172.16.77.101
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        max_connections: 1000
        max_replication_lag: 0
        status: ONLINE
        weight: 1
      mysql-srv2-hg2:
        comment: mysql-srv2-hg2
        hostgroup: 2
        hostname: 172.16.77.102
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        max_connections: 2000
        max_replication_lag: 5
        status: ONLINE
        weight: 1
      mysql-srv3-hg2:
        comment: mysql-srv3-hg2
        hostgroup: 2
        hostname: 172.16.77.103
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        max_connections: 2000
        max_replication_lag: 5
        status: OFFLINE_HARD
        weight: 1
    proxysql_proxysql_servers:
      proxysql-srv-1:
        comment: proxysql-srv-1
        hostname: 172.16.77.201
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        port: 6032
        weight: 0
      proxysql-srv-2:
        comment: proxysql-srv-2
        hostname: 172.16.77.202
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        port: 6032
        weight: 0
    proxysql_replication_hostgroups:
      Cluster:
        comment: Cluster
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        reader_hostgroup: 2
        writer_hostgroup: 1
    proxysql_mysql_users:
      user1:
        active: 1
        backend: 1
        default_hostgroup: 1
        fast_forward: 0
        frontend: 1
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        max_connections: 10000
        password: Passw0rd
        transaction_persistent: 1
        username: user1
      user2:
        active: 1
        backend: 1
        default_hostgroup: 1
        fast_forward: 0
        frontend: 1
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        max_connections: 1000
        password: dr0wssaP
        transaction_persistent: 1
        username: user2
    proxysql_query_rules:
      catchall:
        active: 1
        apply: 1
        destination_hostgroup: 1
        flagIN: 0
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        match_pattern: .*@.*
        negate_match_pattern: 0
        rule_id: 1
      selectforupdate:
        active: 1
        apply: 1
        destination_hostgroup: 1
        flagIN: 0
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        match_pattern: ^SELECT.*FOR UPDATE
        negate_match_pattern: 0
        rule_id: 2
      select:
        active: 1
        apply: 0
        destination_hostgroup: 2
        flagIN: 0
        login_host: "{{ proxysql_login_admin_host }}"
        login_password: "{{ proxysql_login_admin_password }}"
        login_port: "{{ proxysql_login_admin_port }}"
        login_user: "{{ proxysql_login_admin_user }}"
        match_pattern: ^SELECT.*
        negate_match_pattern: 0
        rule_id: 3
  roles:
    - timorunge.proxysql

2) Installation from the official repository

Use the ProxySQL repository (proxysql_use_official_repo is set to True). ProxySQL itself is not providing packages in the repository for Ubuntu > 16.04.

Just set the proxysql_use_official_repo to False for newer Ubuntu releases or use the Percona repository.

Take a look at the testing section for the coverage of the official repository.

- hosts: proxysql
  vars:
    proxysql_use_official_repo: True
    proxysql_use_percona_repo: False
    proxysql_login_admin_host: 127.0.0.1
    proxysql_login_admin_password: admin
    proxysql_login_admin_port: 6032
    proxysql_login_admin_user: admin
    ...

3) Installation from the Percona repository

Use the ProxySQL repository (proxysql_use_percona_repo is set to True).

Take a look at the testing section for the coverage of the Percona repository.

- hosts: proxysql
  vars:
    proxysql_use_official_repo: False
    proxysql_use_percona_repo: True
    proxysql_login_admin_host: 127.0.0.1
    proxysql_login_admin_password: admin
    proxysql_login_admin_port: 6032
    proxysql_login_admin_user: admin
    ...

4) Installation from deb or rpm package

You can install ProxySQL directly from a GitHub release. Just define the proxysql_version. Set proxysql_use_official_repo and proxysql_use_percona_repo to False.

- hosts: proxysql
  vars:
    proxysql_version: 1.4.15
    proxysql_use_official_repo: False
    proxysql_use_percona_repo: False
    proxysql_login_admin_host: 127.0.0.1
    proxysql_login_admin_password: admin
    proxysql_login_admin_port: 6032
    proxysql_login_admin_user: admin
    ...

5) Don't restart ProxySQL after static variable change

If you'd like to restart ProxySQL on your own after a config change of static variables you have to set proxysql_restart_on_static_variables_change to False.

In this case you're hitting an known issue which is no drama. In this case idempotence tests will fail.

You don't have to apply Ansible again after a manual restart.

- hosts: proxysql
  vars:
    proxysql_use_official_repo: False
    proxysql_restart_on_static_variables_change: False
    proxysql_login_admin_host: 127.0.0.1
    proxysql_login_admin_password: admin
    proxysql_login_admin_port: 6032
    proxysql_login_admin_user: admin
    ...

Known issues or: Good to know

1) ProxySQL > 1.4.7 on Ubuntu 16.04 (fixed)

On Ubuntu 16.04 Ansible (version does not matter) / ProxySQL > 1.4.7 seems to have problems to communicate correctly via mysql-python / python-mysqldb.

Example error:

"unable to modify server.. (1045, 'unrecognized token: \"\\'\\n  AND compression = 0\\n  AND weight = 1\\n  AND use_ssl = 0\\n  AND max_connections = 2000\\n  AND max_latency_ms = 0\\n  AND max_replication_lag = 5\"')"

Note:

I've done the following little research with mysql-python installed via pip on Ubuntu 16.04. Don't worry, it's also failing with python-mysqldb.


In the MySQLdb python library the execute method (class BaseCursor) is generating a query in the following way:

query = query % tuple([db.literal(item) for item in args])

db.literal is part of the Connection class and returns single objects as a string and mutliple objects as a sequence while it's converting each sequence as well.

def literal(self, o):
  # ...
  return self.escape(o, self.encoders)

self.escape should escape all special characters in the given object and is using a mapping dict to provide quoting functions for each type. This is self.encoders which - per default and not set different - using MySQLdb.converters.

The mapping for a string is StringType: Thing2Literal. So the string will be escaped with the method Thing2Literal.

def Thing2Literal(o, d):
  # ...
  return string_literal(o, d)

string_literal should convert our string object into a SQL string literal. This means, any special SQL characters are escaped, and it is enclosed within single quotes. In other words, it performs:

"'%s'" % escape_string(str(obj))

During escaping the string the string objects are getting deleted and are returning just a single quote (').

Since in the tests nothing beside the version of ProxySQL changed I assume that a change in ProxySQL (diff 1.4.7 vs. 1.4.8) is causing Ansible to fail. Because ProxySQL itself - if not triggered via Ansible - is working perfectly fine.


Last but not least...

This issue is sorted by installing mysqlclient - which is a fork of MySQLdb - via pip.

2) Packages for Ubuntu > 16.04 (fixed)

ProxySQL itself is not providing "up to date" packages for Ubuntu > 16.04. This Ansible role is working around this by downloading the 16.04 release for Ubuntu > 16.04 and installing the same (this behavoir might change in the future).

There is a package dependency for libcrypto++6 and libssl1.0.0 starting from Ubuntu >= 18.04 (which is sorted out automatically).

3) Non dynamic global variables

ProxySQL has some global_variables which can't be changed during runtime (see proxysql_non_dynamic_variables in vars/main.yml). Having said that, this alone is not a problem since this ProxySQL role is taking care (by generating proxysql.cnf) and provides the possibility to restart automatically if such a variable will change (set proxysql_restart_on_static_variables_change to True).

This role is also setting this value in the ProxySQL database itself and here the problem begins:

If you're changing more than one static variable technically everything is OK. ProxySQL is restarting and taking the new value from proxysql.cnf. But just the first value is changed in the database itself.

It's not an "big issue" since the real value is taken correctly from the configuration file itself but you'll see a changeset in the next Ansible run which will:

  • Restart ProxySQL once again
  • Idempotence tests will fail (if you're not bootstrapping from scratch)

A potential solution could be to not set proxysql_non_dynamic_variables in the ProxySQL database.

4) ProxySQL clustering

The ProxySQL clustering is still experimental. A quote from the clustering documentation: "because this feature is still experimental, the table is not automatically loaded from disk".

For the initialisation from the proxysql.cnf it's important that hostname (obviously) and port (it's not taking the default value) are defined.

5) ProxySQL and the Percona repository on CentOS...

...is not working because of some dependency issues.

Technically we need the following packages to apply Ansible correctly:

proxysql_dependency_pkgs:
  - MySQL-python
  - mysql

With the enabled Percona repository the package dependencies for MySQL-python and mysql are clashing.

yum install MySQL-python
...
============================================================================================
 Package                    Arch     Version                 Repository                Size
============================================================================================
 Installing:
  MySQL-python              x86_64   1.2.5-1.el7             base                      90 k
 Installing for dependencies:
  Percona-Server-shared-56  x86_64   5.6.43-rel84.3.el7      percona-release-x86_64   619 k

Transaction Summary
============================================================================================
Install  1 Package (+1 Dependent package)
yum install mysql
...

============================================================================================
 Package                          Arch    Version             Repository               Size
============================================================================================
 Installing:
  Percona-Server-client-57         x86_64  5.7.24-27.1.el7    percona-release-x86_64  6.8 M
 Installing for dependencies:
  Percona-Server-shared-57         x86_64  5.7.24-27.1.el7    percona-release-x86_64  748 k
  Percona-Server-shared-compat-57  x86_64  5.7.24-27.1.el7    percona-release-x86_64  1.2 M

Transaction Summary
============================================================================================
Install  1 Package (+2 Dependent packages)

As you can see, MySQL-python is depending on Percona-Server-shared-56 while mysql is depending on Percona-Server-shared-57.

Testing

Build Status

Tests are done with Docker and docker_test_runner which brings up the following containers with different environment settings:

  • CentOS 7
  • Debian 9 (Stretch)
  • Ubuntu 16.04 (Xenial Xerus)
  • Ubuntu 18.04 (Bionic Beaver)

Ansible 2.7.9 is installed on all containers and a test playbook is getting applied.

For further details and additional checks take a look at the docker_test_runner configuration and the Docker entrypoint. An high level overview can be found in the following table:

Distribution Version Official repository Percona repository Package
CentOS 7 yes no 1.4.15
Debian 9 yes yes 1.4.15
Ubuntu 16.04 yes yes 1.4.15
Ubuntu 18.04 no yes 1.4.15
# Testing locally:
curl https://raw.githubusercontent.com/timorunge/docker-test-runner/master/install.sh | sh
./docker_test_runner.py -f tests/docker_test_runner.yml

Since the build time on Travis is limited for public repositories the automated tests are limited to:

  • CentOS 7
  • Debian 8 (Jessie)
  • Debian 9 (Stretch)
  • Ubuntu 16.04 (Xenial Xerus)
  • Ubuntu 18.04 (Bionic Beaver)

Dependencies

License

BSD 3-Clause "New" or "Revised" License

Author Information

  • Timo Runge

ansible-proxysql's People

Contributors

timorunge avatar virtualtam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

ansible-proxysql's Issues

proxysql_global_variables issues

It seems to be an issue with variables in the templates. Not 100% why it would be failing. Using the example values.

ansible 2.9.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.7.3 (default, Apr  3 2019, 05:39:12) [GCC 8.3.0]
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793 `" && echo ansible-tmp-1575573481.9743962-172483146853793="` echo /root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793 `" ) && sleep 0'
Using module file /usr/local/lib/python3.7/dist-packages/ansible/modules/database/proxysql/proxysql_global_variables.py
<127.0.0.1> PUT /root/.ansible/tmp/ansible-local-1167956qcxqd5/tmp01oqtk3r TO /root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/AnsiballZ_proxysql_global_variables.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/ /root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/AnsiballZ_proxysql_global_variables.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/AnsiballZ_proxysql_global_variables.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/AnsiballZ_proxysql_global_variables.py", line 102, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/AnsiballZ_proxysql_global_variables.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/AnsiballZ_proxysql_global_variables.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible.modules.database.proxysql.proxysql_global_variables', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib/python2.7/runpy.py", line 188, in run_module
    fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 82, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/ansible_proxysql_global_variables_payload_YwSAYX/ansible_proxysql_global_variables_payload.zip/ansible/modules/database/proxysql/proxysql_global_variables.py", line 266, in <module>
  File "/tmp/ansible_proxysql_global_variables_payload_YwSAYX/ansible_proxysql_global_variables_payload.zip/ansible/modules/database/proxysql/proxysql_global_variables.py", line 231, in main
  File "/tmp/ansible_proxysql_global_variables_payload_YwSAYX/ansible_proxysql_global_variables_payload.zip/ansible/modules/database/proxysql/proxysql_global_variables.py", line 122, in check_config
TypeError: tuple indices must be integers, not str

failed: [127.0.0.1] (item={'key': 'mysql-sessions_sort', 'value': 'True'}) => {
    "ansible_loop_var": "item",
    "changed": false,
    "item": {
        "key": "mysql-sessions_sort",
        "value": "True"
    },
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/AnsiballZ_proxysql_global_variables.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/AnsiballZ_proxysql_global_variables.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1575573481.9743962-172483146853793/AnsiballZ_proxysql_global_variables.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible.modules.database.proxysql.proxysql_global_variables', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib/python2.7/runpy.py\", line 188, in run_module\n    fname, loader, pkg_name)\n  File \"/usr/lib/python2.7/runpy.py\", line 82, in _run_module_code\n    mod_name, mod_fname, mod_loader, pkg_name)\n  File \"/usr/lib/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/tmp/ansible_proxysql_global_variables_payload_YwSAYX/ansible_proxysql_global_variables_payload.zip/ansible/modules/database/proxysql/proxysql_global_variables.py\", line 266, in <module>\n  File \"/tmp/ansible_proxysql_global_variables_payload_YwSAYX/ansible_proxysql_global_variables_payload.zip/ansible/modules/database/proxysql/proxysql_global_variables.py\", line 231, in main\n  File \"/tmp/ansible_proxysql_global_variables_payload_YwSAYX/ansible_proxysql_global_variables_payload.zip/ansible/modules/database/proxysql/proxysql_global_variables.py\", line 122, in check_config\nTypeError: tuple indices must be integers, not str\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

Force initilization after changing admin password

Hi,

I want to deploy proxysql and change the admin password.

It is my understanding that it's currently impossible without stopping and running service proxysql initial manually before launching the playbook again.

Otherwise, the next steps fail with

task path: proxysql/tasks/config/proxysql_global_variables.yml:5
failed: [vm-ubuntu-01] (item={'key': u'admin-mysql_ifaces', 'value': u'127.0.0.1:6032'}) => {"changed": false, "item": {"key": "admin-mysql_ifaces", "value": "127.0.0.1:6032"}, "msg": "unable to connect to ProxySQL A
dmin Module.. (1045, \"ProxySQL Error: Access denied for user 'admin'@'' (using password: YES)\")"}
failed: [vm-ubuntu-01] (item={'key': u'admin-admin_credentials', 'value': u'admin:*****'}) => {"changed": false, "item": {"key": "admin-admin_credentials", "value": "admin:****"}, "msg": "unable to connect to Pr
oxySQL Admin Module.. (1045, \"ProxySQL Error: Access denied for user 'admin'@'' (using password: YES)\")"}

The only workarounds I could find were:

  • inserting that in the tasks/config.yml playbook:
  template:
    dest: /etc/proxysql.cnf
    group: root
    mode: 0400
    owner: root
    src: "{{ proxysql_proxysql_cnf_template }}"
  tags:
    - skip_ansible_lint

- name: Ensure that ProxySQL is stopped before initializing
  service:
    name: proxysql
    state: stopped

- name: Ensure that ProxySQL is initialized
  command: service proxysql initial

- name: Ensure that ProxySQL is running
  service:
    enabled: "{{ proxysql_service_enabled_state | default('yes') }}"
    name: proxysql
    state: "{{ proxysql_service_state | default('started') }}"
  • inserting this kind of file (during the playbook play) to prevent the service being started after installation:
    cat /usr/sbin/policy-rc.d

# This return code prevent systemctl from starting service when
# dpkg calls invoke-rc.d
exit 101

Do you have any other idea?

Cheers,
Arnaud

we are trying to use your role from galaxy

we are getting the following error from your role. Ideas on this?

ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/etc/ansible/roles/timorunge.proxysql/tasks/main.yml': line 23, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:


- include_tasks: facts.yml
  ^ here


The error appears to have been in '/etc/ansible/roles/timorunge.proxysql/tasks/main.yml': line 23, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:


- include_tasks: facts.yml
  ^ here

AttributeError: 'tuple' object has no attribute 'execute'

ansible version 2.13.1

Base info

[root@mysql ~]# cat /etc/redhat-release 
CentOS Linux release 7.9.2009 (Core)
[root@mysql ~]# python -V
Python 2.7.5

Error Info

TASK [proxysql : include_tasks] ************************************************************************************************************************************************************
task path: /home/ansible/roles/proxysql/tasks/config.yml:27
included: /home/ansible/roles/proxysql/tasks/config/proxysql_proxysql_servers.yml for proxysql_node001, proxysql_node002

TASK [proxysql : Manage ProxySQL cluster hosts] ********************************************************************************************************************************************
task path: /home/ansible/roles/proxysql/tasks/config/proxysql_proxysql_servers.yml:5
redirecting module_util ansible.module_utils.mysql to ansible_collections.community.mysql.plugins.module_utils.mysql
redirecting module_util ansible.module_utils.mysql to ansible_collections.community.mysql.plugins.module_utils.mysql
redirecting module_util ansible.module_utils.mysql to ansible_collections.community.mysql.plugins.module_utils.mysql
Using module file /home/ansible/roles/proxysql/library/proxysql_proxysql_servers.py
Pipelining is enabled.
<192.168.60.8> ESTABLISH SSH CONNECTION FOR USER: root
Using module file /home/ansible/roles/proxysql/library/proxysql_proxysql_servers.py
Pipelining is enabled.
<192.168.60.9> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.60.8> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'User="root"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9a07ef05c1"' 192.168.60.8 '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
<192.168.60.9> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'User="root"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/a71210ddbc"' 192.168.60.9 '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
<192.168.60.8> (1, b'', b'Traceback (most recent call last):\n  File "<stdin>", line 107, in <module>\n  File "<stdin>", line 99, in _ansiballz_main\n  File "<stdin>", line 48, in invoke_module\n  File "/usr/lib64/python2.7/runpy.py", line 176, in run_module\n    fname, loader, pkg_name)\n  File "/usr/lib64/python2.7/runpy.py", line 82, in _run_module_code\n    mod_name, mod_fname, mod_loader, pkg_name)\n  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code\n    exec code in run_globals\n  File "/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py", line 412, in <module>\n  File "/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py", line 374, in main\n  File "/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py", line 184, in check_server_config\nAttributeError: \'tuple\' object has no attribute \'execute\'\n')
<192.168.60.8> Failed to connect to the host via ssh: Traceback (most recent call last):
  File "<stdin>", line 107, in <module>
  File "<stdin>", line 99, in _ansiballz_main
  File "<stdin>", line 48, in invoke_module
  File "/usr/lib64/python2.7/runpy.py", line 176, in run_module
    fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 82, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py", line 412, in <module>
  File "/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py", line 374, in main
  File "/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py", line 184, in check_server_config
AttributeError: 'tuple' object has no attribute 'execute'
The full traceback is:
Traceback (most recent call last):
  File "<stdin>", line 107, in <module>
  File "<stdin>", line 99, in _ansiballz_main
  File "<stdin>", line 48, in invoke_module
  File "/usr/lib64/python2.7/runpy.py", line 176, in run_module
    fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 82, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py", line 412, in <module>
  File "/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py", line 374, in main
  File "/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py", line 184, in check_server_config
AttributeError: 'tuple' object has no attribute 'execute'
failed: [proxysql_node001] (item={'key': 'proxysql-srv-1', 'value': {'comment': 'proxysql-srv-1', 'hostname': '192.168.60.8', 'login_host': '127.0.0.1', 'login_password': 'admin', 'login_port': 6032, 'login_user': 'admin', 'port': 6032, 'weight': 0}}) => {
    "ansible_loop_var": "item",
    "changed": false,
    "item": {
        "key": "proxysql-srv-1",
        "value": {
            "comment": "proxysql-srv-1",
            "hostname": "192.168.60.8",
            "login_host": "127.0.0.1",
            "login_password": "admin",
            "login_port": 6032,
            "login_user": "admin",
            "port": 6032,
            "weight": 0
        }
    },
    "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 107, in <module>\n  File \"<stdin>\", line 99, in _ansiballz_main\n  File \"<stdin>\", line 48, in invoke_module\n  File \"/usr/lib64/python2.7/runpy.py\", line 176, in run_module\n    fname, loader, pkg_name)\n  File \"/usr/lib64/python2.7/runpy.py\", line 82, in _run_module_code\n    mod_name, mod_fname, mod_loader, pkg_name)\n  File \"/usr/lib64/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py\", line 412, in <module>\n  File \"/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py\", line 374, in main\n  File \"/tmp/ansible_proxysql_proxysql_servers_payload_QrjULu/ansible_proxysql_proxysql_servers_payload.zip/ansible/modules/proxysql_proxysql_servers.py\", line 184, in check_server_config\nAttributeError: 'tuple' object has no attribute 'execute'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.