Comments (5)
pentago
commented on May 11, 2024
1
Just tested it again a couple of times with the latest image (8.0.12-debian-9-r45) and it works fine. All slaves are consistent. Will close this for now but I'll test occasionally as I totally do not trust it :D
from charts.
juan131
commented on May 11, 2024
Hi @pentago
Could you share the logs of each slave pod? I think there could have been an issue during the initialisation of one of them...
I could not reproduce it locally (check my results below) but I'd like to know the reason why data was not replicated on all your pods since it's a critical issue as you mentioned.
$ helm install --name mysql bitnami/mysql --set slave.replicas=2
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-mysql-master-0 1/1 Running 0 2m
mysql-mysql-slave-0 1/1 Running 0 1m
mysql-mysql-slave-1 1/1 Running 0 39s
$ kubectl exec mysql-mysql-slave-0 ls /bitnami/mysql/data
auto.cnf
ib_buffer_pool
ib_logfile0
ib_logfile1
ibdata1
ibtmp1
my_database
mysql
mysql-bin.000001
mysql-bin.000002
mysql-bin.index
mysql-relay-bin.000003
mysql-relay-bin.000004
mysql-relay-bin.index
mysql_upgrade_info
performance_schema
sys
$ kubectl exec mysql-mysql-slave-1 ls /bitnami/mysql/data
auto.cnf
ib_buffer_pool
ib_logfile0
ib_logfile1
ibdata1
ibtmp1
my_database
mysql
mysql-bin.000001
mysql-bin.000002
mysql-bin.index
mysql-relay-bin.000003
mysql-relay-bin.000004
mysql-relay-bin.index
mysql_upgrade_info
performance_schema
sys
from charts.
pentago
commented on May 11, 2024
The issue occurred when I deleted existing chart installation and try to do the fresh install again by using existing, previously used PVC and volume with pre-existing data on it.
The first slave gets all the data but the second one didn't.
Steps to reproduce:
- Install chart with fresh, pre-provisioned clean volume and PVC
- import some database (larger if possible, my was 500 megs)
- wait for data to replicate to slaves
- delete/purge chart
- install chart again using previously used existing PVC with data
- check the status and compare data on both slaves.
I'd realy like to hear how these exact steps went on your end because on my end it results in different amount of data across slaves, not sure why yet.
from charts.
juan131
commented on May 11, 2024
Hi @pentago
I couldn't reproduce it. I followed these steps:
- Install chart using old version with clean volumes and PVC
$ helm install bitnami/mysql --name mysql --set image.tag=5.7.23-r51,slave.persistence.enabled=false,slave.replicas=2
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-mysql-master-0 1/1 Running 0 6m
mysql-mysql-slave-0 1/1 Running 0 6m
mysql-mysql-slave-1 1/1 Running 0 6m
REPLICA_PASSWORD=$(kubectl get secret --namespace default mysql-mysql -o jsonpath="{.data.mysql-replication-password}" | base64 --decode)
ROOT_PASSWORD=$(kubectl get secret --namespace default mysql-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode)
- Import database (I use this test db: https://github.com/datacharmer/test_db):
- Wait for data to replicate to slaves
$ kubectl exec mysql-mysql-master-0 -- du -h /bitnami/mysql/data/
1.1M /bitnami/mysql/data/performance_schema
8.0K /bitnami/mysql/data/my_database
179M /bitnami/mysql/data/employees
9.7M /bitnami/mysql/data/mysql
676K /bitnami/mysql/data/sys
374M /bitnami/mysql/data/
$ kubectl exec mysql-mysql-slave-0 -- du -h /bitnami/mysql/data/
1.1M /bitnami/mysql/data/performance_schema
8.0K /bitnami/mysql/data/my_database
179M /bitnami/mysql/data/employees
9.7M /bitnami/mysql/data/mysql
676K /bitnami/mysql/data/sys
501M /bitnami/mysql/data/
$ kubectl exec mysql-mysql-slave-1 -- du -h /bitnami/mysql/data/
1.1M /bitnami/mysql/data/performance_schema
8.0K /bitnami/mysql/data/my_database
179M /bitnami/mysql/data/employees
9.7M /bitnami/mysql/data/mysql
676K /bitnami/mysql/data/sys
501M /bitnami/mysql/data/
- delete/purge chart
$ helm delete --purge mysql
- Install chart again using previously used existing PVC with data
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-mysql-mysql-master-0 Bound local-pv-9f804d79 29Gi RWO fast-disks 8m
$ helm install bitnami/mysql --name mysql --set master.persistence.existingClaim=data-mysql-mysql-master-0,slave.persistence.enabled=false,slave.replicas=2,root.password=$ROOT_PASSWORD,replication.password=$REPLICA_PASSWORD
- Check the status and compare data on both slaves.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-mysql-master-0 1/1 Running 0 3m
mysql-mysql-slave-0 1/1 Running 0 1m
mysql-mysql-slave-1 1/1 Running 0 1m
$ kubectl exec mysql-mysql-slave-0 -- du -h /bitnami/mysql/data/
1.1M /bitnami/mysql/data/performance_schema
8.0K /bitnami/mysql/data/my_database
179M /bitnami/mysql/data/employees
9.7M /bitnami/mysql/data/mysql
676K /bitnami/mysql/data/sys
501M /bitnami/mysql/data/
$ kubectl exec mysql-mysql-slave-1 -- du -h /bitnami/mysql/data/
1.1M /bitnami/mysql/data/performance_schema
8.0K /bitnami/mysql/data/my_database
179M /bitnami/mysql/data/employees
9.7M /bitnami/mysql/data/mysql
676K /bitnami/mysql/data/sys
501M /bitnami/mysql/data/
from charts.
juan131
commented on May 11, 2024
Great thanks @pentago !! Keep us in the loop
from charts.
Related Issues (20)
- [bitnami/keycloak] Support ingress for Keycloak admin area HOT 5
- [bitnami/minio] Configuration value metrics.serviceMonitor.namespace has no effect in 12.12.0 HOT 2
- [bitnami/spring-cloud-dataflow] MariaDB JDBC shouldn't have `useMysqlMetadata=true` HOT 3
- [bitnami/mongodb] Unable to carry out safe backups with current CronJob template
- [bitnami/postgresql-ha] backup
- [bitnami/keycloak] keycloak-config-cli: Allow for multiple extraEnvVarsSecret HOT 3
- listeners are not configured or appended properly when am trying to enable extraListeners HOT 1
- listeners are not configured or appended properly when am trying to enable extraListeners HOT 2
- postgresql chart + tls enabled seems to break `cronjob.yaml` HOT 4
- The POSTGRESQL_PASSWORD environment variable is empty or not set HOT 4
- Helm uninstall is not destroying rabbitmq cluster CRD mentioned in extraDeploy HOT 4
- Minio Helm chart uses default "/bitnami/minio/data" path even when "persistence.mountPath" is provided HOT 10
- auth.existingSecret does not work for PostgreSQL chart HOT 2
- [bitnami/jupyterhub]cannot disable the xsrf check with configuration HOT 1
- [bitnami/mariadb] Unable to upgrade without passing the rootPassword to values.yaml HOT 5
- [bitnami/nginx] HTML Structure Folde HOT 2
- [bitnami/gitea] add cache handling with redis HOT 3
- [bitnami/rabbitmq] Readiness failed when memoryHighWatermark enabled HOT 1
- Deploy Redis Cluster with redis-stack modules HOT 1
- Kafka controller: cannot increase persistence volume size HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from charts.