Comments (10)
fmulero
commented on September 25, 2024
1
Hi @acharbha
I am trying to reproduce your issue. At the moment I've seen that the documentation about the value sasl.users.passwords seems not valid. That value should be set in this manner (with a list):
sasl:
client:
users:
- user1
passwords:
- "user1pass"I hope to come back soon with more news.
from charts.
javsalgar
commented on September 25, 2024
Hi,
Could you check the rendered configuration that the chart generates? It should be in configmaps and secrets.
from charts.
acharbha
commented on September 25, 2024
Following values is working to have stable pods with no restart
helm.exe -n my-kafka get values my-kafka
USER-SUPPLIED VALUES:global:
storageClass: "nfs-lab"
extraConfig: |
allow.everyone.if.no.acl.found=true
super.users=User:admin;User:controller_user;User:dbaassuperadmin
auto.create.topics.enable=TRUE
default.replication.factor=1
delete.topic.enable=FALSE
authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer
sasl:
client:
users:
- user1
passwords: "user1pass"
controller:
podSecurityContext:
enabled: false
containerSecurityContext:
enabled: false
logPersistence:
enabled: true
automountServiceAccountToken: true
broker:
podSecurityContext:
enabled: false
containerSecurityContext:
enabled: false
logPersistence:
enabled: true
automountServiceAccountToken: true
externalAccess:
enabled: true
autoDiscovery:
enabled: true
controller:
service:
loadBalancerIPs:
- 10.11.XX.Y1
- 10.11.XX.Y2
- 10.11.XX.Y3
publishNotReadyAddresses: true
broker:
service:
publishNotReadyAddresses: true
rbac:
create: true
However, when I start, I try to publish using kafka client its failing SASL auth
kubectl exec --tty -i my-kafka-client --namespace my-kafka -- bash
I have no name!@my-kafka-client:/$ cat /tmp/client.properties
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
username="user1" \
password="user1pass";
I have no name!@my-kafka-client:/$I have no name!@my-kafka-client:/$ kafka-console-producer.sh \
--producer.config /tmp/client.properties \
--broker-list my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092,my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092,my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 \
--topic test
>[2024-05-24 09:22:21,948] ERROR [Producer clientId=console-producer] Connection to node -1 (my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local/10.42.97.8:9092) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-256 (org.apache.kafka.clients.NetworkClient)
[2024-05-24 09:22:21,949] WARN [Producer clientId=console-producer] Bootstrap broker my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2024-05-24 09:22:22,276] ERROR [Producer clientId=console-producer] Connection to node -2 (my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local/10.42.203.111:9092) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-256 (org.apache.kafka.clients.NetworkClient)
[2024-05-24 09:22:22,276] WARN [Producer clientId=console-producer] Bootstrap broker my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 (id: -2 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2024-05-24 09:22:22,591] ERROR [Producer clientId=console-producer] Connection to node -3 (my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local/10.42.205.225:9092) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-256 (org.apache.kafka.clients.NetworkClient)Even with plaintext protocol I cannot produce the messages getting error -
listeners:
client:
protocol: PLAINTEXT
controller:
protocol: PLAINTEXT
interbroker:
protocol: PLAINTEXT
external:
protocol: PLAINTEXTkafka-console-producer.sh \
--broker-list my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092,my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092,my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 \
--topic test
>[2024-05-24 09:36:58,131] WARN [Producer clientId=console-producer] Connection to node -3 (my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local/10.42.205.33:9092) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-05-24 09:36:58,131] WARN [Producer clientId=console-producer] Bootstrap broker my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 (id: -3 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2024-05-24 09:36:58,139] WARN [Producer clientId=console-producer] Connection to node -1 (my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local/10.42.97.164:9092) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-05-24 09:36:58,139] WARN [Producer clientId=console-producer] Bootstrap broker my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2024-05-24 09:36:58,247] WARN [Producer clientId=console-producer] Connection to node -2 (my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local/10.42.203.186:9092) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)
[2024-05-24 09:36:58,248] WARN [Producer clientId=console-producer] Bootstrap broker my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 (id: -2 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)from charts.
acharbha
commented on September 25, 2024
There's something basic which is not working - javsalgar.
I built kafka with all default value still can't produce the messages -
PS C:\Users\acharbha> helm -n my-kafka install my-kafka bitnami/kafka
NAME: my-kafka
LAST DEPLOYED: Fri May 24 19:22:30 2024
NAMESPACE: my-kafka
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: kafka
CHART VERSION: 28.2.4
APP VERSION: 3.7.0
** Please be patient while the chart is being deployed **
Kafka can be accessed by consumers via port 9092 on the following DNS name from within your cluster:
my-kafka.my-kafka.svc.cluster.local
Each Kafka broker can be accessed by producers via port 9092 on the following DNS name(s) from within your cluster:
my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092
my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092
my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092
The CLIENT listener for Kafka client connections from within your cluster have been configured with the following security settings:
- SASL authentication
To connect a client to your Kafka, you need to create the 'client.properties' configuration files with the content below:
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
username="user1" \
password="$(kubectl get secret my-kafka-user-passwords --namespace my-kafka -o jsonpath='{.data.client-passwords}' | base64 -d | cut -d , -f 1)";
To create a pod that you can use as a Kafka client run the following commands:
kubectl run my-kafka-client --restart='Never' --image docker.io/bitnami/kafka:3.7.0-debian-12-r6 --namespace my-kafka --command -- sleep infinity
kubectl cp --namespace my-kafka /path/to/client.properties my-kafka-client:/tmp/client.properties
kubectl exec --tty -i my-kafka-client --namespace my-kafka -- bash
PRODUCER:
kafka-console-producer.sh \
--producer.config /tmp/client.properties \
--broker-list my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092,my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092,my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 \
--topic test
CONSUMER:
kafka-console-consumer.sh \
--consumer.config /tmp/client.properties \
--bootstrap-server my-kafka.my-kafka.svc.cluster.local:9092 \
--topic test \
--from-beginning
WARNING: There are "resources" sections in the chart not set. Using "resourcesPreset" is not recommended for production. For production installations, please set the following values according to your workload needs:
- controller.resources
+info https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/.\kubectl.exe get pods -n my-kafka
NAME READY STATUS RESTARTS AGE
my-kafka-client 1/1 Running 0 11h
my-kafka-controller-0 1/1 Running 0 16m
my-kafka-controller-1 1/1 Running 0 16m
my-kafka-controller-2 1/1 Running 0 16mkubectl get secret my-kafka-user-passwords --namespace my-kafka -o jsonpath='{.data.client-passwords}' | base64 -d | cut -d , -f 1
CvqPI061lNkubectl exec --tty -i my-kafka-client --namespace my-kafka -- bash
I have no name!@my-kafka-client:/$ cat /tmp/client.properties
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
username="user1" \
password="CvqPI061lN";root@acharbha-mobl1:~# kubectl exec --tty -i my-kafka-client --namespace my-kafka -- bash
I have no name!@my-kafka-client:/$ kafka-console-producer.sh \
--producer.config /tmp/client.properties \
--broker-list my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092,my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092,my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 \
--topic test
>[2024-05-24 14:07:47,063] ERROR [Producer clientId=console-producer] Connection to node -3 (my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local/10.42.205.55:9092) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-256 (org.apache.kafka.clients.NetworkClient)
[2024-05-24 14:07:47,063] WARN [Producer clientId=console-producer] Bootstrap broker my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 (id: -3 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2024-05-24 14:07:47,072] ERROR [Producer clientId=console-producer] Connection to node -2 (my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local/10.42.203.157:9092) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-256 (org.apache.kafka.clients.NetworkClient)
[2024-05-24 14:07:47,072] WARN [Producer clientId=console-producer] Bootstrap broker my-kafka-controller-1.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 (id: -2 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2024-05-24 14:07:47,387] ERROR [Producer clientId=console-producer] Connection to node -1 (my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local/10.42.97.186:9092) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-256 (org.apache.kafka.clients.NetworkClient)
[2024-05-24 14:07:47,387] WARN [Producer clientId=console-producer] Bootstrap broker my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2024-05-24 14:07:47,697] ERROR [Producer clientId=console-producer] Connection to node -3 (my-kafka-controller-2.my-kafka-controller-headless.my-kafka.svc.cluster.local/10.42.205.55:9092) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-256 (org.apache.kafka.clients.NetworkClient)
from charts.
acharbha
commented on September 25, 2024
kubectl exec --tty -i my-kafka-controller-0 --namespace my-kafka -- bash
/opt/bitnami/kafka/bin$ kafka-metadata-quorum.sh --bootstrap-server my-kafka-controller-0.my-kafka-controller-headless.my-kafka.svc.cluster.local:9092 describe --status
org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: listNodes
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: listNodes
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)
at org.apache.kafka.tools.MetadataQuorumCommand.handleDescribeStatus(MetadataQuorumCommand.java:210)
at org.apache.kafka.tools.MetadataQuorumCommand.execute(MetadataQuorumCommand.java:111)
at org.apache.kafka.tools.MetadataQuorumCommand.mainNoExit(MetadataQuorumCommand.java:62)
at org.apache.kafka.tools.MetadataQuorumCommand.main(MetadataQuorumCommand.java:57)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: listNodesfrom charts.
acharbha
commented on September 25, 2024
I appreciate your offer to help with this. I expect that at least the default configuration should be in a working state.
#26387 (comment)
from charts.
acharbha
commented on September 25, 2024
@fmulero any update ?
from charts.
github-actions
commented on September 25, 2024
This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.
from charts.
fmulero
commented on September 25, 2024
Sorry about my very late response.
About the problem you reported here with the default values, Did you removed previous PV? I am not able to reproduce that problem and I think you have a conflict coming from existing PV.
I also tried to reproduce the original issue with latest version of the chart and I am not able to do it, not sure if changes applied on #27097 could fix this issue also. Could you give it a try?
from charts.
github-actions
commented on September 25, 2024
Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.
from charts.
Related Issues (20)
- [bitnami/apache] cloneHtdocsFromGit causes CrashLoopBackOff in git-repo-syncer HOT 3
- [bitnami/airflow] Worker logs can't be viewed/reached from the UI HOT 1
- [bitnami/discourse] Allow configuration of POSTGRESQL_CLIENT_CREATE_DATABASE_EXTENSIONS for specific plugins HOT 1
- [bitnami/discourse] Plugins are not installed when added in the helm-release HOT 7
- [bitnami/mlflow] password authentication failed for user "bn_mlflow" when deploying with ArgoCD HOT 8
- [bitnami/redis-cluster] Tekton Sidecars redis-cluster change port HOT 1
- Thanos Receive not mapping StorageClass to PVC
- [bitnami/etcd] Upgrading from older chart version of etcd such as 8.5.8 to the latest 10.2.11 with no downtime/data loss. HOT 2
- [bitnami/redis] openssl not found in redis chart HOT 1
- Bitnami helm chart for dragonflydb HOT 1
- [bitnami/keycloak] Not able to reuse pvc/pv from previous deploy HOT 2
- [bitnami/postgresql] Add support for setting custom podantiaffinity (or alternative) HOT 2
- [bitnami/grafana] Is grafana helm error? HOT 3
- [bitnami/postgresql-ha] pgpool can't be restored to health HOT 5
- [bitnami/mlflow] GCS storage for MLFow HOT 2
- [bitnami/postgresql-ha] Existing secret enforces key "postgres-password" even if user is postgres HOT 2
- [bitnami/redis] Can Redis provide a fixed external address for access
- [bitnami/redis] Can Redis provide a fixed external address for access HOT 2
- [bitnami/mongodb] Add client.pem in mongoDB chart with TLS enabled HOT 3
- tls.extraDnsNames is wrongly documented in values.yaml HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from charts.