New installation.
I see 401 Unauthorized
error in logs. Also, Kibana asks for credentials.
To Reproduce
Steps to reproduce the behavior:
- Install
- check logs
logstash | [WARN ] 2022-04-26 11:28:56.121 [Ruby-0-Thread-10: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:213] elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://pfelk_logstash:xxxxxx@es01:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://es01:9200/'"}
Screenshots
If applicable, add screenshots to help explain your problem.
Operating System (please complete the following information):
Linux 5.15.0-27-generic x86_64
PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
- Version of Docker (docker version 20.10.12, build 20.10.12-0ubuntu4)
- Version of Docker-Compose (docker-compose version 1.29.2, build unknown)
**Elasticsearch, Logstash, Kibana (please complete the following information):**
- Version of ELK (ELK_VERSION=8.1.1)
**Service logs
- `docker-compose logs pfelk01`
{"@timestamp":"2022-04-26T11:39:31.318Z", "log.level": "INFO", "message":"Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es01][transport_worker][T#3]","log.logger":"org.elasticsearch.xpack.security.authc.RealmsAuthenticator","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"sghCi4faTMq-0HWfuHE6Wg","elasticsearch.node.name":"es01","elasticsearch.cluster.name":"es-docker-cluster"}
- `docker-compose logs pfelk02`
{"@timestamp":"2022-04-26T11:27:10.187Z", "log.level": "INFO", "message":"added {{es03}{he_uTsJLRVqVdt94ZJZedQ}{XIGGdlwDTu6hmdcuNqYScQ}{172.18.0.5}{172.18.0.5:9300}{cdfhilmrstw}}, term: 2, version: 30, reason: ApplyCommitRequest{term=2, version=30, sourceNode={es01}{sghCi4faTMq-0HWfuHE6Wg}{29Cab6DxSUeO4rLOk3WPYw}{172.18.0.3}{172.18.0.3:9300}{cdfhilmrstw}{ml.machine_memory=16754929664, xpack.installed=true, ml.max_jvm_size=536870912}}", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.service.ClusterApplierService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"LW-kJ81hQICwVKDyxvp-lA","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:14.199Z", "log.level": "INFO", "message":"license [9d9bf110-6edb-43d5-80ed-ef06331a806b] mode [basic] - valid", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.license.LicenseService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"LW-kJ81hQICwVKDyxvp-lA","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:14.200Z", "log.level": "INFO", "message":"license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.xpack.security.authc.Realms","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"LW-kJ81hQICwVKDyxvp-lA","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:17.829Z", "log.level": "INFO", "message":"retrieve geoip database [GeoLite2-ASN.mmdb] from [.geoip_databases] to [/tmp/elasticsearch-8501814693674185395/geoip-databases/LW-kJ81hQICwVKDyxvp-lA/GeoLite2-ASN.mmdb.tmp.gz]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"LW-kJ81hQICwVKDyxvp-lA","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:18.677Z", "log.level": "INFO", "message":"successfully loaded geoip database file [GeoLite2-ASN.mmdb]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][generic][T#3]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"LW-kJ81hQICwVKDyxvp-lA","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:34.902Z", "log.level": "INFO", "message":"retrieve geoip database [GeoLite2-City.mmdb] from [.geoip_databases] to [/tmp/elasticsearch-8501814693674185395/geoip-databases/LW-kJ81hQICwVKDyxvp-lA/GeoLite2-City.mmdb.tmp.gz]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"LW-kJ81hQICwVKDyxvp-lA","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:36.504Z", "log.level": "INFO", "message":"retrieve geoip database [GeoLite2-Country.mmdb] from [.geoip_databases] to [/tmp/elasticsearch-8501814693674185395/geoip-databases/LW-kJ81hQICwVKDyxvp-lA/GeoLite2-Country.mmdb.tmp.gz]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"LW-kJ81hQICwVKDyxvp-lA","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:36.716Z", "log.level": "INFO", "message":"successfully loaded geoip database file [GeoLite2-Country.mmdb]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][generic][T#4]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"LW-kJ81hQICwVKDyxvp-lA","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:36.974Z", "log.level": "INFO", "message":"successfully loaded geoip database file [GeoLite2-City.mmdb]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es02][generic][T#1]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"LW-kJ81hQICwVKDyxvp-lA","elasticsearch.node.name":"es02","elasticsearch.cluster.name":"es-docker-cluster"}
- `docker-compose logs pfelk03`
{"@timestamp":"2022-04-26T11:27:10.767Z", "log.level": "INFO", "message":"publish_address {172.18.0.5:9200}, bound_addresses {0.0.0.0:9200}", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.http.AbstractHttpServerTransport","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"he_uTsJLRVqVdt94ZJZedQ","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:10.768Z", "log.level": "INFO", "message":"started", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.node.Node","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"he_uTsJLRVqVdt94ZJZedQ","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:14.211Z", "log.level": "INFO", "message":"license [9d9bf110-6edb-43d5-80ed-ef06331a806b] mode [basic] - valid", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es03][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.license.LicenseService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"he_uTsJLRVqVdt94ZJZedQ","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:14.212Z", "log.level": "INFO", "message":"license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es03][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.xpack.security.authc.Realms","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"he_uTsJLRVqVdt94ZJZedQ","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:17.828Z", "log.level": "INFO", "message":"retrieve geoip database [GeoLite2-ASN.mmdb] from [.geoip_databases] to [/tmp/elasticsearch-18255990313510465210/geoip-databases/he_uTsJLRVqVdt94ZJZedQ/GeoLite2-ASN.mmdb.tmp.gz]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es03][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"he_uTsJLRVqVdt94ZJZedQ","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:18.665Z", "log.level": "INFO", "message":"successfully loaded geoip database file [GeoLite2-ASN.mmdb]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es03][generic][T#1]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"he_uTsJLRVqVdt94ZJZedQ","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:34.906Z", "log.level": "INFO", "message":"retrieve geoip database [GeoLite2-City.mmdb] from [.geoip_databases] to [/tmp/elasticsearch-18255990313510465210/geoip-databases/he_uTsJLRVqVdt94ZJZedQ/GeoLite2-City.mmdb.tmp.gz]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es03][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"he_uTsJLRVqVdt94ZJZedQ","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:36.516Z", "log.level": "INFO", "message":"retrieve geoip database [GeoLite2-Country.mmdb] from [.geoip_databases] to [/tmp/elasticsearch-18255990313510465210/geoip-databases/he_uTsJLRVqVdt94ZJZedQ/GeoLite2-Country.mmdb.tmp.gz]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es03][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"he_uTsJLRVqVdt94ZJZedQ","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:36.693Z", "log.level": "INFO", "message":"successfully loaded geoip database file [GeoLite2-Country.mmdb]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es03][generic][T#3]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"he_uTsJLRVqVdt94ZJZedQ","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"es-docker-cluster"}
{"@timestamp":"2022-04-26T11:27:37.062Z", "log.level": "INFO", "message":"successfully loaded geoip database file [GeoLite2-City.mmdb]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es03][generic][T#2]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"ynFu1V-JT7S131Cr1K0OSw","elasticsearch.node.id":"he_uTsJLRVqVdt94ZJZedQ","elasticsearch.node.name":"es03","elasticsearch.cluster.name":"es-docker-cluster"}
- `docker-compose logs logstash`
[WARN ] 2022-04-26 11:41:31.580 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:213] licensereader - Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://elastic:xxxxxx@es01:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://es01:9200/'"}
[WARN ] 2022-04-26 11:41:31.996 [Ruby-0-Thread-10: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:213] elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://pfelk_logstash:xxxxxx@es01:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://es01:9200/'"}
[WARN ] 2022-04-26 11:41:37.001 [Ruby-0-Thread-10: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:213] elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://pfelk_logstash:xxxxxx@es01:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://es01:9200/'"}
[WARN ] 2022-04-26 11:41:42.016 [Ruby-0-Thread-10: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:213] elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://pfelk_logstash:xxxxxx@es01:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://es01:9200/'"}
[WARN ] 2022-04-26 11:41:47.021 [Ruby-0-Thread-10: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:213] elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://pfelk_logstash:xxxxxx@es01:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://es01:9200/'"}
- `docker-compose logs kibana`
[2022-04-26T11:26:36.862+00:00][INFO ][plugins-service] Plugin "metricsEntities" is disabled.
[2022-04-26T11:26:36.978+00:00][INFO ][http.server.Preboot] http server running at http://0.0.0.0:5601
[2022-04-26T11:26:37.028+00:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
[2022-04-26T11:26:37.061+00:00][WARN ][config.deprecation] The default mechanism for Reporting privileges will work differently in future versions, which will affect the behavior of this cluster. Set "xpack.reporting.roles.enabled" to "false" to adopt the future behavior before upgrading.
[2022-04-26T11:26:37.269+00:00][INFO ][plugins-system.standard] Setting up [112] plugins: [translations,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,usageCollection,taskManager,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,sharedUX,share,embeddable,uiActionsEnhanced,screenshotMode,screenshotting,banners,telemetry,newsfeed,fieldFormats,expressions,dataViews,charts,esUiShared,bfetch,data,savedObjects,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,watcher,licenseManagement,advancedSettings,spaces,security,savedObjectsTagging,reporting,lists,fileUpload,ingestPipelines,encryptedSavedObjects,dataEnhanced,cloud,snapshotRestore,eventLog,actions,alerting,triggersActionsUi,transform,stackAlerts,ruleRegistry,savedObjectsManagement,console,controls,graph,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,visualizations,canvas,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,rollup,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,dashboard,maps,dashboardEnhanced,expressionTagcloud,expressionPie,visTypePie,expressionMetricVis,expressionHeatmap,expressionGauge,dataViewFieldEditor,lens,cases,timelines,discover,osquery,observability,discoverEnhanced,dataVisualizer,ml,uptime,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,dataViewManagement]
[2022-04-26T11:26:37.308+00:00][INFO ][plugins.taskManager] TaskManager is identified by the Kibana UUID: c40fdeb9-0c5c-4299-b81a-7a6e03fc4c30
[2022-04-26T11:26:37.556+00:00][WARN ][plugins.security.config] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2022-04-26T11:26:37.557+00:00][WARN ][plugins.security.config] Session cookies will be transmitted over insecure connections. This is not recommended.
[2022-04-26T11:26:37.579+00:00][WARN ][plugins.security.config] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2022-04-26T11:26:37.583+00:00][WARN ][plugins.security.config] Session cookies will be transmitted over insecure connections. This is not recommended.
[2022-04-26T11:26:37.619+00:00][WARN ][plugins.reporting.config] Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2022-04-26T11:26:37.626+00:00][WARN ][plugins.reporting.config] Found 'server.host: "0.0.0.0"' in Kibana configuration. Reporting is not able to use this as the Kibana server hostname. To enable PNG/PDF Reporting to work, 'xpack.reporting.kibanaServer.hostname: localhost' is automatically set in the configuration. You can prevent this message by adding 'xpack.reporting.kibanaServer.hostname: localhost' in kibana.yml.
[2022-04-26T11:26:37.636+00:00][WARN ][plugins.encryptedSavedObjects] Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2022-04-26T11:26:37.654+00:00][WARN ][plugins.actions] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2022-04-26T11:26:37.671+00:00][WARN ][plugins.alerting] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2022-04-26T11:26:37.714+00:00][INFO ][plugins.ruleRegistry] Installing common resources shared between all indices
[2022-04-26T11:26:38.549+00:00][INFO ][plugins.screenshotting.config] Chromium sandbox provides an additional layer of protection, and is supported for Linux Ubuntu 20.04 OS. Automatically enabling Chromium sandbox.
[2022-04-26T11:26:42.165+00:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. connect ECONNREFUSED 172.18.0.3:9200
[2022-04-26T11:26:44.118+00:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/x-pack/plugins/screenshotting/chromium/headless_shell-linux_x64/headless_shell
[2022-04-26T11:27:07.675+00:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. security_exception: [security_exception] Reason: unable to authenticate user [kibana_system] for REST request [/_nodes?filter_path=nodes..version%2Cnodes..http.publish_address%2Cnodes.*.ip]
[2022-04-26T11:27:23.854+00:00][INFO ][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
[2022-04-26T11:27:23.855+00:00][INFO ][savedobjects-service] Starting saved objects migrations
[2022-04-26T11:27:24.626+00:00][INFO ][savedobjects-service] [.kibana] INIT -> CREATE_NEW_TARGET. took: 94ms.
[2022-04-26T11:27:24.673+00:00][INFO ][savedobjects-service] [.kibana_task_manager] INIT -> CREATE_NEW_TARGET. took: 139ms.
[2022-04-26T11:27:34.506+00:00][INFO ][savedobjects-service] [.kibana_task_manager] CREATE_NEW_TARGET -> MARK_VERSION_INDEX_READY. took: 9833ms.
[2022-04-26T11:27:34.668+00:00][INFO ][savedobjects-service] [.kibana] CREATE_NEW_TARGET -> MARK_VERSION_INDEX_READY. took: 10042ms.
[2022-04-26T11:27:34.756+00:00][INFO ][savedobjects-service] [.kibana_task_manager] MARK_VERSION_INDEX_READY -> DONE. took: 250ms.
[2022-04-26T11:27:34.756+00:00][INFO ][savedobjects-service] [.kibana_task_manager] Migration completed after 10222ms
[2022-04-26T11:27:34.825+00:00][INFO ][savedobjects-service] [.kibana] MARK_VERSION_INDEX_READY -> DONE. took: 157ms.
[2022-04-26T11:27:34.825+00:00][INFO ][savedobjects-service] [.kibana] Migration completed after 10293ms
[2022-04-26T11:27:35.306+00:00][INFO ][status] Kibana is now unavailable
[2022-04-26T11:27:35.306+00:00][INFO ][plugins-system.preboot] Stopping all plugins.
[2022-04-26T11:27:35.307+00:00][INFO ][plugins-system.standard] Starting [112] plugins: [translations,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,usageCollection,taskManager,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,sharedUX,share,embeddable,uiActionsEnhanced,screenshotMode,screenshotting,banners,telemetry,newsfeed,fieldFormats,expressions,dataViews,charts,esUiShared,bfetch,data,savedObjects,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,watcher,licenseManagement,advancedSettings,spaces,security,savedObjectsTagging,reporting,lists,fileUpload,ingestPipelines,encryptedSavedObjects,dataEnhanced,cloud,snapshotRestore,eventLog,actions,alerting,triggersActionsUi,transform,stackAlerts,ruleRegistry,savedObjectsManagement,console,controls,graph,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,visualizations,canvas,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,rollup,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,dashboard,maps,dashboardEnhanced,expressionTagcloud,expressionPie,visTypePie,expressionMetricVis,expressionHeatmap,expressionGauge,dataViewFieldEditor,lens,cases,timelines,discover,osquery,observability,discoverEnhanced,dataVisualizer,ml,uptime,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,dataViewManagement]
[2022-04-26T11:27:36.864+00:00][INFO ][plugins.fleet] Beginning fleet setup
[2022-04-26T11:27:36.882+00:00][INFO ][plugins.monitoring.monitoring] config sourced from: production cluster
[2022-04-26T11:27:37.827+00:00][INFO ][http.server.Kibana] http server running at http://0.0.0.0:5601
[2022-04-26T11:27:38.412+00:00][INFO ][plugins.monitoring.monitoring.kibana-monitoring] Starting monitoring stats collection
[2022-04-26T11:27:39.366+00:00][INFO ][status] Kibana is now degraded (was unavailable)
[2022-04-26T11:27:40.399+00:00][INFO ][plugins.ruleRegistry] Installed common resources shared between all indices
[2022-04-26T11:27:40.399+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.uptime.alerts
[2022-04-26T11:27:40.400+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-security.alerts
[2022-04-26T11:27:40.400+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .preview.alerts-security.alerts
[2022-04-26T11:27:40.400+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.logs.alerts
[2022-04-26T11:27:40.400+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.metrics.alerts
[2022-04-26T11:27:40.400+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.apm.alerts
[2022-04-26T11:27:40.838+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.uptime.alerts
[2022-04-26T11:27:40.980+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.logs.alerts
[2022-04-26T11:27:41.107+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.metrics.alerts
[2022-04-26T11:27:41.436+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-security.alerts
[2022-04-26T11:27:41.714+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.apm.alerts
[2022-04-26T11:27:43.429+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found
[2022-04-26T11:27:43.484+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .preview.alerts-security.alerts
[2022-04-26T11:27:45.059+00:00][INFO ][plugins.fleet] Fleet setup completed
[2022-04-26T11:27:45.103+00:00][INFO ][plugins.securitySolution] Dependent plugin setup complete - Starting ManifestTask
[2022-04-26T11:27:47.738+00:00][INFO ][status] Kibana is now available (was degraded)
[2022-04-26T11:27:47.757+00:00][INFO ][plugins.reporting.store] Creating ILM policy for managing reporting indices: kibana-reporting
**Additional context**
Add any other context about the problem here.