Comments (10)
msteed
commented on May 30, 2024
1
The fix will ship in the upcoming 8.16.1 release. For anyone interested in trying it before then, the patch should apply cleanly to recent releases of the plugin and work with existing releases of salt:
Author: Michael Steed <[email protected]>
Date: Fri Jan 19 11:11:05 2024 -0700
fix spurious imports
diff --git a/sseape/utils/plugin.py b/sseape/utils/plugin.py
index f2f9ce7..3df873e 100644
--- a/sseape/utils/plugin.py
+++ b/sseape/utils/plugin.py
@@ -12,7 +12,6 @@ import salt.client
import salt.config
import salt.utils.process
import salt.exceptions
-import salt.ext.tornado.ioloop
import salt.utils.files
import salt.utils.hashutils
import sseape.version
diff --git a/sseape/utils/rmq.py b/sseape/utils/rmq.py
index bb0a92d..1ef58e7 100644
--- a/sseape/utils/rmq.py
+++ b/sseape/utils/rmq.py
@@ -21,9 +21,9 @@ from pika.adapters.select_connection import SelectConnection
from pika.exchange_type import ExchangeType
# Import salt libs
-import salt.ext.tornado
-import salt.ext.tornado.gen
-import salt.ext.tornado.platform.asyncio
+import salt.client
+import salt.loader
+import salt.utils.event
# Import SSEAPE libs
import sseape.utils.config as sseape_configfrom salt.
msteed
commented on May 30, 2024
1
@mastermel This failure was fixed in sseape, the enterprise plugin, version 8.16.1. If you have sseape installed, you can either update it or apply the above patch. If you're not using that plugin, the failure you are seeing is something different.
from salt.
welcome
commented on May 30, 2024
Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey.
Please be sure to review our Code of Conduct. Also, check out some of our community resources including:
- Community Wiki
- Salt’s Contributor Guide
- Join our Community Slack
- IRC on LiberaChat
- Salt Project YouTube channel
- Salt Project Twitch channel
There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar.
If you have additional questions, email us at [email protected]. We’re glad you’ve joined our community and look forward to doing awesome things with you!
from salt.
whytewolf
commented on May 30, 2024
@frogunder this is strange cause almost none of the posted errors besides the minion error seem to be actual errors in salt. and sseapi shouldn't be causing errors in the minion.
master log
2024-01-10 18:10:39,220 [salt.client :1210][WARNING ][28793] Malformed event return: salt/job/20240110181038590782/returning
2024-01-10 18:10:40,456 [sseapiclient.apiclient:309 ][ERROR ][20288] riq 8: ret.save_load: code 4000: No master was found by the ID: test_ubu20
these two errors. the first one happens in 3006.5 and is just a logging issue that needs to be taken care of. but the effect the logging issue has was happening for as long as SSEAPI has been sending that returning event. the second is a message from raas. raas can't find the master_id. so can't save the event. which is odd since you show the master as accepted.
Are there any other errors in the master? something earlier that might have been missed?
the fact the minion doesn't show up in raas tells me the master isn't sending information to raas. cause if the minion is accepted in the master it should just show up, all that happens from the master.
from salt.
frogunder
commented on May 30, 2024
I create a new env to chcek again. I created a raas server and 3007.0rc1 and 3006.5 machines.
3006.5 connects just fine to raas while 3007.0rc1 does not.
The 3006.5 master/minion logs are clean
This is what I see in the 3007.0rc1 logs
master
root@salt:~# cat /var/log/salt/master
2024-01-16 19:56:12,206 [salt.transport.tcp:1405][ERROR ][18092] Publish server binding pub to /var/run/salt/master/master_event_pub.ipc ssl=None
2024-01-16 19:59:45,347 [salt.utils.parsers:1062][WARNING ][17973] Master received a SIGTERM. Exiting.
2024-01-16 19:59:47,326 [salt.transport.tcp:1405][ERROR ][19047] Publish server binding pub to /var/run/salt/master/master_event_pub.ipc ssl=None
2024-01-16 20:03:51,185 [salt.utils.parsers:1062][WARNING ][18997] Master received a SIGTERM. Exiting.
2024-01-16 20:03:52,646 [salt.transport.tcp:1405][ERROR ][19988] Publish server binding pub to /var/run/salt/master/master_event_pub.ipc ssl=None
root@salt:~#
minion
root@salt:~# cat /var/log/salt/minion
2024-01-16 19:56:12,365 [salt.transport.tcp:1405][ERROR ][18049] Publish server binding pub to /var/run/salt/minion/minion_event_09506b349f_pub.ipc ssl=None
2024-01-16 19:56:14,638 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:56:24,651 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:56:34,682 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:56:44,713 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:56:54,743 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:57:04,776 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:57:12,334 [salt.minion :1106][ERROR ][18049] Minion unable to successfully connect to a Salt Master.
2024-01-16 19:57:14,788 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:57:24,810 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:57:34,829 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:57:44,851 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:57:54,871 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:58:04,891 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:58:14,910 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:58:24,930 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:58:34,949 [salt.crypt :1022][ERROR ][18049] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:58:38,960 [salt.utils.parsers:1062][WARNING ][18049] Minion received a SIGTERM. Exiting.
2024-01-16 19:58:42,784 [salt.transport.tcp:1405][ERROR ][18803] Publish server binding pub to /var/run/salt/minion/minion_event_38a810ebdd_pub.ipc ssl=None
2024-01-16 19:58:42,869 [salt.crypt :1022][ERROR ][18803] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:58:52,878 [salt.crypt :1022][ERROR ][18803] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:58:53,743 [salt.utils.parsers:1062][WARNING ][18803] Minion received a SIGTERM. Exiting.
2024-01-16 19:58:57,948 [salt.transport.tcp:1405][ERROR ][18853] Publish server binding pub to /var/run/salt/minion/minion_event_38a810ebdd_pub.ipc ssl=None
2024-01-16 19:58:58,034 [salt.crypt :1022][ERROR ][18853] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 19:59:50,476 [salt.utils.parsers:1062][WARNING ][18853] Minion received a SIGTERM. Exiting.
2024-01-16 19:59:53,205 [salt.transport.tcp:1405][ERROR ][19621] Publish server binding pub to /var/run/salt/minion/minion_event_09506b349f_pub.ipc ssl=None
2024-01-16 19:59:53,586 [salt.crypt :1022][ERROR ][19621] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-16 20:00:47,872 [salt.transport.zeromq:396 ][ERROR ][19621] Exception while running callback
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/salt/transport/zeromq.py", line 394, in consume
await callback(msg)
File "/usr/local/lib/python3.8/dist-packages/salt/channel/client.py", line 486, in wrap_callback
await callback(decoded)
TypeError: object NoneType can't be used in 'await' expression
root@salt:~#
from salt.
whytewolf
commented on May 30, 2024
are there any errors logged in raas? maybe an event was changed and raas no longer recognizes the events.
from salt.
frogunder
commented on May 30, 2024
This is when a 3006.5 machine is connected to raas and work fine and when a 3007.0rc1 is not working.
These logs are from the raas server
raas log is empty
root [ ~ ]# cat /var/log/raas/raas
root [ ~ ]#
master log
root [ ~ ]# cat /var/log/salt/master
2024-01-18 16:54:30,419 [sseape.utils.engine:326 ][ERROR ][1590] sseapi_event_queue: could not connect to SSE server: [Errno 111] Connection refused
2024-01-18 16:54:30,421 [sseape.utils.engine:326 ][ERROR ][1591] sseapi_rpc_queue: could not connect to SSE server: [Errno 111] Connection refused
2024-01-18 16:54:31,035 [salt.transport.zeromq:328 ][ERROR ][1663] ReqServer clients tcp://0.0.0.0:4506
2024-01-18 16:54:31,036 [salt.transport.zeromq:330 ][ERROR ][1663] ReqServer workers ipc:///var/run/salt/master/workers.ipc
2024-01-18 16:54:31,075 [salt.loaded.ext.fileserver.sseapi_fs:145 ][ERROR ][1585] Failed to get envs: [Errno 111] Connection refused
2024-01-18 16:54:31,128 [salt.loaded.ext.fileserver.sseapi_fs:145 ][ERROR ][1674] Failed to get envs: [Errno 111] Connection refused
2024-01-18 16:54:31,144 [salt.loaded.ext.fileserver.sseapi_fs:145 ][ERROR ][1585] Failed to get envs: [Errno 111] Connection refused
2024-01-18 16:54:36,849 [salt.utils.parsers:1060][WARNING ][951] Master received a SIGTERM. Exiting.
2024-01-18 16:54:39,661 [sseape.utils.engine:326 ][ERROR ][2774] sseapi_event_queue: could not connect to SSE server: [Errno 104] Connection reset by peer
2024-01-18 16:54:39,661 [sseape.utils.engine:326 ][ERROR ][2775] sseapi_rpc_queue: could not connect to SSE server: [Errno 104] Connection reset by peer
2024-01-18 16:54:40,087 [salt.transport.zeromq:328 ][ERROR ][2892] ReqServer clients tcp://0.0.0.0:4506
2024-01-18 16:54:40,090 [salt.transport.zeromq:330 ][ERROR ][2892] ReqServer workers ipc:///var/run/salt/master/workers.ipc
2024-01-18 16:54:40,160 [salt.loaded.ext.fileserver.sseapi_fs:145 ][ERROR ][2922] Failed to get envs: [Errno 111] Connection refused
2024-01-18 16:54:40,338 [salt.loaded.ext.fileserver.sseapi_fs:145 ][ERROR ][2772] Failed to get envs: [Errno 111] Connection refused
2024-01-18 16:54:40,403 [salt.loaded.ext.fileserver.sseapi_fs:145 ][ERROR ][2772] Failed to get envs: [Errno 111] Connection refused
2024-01-18 16:54:41,473 [salt.loaded.ext.fileserver.sseapi_fs:292 ][ERROR ][2772] Failed to get the file listing for saltenv(base): [Errno 111] Connection refused
2024-01-18 16:54:44,667 [sseape.utils.engine:326 ][ERROR ][2774] sseapi_event_queue: could not connect to SSE server: [Errno 111] Connection refused
2024-01-18 16:54:44,667 [sseape.utils.engine:326 ][ERROR ][2775] sseapi_rpc_queue: could not connect to SSE server: [Errno 111] Connection refused
2024-01-18 16:54:49,676 [sseape.utils.engine:326 ][ERROR ][2774] sseapi_event_queue: could not connect to SSE server: [Errno 111] Connection refused
2024-01-18 16:54:49,677 [sseape.utils.engine:326 ][ERROR ][2775] sseapi_rpc_queue: could not connect to SSE server: [Errno 111] Connection refused
2024-01-18 16:54:50,433 [salt.loaded.ext.fileserver.sseapi_fs:145 ][ERROR ][2772] Failed to get envs: [Errno 111] Connection refused
2024-01-18 16:54:51,504 [salt.loaded.ext.fileserver.sseapi_fs:292 ][ERROR ][2772] Failed to get the file listing for saltenv(base): [Errno 111] Connection refused
2024-01-18 16:54:56,427 [sseape.utils.engine:99 ][WARNING ][2774] sseapi_event_queue: queue sseapi-events: discarded 20 items due to age limit
2024-01-18 16:54:56,447 [sseape.utils.engine:99 ][WARNING ][2775] sseapi_rpc_queue: queue sseapi-rpc: discarded 2 items due to age limit
root [ ~ ]#
minion log
root [ ~ ]# cat /var/log/salt/minion
2024-01-18 16:54:38,048 [salt.utils.parsers:1060][WARNING ][1322] Minion received a SIGTERM. Exiting.
2024-01-18 16:54:43,896 [salt.crypt :854 ][ERROR ][2684] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2024-01-18 16:54:53,912 [salt.crypt :854 ][ERROR ][2684] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
from salt.
whytewolf
commented on May 30, 2024
found the issue. looks like a code fix on the sseapi side of things to account for us dropping the vendor tornado.
from salt.
mastermel
commented on May 30, 2024
Hey friends, I just updated a 3006.5 master to 3007.0 and am no longer able to send commands to any minion (all still running 3006.5). After starting the salt-master service I get the following entry in its log and then nothing else:
2024-03-07 09:22:12,327 [salt.transport.tcp:1405][ERROR ][3719945] Publish server binding pub to /var/run/salt/master/master_event_pub.ipc ssl=None
This seems to be a similar error as reported by @frogunder. Is there a fixed version available yet?
from salt.
whytewolf
commented on May 30, 2024
since 8.16.2 was released going to go ahead and close this.
from salt.
Related Issues (20)
- [BUG] job cache is saved locally even if job_cache is set to False in master config HOT 1
- [BUG] salt-master defunct after API calls using runner client HOT 3
- [BUG] Using IPv6 in 3007.0 causes TCP PublishServer to crash HOT 1
- [BUG] Channels are closed while listened to in 3007.0 HOT 1
- [BUG] salt-ssh breaks on FreeBSD if IFCONFIG_FORMAT is set to return CIDR HOT 1
- [BUG] MySQL Version caching ignores connection_args breaking states the use multiple MySQL servers
- [DOCS] Link to Slack community only works if you have a saltstack.com email address HOT 2
- [BUG] [3007] Salt-master doesn't start when ssl is enabled HOT 2
- [BUG] FIPS mode uses non FIPS algorithms.
- [DOCS] state_top_saltenv documention is inaccurate in regard to when it applies
- [BUG] file_roots to relative path seems broken HOT 1
- [BUG] states from gitfs fail with AttributeError: '_pygit2.Blob' object has no attribute 'oid' with pygit2 1.15.0 HOT 3
- [BUG] [3007] Problem running salt function in Jinja template: Cannot run the event loop while another loop is running HOT 2
- [BUG] [3007.1] startup_states: highstate stop working HOT 1
- [BUG] [Regression] yaml output broken in 3006.8
- [BUG] Error trying to install packages with zypper in AutoYaST process HOT 2
- [BUG] Unless or Onlyif requisite isn't working anymore in 3007.1 with cmd.run and shell powershell. HOT 3
- [BUG] 3007.1 fail to write to vault when using self-signed certificates (verify config not honored) HOT 2
- [BUG] Salt-SSH: Accessing modules in Jinja templates via attributes disables wrappers
- [BUG] DeprecationWarning: ssl.PROTOCOL_TLS is deprecated
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from salt.