Comments (9)
Thank you @edwbuck for opening this issue.
It seems that the mechanism using the plugin_data_file
setting with a SIGUSR1 signal is not a good fit for your use case. Could you please provide a short description of your use case so we can better understand the need for this RPC? Thanks!
from spire.
The use case is that in addition to reloading plugins on disk, there is a desire to expose the plugin configuration to an API.
This permits one to perform an API call, from a remote system, to change the plugin configuration. The end result for success would look like:
- The plugin received the new configuration data and accepted it.
- The plugin new configuration data is on disk so the system, should it restart, restarts with the new configuration data.
And in a failure scenario:
- For one reason or another the API call fails (bad config, unchangeable setting, etc.)
- The plugin data on disk is not updated, so the system won't restart with a bad configuration.
Ideally, only plugins that have been reconfigured to use plugin_data_file
would be impacted.
the SIGUSR1
proposal that was put forward by sorin was woefully out-of-date, but the push to make reconfiguration possible was welcomed. It is now a matter of exposing the reconfiguration approach to an API so external management tools can alter plugin configurations without launching a shell and sending a SIGUSR1.
This is not a proposal to remove SIGUSR1, as users that lack company-specific management tooling should still enjoy the benefits of reconfiguration.
from spire.
Thank you Edwin for the details. Maybe @sorindumitru can provide some more context for this request? Particularly, what's the main obstacle to use the SIGUSR1 approach in your use case?
from spire.
My need for this would be to get some feedback from the reload process to see if the configuration is ok or not, to allow for some kind of automated recovery (e.g. roll back configuration to previous config).
At the moment I can alarm based on the log file, but reacting to the error in an automated way is a bit error prone.
from spire.
Can you provide some more context for this request?
Yes, the context is that after asking SPIRE to reconfigure, the config files ideally should be in the state that permits SPIRE server to relaunch in the same way it was running when it shutdown.
Under the current SIGUSR1 approach, monitoring and applying these configuration changes involves many different steps, all which must be coordinated:
- Backup the file
- Write the file
- Send SIGUSR1
- In the Catalog component, sent the configuration to the plugin.
- In the Catalog component, see if an error response was returned by the plugin. (which will be logged as a side-effect).
- Monitor the log (to detect the side effect of failed plugin reconfiguration).
- If the configuration failed, restore the backup.
Steps 1, 2, and 7 require a sysadmin (or tooling) to run the file commands "cp", "cat", and "mv".
Steps 3 require a sysadmin to send the "kill" command
Steps 4 and 5 are internal to SPIRE
Step 6 requires a scanner to scan the log files, and that the SPIRE system is configured with log levels high enough to log the reconfiguration in the first place.
This means there are 4 or 5 command line calls, a need to build a log file parser, and a need to ensure that SPIRE is configured to emit the necessary logged messages. A failure in any one of these items fails to set the system in a state where it is likely to startup with the same running configuration it had when it shutdown.
With an API as the entry point of the new configuration file, we can reorder this as:
- Call the API.
- In the Catalog component, send the configuration to the plugin.
- In the Catalog component, see if an error response was returned by the plugin. (better than scanning log files)
- If the plugin didn't raise an error, rewrite the plugin's
plugin_data_file
with the same string used in step #2. - The return error is sent back to the API caller.
Step 1 is done by an external system, or perhaps even a dedicated tool. It waits till it gets a GRPC response.
Step 2 and 3 are done by SPIRE internally, same as before. Additionally Step 4 is done by SPIRE, and Step 5 is done by SPIRE.
This means that there are fewer independent processes involved, and less need / worry to coordinate them. The config file is only changed AFTER the plugin is successfully reconfigured, meaning that there is a smaller opportunity for the config file and the running server to be different when the plugin fails to accept the new configuration.
Particularly, what's the main obstacle to use the SIGUSR1 approach in your use case?
The need to coordinate the steps above to restore the configuration file to the one that is running in memory.
from spire.
Thank you for the details.
We have discussed this request and we think that providing a way to validate a configuration of a plugin would be good to have. Instead of exposing a new API in the server/agent, this could be achieved through a command in the SPIRE Server CLI and SPIRE Agent CLI, that would output the result of the validation of the configuration of a specified plugin.
At the same time, we don't think that SPIRE should take care of writing/persisting configuration files, as managing configuration files is outside of the scope of SPIRE. We think that this handling should happen outside of SPIRE.
from spire.
Instead of exposing a new API in the server/agent, this could be achieved through a command in the SPIRE Server CLI and SPIRE Agent CLI, that would output the result of the validation of the configuration of a specified plugin.
This sounds reasonable, and will re-establish some aspects of the two-phase (verify and then apply) change pattern that was initially proposed. One remaining item is where to place the new configuration that is being verified. Ideally, it would not be overwriting the running configuration, because that would simply re-establish the issue that we are trying to avoid (overwriting the configuration file until it is applied).
As with any file placement strategy, there are many different patterns. Please consider how the new configuration files can rest on disk without overwriting the old. There are a few obvious choices, and probably a few that I didn't think to mention:
- Putting all the files of the new configuration into a different directory, and using a cli option to relocate the "verify" command to the new directory.
- Putting a prefix or suffix on the new files, and using a cli option to specify the matching "new configuration" files.
- Some other pattern
At the same time, we don't think that SPIRE should take care of writing/persisting configuration files, as managing configuration files is outside of the scope of SPIRE. We think that this handling should happen outside of SPIRE.
Thank you for this design decision. I'll keep it in mind for future efforts.
from spire.
One remaining item is where to place the new configuration that is being verified. Ideally, it would not be overwriting the running configuration, because that would simply re-establish the issue that we are trying to avoid (overwriting the configuration file until it is applied).
The SPIRE CLI command would just take the location of the configuration of a specific plugin that is being verified, and return the result of the verification. I think that is up to the user to choose the best strategy to store those configuration files that are being used for verification, and SPIRE shouldn't really depend on that strategy.
from spire.
Closing this issue in favor of #5253 and #5268.
from spire.
Related Issues (20)
- Nested SPIRE Architecture, NestedA workload invoke NestedB worload error in one case. HOT 6
- server, agent: Allow specifying CA when fetching bundles HOT 3
- Release SPIRE v1.10.1
- spire-agent health check report spire_agent_rpc_workload_api_fetch_x509_bundles{status="PermissionDenied"} metrics HOT 1
- ReplayMissedEvents Incorrectly assumes autoincrement value of 1 for all deployments. HOT 1
- [Event-based entry cache] replayMissedEvents queries DB in loop for every missed ID HOT 4
- [Event-based entry cache] New server config for `sql_transaction_timeout` is not propagated to endpoints config
- 929 SDS connections for stress testing, Spire agent experience memory leaks. HOT 5
- Filter cache update reads to only read each entry once, when an entry has two events in the same refresh cycle. HOT 1
- RFC: Hybrid Node Attestor
- Enable SyncAuthorizedEntries by default
- The spire-agent k8s workload attestor wont refresh kubelet ca if it changes on disk HOT 2
- Force rotation of intermediates using a tainted authority
- Feature to delete individual keys from trust bundle HOT 4
- Document SyncAuthorizedEntries experimental flag in agent
- Remove public key from stored slots
- Propagation of the tainted bit in bundleutil and server/api functions
- s3 bundlePublisher for other implementations
- Allow enabling the LocalAuthority API and related CLI commands through the `forced_rotation` feature flag
- Allow enabling the LocalAuthority API and related CLI commands through an experimental setting instead of a feature flag
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spire.