saltstack / community Goto Github PK

Same steps as #47 but build an ARM package

Modify the build scripts
Make sure dependencies build
Go to the latest version available

I am thinking that wg-cloud would be the perfect one to go first, and maybe one other. Obviously, we have some work to do before we actually launch, announce, etc. But deciding what should go first now will help with planning.

@gtmanfred How do you feel about this?

What other group might also be good to launch in the first round? Or should we only do one first?

The other possibilities are (that have already been partially defined):

• wg-formulas (led by Daniel)
• wg-transport (led by Mike)
• wg-windows (led by Shane)

I spoke with Shane about this WG idea and he said he'd like to see how some of the other meetings go before jumping in with the Windows group. I have not spoken with Mike or Daniel about these other groups yet.

It's possible to arrange a new one as well if there are other ideas.

Please let me know your thoughts @Ch3LL, @gtmanfred, and @garethgreenaway?

How does it work to add and remove members to a WG? Let's define this and document it.

These docs: https://github.com/saltstack/community/blob/master/proposed-wg/README.md are a good start, but they could use some more information.

Let's get the information about how to propose a WG added here. Let's make these steps very clear. For example, the following sentence in those docs:

To request ownership, please comment on the GitHub issue for the working group for which you wish to volunteer and thanks!

Is unclear to me. Which GitHub issue? Is that how proposed groups are defined first? (That's a fine process to me, but we should state how to do that somewhere.)

Also, the first sentence,

This directory contains proposals for working groups which do not yet have enough active leadership to be considered an active group.

How can one become a leader? Let's link to the doc we create about leader responsibilities here as well.

What else can we add here that will be useful?

After looking through these issues, it's clear that there are a lot of good ideas here.

I don't think all of them are necessary to complete prior to rolling this out, however. Could we go through these and pick out which ones are strictly necessary prior to launching this project? It would be nice to have a clear set of steps to get this out the door.

Refs #20

The following artifacts will need to be archived and publicly available:

• Agendas
• Meeting Notes
• Recordings of the Meetings

Look into using Google Docs for this and maybe YouTube for the meeting recordings. Or maybe Zoom offers something in that vein.

And, of course, document these processes!

Rigel currently has a wrapper around the service execution module for creating services.
Add the ability to create a service in a stateful manner to the states/service.py state module.

https://docs.saltstack.com/en/latest/ref/states/writing.html

Mircea Ulinic doesn't link to his git hub profile

We need to document what is expected of the WG leaders and what that means. There's some initial ideas in the table of the Community Membership page as well as in the bullet list of the Proposing a WG section.

We should define this more clearly and all in one place.

Now that we have slack set up for the community, let's make sure we document how to join the slack group (and associated channels) like we do for IRC.

Look at the following:

• Ability to log to the Windows Event Logs (log handler)
• Ability to Query the Windows Event Logs
• Ability to configure logging
• Shorten the names of the module and the functions

• Update the current ssh_ext_alternatives feature in Salt-SSH to automatically attempt to find the necessary python libraries that Salt-SSH uses to copy over and check to see if they are importable. If a user needs additional libraries other than the default they will need to add those manually to the configuration.

• Add tests for anything changed in the ssh_ext_alternatives feature

• Document changes

• Research ssh_ext_alternatives to see if there are any other improvements we can make to make the python3 migration for salt-ssh easier. If there is significant changes found during this step it might require writing another SEP.

Part of work towards adding this SEP: saltstack/salt-enhancement-proposals#11

Must find leadership for networking WoG.

• Leader (2-3 hrs/week)
• Secondary (1-2 hrs/week)

Port waiting for review: saltstack/salt#55237

Dependent on this work: #47

• Add logic into salt-ssh so it can use the python3 binary being built from #47

• If the ARM architecture is detected we will include a warning to the user to download the binary. To make this download or upgrade of the binary easier, a switch/arg will be added to the salt-ssh cli tool that will handle this logic for both binaries.

saltstack/salt#51863

It's generally good practice in GH to have a contributing file. I am not sure what this will looking like for this repo yet, but we should make sure we have to and direct people to right places.

All WG meetings should have an agenda defined ahead of time by the WG leaders. There should be documentation about the need for this, what kinds of things might be good to have on an agenda, and how different groups might think about having other people interested in (or already a member of) the WG participate or propose agenda items.

• Document changes including the different solutions for using salt-ssh via python3 on a python2 host in both the Sodium release notes and Salt-SSH docs.

• Bring MSI installer features in line with Nullsoft installer.
• Automate MSI build process

I am using a simple salt state to send (file.managed) and execute (cmd.run) a shell script on a minion/target. No matter what exit or return value the shell script sends, the salt master is interpreting the result as successful.

I tried using cmd.script, but keep getting a permission denied error on the temp version of the file under /tmp. Filesystem is not mounted with noexec so we can't figure out why it won't work. I saw earlier references to a bug for this but was supposedly fixed in 2012.

For cmd.run, stdout in the job output shows the failed return code and message but Salt still says Success. Running the script locally on the minion reports the return/exit code as expected.

I tried adding stateful: True into the cmd.run block and formatted the key value pairs at the end of the shell script as demonstrated in the docs.

Running against 2 minion target, 1 fail 1 succeed. Both report Result as True but correctly populate Comment with my key value pair.

I've tried YES/NO, TRUE/FALSE, 0/1 - nothing works.

The end of my shell script, formatted as shown in the docs.

echo Return_Code=${STATUS}

if [[ ${STATUS} -ne 0 ]]
   then
      echo ""
      echo "changed=False comment='Failed'"
   else
      echo ""
      echo "changed=True comment='Success'"
fi

The SLS block:

stop_oracle:
  cmd.run:
    - name: {{scriptDir}}/{{scriptName}}{{scriptArg}}
    - stateful: True
    - failhard: True

SLS output from Successful minion:

----------
          ID: stop_oracle
    Function: cmd.run
        Name: /u01/orastage/oraclepsu/scripts/oracle_ss_wrapper.ksh stop
      Result: True
     Comment: Success
     Started: 14:37:44.519131
    Duration: 18930.344 ms
     Changes:
              ----------
              changed:
                  True
              pid:
                  26195
              retcode:
                  0
              stderr:
                  stty: standard input: Inappropriate ioctl for device
              stdout:
                  Script running under ROOT

                  Mon Jul  1 14:38:03 EDT 2019 : Successful
                  Return_Code=0

SLS output from Failed minion:

----------
          ID: stop_oracle
    Function: cmd.run
        Name: /u01/orastage/oraclepsu/scripts/oracle_ss_wrapper.ksh stop
      Result: True
     Comment: Failed
     Started: 14:07:14.153940
    Duration: 38116.134 ms
     Changes:

Output from shell script run locally on fail target:

$ /u01/orastage/oraclepsu/scripts/oracle_ss_wrapper.ksh stop

Mon Jul  1 15:29:18 EDT 2019 : There are errors in the process
Return_Code=1

changed=False comment='Failed'

Output from shell script run locally on success target:

$ /u01/orastage/oraclepsu/scripts/oracle_ss_wrapper.ksh stop

Mon Jul  1 16:03:18 EDT 2019 : Successful
Return_Code=0

changed=True comment='Success'

Version Information:

Master: Oracle linux, 2018.3.2 (Oxygen) 
Minion: Redhat Linux, 2017.7.4 (Nitrogen) 
Minion: AIX, 2017.7.2 (Nitrogen) 

Outside of the WG meetings, it would be a good idea to hold main community meetings. This is where the leaders of each WG could come and do a standup-type meeting about what they're working on and how things are going.

We could do this monthly or every other week or something like that.

Let's flesh out what this looks like and get it planned.

@gtmanfred @garethgreenaway @Ch3LL any initial thoughts on this idea?

Work with Rhett to get some information added about the WGs on our main website. I like what Kubernetes has done here: https://kubernetes.io/community/

I like that they have the calendar there showing all of the registered WG meetings. I also like how they explicitly state that these ideas meetings are open for any one to join.

Let's get something like this up on saltstack.com once we have several groups up and running.

We will need to monitor and make sure our Python 3 binary and other built libraries are kept up to date with any security releases. Since the Python 3 binary will be managed outside of the salt repo we can do releases outside of the Salt release cycle to include these new patches and updates quickly. We will add a tool to help monitor if there are vulnerabilities in our dependencies. Either pyup or
github security alerts

• add a tool to help monitor for vulnerabilities in the python3 binary

Part of work towards adding this SEP: saltstack/salt-enhancement-proposals#11

Requirement

Below is a list of slowest tests in Salt new master branch
https://docs.google.com/spreadsheets/d/18a2jlsEwrBH73emohkEsdPVbThfuhg5TbMLbEANu-Uw/edit?usp=sharing

Pending Decisions

Benchmarks for acceptable test duration

Script

Below script can be used to generate the slowest tests and also to test the duration after improvement
https://github.com/waynew/salt/blob/utils/tools/slow-test-report.py

Maintain and build a Python 3 binary that will be copied over with the tarball to be included in the salt-call run. This will also be gated by a config option that a user will need to enable if they always want to use the binary option.

How are we going to build the Python 3 binary?
We will build the binary statically for both x86_64 and ARM architecture. We will include the x86_64 binary by default in the Salt-SSH package. If the ARM architecture is detected we will include a warningto the user to download the binary. To make this download or upgrade of the binary easier, a switch/arg will be added to the salt-ssh cli tool that will handle this logic for both binaries. We will include the required libraries to run salt-call and include the pip binary. If users want to include other dependencies on top of this binary they can use the ssh_ext_alternatives feature to include the additional dependencies.

Python Binary Security Releases:
We will need to monitor and make sure our Python 3 binary and other built libraries are kept up to date with any security releases. Since the Python 3 binary will be managed outside of the salt repo we can do releases outside of the Salt release cycle to include these new patches and updates quickly. We will add a tool to help monitor if there are vulnerabiltiies in our dependencies. Either pyup or github security alerts

• Research best way to build x86_64 python3 statically.

• Need to review the licenses of all the libraries included in the build to see if there are any licensing conflict.

• If there are any issues found during the research of this task which requires significant changes to the original plan might need to open a new SEP.

Part of work towards adding this SEP: saltstack/salt-enhancement-proposals#11

Instructions are provided in docs/README.md file on how to change any files.

There are a few things that could make this page more friendly.

• Include a link to the formulas repo
• Make it clear that the first link under Meetings opens up a Google HangOut. It's a bit jarring to click on it thinking "what does this do?" and have it open a HangOut.
• Include links to where meeting notes, agendas, and recordings can be found
• Add contact information links (slack channels, mailing list, open issues link)
• List the GH teams involved here
• List the GH labels involved here

I used this page as an example: https://github.com/kubernetes/community/tree/master/sig-architecture

The main README is what everyone will see first when they come to this repo - let's make it simple yet informative of what we're trying to accomplish with these working groups. Let's leverage this area to link to the other documents about how to get involved.

• Leader (2-3 hrs/week)
• Secondary leader (1-2 hrs/week)

Each WG will need a label in the salt repo. Currently, there are two labels related to salt-cloud:

• Salt-Cloud
• team-cloud

What is the purpose of either of these labels? Why are there two? Do we need two? Can we standardize on one and use it for wg-cloud?

@gtmanfred ^^ :)

We need to create a document that explains how to create a new WG. This has been partially done in community-membership.md, but I think we can make this better.

We need to define what it really takes in more clear steps what creating a new WG requires. We can base this on what kubernetes does in their sig-governance doc.

What needs to be done? The WG needs, at a minimum:

• name and purpose - this should be proposed somewhere. In an issue maybe? general salt-users?
• label in salt (or related repo?) - how do you get one? what should it be named?
• leader (or two) defined
• where are discussions happening? contact info: slack? google user group?
• Meeting schedule (calendar!)

How can we adapt some of the checklist kubernetes has to work for the salt repo most effectively?

The best place to add this to start would be to Salt's README. But there are probably other places we can point back to this repository as well.

This is related somewhat to #16.

Verify that removing the uncommented portions of the config file on Windows has no effect and that the values set in the config are actually the default values.

https://trello.com/c/iIEhrJnA/361-follow-the-missing-patch-from-develop-to-master

Dependent on this issue: #47

• Automate the build of python3 (with drone, jenkins or travisci) for salt-ssh

• Create a test plan. Tests need to run on any new builds of the python3 binary, testing all salt branches that use this binary.

This binary will need to be managed in a repo outside of salt so we can release outside of the salt release cycle.

We need to add documentation about community meetings. This can be added to an existing document, or it can live in it's own page.

Regardless of what we decide, we need to make sure we link to this documentation from the necessary pages. Review the docs we currently have available/proposed and decide where adding mentions of this would be useful.

The document should include:

• What the community meeting is
• Who is involved in the community meeting
• What is discussed at the community meeting/general agendas
• When the community meeting is held

This is largely dependent on #14.

• add tests that use the pip module/state for all of the different salt-ssh options including: python3 binary, ssh_ext_alternatives, pre_flight cmd.

saltstack/salt#49277

• make sure if python versions match we will simply use the system python and Salt-SSH will run as it always has.

• If the python versions do not match and an alternative option is not set, when Salt-SSH is run a message will be printed out asking the user if they want to copy over the binary, and informing them of the other alternative options pre_flight and ssh_ext_alternatives.

• Add tests for changes above

Create channels for all approved WOGs that don't already have groups.

• Add a Salt-SSH configuration (for example: pre_flight) to run raw commands before the Salt-SSH command. This will allow a user to create a custom script they can run to install Python 3. It will only run these pre_flight commands if the tarball is not copied over.

• Add test coverage for this new feature

• Document the new feature for sodium release

Part of work towards adding this SEP: saltstack/salt-enhancement-proposals#11