tholemu / concord-ansible-waf Goto Github PK

Collection of Ansible playbooks to onboard common F5 WAF Configurations.

Optionally uses Concord to run ansible tasks: https://concord.walmartlabs.com/docs/getting-started/installation.html

The included concord.yml file will instruct concord to run the associated ansible playbook to one or more Big-IP instances.

The ansible playbooks can be run without concord, but you would need to supply the proper variables and ansible flags which concord is doing for us. Concord just runs ansible-playbook command under the hood with appropriate flags so 100% of the ansible functionality is possible without concord.

Ansible will import a number of Big-IP configuration items which will be hosted in a git repo. For demo purposes, the following repo is available for reference: https://github.com/megamattzilla/f5PublicConfigCommon

Current Import Playbooks:

install_atc_rpm.yml #Install DO, AS3, and TS RPM files to remote Big-IPs
do.yml #Create and push DO declaration to remote Big-IPs
import_custom_attack_signatures.yml #Pulls F5 signature .im file from external git repo and pushes to remote Big-IPs
import_attack_signatures.yml #Pulls custom signature .xml file from external git repo and pushes to remote Big-IPs
import_tmsh_config.yml #Pulls .conf file configuration from external git repo(s) and pushes to remote Big-IPs
as3.yml #Create and push an AS3 declaration to remote Big-IPs
all.yml #Run all import playbooks above in order.

Current Export Playbooks:

export_config.yml #Exports WAF (ASM) policies to github main branch and version tag.

Coming soon:

non-concord ansible examples  
playbook to export additional F5 config to git repo
playbook to promote repo branch to new tag version

TMOS v15.1.+

Download the AnO Tool chain RPMs you would like to use and place into playbooks/files/ folder. Example:

└── playbooks  
    ├── files  
    │   ├── f5-appsvcs-3.28.0-3.noarch.rpm  
    │   ├── f5-declarative-onboarding-1.21.0-3.noarch.rpm  
    │   └── f5-telemetry-1.20.0-3.noarch.rpm  

Concord.yml has a few required variables in order for the playbook to run. Either add a Concord secret via UI with the passphrase or replace the secrets reference with your own passphrase here (no encryption).

#Lookup F5 admin password from Concord secrets vault: https://concord.walmartlabs.com/docs/plugins/crypto.html
f5adminpassword: "${crypto.exportAsString('Default', 'f5adminpassword', null)}"
#Lookup github personal access token for AS3 to access private repo
As3Token: "${crypto.exportAsString('Default', 'githubAs3Token', null)}"

By default, this concord file will want to query inventory from concord json store. Note that we are still using inventory not the JSON store although they are very similar.

inventory: "${inventory.ansible('Default', 'ansibleinv', 'ansiblegroup', 'ansiblequery2')}"

Change this to an inline inventory for testing, or better yet, setup a concord inventory below!

Some of these playbooks make use of external github repositories to store F5 related configuration data.
In order for ansible to access the external github repositories, you need to create an SSH key with access to the github repositories.
The key can be generated by concord secrets vault, or imported to secrets vault to make available to ansible.
Once the key is stored in concord secrets vault, it can be passed to ansible by using the concord.yml config below:

flows:
  #Name of Flow visible in Concord
  ansible_2.9.20:
    #Export key from secrets vault to a file. Syntax is: ('Organization', 'SSH key name in secrets vault', 'required store password' )
    - expr: ${crypto.exportKeyAsFile('Default', 'concord-config-key3', 'Bestpassword1')}
    ...
    ...
    - task: ansible
      in:
        auth:
          privateKey: 
            path: "${myKeys.private}"
    ...
    ...
        extraVars:
          SSH_KEY_PATH: "${myKeys.private}"

You could then use the SSH key {{ SSH_KEY_PATH }} in your ansible playbooks

My pain is your gain! Here is exactly how to setup a concord inventory from scratch. This can only be done via API call. I included a postman collection as well to do this.

POST http://concord.host.example/api/v1/org/Default/inventory

POST /api/v1/org/Default/inventory HTTP/1.1
Authorization: REDACTED {{concord API token here}}
Content-Type: application/json
User-Agent: PostmanRuntime/7.26.10
Accept: */*
Postman-Token: REDACTED
Host: concord.host.example
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 254
Cookie: JSESSIONID=REDACTED

{
"name": "newansibleinventory",
"orgId": "0fac1b18-d179-11e7-b3e7-d7df4543ed4f",
"orgName": "Default",
"visibility": "PUBLIC",
"owner": {
"id": "230c5c9c-d9a7-11e6-bcfd-bb681c07b26c",
"username": "Admin"
}
}

Expect 200 Ok response.

POST http://concord.host.example:8001/api/v1/org/Default/inventory/newansibleinventory/data/bigip-waf-01

POST /api/v1/org/Default/inventory/newansibleinventory/data/bigip-waf-01 HTTP/1.1
Authorization: REDACTED {{concord API token here}}
Content-Type: application/json
User-Agent: PostmanRuntime/7.26.10
Accept: */*
Postman-Token: REDACTED
Host: concord.host.example
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 288
Cookie: JSESSIONID=REDACTED

{
"data": {
"ipv4": {
"extaddress": "10.10.10.8",
"mgmtmacaddress": "00-0D-3A-EC-4C-44"
},
"location": "scus",
"mgmtaddress": "10.10.11.7",
"environment": "dev"
},
"host": "bigip-waf-01"
}

Expect 200 ok response.

Repeat for each item that needs added. I am still looking for a way to bulk-upload. As the URI needs to point to a key, im not 100% sure this is possible but its trivial to do this on a loop for each Big-IP item anyways.

This part was confusing. Concord has the data we need in an inventory, but we cannot directly call that data via a concord flow. Instead the concord flow will call a query, which we will create now. This query will add ALL Big-IPs in the inventory to the ansible host group when this query is called. To read more about query filtering, see Inventory Query.

POST http://concord.host.example/api/v1/org/Default/inventory/newansibleinventory/query/ansiblequery2

POST /api/v1/org/Default/inventory/newansibleinventory/query/ansiblequery2 HTTP/1.1
Authorization: REDACTED {{concord API token here}}
Content-Type: text/plain
User-Agent: PostmanRuntime/7.26.10
Accept: */*
Postman-Token: REDACTED
Host: concord.host.example
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 54
Cookie: JSESSIONID=REDACTED

SELECT CAST(item_data as varchar) FROM inventory_data;

Expect HTTP 200 Ok response.

We are done with the API only part.

"${inventory.ansible('Default', 'newansibleinventory', 'ansiblegroup', 'ansiblequery2')}"

The syntax is:
Default = #Concord organization where this inventory is created
newansibleinventory = #the inventory name created in step #1
ansiblegroup = #the new ansible group to create. This is the host group ansible playbooks will target.
ansiblequery2 = #the inventory query created on step #3

Concord recommends to initialize the inventory in a logging statement under the flow before its also called as an inventory. Example:

flows:
  #Name of Flow visible in Concord
  ansible_2.9.20:
    - log: "Inventory: ${inventory.ansible('Default', 'ansibleinv', 'ansiblegroup', 'ansiblequery2')}"
    ...
    log first. then later in concord file specify the actual inventory reference for ansible. 
    ...
        inventory: "${inventory.ansible('Default', 'ansibleinv', 'ansiblegroup', 'ansiblequery2')}"

Follow concord quickstart to point to this concord.yml file:
https://concord.walmartlabs.com/docs/getting-started/quickstart.html

Then define your secrets in concord, modify concord.yml variables to point to those secrets, and then start your Concord process!

Ansible playbooks for DO and AS3 make extensive use of Jinja templates. The Jinja template is pulling from variables defined in group_var files.

Group_vars all.yml and ansiblegroup.yml store most of the ansible playbook vars.
group_vars file for ansiblegroup is the group of F5s to receive a common configuration set sourced from a github repo(s).
group_vars file for gold is a single F5 where configuration data can be collected and exported to git repo(s).

The included dockerfile in dockerfile/Dockerfile will allow you to create a custom ansible container based on walmartlabs/concord-ansible. The example docker file updates ansible version to 2.9.20.