hewlettpackard / squest Goto Github PK

View Code? Open in Web Editor NEW

276.0 19.0 47.0 17.26 MB

Service request portal on top of Ansible Tower/AWX

Home Page: https://hewlettpackard.github.io/squest/latest

License: Apache License 2.0

Python 85.87% Shell 0.04% Dockerfile 0.09% CSS 0.25% JavaScript 0.74% HTML 13.01%

ansible tower catalog portal django awx rhaap

squest's Introduction

Self service portal on top of Red Hat Ansible Automation Platform

Squest is a sophisticated framework that empowers you to create a web portal offering a comprehensive Everything-as-a-Service experience.

In Squest, the services made available are not hardcoded. Instead, each service serves as a reference to your automation capabilities within the backend engine, Red Hat Ansible Automation Platform (RHAAP)/AWX (formerly known as Ansible Tower). This essentially means that anything, as long as it exists as an automation script, can be transformed into a service. Your only limitation lies in your own proficiency in automation.

Going beyond mere deployment, Squest maintains a record of every instance of services it provisions in its database. This allows you to augment these instances with additional automation, enabling your users to effectively manage the lifecycle of their resources. Each operation performed corresponds to yet another link to an automation process in the backend. This empowering feature enhances user autonomy, relieving you of the burden of providing extensive support and managing object updates.

If you want an idea of what you can do with Squest, click on the image below.

Features

Service catalog:
- Add services to your catalog based on job templates you have in your Red Hat Ansible Automation Platform/AWX instance
- Manage requests for services (review, update, approve and process)
- Manage lifecycle of each instance of a service
- Integrated or external support page
- Organization, team and user management
- Approval workflows
- Custom on-boarding documentation
- RBAC (Roles, Permissions, Role bindings)
Resource tracking:
- Create generic objects
- Link objects to compose layers of your infrastructure (Physical servers, Virtualization, Containers, Projects, Tenants,...)
- Visualize pool of resource of what you have provided (CPU, vCPU, memory, disk...)
- Check consumptions before approving new requests
- Graph representation of resource layers
- Set quotas on each attribute

Links

📘 Documentation
💬 Chat on Gitter
🎥 Demo video
🐵 Our Ansible playbook testing framework

If you like the project, star it ⭐, it motivates us a lot 🙂

squest's People

Contributors

Stargazers

Watchers

Forkers

a-belhadj dmsimard sispheor joshypj jeffkight eliasboulharts shannara rjonss chris93111 tinle behelit1582 papapillon jsianard classicvalues silvinux nagarajgandge syedmaseerulla seancmasters kamaib24 nikeyol secden sandeepvc22 slarimore02 miketoupin bnoahpro nsxsoft vinicius-alcantara rcarrillocruz quynhlab hinotori74 thomaspreischl malsey octodemo mwy3510 giulianoquites gdmtt leosavio clebersonsouza quentinlegay beerfranz thiagolinhares migounette be-secure klaas- gxkch lacoski

squest's Issues

Github action CI/CD

Feature: Admin can list user + filter by billing group (billing _group=None)

And link to the dashboard

Error when creating 2 different service that use the same template name from 2 different AWX

How to reproduce:

Create 2 AWX/Tower instance
Create a service that use the "Demo job template" from AWX A
Create another service that use the "Demo job template" from AWX B

Traceback:

Traceback (most recent call last):
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/nico/Documents/squest/service_catalog/views/admin/catalog.py", line 20, in add_service
    if form.is_valid():
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 177, in is_valid
    return self.is_bound and not self.errors
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 172, in errors
    self.full_clean()
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 374, in full_clean
    self._clean_fields()
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 392, in _clean_fields
    value = field.clean(value)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/fields.py", line 149, in clean
    value = self.to_python(value)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/models.py", line 1274, in to_python
    value = self.queryset.get(**{key: value})
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/db/models/query.py", line 433, in get
    raise self.model.MultipleObjectsReturned(
service_catalog.models.job_templates.JobTemplate.MultipleObjectsReturned: get() returned more than one JobTemplate -- it returned 2!
ERROR 2021-07-13 13:00:37,743 log Internal Server Error: /settings/catalog/service/add_service/
Traceback (most recent call last):
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/nico/Documents/squest/service_catalog/views/admin/catalog.py", line 20, in add_service
    if form.is_valid():
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 177, in is_valid
    return self.is_bound and not self.errors
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 172, in errors
    self.full_clean()
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 374, in full_clean
    self._clean_fields()
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 392, in _clean_fields
    value = field.clean(value)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/fields.py", line 149, in clean
    value = self.to_python(value)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/models.py", line 1274, in to_python
    value = self.queryset.get(**{key: value})
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/db/models/query.py", line 433, in get
    raise self.model.MultipleObjectsReturned(
service_catalog.models.job_templates.JobTemplate.MultipleObjectsReturned: get() returned more than one JobTemplate -- it returned 2!
[13/Jul/2021 13:00:37] "POST /settings/catalog/service/add_service/ HTTP/1.1" 500 109972

CI/CD is broken du too dependency error

We have a failure during the Docker image build

E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/libu/libuv1/libuv1-dev_1.34.2-1ubuntu1.1_amd64.deb  404  Not Found [IP: 52.250.76.244 80]

This package has been replaced by libuv1-dev_1.34.2-1ubuntu1.3_amd64.deb.

Possible workaround

downloading this new package and installing it before calling the system packages installation
Install the package alone before (sudo apt-get install libuv1-dev)

Doc: resource tracking

Add resource tracking doc to mkdoc

Feature: Hook job template on request/instance state update

When creating a service, we only select the job template that will be used for provisioning.
It would be interesting to add other job templates that could be executed following the request or instance state.

on request submitted
on request canceled
on request need_info
on request processing
...

And on instance change

on instance provisioning
on instance deleting
...

Inconsistent mail notifications?

I have configured a service for the service catalog and the mail notifications "from" address is different:
SUBMITTED: from user that requested the service
ACCEPTED: from "admin mail address"
PROCESSING: from "admin mail address"
COMPLETED: from "squest@SQUEST_HOST"

Works as expected? I would expect that "from" should be always squest@SQUEST_HOST

Add AWX docker image to the dev Docker compose

So it can be used when dev and for integration test

add AWX to compose
Add an Ansible playbook that print extra vars sent by squest and call the squest api to update the instance (URI module)
Add Ansible playbook that configure AWX (create job template, create squest credential type, squest credential aka token, create Tower admin token)

Feature: user profile page

As a USER of the portal
I want to READ my profile info

Example of info:

email
first name
last name
last login
date created

From this page we will be able to create as well token (related to #25 )

Change help links or refactor with footer

Git repo
Chat
Doc
API doc

LDAP not working

I tried different settings in ldap_config.py and also read the django docs and double check the config.
But every time I receive "AUTH_LDAP_USER_SEARCH must be an LDAPSearch instance". Can you check this? Manually running a ldapsearch on command line works fine.

Regards,
Stephan

Onboarding page on service

As a end user I need a page that explain me how to use the service I ordered

Display service catalog fail if image is not set

Images are not compulsory when adding a new service but sQuest crash when we list services

Feature Request: Assign permissions to services

I am new to django and wanted to ask if it's possible to show only some services for a logged in user and not all services in the catalog view.

Regards,
Stephan

Bug: Edit service does not set back the select billing group

Documentation to hide survey fields from end user

In your video I can see that the end user doesn't see all survey fields from tower. There is one dropdown at the top that the admin can see. How can I achive this?

Regards,
Stephan

Feature Request: Support different survey types

Currently it seems that it will only collect tower survey from type "text" and not "multiselect" or "password. Can you add this, too?

Regards,
Stephan

Feature: support of workflow job template

So far Squest only support job templates.
It would be interesting to support as well workflows.

Feature: admin can list all support (and filter by opened or closed)

Feature: over commitment ratio on attributes

Now:
80 physical server CPU ---> 80 vCPU in a vCenter pool

With ratio set to 2:
80 physical server CPU ---> 160 vCPU in a vCenter pool

Feature: API token management

As a user I need to be able to create an API token that can be used in a playbook to update instance spec.

Wrong URL in service-delete.html

NoReverseMatch at /settings/catalog/service/4/delete/

Reverse for 'settings_catalog' not found. 'settings_catalog' is not a valid view function or pattern name.

Feature: end user gives permission on instance/request to other user

As a End User of the portal
I want to be able to give permissions (Read or Update)
on my instances
to other users of the portal

Enhancement: update the button that show number of job template after a sync

The sync job template button does not update the number of job template once the ajax request is complete.
We need to refresh the page.

update the ajax call to update the button

Feature: Announcements

Admin can create an announcement message with start and end date to notify users of an incoming maintenance

Enhancement: add resource group template

As an admin
I can define a template of a resource group that can be copied to become a resource group.

E.g: define a "physical server" with CPU, Memory, Disk

Feature: Resource tracking global view

The resource tracking feature is in progress.
We have so far:

CRUD on resource group
CRUD on resource pool

We need to add a view that give a visual map of resource group with link on attributes of a resource pool.

The idea is to generate a page with graphviz.

Example of code with dot language (can be tested in the online editor here)

digraph {
  node [shape=plaintext]
    rg1 [label=
    <
    <TABLE BORDER="0" CELLBORDER="1" CELLSPACING="0">
      <TR>
          <TD COLSPAN="12">Resource Group</TD>
      </TR>
      <TR>
          <TD COLSPAN="6">Name</TD>
          <TD COLSPAN="6">Server</TD>
      </TR>
      <TR>
          <TD COLSPAN="6">Resource</TD>
          <TD COLSPAN="6">34</TD>
      </TR>
      <TR>
          <TD COLSPAN="12">Attributes</TD>
      </TR>
      <TR>
        <TD COLSPAN="6">Name</TD>
        <TD COLSPAN="6">Total</TD>
      </TR>
      <TR>
        <TD PORT="rg1_vCPU" COLSPAN="6">vCPU</TD>
        <TD COLSPAN="6">60</TD>
      </TR>
      <TR>
        <TD PORT="rg1_memory" COLSPAN="6">Memory</TD>
        <TD COLSPAN="6">34</TD>
      </TR>
    </TABLE>
    >
    ];
    
  node [shape=plaintext]
    rp1 [label=
    <
    <TABLE BORDER="0" CELLBORDER="1" CELLSPACING="0">
      <TR>
          <TD COLSPAN="12">Resource Pool</TD>
      </TR>
      <TR>
          <TD COLSPAN="6">Name</TD>
          <TD COLSPAN="6">vCenter</TD>
      </TR>
      <TR>
          <TD COLSPAN="12">Attributes</TD>
      </TR>
      <TR>
        <TD COLSPAN="3">name</TD>
        <TD COLSPAN="3">Produced</TD>
        <TD COLSPAN="3">Consumed</TD>
        <TD COLSPAN="3">Available</TD>
      </TR>
      <TR>
        <TD PORT="rp1_vCPU" COLSPAN="3">vCPU</TD>
        <TD COLSPAN="3">40</TD>
        <TD COLSPAN="3">30</TD>
        <TD COLSPAN="3">10</TD>
      </TR>
      <TR>
        <TD PORT="rp1_memory" COLSPAN="3">Memory</TD>
        <TD COLSPAN="3">100</TD>
        <TD COLSPAN="3">20</TD>
        <TD COLSPAN="3">80</TD>
      </TR>
    </TABLE>
    >
    ]; 
    
    rg2 [label=
    <
    <TABLE BORDER="0" CELLBORDER="1" CELLSPACING="0">
      <TR>
          <TD COLSPAN="12">Resource Group</TD>
      </TR>
      <TR>
          <TD COLSPAN="6">Name</TD>
          <TD COLSPAN="6">VM</TD>
      </TR>
      <TR>
          <TD COLSPAN="6">Resource</TD>
          <TD COLSPAN="6">34</TD>
      </TR>
      <TR>
          <TD COLSPAN="12">Attributes</TD>
      </TR>
      <TR>
        <TD COLSPAN="6">Name</TD>
        <TD COLSPAN="6">Total</TD>
      </TR>
      <TR>
        <TD PORT="rg2_vCPU" COLSPAN="6">vCPU</TD>
        <TD COLSPAN="6">60</TD>
      </TR>
      <TR>
        <TD PORT="rg2_memory" COLSPAN="6">Memory</TD>
        <TD COLSPAN="6">34</TD>
      </TR>
    </TABLE>
    >
    ];
    
    node [shape=plaintext]
    rg3 [label=
    <
    <TABLE BORDER="0" CELLBORDER="1" CELLSPACING="0">
      <TR>
          <TD COLSPAN="12">Resource Group</TD>
      </TR>
      <TR>
          <TD COLSPAN="3">Name</TD>
          <TD COLSPAN="3">Resources</TD>
          <TD COLSPAN="3">vCPU</TD>
          <TD COLSPAN="3">Memory</TD>
      </TR>
      <TR>
        <TD COLSPAN="3">vCenter</TD>
        <TD COLSPAN="3">23</TD>
        <TD COLSPAN="3">30</TD>
        <TD COLSPAN="3">45</TD>
      </TR>
    </TABLE>
    >
    ];
    
    rg1:rg1_vCPU -> rp1:rp1_vCPU [color = green]
    rg1:rg1_memory -> rp1:rp1_memory [color = green]
    
   rp1:rp1_vCPU  -> rg2:rg2_vCPU [color = red]
   rp1:rp1_memory -> rg2:rg2_memory [color = red]
    
}

Here is the result:

Todo:

New link from the sidebar that point to this new page
Method that generate the view from current data in the DB
Color on attribute links (red when consumer, green when producer)
On resource group: Link on the number of resource that point to resource_group_resource_list view
On resource pool: Link on producer that point to resource_pool_attribute_producer_list
On resource pool: Link on consumer that point to resource_pool_attribute_consumer_list
Add background color on TD element with same color as our bootstrap framework

Usage of day 2 operations?

How can I use day 2 operations? Do I need special extra_vars in ansible tower or in the playbook (for create/update/delete)?

Add web socket to immediately receive updates from the backend

Use cases:

Tower job template successfully complete --> The instance status switch to "available" without refreshing the page
Multiple admin working in the same time: if one admin approve a request, other connected admins receive the update directly

Lib for that: https://channels.readthedocs.io/

Unit tests coverage > 80%

Tests to do:

Enhancement: when adding a tower server, automatically run a first sync of job template

Duplicated Job Templates on sQuest when starting celery after two sync

Steps to reproduce:

Launch development server without celery worker
Add a tower
Sync job templates
Sync job templates again
Start the celery worker: celery -A service_catalog worker -l info

DB Screenshot:

Currently on commit : 46302ff (upstream/resoure_tracker)

Enhancement: main dashboard update

number of request per state
total instances per type of service (X number of VMware Vm, X Openshift project, X openstack tenant,...)
Total users
per billing group --> pool attribute consumption. Pie chart with consumption of each attribute for a RP if the attribute is linked to an instance and the instance to a group.
User with no billing group
Opened support cases

if RP --> attribute RG ---> Resource ---> Instance ---> Billing group THEN new category with Organization name

Docker image: permission error on /app/media

Error when creating a service. /app/media seems not allowed.

Enhancement: Update Tower server creation form

add an example of host with port as placeholder
add https to "is_secure"
Detect is a trailing slash is present at the end of the given URL and remove it before the save

Fix : Favicon 404 error

We must add a favicon.

API: Resource tracking create resource in resource group

CI/CD: Add mkdoc deploy on merge master

Github action here

CI/CD: Add integration test to test the day 2 operations

Squest call to Tower
Tower send spec update on the target instance
Squest new call with updated spec

Add tag on resource and resource pool for filtering

https://github.com/jazzband/django-taggit

Refactor : generic template for create/delete/edit

All all of our template are the same.
We could maybe send some data from the view like the target URL, button name,...

Production deployment (docker-compose)

Create a production deployment
The first version can be done with docker compose.

Implementation example: https://github.com/netbox-community/netbox-docker

Feature:

deployment
update (pull last version + build image)
graceful shutdown
backup
TLS
LDAP
Django prod settings

Feature: group and billing group management

As an admin I can:

CRUD on Groups
add/delete a user to a group
Create Billing Group model
CRUD on Billing Group
add / delete user from Billing Group
instance linked to billing group

Spec:

Foreign key on Instance to BillingGroup
User --> 1,n -- 0,n BillingGroup (create default billing group: fixtures)
manage the billing group of instance with survey field

When user create instance
if one billingroup
then instance billing == billingroup
else
need to select billingroup

describe in documentation default billing group for a new user

Bug when I process a request

Bug appear after processing a request with the "Demo job template"

Environment:


Request Method: POST
Request URL: http://127.0.0.1:8000/admin/request/10/process/

Django Version: 3.1.7
Python Version: 3.8.10
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'rest_framework',
 'fontawesome-free',
 'django_celery_results',
 'django_celery_beat',
 'guardian',
 'django_node_assets',
 'django_filters',
 'drf_yasg',
 'service_catalog',
 'resource_tracker']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback (most recent call last):
  File "/home/anthony/.cache/pypoetry/virtualenvs/tower-service-catalog--Kn3-9f8-py3.8/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/anthony/.cache/pypoetry/virtualenvs/tower-service-catalog--Kn3-9f8-py3.8/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/anthony/.cache/pypoetry/virtualenvs/tower-service-catalog--Kn3-9f8-py3.8/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/anthony/HPE/squest/service_catalog/views/admin/request.py", line 162, in admin_request_process
    target_request.perform_processing()
  File "/home/anthony/.cache/pypoetry/virtualenvs/tower-service-catalog--Kn3-9f8-py3.8/lib/python3.8/site-packages/django_fsm/__init__.py", line 520, in _change_state
    return fsm_meta.field.change_state(instance, func, *args, **kwargs)
  File "/home/anthony/.cache/pypoetry/virtualenvs/tower-service-catalog--Kn3-9f8-py3.8/lib/python3.8/site-packages/django_fsm/__init__.py", line 326, in change_state
    result = method(instance, *args, **kwargs)
  File "/home/anthony/HPE/squest/service_catalog/models/request.py", line 105, in perform_processing
    tower_job_id = self.operation.job_template.execute(extra_vars=tower_extra_vars)
  File "/home/anthony/HPE/squest/service_catalog/models/job_templates.py", line 22, in execute
    return tower_job_run.id

Exception Type: AttributeError at /admin/request/10/process/
Exception Value: 'NoneType' object has no attribute 'id'

Add question description from tower/awx as help_text in squest

I think that should be possible like this: help_text=survey_definition['question_description']
So that the end user get a little bit more information what data is needed.

Feature: custom field on Resource group

So far Squest only manage integer attribute. We need to be able to add string like a comment.

Add versioning plugin to mkdoc page

Need to be done for the first release

https://squidfunk.github.io/mkdocs-material/setup/setting-up-versioning/

Add navigation on top of page

Pure bootstrap: https://getbootstrap.com/docs/4.0/components/breadcrumb/

Maybe a lib can do the trick: https://django-bootstrap-breadcrumbs.readthedocs.io/en/latest/

Or just a snippet: https://www.bedjango.com/blog/snippet-breadcrumbs/

To be tested, compared and implemented.

Feature: Service catalog operation details (job template info)

As an admin I need to have a page that print all info concerning a job template for all operation of a service.

This feature will be mostly used to compare the survey present in Squest VS the survey declared in Tower/AWX

Enhancement: add a button to disable/enable a service from the catalog

After changing Tower/AWX survey, we must synchronize job templates on squest.
Do we update survey when is updated on Tower automatically or do we ask to update it manually?

hewlettpackard / squest Goto Github PK

squest's Introduction

Self service portal on top of Red Hat Ansible Automation Platform

Features

Links

squest's People

Contributors

Stargazers

Watchers

Forkers

squest's Issues

Recommend Projects

Recommend Topics

Recommend Org