hewlettpackard / squest Goto Github PK

View Code? Open in Web Editor NEW

279.0 19.0 48.0 17.7 MB

Service request portal on top of Ansible Tower/AWX

Home Page: https://hewlettpackard.github.io/squest/latest

License: Apache License 2.0

Python 85.90% Shell 0.04% Dockerfile 0.07% CSS 0.25% JavaScript 0.74% HTML 13.00%

ansible tower catalog portal django awx rhaap

squest's Issues

Add AWX docker image to the dev Docker compose

So it can be used when dev and for integration test

add AWX to compose
Add an Ansible playbook that print extra vars sent by squest and call the squest api to update the instance (URI module)
Add Ansible playbook that configure AWX (create job template, create squest credential type, squest credential aka token, create Tower admin token)

Feature Request: Assign permissions to services

I am new to django and wanted to ask if it's possible to show only some services for a logged in user and not all services in the catalog view.

Regards,
Stephan

Feature: Resource tracking global view

The resource tracking feature is in progress.
We have so far:

CRUD on resource group
CRUD on resource pool

We need to add a view that give a visual map of resource group with link on attributes of a resource pool.

The idea is to generate a page with graphviz.

Example of code with dot language (can be tested in the online editor here)

digraph {
  node [shape=plaintext]
    rg1 [label=
    <
    <TABLE BORDER="0" CELLBORDER="1" CELLSPACING="0">
      <TR>
          <TD COLSPAN="12">Resource Group</TD>
      </TR>
      <TR>
          <TD COLSPAN="6">Name</TD>
          <TD COLSPAN="6">Server</TD>
      </TR>
      <TR>
          <TD COLSPAN="6">Resource</TD>
          <TD COLSPAN="6">34</TD>
      </TR>
      <TR>
          <TD COLSPAN="12">Attributes</TD>
      </TR>
      <TR>
        <TD COLSPAN="6">Name</TD>
        <TD COLSPAN="6">Total</TD>
      </TR>
      <TR>
        <TD PORT="rg1_vCPU" COLSPAN="6">vCPU</TD>
        <TD COLSPAN="6">60</TD>
      </TR>
      <TR>
        <TD PORT="rg1_memory" COLSPAN="6">Memory</TD>
        <TD COLSPAN="6">34</TD>
      </TR>
    </TABLE>
    >
    ];
    
  node [shape=plaintext]
    rp1 [label=
    <
    <TABLE BORDER="0" CELLBORDER="1" CELLSPACING="0">
      <TR>
          <TD COLSPAN="12">Resource Pool</TD>
      </TR>
      <TR>
          <TD COLSPAN="6">Name</TD>
          <TD COLSPAN="6">vCenter</TD>
      </TR>
      <TR>
          <TD COLSPAN="12">Attributes</TD>
      </TR>
      <TR>
        <TD COLSPAN="3">name</TD>
        <TD COLSPAN="3">Produced</TD>
        <TD COLSPAN="3">Consumed</TD>
        <TD COLSPAN="3">Available</TD>
      </TR>
      <TR>
        <TD PORT="rp1_vCPU" COLSPAN="3">vCPU</TD>
        <TD COLSPAN="3">40</TD>
        <TD COLSPAN="3">30</TD>
        <TD COLSPAN="3">10</TD>
      </TR>
      <TR>
        <TD PORT="rp1_memory" COLSPAN="3">Memory</TD>
        <TD COLSPAN="3">100</TD>
        <TD COLSPAN="3">20</TD>
        <TD COLSPAN="3">80</TD>
      </TR>
    </TABLE>
    >
    ]; 
    
    rg2 [label=
    <
    <TABLE BORDER="0" CELLBORDER="1" CELLSPACING="0">
      <TR>
          <TD COLSPAN="12">Resource Group</TD>
      </TR>
      <TR>
          <TD COLSPAN="6">Name</TD>
          <TD COLSPAN="6">VM</TD>
      </TR>
      <TR>
          <TD COLSPAN="6">Resource</TD>
          <TD COLSPAN="6">34</TD>
      </TR>
      <TR>
          <TD COLSPAN="12">Attributes</TD>
      </TR>
      <TR>
        <TD COLSPAN="6">Name</TD>
        <TD COLSPAN="6">Total</TD>
      </TR>
      <TR>
        <TD PORT="rg2_vCPU" COLSPAN="6">vCPU</TD>
        <TD COLSPAN="6">60</TD>
      </TR>
      <TR>
        <TD PORT="rg2_memory" COLSPAN="6">Memory</TD>
        <TD COLSPAN="6">34</TD>
      </TR>
    </TABLE>
    >
    ];
    
    node [shape=plaintext]
    rg3 [label=
    <
    <TABLE BORDER="0" CELLBORDER="1" CELLSPACING="0">
      <TR>
          <TD COLSPAN="12">Resource Group</TD>
      </TR>
      <TR>
          <TD COLSPAN="3">Name</TD>
          <TD COLSPAN="3">Resources</TD>
          <TD COLSPAN="3">vCPU</TD>
          <TD COLSPAN="3">Memory</TD>
      </TR>
      <TR>
        <TD COLSPAN="3">vCenter</TD>
        <TD COLSPAN="3">23</TD>
        <TD COLSPAN="3">30</TD>
        <TD COLSPAN="3">45</TD>
      </TR>
    </TABLE>
    >
    ];
    
    rg1:rg1_vCPU -> rp1:rp1_vCPU [color = green]
    rg1:rg1_memory -> rp1:rp1_memory [color = green]
    
   rp1:rp1_vCPU  -> rg2:rg2_vCPU [color = red]
   rp1:rp1_memory -> rg2:rg2_memory [color = red]
    
}

Here is the result:

Todo:

New link from the sidebar that point to this new page
Method that generate the view from current data in the DB
Color on attribute links (red when consumer, green when producer)
On resource group: Link on the number of resource that point to resource_group_resource_list view
On resource pool: Link on producer that point to resource_pool_attribute_producer_list
On resource pool: Link on consumer that point to resource_pool_attribute_consumer_list
Add background color on TD element with same color as our bootstrap framework

Feature: admin can list all support (and filter by opened or closed)

Feature: Announcements

Admin can create an announcement message with start and end date to notify users of an incoming maintenance

Enhancement: add resource group template

As an admin
I can define a template of a resource group that can be copied to become a resource group.

E.g: define a "physical server" with CPU, Memory, Disk

Feature: over commitment ratio on attributes

Now:
80 physical server CPU ---> 80 vCPU in a vCenter pool

With ratio set to 2:
80 physical server CPU ---> 160 vCPU in a vCenter pool

Bug when I process a request

Bug appear after processing a request with the "Demo job template"

Environment:


Request Method: POST
Request URL: http://127.0.0.1:8000/admin/request/10/process/

Django Version: 3.1.7
Python Version: 3.8.10
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'rest_framework',
 'fontawesome-free',
 'django_celery_results',
 'django_celery_beat',
 'guardian',
 'django_node_assets',
 'django_filters',
 'drf_yasg',
 'service_catalog',
 'resource_tracker']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback (most recent call last):
  File "/home/anthony/.cache/pypoetry/virtualenvs/tower-service-catalog--Kn3-9f8-py3.8/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/anthony/.cache/pypoetry/virtualenvs/tower-service-catalog--Kn3-9f8-py3.8/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/anthony/.cache/pypoetry/virtualenvs/tower-service-catalog--Kn3-9f8-py3.8/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/anthony/HPE/squest/service_catalog/views/admin/request.py", line 162, in admin_request_process
    target_request.perform_processing()
  File "/home/anthony/.cache/pypoetry/virtualenvs/tower-service-catalog--Kn3-9f8-py3.8/lib/python3.8/site-packages/django_fsm/__init__.py", line 520, in _change_state
    return fsm_meta.field.change_state(instance, func, *args, **kwargs)
  File "/home/anthony/.cache/pypoetry/virtualenvs/tower-service-catalog--Kn3-9f8-py3.8/lib/python3.8/site-packages/django_fsm/__init__.py", line 326, in change_state
    result = method(instance, *args, **kwargs)
  File "/home/anthony/HPE/squest/service_catalog/models/request.py", line 105, in perform_processing
    tower_job_id = self.operation.job_template.execute(extra_vars=tower_extra_vars)
  File "/home/anthony/HPE/squest/service_catalog/models/job_templates.py", line 22, in execute
    return tower_job_run.id

Exception Type: AttributeError at /admin/request/10/process/
Exception Value: 'NoneType' object has no attribute 'id'

Bug: Edit service does not set back the select billing group

Documentation to hide survey fields from end user

In your video I can see that the end user doesn't see all survey fields from tower. There is one dropdown at the top that the admin can see. How can I achive this?

Regards,
Stephan

I tried different settings in ldap_config.py and also read the django docs and double check the config.
But every time I receive "AUTH_LDAP_USER_SEARCH must be an LDAPSearch instance". Can you check this? Manually running a ldapsearch on command line works fine.

Regards,
Stephan

Feature: custom field on Resource group

So far Squest only manage integer attribute. We need to be able to add string like a comment.

Feature: user profile page

As a USER of the portal
I want to READ my profile info

Example of info:

email
first name
last name
last login
date created

From this page we will be able to create as well token (related to #25 )

CI/CD is broken du too dependency error

We have a failure during the Docker image build

E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/libu/libuv1/libuv1-dev_1.34.2-1ubuntu1.1_amd64.deb  404  Not Found [IP: 52.250.76.244 80]

This package has been replaced by libuv1-dev_1.34.2-1ubuntu1.3_amd64.deb.

Possible workaround

downloading this new package and installing it before calling the system packages installation
Install the package alone before (sudo apt-get install libuv1-dev)

Add versioning plugin to mkdoc page

Need to be done for the first release

https://squidfunk.github.io/mkdocs-material/setup/setting-up-versioning/

Feature: group and billing group management

As an admin I can:

CRUD on Groups
add/delete a user to a group
Create Billing Group model
CRUD on Billing Group
add / delete user from Billing Group
instance linked to billing group

Spec:

Foreign key on Instance to BillingGroup
User --> 1,n -- 0,n BillingGroup (create default billing group: fixtures)
manage the billing group of instance with survey field

When user create instance
if one billingroup
then instance billing == billingroup
else
need to select billingroup

describe in documentation default billing group for a new user

Unit tests coverage > 80%

Tests to do:

Fix : Favicon 404 error

We must add a favicon.

Inconsistent mail notifications?

I have configured a service for the service catalog and the mail notifications "from" address is different:
SUBMITTED: from user that requested the service
ACCEPTED: from "admin mail address"
PROCESSING: from "admin mail address"
COMPLETED: from "squest@SQUEST_HOST"

Works as expected? I would expect that "from" should be always squest@SQUEST_HOST

Change help links or refactor with footer

Git repo
Chat
Doc
API doc

Add tag on resource and resource pool for filtering

https://github.com/jazzband/django-taggit

Usage of day 2 operations?

How can I use day 2 operations? Do I need special extra_vars in ansible tower or in the playbook (for create/update/delete)?

Feature: API token management

As a user I need to be able to create an API token that can be used in a playbook to update instance spec.

Production deployment (docker-compose)

Create a production deployment
The first version can be done with docker compose.

Implementation example: https://github.com/netbox-community/netbox-docker

Feature:

deployment
update (pull last version + build image)
graceful shutdown
backup
TLS
LDAP
Django prod settings

Add navigation on top of page

Pure bootstrap: https://getbootstrap.com/docs/4.0/components/breadcrumb/

Maybe a lib can do the trick: https://django-bootstrap-breadcrumbs.readthedocs.io/en/latest/

Or just a snippet: https://www.bedjango.com/blog/snippet-breadcrumbs/

To be tested, compared and implemented.

Enhancement: main dashboard update

number of request per state
total instances per type of service (X number of VMware Vm, X Openshift project, X openstack tenant,...)
Total users
per billing group --> pool attribute consumption. Pie chart with consumption of each attribute for a RP if the attribute is linked to an instance and the instance to a group.
User with no billing group
Opened support cases

if RP --> attribute RG ---> Resource ---> Instance ---> Billing group THEN new category with Organization name

CI/CD: Add mkdoc deploy on merge master

Github action here

Enhancement: update the button that show number of job template after a sync

The sync job template button does not update the number of job template once the ajax request is complete.
We need to refresh the page.

update the ajax call to update the button

Feature: support of workflow job template

So far Squest only support job templates.
It would be interesting to support as well workflows.

API: Resource tracking create resource in resource group

Display service catalog fail if image is not set

Images are not compulsory when adding a new service but sQuest crash when we list services

Onboarding page on service

As a end user I need a page that explain me how to use the service I ordered

Error when creating 2 different service that use the same template name from 2 different AWX

How to reproduce:

Create 2 AWX/Tower instance
Create a service that use the "Demo job template" from AWX A
Create another service that use the "Demo job template" from AWX B

Traceback:

Traceback (most recent call last):
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/nico/Documents/squest/service_catalog/views/admin/catalog.py", line 20, in add_service
    if form.is_valid():
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 177, in is_valid
    return self.is_bound and not self.errors
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 172, in errors
    self.full_clean()
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 374, in full_clean
    self._clean_fields()
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 392, in _clean_fields
    value = field.clean(value)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/fields.py", line 149, in clean
    value = self.to_python(value)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/models.py", line 1274, in to_python
    value = self.queryset.get(**{key: value})
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/db/models/query.py", line 433, in get
    raise self.model.MultipleObjectsReturned(
service_catalog.models.job_templates.JobTemplate.MultipleObjectsReturned: get() returned more than one JobTemplate -- it returned 2!
ERROR 2021-07-13 13:00:37,743 log Internal Server Error: /settings/catalog/service/add_service/
Traceback (most recent call last):
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/nico/Documents/squest/service_catalog/views/admin/catalog.py", line 20, in add_service
    if form.is_valid():
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 177, in is_valid
    return self.is_bound and not self.errors
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 172, in errors
    self.full_clean()
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 374, in full_clean
    self._clean_fields()
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/forms.py", line 392, in _clean_fields
    value = field.clean(value)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/fields.py", line 149, in clean
    value = self.to_python(value)
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/forms/models.py", line 1274, in to_python
    value = self.queryset.get(**{key: value})
  File "/home/nico/.cache/pypoetry/virtualenvs/tower-service-catalog-X_7Nmek_-py3.8/lib/python3.8/site-packages/django/db/models/query.py", line 433, in get
    raise self.model.MultipleObjectsReturned(
service_catalog.models.job_templates.JobTemplate.MultipleObjectsReturned: get() returned more than one JobTemplate -- it returned 2!
[13/Jul/2021 13:00:37] "POST /settings/catalog/service/add_service/ HTTP/1.1" 500 109972

Add web socket to immediately receive updates from the backend

Use cases:

Tower job template successfully complete --> The instance status switch to "available" without refreshing the page
Multiple admin working in the same time: if one admin approve a request, other connected admins receive the update directly

Lib for that: https://channels.readthedocs.io/

Enhancement: add a button to disable/enable a service from the catalog

After changing Tower/AWX survey, we must synchronize job templates on squest.
Do we update survey when is updated on Tower automatically or do we ask to update it manually?

Wrong URL in service-delete.html

NoReverseMatch at /settings/catalog/service/4/delete/

Reverse for 'settings_catalog' not found. 'settings_catalog' is not a valid view function or pattern name.

Feature: Hook job template on request/instance state update

When creating a service, we only select the job template that will be used for provisioning.
It would be interesting to add other job templates that could be executed following the request or instance state.

on request submitted
on request canceled
on request need_info
on request processing
...

And on instance change

on instance provisioning
on instance deleting
...

Feature: end user gives permission on instance/request to other user

As a End User of the portal
I want to be able to give permissions (Read or Update)
on my instances
to other users of the portal

CI/CD: Add integration test to test the day 2 operations

Squest call to Tower
Tower send spec update on the target instance
Squest new call with updated spec

Feature: Admin can list user + filter by billing group (billing _group=None)

And link to the dashboard

Enhancement: when adding a tower server, automatically run a first sync of job template

Enhancement: Update Tower server creation form

add an example of host with port as placeholder
add https to "is_secure"
Detect is a trailing slash is present at the end of the given URL and remove it before the save

Launch development server without celery worker
Add a tower
Sync job templates
Sync job templates again
Start the celery worker: celery -A service_catalog worker -l info

DB Screenshot:

Currently on commit : 46302ff (upstream/resoure_tracker)

hewlettpackard / squest Goto Github PK

squest's Issues

Recommend Projects

Recommend Topics

Recommend Org