Code Monkey home page Code Monkey logo

oar's Introduction

OAR Logo OAR Logo OAR Logo OAR Logo OAR Logo

The OAR Framework for Software Test Reporting

Outcome, Analysis, Resolution

See the GitHub wiki for more info on project background: https://github.com/ryandem1/oar/wiki/Background

Links

Overview

The core of OAR is the oar-service. It serves as the backend for test ingestion/enrichment/querying.

Currently, the OAR service is provided in a minimal Docker image. All parts of OAR are open-source and are assumed to be self-hosted. The oar-service image strives to be compatible with a wide variety of hosting environments, although exact steps for hosting OAR will vary and not be covered in this documentation.

๐Ÿ”Œ๐Ÿ”Œ OAR Integrations ๐Ÿ”Œ๐Ÿ”Œ

There are the following packages to easily integrate OAR ingestion into common testing frameworks:

  • pytest-oar: OAR ingestion PyTest plugin and general oar-service Python interface.

๐Ÿš€๐Ÿš€ The following components are coming in the future ๐Ÿš€๐Ÿš€:

  • OAR analytics UI: A data visualization UI that connects to an instance of an OAR DB to provide fast, rich aggregate statistics around software/test quality.
  • More ingestion integrations (maybe some JS UI frameworks?)

Getting Familiar with the OAR Service

Everything is built around the OAR service (or even more abstractly, the OAR data model). So getting familiar with the service will get you familiar with any other OAR component.

The best place to get started with the oar-service api is to look at the oar-service-spec.json. This is the OpenAPI v3 schema for the service; copy and pasting it in swagger will provide an interface: https://editor.swagger.io/.

Getting Familiar with the OAR Enrich UI

The OAR process encourages test enrichment; meaning to analyze test results (specifically failures) and adding addtional metadata regarding the outcome of the test. This includes indicating if it was a true/false positive, if a ticket came out of it, ticket reference, notes, diagnostic steps to make it easier to debug next time, etc.

All data will go back into the OAR DB, and can be captured in any sort of reporting layer. This is only as valuable as the data you add. Enrichment will cost time, but the idea is that the extra insights are vital and having them will save time in the end.

The UI is one way to perform this enrichment, but there are potentially some lighter-weight alternatives like some sort of Slack channel/plugin combo to enrich results.

Starting local instance for development/sandbox

To start a local instance of OAR, you could either pull and run the image (would need to configure environment variables), or more simply, clone this repository and use a few make commands:

make clean db build-service test-service service enrich-ui-dev

To break it down:

  • clean: Will delete orphan database volumes, teardown existing service, and perform other environment cleanups.
  • db: Starts a new instance of Postgres locally, waits for startup process, and will run the init-postgres.sql to create the OAR table.
  • build-service: Builds the oar-service image
  • test-service: Will run unit tests that do not do database cleanup, so it serves as a helpful way to seed the DB with some data for experimenting
  • service: Starts a local oar-service container and will port-forward the service to your localhost.
  • enrich-ui-dev: Starts a dev instance of the oar-enrich-ui, which will run on http://localhost:5173. Configure the base URL of the service in the settings.

After running these commands, use the Swagger UI or some other HTTP client to start adding tests!

oar's People

Contributors

dependabot[bot] avatar ryandem1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

oar's Issues

Save Queries

Add Query String as URL Parameter is required to be completed before this one.

So, with the ability to save/share queries as base64 encoded strings, saving them becomes more straight-forward.

image

A new text field/button should be added underneath the "Filter Tests" section of the modal.

  • The value of the text field will be the readable name of the query. It is not nullable, should be limited to 40 characters, and should have its surrounding spaces trimmed.
  • The button should be the "Save for Later" button, and it will save the new query

There should also be a new dropdown to select a saved query instead of creating a new one. Here are some properties of the dropdown:

  • The default placeholder should be "Enter or select query..."
  • The dropdown values should be the saved query labels
  • When a dropdown value is selected, it should populate the JSON editor with the decoded base64 query string value

To save data...

For now, we will utilize the browser's localStorage as a way of data persistence. This means that queries will not be persisted if the user clears their browser's storage, which I am sure will be somewhat annoying. Right now, OAR only requires a single Postgres table for test results, and we are going to stretch that for now. In the future I think implementing an optional redis instance might be a good way to go, but not necessary for now.

There is a writable store that connects to localStorage that might be appropriate for this use case from Skeleton UI: https://www.skeleton.dev/utilities/local-storage-stores

Update README.md to include new make commands and enrich-ui

Now that the enrich UI is at MVP, it should be included in the README.md, as well as the new make commands to work with the enrich-ui locally. Should also include some documentation around the full OAR process with enrichment included, as I still think it is unclear from current documentation.

Add Query String as URL Parameter

So filters on the enrich-ui need an easier way of being saved.

Here is an example query:

{
    "resolutions": ["Unresolved"],
    "docs": [
        {"owner": "Patrick Star"}
    ]
}

This gets base64 encoded and sent to the GET /tests endpoint to get a filtered set of tests.

In the UI, this process is performed in the filter modal:

image

As a UX improvement, it would be nice to have the filter that is applied to the test table to be dictated by a query URL parameter that is any base64 encoded query. This way, when a new filter gets applied to the test table, the URL will reflect the query.

This will make it easier to share, save, and modify test queries.

Test table pagination must reset when table is refreshed

Whenever the enrich-ui table resets, it must reset the pagination offset too.

For example, if you scroll to page 5 on the pagination and then apply a filter that only returns enough results for 2 pages, the user has to manually scroll back 3 pages to get to the results. It sort of looks like there are no results at first glance when there really is.

Marching ants test selection animation doesn't work across multiple pages

Marching ants test selection logic currently only works across a single page. The logic needs to be updated to work on each pagination page independently.

Minimum reproduction

  1. Start with enough tests in the tests table to enable at least 2 paginated pages
  2. Select the test in the first table index
  3. Navigate to another page of tests
  4. Select the test in the second table index

Issue: The selection animation in the second index behaves as if the test above it was selected, when it was actually only selected on the first page. It becomes more problematic if you are doing a lot of selecting/deselecting across multiple pages because the animation gets all kinds of screwed up.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.