automattic / vip-go-harmonia Goto Github PK

1.0 6.0 0.0 942 KB

Harmonia (formerly Preflight Checks) is a tool that provides a set of automated tests (or checks) that can test and validate the application functionality and responsiveness with an environment as close to production, before the actual deployment to an environment.

License: MIT License

TypeScript 67.20% JavaScript 27.87% Dockerfile 2.49% Shell 2.44%

vip vip-go

vip-go-harmonia's Introduction

Harmonia

Formely known as Preflight Checks. The goal of this project is to provide a set of automated tests (or checks) that can test and validate the application functionality and responsiveness with an environment as close to production as possible, before the actual deployment to an environment.

Building

$ npm run build

Running the CLI

After building Harmonia, it should be possible to run the CLI directly using the following command:

$ node dist/cli.js

The command can be linked and installed on the environment using npm link:

$ npm link
$ harmonia --help

Contributing

For details about contributing to this project (e.g. publishing a release), please see CONTRIBUTING.md.

vip-go-harmonia's People

Contributors

Stargazers

Watchers

vip-go-harmonia's Issues

New check: Warn when `node_modules` is staged for commit

In order to prevent committing the node_modules to a deploy branch, create a new check that will validate:

If node_modules exists (and maybe if current branch == deploy branch)
- Check if it's ignored in .gitignore
- Check if it's not staged for commit
- Check if it's not already commited

If so, it should throw a Warning, advising to either remove the node_modules from the repository (or staging), and/or ignore it in .gitignore, depending on the situation.

Test Report: Simplify/declutter the GitHub report

The current GitHub report has a lot of information that can overwhelm the developer trying to understand next steps. The full report should be hidden by default, and the important issues should be visible in a first glance.

Additionally, the Description should be changed to something like "Action Required", providing some more useful insight, and linking to the relevant documentation for further details.

VIP Harmonia Feedback Example

Harmonia Results

This is an example of a small paragraph that we can include in the pull request comment to give further information about Harmonia, the tests and the results. Expand each test suite to view detailed results (failures/blockers are expanded by default).

TESTS SUMMARY

✅    PASSED - 6 tests
⚠️    PARTIAL SUCCESS - 4 tests
❌    FAILURE - 0 tests

Test suite details

Node.JS
Test a node.JS site

⚠️ PARTIAL SUCCESS. Click to toggle details view - :arrow_down:

	Test	Description	Issues
✅	Check NPM script in the project	Checks your package.json for `npm build` and `npm start`	0
⚠️	Check usage of required and recommended packages	Checks if the project is using the necessary packages in the production dependencies	1

Issues Found:

⚠️ Warning - Missing required package @automattic/vip-go
In Check usage of required and recommended packages
Read more in the documentation

Docker
Builds and executes the docker environment

✅ PASSED. Click to toggle details view - :arrow_down:

	Test	Description
✅	Builds the Docker image	Tries to build the docker image and the application
✅	Start and run the Docker image	Tries to start the application in the built Docker image
✅	Testing application availability and health-check endpoint	Checking if your app accepts a PORT and responds to /cache-healthcheck

Issues Found:
None

Health Check
Tests a number of HTTP requests and looks at the response time

⚠️ PARTIAL SUCCESS. Click to toggle details view - :arrow_down:

	Test	Description	Issues
✅	Testing availability and response time for static URLs	Checks if your app responds to requests to a subset of static URLs	0
⚠️	Testing with home URLs	Fetches URLS from the index and check their availability	2

Issues Found:

⚠️ Warning - http://localhost:3991/tag/redis responded with a 404 - Not Found in 305ms
In Testing with home URLs
⚠️ Warning - http://localhost:3991/tag/node-js responded with a 404 - Not Found in 408ms
In Testing with home URLs

Create factory for SiteConfig and EnvironmentVariables

The SiteConfig and EnvironmentVariables objects are a requirement to bootstrap Harmonia.

const harmonia = new Harmonia();

const siteOptions = new SiteConfig( {
    siteID: opt.app.id,
    nodejsVersion: arg[ 'node-version' ],
    repository: opt.app.repo,
    baseURL: 'http://localhost:' + opt.port,
} );

const envVars = EnvironmentVariables( {
    PORT: opt.port,
} );

harmonia.bootstrap( siteOptions, envVars );

Since these objects are not exported in the main file, they cannot be instantiated when using Harmonia as a npm module. Therefore, the main Harmonia class, exported in harmonia.ts, needs to have a factory for these objects. Something like

const siteOptions = Harmonia.buildSiteConfig( { ...initialArgs } );
siteOptions.setPackageJSON( ... ); // Using the newly built object

const envVars = Harmonia.buildEnvVars( { ...initialEnvVars } ) ;
envVars.set( 'NEW_ENV_VAR', 'example' ); 

harmonia.bootstrap( siteOptions, envVars );

Test Summary: Count Aborted as Failed & Improve counter

If a Harmonia test is aborted, it should count as failed.

It failed, it should be counted as a failure. Any tests that come after should be listed and shown as aborted. (We might change that word to "cancelled" in the UI.)