easydynamics / oscal-editor-deployment Goto Github PK

As a consumer of the EasyGRC all-in-one deployment, I want to be able to quickly understand how to simply pull and run the Docker container with the required OSCAL content folder structure so that I can use the tool locally.

Acceptance Criteria

• The root README contains a brief description and link to the all-in-one deployment
• The all-in-one deployment README focuses simple instructions for creating the OSCAL content folder structure then pulling and running the Docker image from ghcr first with subsequent sections for building and testing
• All steps can be run without checking out the easygrc-deployment repo

In the web application, one of the many ways to navigate to different editors using their unique URLs. The current e2e tests for the OSCAL Drawer Component do not have this implemented yet.

Create e2e test cases that would allow a user to successfully navigate to the different editors in the Profiles, Catalog, Component, and SSP sections in the drawer by their unique URLs.

The cypress e2e tests are functioning however there are a few tests that have unreasonably long run times. This could possibly be because:
-GitHub Actions can be a slow runner
-our app that slow?
Poorly structured tests
GitHub is slow?

There are also cy.wait calls which make the tests longer than needed. Implementing a feature to make the tests run using a timeout feature will optimize the testing environment and make the tests run at better efficiency.

Description

For the past couple of weeks, the cypress tests on the build, test, deploy workflow have been failing more frequently.

Steps to Reproduce

View the workflow failures such as

• https://github.com/EasyDynamics/oscal-editor-deployment/actions/runs/4354049162/jobs/7608869123
• https://github.com/EasyDynamics/oscal-editor-deployment/actions/runs/4323346922/jobs/7546847878

What has been done so far

• Docker logs do not provide any obvious problems
• Cypress results also just show normal loading issues like seen in the past
• The tests failing usually mention a time out of some sort
• The cypress dependency recently updated from 12.6.0 to 12.7.0, but it doesn't seem like that would cause issues.

Expected Behavior

The Drawer.cy.js tests rarely fail and all other tests pass.

Ideally the tests should never fail if things are working correctly, but due to some issues with loading speed 100% passing is not always the case. It should pass with a second attempt.

Actual Behavior

The tests are failing pretty regular and sometimes even beyond a second rerun of the workflow. What is even more odd is that the failures are no longer constrained to just Drawer.cy.js tests.

As a user running the OSCAL Editor all-in-one Docker deployment, I want to consume as small an image as possible so that I can more quickly deploy the app and save space on my machine / Docker registry.

As an IT administrator, I want the OSCAL Editor all-in-one Docker image to be as small as possible so that I can optimize costs.

For the deployment of the OSCAL Editor all-in-one Docker image, we are relying on the use of GitHub Packages, with multiple artifacts being pulled into the Docker image as well as publishing the image to GitHub Packages. We should look into how the size of the image can be reduced to maintain an efficient CI/CD pipeline and cost-optimized use of GitHub Packages.

Implementation Notes

• We should investigate the base image being used and ensure we're not adding unnecessary artifacts to the image layers

Acceptance Criteria

• Either the Docker image has been optimized to result in a smaller sized image at the end of the build process or justification is provided for why that is not feasible
• Documentation exists for the process used for slimming the image further
• Metrics are provided for before/after optimization

As a developer looking for OSCAL editing tools, I want a descriptive Overview section for the OSCAL Editor All-In-One Docker image published on Docker Hub so that I can more easily find and understand how to use the tool.

Acceptance Criteria

• A short description (returned in searches, etc.)
• Overview Section
  • A basic description of what the tool does (which also accounts for SEO)
  • Commands on how to use the image
  • Supported tags and respective Dockerfile links
  • Configuration options
  • Link to report issues
• Source repository Section
• No automated testing is required for this story/task

Description

When running the all-in-one deployment with an SSP or Profile that uses a relative link to import a Catalog/Profile from local files, the link will not be resolved correctly in the OSCAL Editor, so the fetch will fail - meaning no controls are displayed on the page.
This is necessary to fix as some users will need to view and edit custom controls that are not found online.

Acceptance Criteria

An SSP or Profile that imports a Catalog/Profile from other files in the local oscal-content directory will correctly display controls from that file when run from the all-in-one deployment image

Implementation Notes

As an example, a Profile can be loaded that contains an element in its imports list that points to a Catalog in back-matter with the rlink set to ../catalogs/my-catalog.json. At oscal-content/catalogs/my-catalog.json there is a valid OSCAL Catalog with some control.

When running both the REST Service and OSCAL Editor locally (outside of the Docker deployment) the OSCAL Editor makes the fetch as intended. The controls can be viewed, and developer tools shows that the OSCAL Editor makes a fetch request to:
http://localhost:8080/oscal/v1/catalogs/my-catalog.json

When running from the all-in-one image, however -
Developer tools shows that the OSCAL Editor makes an invalid fetch request to:
http://localhost:8080/oscal/v1/profiles//oscal/v1/catalogs/my-catalog.json

The discrepancy between the behavior in the Docker deployment and in the example build is due to the REST_APP_BASE_URL environment variable being set differently.

The all-in-one Dockerfile sets this variable to oscal/v1 as shown here.

The README for the OSCAL React Library instructs a user to set this variable to http://localhost:8080/oscal/v1

These are both correct, because the all-in-one Dockerfile needs to be agnostic of the port that it is running on, so that the user can choose any port they want as they run the container. However, in the case of the all-in-one deployment, it causes an issue with how the OSCAL React Library interprets links that are missing the http:// prefix.

In the React Library, the environment variable is passed into OSCALLoader and used to build the URL for REST requests as shown here

This all works as intended and the REST requests are built properly; when a Profile is resolving controls from a local catalog, the URL is passed down as parentUrl in OSCALProfileResolver which is meant to resolve the relative link.
The issue comes from how the link is interpreted as it passed into resolveLinkHref here. On line 15, a relative link (ie. one that does not start with "http") is appended onto parentUrl.
We end up with a final link for the fetch request that looks like:
oscal/v1/catalogs/my-catalog.json

This link is now still missing the http:// prefix, so the browser interprets it as a link relative to its current page, resulting in a final request to something screwy like http://localhost:8080/oscal/v1/profiles//oscal/v1/catalogs/my-catalog.json

When running the editor via the All-in-One Docker image from DockerHub, the component-uuid is not set. This originally occurred in issues EasyDynamics/oscal-rest-service#100 and EasyDynamics/oscal-react-library#421.

When running in the rest-service and oscal-react-library-example, it seems that these two issues are fixed. removes is displayed in the SSP and added parameters are given a component-uuid. These things do not work when I tried to build the Docker image myself nor when I pulled the Docker image.

Steps to Reproduce

• docker pull ghcr.io/easydynamics/oscal-editor-all-in-one or docker build --tag oscal-editor-all-in-one .
• docker run -p 8080:8080 -v "$(pwd)"/oscal-content:/app/oscal-content oscal-editor-all-in-one
• Navigate to http://localhost:8080/system-security-plan/cff8385f-108e-40a5-8f7a-82f3dc0eaba8
• Click on Enterprise Logging, Monitoring, and Alerting Policy under AC-2 Account Management
• OPTIONAL: Attempt to edit the params in something like AC-2.4 Automated Audit Actions

Expected behavior

• removes is displayed underneath AC-2.2 Removal of Temporary / Emergency Accounts
• Params you edited are displayed and their json definition contain the correct component-uuid tag

Actual

• removes is not displayed
• Params you edited are displayed for all components and do not have a component-uuid tag.

Implementation notes

I will look into fixing this error as I feel pretty familiar with it. It would be helpful for someone to confirm this issue is not just a local problem for me.

Acceptance Criteria

• Test loading via uuid in the address bar to navigate to an example SSP, Catalog, Profile, & Component Definition.
• Test loading those same examples by using the Drawer Selector
  • url should also update to reflect the loaded object's uuid
  • Test that multiple of an oscal object are properly listed. For example, multiple catalogs. This may require adding some more example data.
• Ensure that removing the LoaderForm while in REST mode, does not break functionality of tests.

Need to verify that the local URLs for component definition and SSP do in fact render the expected pages.

As a developer contributing to the OSCAL Editor Deployment, I want to be able to test custom/feature branches of the REST service just as easily as I can for custom/feature branches of the Viewer application so that I can test corresponding changes to both applications easier.

Acceptance Criteria

• A developer can perform a docker build command that uses a custom branch of the oscal-rest-service repository without having to run any mvn commands manually.

Implementation Notes

• We would need a process to clone the repo and build it within an additional stage within the Dockerfile (like we do for the viewer), we should not build within the actual app container
• The process would be somewhat similar to the viewer, ideally, where you either provide a JAR or a repo and branch (with similar defaults)
• The packages_pull.sh script would need to be run within one of the phases when the JAR isn't provided

As a developer in the OSCAL community, I want to be able to access and contribute to the OSCAL Editor so that I can view and edit OSCAL files without building and configuring separate components.

Note this is blocked by EasyDynamics/easygrc#25.

Acceptance Criteria

• The oscal-editor-deployment GitHub repo is public
• A license has been specified
• Copyright has been defined if not public domain
• A contribution page exists
• The Docker container is available on Docker Hub (as a result of an automated pipeline)
  • Note that this should be the last part of implementation of this story, i.e. should not hold up making the repository open source and public. If so, we can move this to a future story.

Implementation Notes

• Contributing and code of conduct should pull from our organization default

As a developer contributing to or receiving contributions on the OSCAL Editor Deployment, I want to be able to test PRs through an automated workflow so that I can have confidence that the changes won't break functionality.

Now that the repository is public, we should have a workflow that builds the container and runs the end-to-end tests as part of the PR process.

Acceptance Criteria

• A basic workflow is added to ensure that the Docker container builds and that the end-to-end tests run on every PR

Implementation Notes

• This may depend on #17 so that we can pull from a branch of oscal-rest-service rather than relying on a token with packages:read (but we may not need that anymore since oscal-rest-service is public).

Description

When running the all-in-one deployment with an SSP or Profile that uses a relative link to import a Catalog/Profile from local files, the link will not be resolved correctly in the OSCAL Editor, so the fetch will fail - meaning no controls are displayed on the page.
This is necessary to fix as some users will need to view and edit custom controls that are not found online.

Acceptance Criteria

• An SSP or Profile that imports a Catalog/Profile from other files in the local oscal-content directory will correctly display controls from that file when run from the all-in-one deployment image

Implementation Notes

This seems to be an issue with the configuration of the OSCAL Editor in Spring Boot within the all-in-one deployment.

As an example, a Profile can be loaded that contains an element in its imports list that points to a Catalog in back-matter with the rlink set to ../catalogs/my-catalog.json. At oscal-content/catalogs/my-catalog.json there is a valid OSCAL Catalog with some control.

When running both the REST Service and OSCAL Editor locally (outside of the Docker deployment) the OSCAL Editor makes the fetch as intended. The controls can be viewed, and developer tools shows that the OSCAL Editor makes a fetch request to:
http://localhost:8080/oscal/v1/catalogs/my-catalog.json

When running from the all-in-one image, however -
Developer tools shows that the OSCAL Editor makes an invalid fetch request to:
http://localhost:8080/oscal/v1/profiles//oscal/v1/catalogs/my-catalog.json
Causing the UI to render:

See parent EasyDynamics/easygrc#4.

Package the oscal-rest-service and example app

As a user of the all-in-one OSCAL Editor, I want to run the image from Docker Hub or the GitHub Container Registry on my Raspberry Pi (or other arm64/aarch64 machine) and Graviton2 EC2 instance.

Acceptance Critera

• An image supporting 64-bit ARM is also pushed to the image repositories on GitHub and Docker Hub

Implementation Notes

• We probably want to consider docker/setup-qemu-action@v1 for this
• If trivial we may want to support all architectures
• If this slows down the build process, we may want to only do this for the final build of the container (not the earlier build that gets used for testing) or use a more limited set of architectures (such as: arm, amd, riscv)

Subtask of EasyDynamics/easygrc#6.

As a developer wanting to use the all-in-one Docker image, I want to be able to specify the base URL of the OSCAL REST service for the OSCAL Viewer to send requests to as a build argument so that I can deploy the container to a shared environment.

Acceptance Criteria

• An OSCAL_REST_BASE_URL build argument can be specified in the Docker run command that configures the OSCAL viewer's base REST URL
• If no OSCAL_REST_BASE_URL build argument is specified a default of http://localhost:8080/oscal/v1 should be assumed
• End-to-End tests have been updated to accommodate the resulting REST mode of the OSCAL Viewer

Implementation Notes

• As a result of EasyDynamics/oscal-react-library#226 and due to the way React environment variables work, this will likely involve building the example app as part of the Docker image build

As a user of the OSCAL Editor All-in-One Docker deployment, I want the UI to support a backend URL other than localhost so that I can access the UI from a device other than the one running the container.

Acceptance Criteria

• The container can be run on one machine and the UI can be accessed from another machine

Original Issue Report

I have a two PC setup where the linux workstation runs the docker stuff and one windows PC where I access via browser the docker offered services.
At least the start page of this editor shows up, but then fails to load the oscal files from the folder.
In the browser networking I can see that the web stuff is loaded from my private IP range where the docker is running, but the API seems hardcoded to use localhost as a target.

As the other folders can already be optionally defined by docker environment variables, it should be handled the same way for the API mountpoint. And it should be mentioned in the READMEs, that it is currently not possible to use it outside of localhost browser access.

Example request of the web stuff (working):

And the failing API request afterwards:

After moving elements of our React application, some of our Cypress tests report the uncaught exception: "ResizeObserver loop limit exceeded." Most of the OSCAL_Edit_Tests.cy.js will display this error. Please note, when running our workflow tests will fail to this exception, although tests will not fail locally in Cypress but still display the exception.

Acceptance Criteria

"ResizeObserver loop limit exceeded" is no longer avoided and the uncaught exception does not show when running tests.

Implementation Notes

This error may be related to the communication between OSCALLoader and App in our React application.

Bug: Specifying "include-all" instead of "include-controls" in a profile import causes an error and doesn't render.

Yikes! Something went wrong loading the OSCAL data.
e is undefined

I've tested this with my own profiles as well as established working profiles from AJ Stein from NIST, e.g. https://github.com/ImportantFederalAgency/catalog/blob/main/dist/content/baselines/rev5/json/Step2_IFA_Baseline_High-Impact_profile.json

Description

As a contributor to the oscal-editor, I want to be able to more strictly enforce consistent code standards. Adding a linting check to all pr's will ensure a minimum level of code elegance.

Acceptance Criteria

• Ability to lint code locally before pushing to GitHub
• Each PR must check linting before allowing code to be merged in

#122, #119 & #113 have served to make cypress tests pass at the cost of speed. This is primarily due to the Rev5 Catalog's size making it take a while to load. Having the tests take such a long time to run slows down development and having them randomly fail is not sustainable.

Ideally we can completely remove the wait() calls we added in these prs. It would be acceptable to decrease there time significantly. This will most likely involve changes with the oscal-react-library loading these files.

Acceptance Criteria

• Significant reduction in wait times if not outright deletion
• Consider removing all retries of the tests
• Complete removal of intermittent failures.

This project looks great, thanks!

I am using the docker container, which I have started with the recommended directory structure and a single XML file defining a catalog. When I select my catalog, the UI shows:

Yikes! Something went wrong loading the OSCAL data.
Cannot read properties of undefined (reading 'uuid')

The docker log shows:

c.e.o.d.r.file.OscalCatalogRepoFileImpl : Unparsable content found at oscal-content/catalogs/cd2.xml

It appears that it is unhappy with my uuid. The relevant snippet of the XML file starts:

<?xml version="1.0" encoding="UTF-8"?>
<catalog xmlns="http://csrc.nist.gov/ns/oscal/1.0"
  uuid="fdaa19ab-f9f2-46ef-8c91-ec6902850a44">
  <metadata>
....
</catalog>

Any clues?

Thanks