Now, spectest support below method;

• GET
• POST
• PUT
• DELETE
• PATCH

These method are defined in RFC 7231 section 4.3. The spectest is not supported them.

• HEAD
• CONNECT
• OPTIONS
• TRACE

Is your feature request related to a problem? Please describe.

The spectest produces reports in Markdown format on a per-test-case basis. It is hard for us to find the desired report among many Markdowns.

Therefore, we will add a function to index markdowns located under the specified directory.

Describe the solution you'd like

jbrown1618/markdown-index is close to the function we are looking for. However, some features are missing.

For example,

• change generating file name.
• change index markdown title
• change url text (now, it is same as file name)

Additional context

This feature will implement as cli command.
In addition, add the ability to generate indexes with go generate.

Introduction

The spectest is a package forked from steinfletcher/apitest. The apitest has the functionality to generate HTML documents from end-to-end test results, and the development of spectest was initiated to enhance this feature.

One month after the fork, spectest has added the capability to output documents in markdown format (I have also conducted refactoring). While there are several feature ideas, developing them alone would be time-consuming.

Therefore, I have decided to clearly outline spectest's roadmap and record feature ideas as issues. If you are interested in spectest, feel free to implement some of these issues.

Goal

1. Generate Markdown documents from E2E test results to reduce the time developers spend on document creation.
2. Support BDD-style testing methods.
3. Become a versatile library that allows easy writing of unit tests for entities other than APIs.
4. Provide CLI commands to enhance the developer experience during unit test execution.

Roadmap

• Roadmap 1. Refactoring
• Roadmap 2. Adding Functionality to Simplify Unit Testing
• Roadmap 3. Enhancement of Documentation Feature
• Roadmap 4. Creating Mocks for External Systems Such as AWS
• Roadmap 5. Making Spectest Usable for Simple Unit Tests
• Roadmap 6. Creating a Website to Explain How to Use Spectest

Roadmap 1. Refactoring

• #18
• #25 (This issue is related roadmap 4)
• #29
• #34
• #35
• #145
• #146
• #148
• #164

Roadmap 2. Adding Functionality to Simplify Unit Testing

• #140
• #141
• #162

Roadmap 3. Enhancement of Documentation Feature

• #100
• #139
• #168

Roadmap 4. Creating Mocks for External Systems Such as AWS

• #143
• ??? Adding another service mock such as firebase.

Roadmap 5. Making Spectest Usable for Simple Unit Tests

• #97

Roadmap 6. Creating a Website to Explain How to Use Spectest

• #144

Is your feature request related to a problem? Please describe.

I modify the code so that unit tests pass in a Windows environment.
I know that the difflib package is not compatible with windows. I think we should export for difflib package as an external repository.

Is your feature request related to a problem? Please describe.

A feature has been added to change the output of test results when using spectest ./... instead of go test ./.... For instance, successful tests will be displayed as green dots, while failed tests will be represented by red dots. This functionality is inspired by ideas from onsi/ginkgo and shellspec/shellspec.

Describe the solution you'd like

WIP

Describe alternatives you've considered

WIP

Additional context

WIP

Is your feature request related to a problem? Please describe.

The spectest package generates sequence diagrams.
This diagram has a hash as the file name. I want to store sequence diagrams in a document directory to share them with developers. However, using hash names for file titles is a bit inconvenient.

So, I wanted to give custom names to the sequence diagrams.

Describe the solution you'd like

The solution is to add CustomeSequenceDiagramName().
（Method name may not be appropriate）

example

				Report(spectest.SequenceDiagram()).
				CustomeSequenceDiagramName("diagram_name_is_here").
				// ...
				End()

CustomSequenceDiagramName set APITest.meta["diagramName"]. If meta['diagramName'] is set, the hash name will not be used when outputting the sequence diagram.
The problem with this approach is that if you execute Meta() after executing CustomSequenceDiagramName(), the settings from CustomSequenceDiagramName() are lost.

Describe alternatives you've considered

The alternative way is implementing new ReportFormatter interface.
This method has a large modification scale. However, it has the advantage of being able to be implemented outside of the spectest package (I would adopt this idea if no additional features were added).

Additional context

nothing

Is your feature request related to a problem? Please describe.

The apitest package, which is the source of the fork, provided an example at github.com/99designs/gqlgens. However, they were broken and did not work. We will re-create the examples again.

Currently, we are generating separate HTML files for each test case, but I would like to generate a single HTML document that consolidates documentation for each endpoint.

Is your feature request related to a problem? Please describe.

After generating the index markdown, spectest index does not output anything. I think it is difficult for the user to understand what happened with this specification. Therefore, the generated index file path will be outputted to the standard output.

Describe the solution you'd like

If markdown.GenerateIndex() completes successfully, it will output the index file path to the standard output. If an error occurs, it will directly return the error to the higher-level function.
https://github.com/go-spectest/spectest/blob/26de85c47a740f596035376cf4b84e09ac9c7fc2/cmd/spectest/sub/index.go#L59-L62

To proceed with refactoring, it is necessary to ensure the current behavior through unit tests. First, create unit tests for functions that are not covered by test coverage. Since unit tests are created incrementally, it is expected that multiple pull requests will be made.

You can check the unit test coverage with the following command.

$ make test
$ open coverage.html 

Is your feature request related to a problem? Please describe.

The spectest/x/db package is a wrapper for the database/sql package. However, there are wrappers for deprecated methods, and some parts do not work as wrappers for the current database/sql package.

So, update spectest/x/db.

Is your feature request related to a problem? Please describe.

We are implementing mocks for the official AWS package (SDK) in go-spectest/aws. The package was severely broken at the time of forking, and there was also an issue with the limited number of supported AWS resources.

Therefore, we believe it would be best to start fresh by adding AWS SDK mocks anew. With over 100 types of AWS resources, manually adding mocks would be daunting. We are considering adopting a method to automatically generate mocks from the interfaces exposed by the AWS SDK.

Additional context

Eventually, go-spectest/aws is planned to be integrated into go-spectest/spectest.

Why we need side project

I believe that the best way to improve the spectest package is to actually use it. Since spectest is a testing framework, it requires something to test. However, it doesn't make sense to use it in very small projects.

Therefore, I plan to develop a client-server application and test its server-side components using spectest. The goal is to reach a state where spectest is used for all unit tests (currently, spectest provides functionality only for E2E testing).

The server and client will be developed in separate repositories. The client is intended for Android (kotlin) or Web (elm) platforms.

First Decisions

Deciding on the application concept is the first step.
We plan to create an app for upscaling images and videos, but this application will require Deep Learning training. Additionally, we need to consider other app ideas that do not require as much development effort.

Is your feature request related to a problem? Please describe.

We often manually create expectations for response data. This process is very tedious. Therefore, we will add a function to generate the expected value data in a specified path when the test is executed with a specific procedure.

Describe the solution you'd like

We would add the Golden() method. This method writes response body data to the specified file path. However, the write operation is performed only when certain flags are enabled.

The method to enable the flags is under consideration. For example, we can create a spectest cli command, and the Golden() method will generate a file if the --golden option of this command is enabled.

Introduce gosec. For example;

name: Gosec
on:
  workflow_dispatch:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  tests:
    runs-on: ubuntu-latest
    env:
      GO111MODULE: on
    steps:
      - name: Checkout Source
        uses: actions/checkout@v4
      - name: Run Gosec Security Scanner
        uses: securego/gosec@master
        with:
          args: ./...

Is your feature request related to a problem? Please describe.

Related #10

We need to add the following HTTP method mock methods to the Mock structure.

• HEAD
• CONNECT
• OPTIONS
• TRACE

Is your feature request related to a problem? Please describe.

Many projects managed by git submodule are tightly integrated with spectest. Therefore, whenever code refactoring is completed within each repository, it is incorporated into spectest.

Is your feature request related to a problem? Please describe.

"Spectest" has the functionality to create sequence diagrams. However, due to the coarse granularity of the generated sequence diagrams, I want to create sequence diagrams based on method calls.

When creating function/method-based sequence diagrams, it is straightforward to perform static analysis of the code. This method eliminates the need to execute it every time during testing, addressing concerns about slow performance.

Therefore, a command-based sequence diagram generation feature is developed to facilitate verification.

Is your feature request related to a problem? Please describe.

The apitest package, which is the source of the fork, provided an example at github.com/kataras/iris. However, they were broken and did not work. We will re-create the examples again.

Is your feature request related to a problem? Please describe.

It seems that "\n" is hard-coded as a newline code, so use "\n" on Linux and Mac, and "\r\n" on Windows.

Spectest has many fields within a single struct. I believe these fields can be encapsulated into smaller structs. By creating new structs and associating methods with them, the code readability and organization can be significantly improved.

In cases where you return random values like ULID, you can use a user-defined function to define custom Assert criteria and perform the Assert operation.

Is your feature request related to a problem? Please describe.

The spectest display response data in the document according to Content Type.
For example, spectest will display binary data in the document if the response data is an image. If possible, i would like to display images in the document.

Describe the solution you'd like

Determine Content-Type from HTTP headers and create HTML template generation process according to Content-Type.

Describe alternatives you've considered

The method I suggested has the potential to significantly rewrite the HTML template.

Additional context

nothing

Is your feature request related to a problem? Please describe.

Due to heavy use of method chaining, most of the error handling in apitest relies on panic(). However, even with method chaining, errors can be returned to the calling function or evaluated later.

In spectest, the error handling approach will be changed.

Is your feature request related to a problem? Please describe.

Some functions in spectest are hard to understand at a glance. To address this, I will add documentation. Additionally, due to the lengthy lines of code within the functions, I will refactor them into smaller, more manageable functions.

Is your feature request related to a problem? Please describe.

If the expected value does not match the obtained value, the following message appears. We want to display this message in red text to make it easier for users to understand the error content.

                Error:          Not equal: 
                                expected: true
                                actual  : false
                Test:           TestNewApp/Width_test
                Messages:       mismatched values for header 'Content-Type'. Expected image/png but received text/plain; charset=utf-8
                Name:           width,height is zero

Use fatih/color to change text color for cross-platform support.