bastion-dev / bastion Goto Github PK

We currently do the following when releasing a new version of Bastion:

• Merge the develop branch into master branch.
• Set the POM file's version to the next release version number.
• Run mvn deploy -Prelease.
• Commit.
• Tag the current commit on master for the version we're releasing.
• Merge master into develop.
• Set the POM file's version to the next SNAPSHOT version.
• Commit.
• Push master, develop and the new tag.

We need to automate/script this so that we can configure Travis to run the whole process by itself.

The JsonSchemaAssertions object does not assert for application/json in the content-type header. It should behave similar to JsonResponseAssertion in that it checks that the content-type is application/json. It should also allow the user to override the expected content-type using overrideContentType().

At the moment, Bastion.request() requires descriptive text. It would be good to have an alternative Bastion.request() method which does not require descriptive text (the descriptive text is auto-generated somehow).

We need a way for users to specify Global HTTP attributes that will be sent with each request sent by Bastion. The attributes include:

• HTTP Query Params
• HTTP Headers
• Route Parameters

I think it's best if the API to modify these global attributes goes into the Bastion entry-point class, Bastion.

java.nio.file.InvalidPathException: Illegal char <:> at index 9: classpath:/json/create_sushi_request.json
        at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
        at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
        at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
        at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
        at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
        at java.nio.file.Paths.get(Paths.java:84)
        at rocks.bastion.core.FileRequest.guessResourceMimeType(FileRequest.java:274)

All tests in FileRequestTest are failing due to this exception.

JsonRequest requires a JsonRequest.get(url) utility method which functions the same way as GeneralRequest.get() but sets the content type to application/json.

This will also allow the .bind() to properly decode the model of the response from the GET into its json representation. The current alternative to this is using GeneralRequest.get(url).setContentType("application/json") .

Trying to assert on the HTTP statusCode using a lambda function as below:

.withAssertions((statusCode, response, model) -> Assertions.assertThat(statusCode).isEqualTo(200))

Will throw the following exception:

java.lang.ClassCastException: com.fasterxml.jackson.databind.node.ObjectNode cannot be cast to java.lang.String at rocks.bastion.core.BastionBuilderImpl.executeAssertions(BastionBuilderImpl.java:168) at rocks.bastion.core.BastionBuilderImpl.call(BastionBuilderImpl.java:101) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

We need a User Guide for Bastion similar to Hibernate's user guide which explains all the different features of Bastion in detail.

It should be Markdown generated documentation. Any code that appears in the documentation should be working and tested code.

The test contentType_jsonFileType_contentTypeShouldBeJson is failing when building Bastion on Mac. Bastion is failing to detect the file type and is therefore resorting to the default text/plain filetype instead of the expected application/json.

I have discussed this with @KPull who found the following related links:

• http://stackoverflow.com/questions/12407479/why-does-files-probecontenttype-return-null
• http://bugs.java.com/bugdatabase/view_bug.do?bug_id=7133484

Similar to #21, we need to add a fromObject() method to the FormUrlEncodedRequest class. This will serialize the given object into a URL encoded string to be sent with the request.

The name associated with the HttpRequest is meant to describe the request type (eg. 'Register a User'). The descriptive text with the Bastion test builder is meant to add additional info about the specific test (eg. 'Invalid Registration').

These two should appear in test reports in BastionRunner using the method BastionBuilderImpl#getDescriptiveText, however this is not currently being used inside the Bastion test builder.

As discussed in KPull/Bastion#34.

We should create a code style file, using IntelliJ IDEA at least, for use when formatting Bastion source code. This will ensure that the code is consistent across everyone's contributions. The IntelliJ defaults are already a good starting point, but we might want to tweak some options.

Implement the option to ignore the order of values in a JSON Assertion for particular array fields in case an API does not guarantee such ordering, while still reporting errors in case of missing or extra values in the array.

Examples:

{ "array":["first","second","third"] } == { "array":["third","first","second"] }
{ "array":["first","second","third"] } != { "array":["third","first"] }
{ "array":["first","second","third"] } != { "array":["third","first","second","fourth"] }

Since we are using ObjectMapper for JSON de/serialization we should provide a way for Bastion users to configure the ObjectMapper we use for Bastion; ideally through Bastion.config().

Therefore we should also make sure that we use the same instance of ObjectMapper whenever we de/serialize.

I think we can have Bastion.config().objectMapper() provide an instance which can be configured.

We need to add and support a new type of request: MultipartRequest. This is important because it is usually used by web applications to upload files to a web server.

The MultipartRequest should support:

• addRequest: Adds an HTTP request to the multipart request. Remember that a multipart request is one big HTTP request that contains smaller "HTTP requests" within it.
• addAttachment: A convenience function which calls addRequest above with a GeneralRequest object that sends the given file.

Finish the implementation of the JSONRequestTest class which should cover all possible paths through the JSONRequest.

Assertion objects have some common capabilities. For example, both the JsonSchemaAssertions and the JsonResponseAssertions allow the user to set an expected content type. The code that handles this kind of common functionality is currently duplicated in these requests. There may be other cases of this. At the end of the day, these objects are all making assertions on HTTP responses, so they are bound to have some common functionality.

We need to refactor the assertions to abstract this common functionality.

Suggestion from @KPull: We can abstract this into a CommonAssertions object (just like we have CommonRequestAttributes), and have the applicable assertions inherit from CommonAssertions by composition.

Implement a JsonResponseAssertion as a subclass of Assertions. This class will first verify that the response body returned is valid JSON and takes an expected JSON string. The assertion will not compare the response with the expected JSON string for being identical but will compare that their structure is identical.

The JsonResponseAssertion must also support comparing for the correct status code, optional headers and content type.

Given that I have a server which only accepts application/x-www-form-urlencoded (and will return an exception message in the response body if the content-type header is anything different) and I make the following request:

        Bastion.request(JsonRequest.postFromString(wizardsEndpoint, "{\"name\":\"Harry Potter\"}"))
                .bind(Wizard.class)
                .withAssertions((statusCode, response, model) -> {
                    assertThat(statusCode).isEqualTo(200);
                })
                .call();

then the following exception is thrown:

java.lang.AssertionError: Could not parse response into model object of type com.hogwarts.exceptions.UnsupportedMediaTypeExceptionResponse

	at rocks.bastion.core.BastionBuilderImpl.decodeModel(BastionBuilderImpl.java:188)
	at rocks.bastion.core.BastionBuilderImpl.call(BastionBuilderImpl.java:101)

Decoding of the exception into a Wizard failed and the actual response from the server was not logged at any point. It is not clear whether the server sent back an exception message or if Bastion was not able to decode the response.

This is important to have fully-reproducible automated tests.

The global timeout value, set using Bastion.globals().timeout() is not used by Bastion at the moment. We need to fix this and apply the timeout, only if the user has not specified a different timeout in their request.

Maybe we could have a USE_GLOBAL_TIMEOUT constant which the RequestExecutor checks for. When this constant is returned by HttpRequest.timeout(), the timeout will be whatever is set in Bastion's globals.

Our Request and Assertions subclasses are not final in the sense that anyone can subclass them. This is NOT the intended use of these classes and should be marked final. If someone would like to extend their functionality, they should compose the classes rather than subclass them.

At the moment, Bastion does not output any information related to the request sent or response received (unless the configured Assertions or Callback do so). We need a more universal way of outputting the full HTTP requests/responses using the Bastion builder. This should ideally be the default behaviour when a Bastion request fails (through an assertion or otherwise).

The Bastion fluent-build has a method, thenDo() which takes a Callback function object which gets executed after the request is performed. Now that Bastion has been active for quite a while, it seems this method is not really being used for anything and could be removed for our initial release. Remember that if someone wants to do something after a request, they can just type out their code as normal after the Bastion.request()...call() statement.

Alternatively, we can also change the concept into something like a withPlugins() method which takes a series of Plugin objects. This let's you configure how Bastion performs the request and what to do when it receives a response. A use-case I can think of, which is available in other REST libraries, would be something like a CookieHandler which automatically sets new cookies in the response into Bastion's globals().

We need an official logo for Bastion. The logo will be used for the Bastion User Guide and the GitHub Repository. A small version of the logo is also needed so that we can use it for the Maven Central Repository and related websites such as http://mvnrepository.com/artifact/rocks.bastion/bastion.

We need to implement a SOAPRequest class (implement Request) which is similar to the JSONRequest class but works for XML requests using SOAP.

Currently Bastion is just throwing a NPE

We incorrectly refer to the content data inside HTTP message as the content 'body'. The HTTP specification calls this the "HTTP Entity". Even though this is a breaking API change, I believe that, given the library is still in BETA, we change all references to "body" and update them to "entity" instead to better reflect the standard.

At the moment, the URL http://bastion.rock takes you to Bastion's GitHub repository. We need some sort of website containing the first crucial pieces of information for someone planning on using Bastion for the first time. The website should be attractive and mobile-friendly. There should also be a link to the JavaDocs and to the GitHub repository. Also, the website should ideally be hosted through GitHub pages: therefore, it needs to be in a folder named /docs on the master branch.

When running all tests in IntelliJ, the groovy test seems to fail with a NPE.

Similarly to #24, we need to add a new type of assertions object, FileResponseAssertions, that will load the binary data from the given file or resource and compare it to the content body received from the server.

The FileResponseAssertions should try to guess what the Content-Type header should be depending on the file chosen (some files will even have a MIME type embedded within them).

At the moment, the Bastion repository is under the KPull account. In the spirit of open-source and, as an effort for building a community that actively contributes to Bastion, I believe we should split the Bastion repository under its own Bastion organisation, separate from the KPull account.

The main Bastion repository will always have the latest Bastion version as its master branch. I would then make pull requests to the main repository as everyone else, from my own account.

Does anyone have any better suggestions for managing this?

We need to set up TravisCI on Github to automatically build pull requests and verify that all tests pass.

There's a problem with StatusCodeAssertions.expecting() method which is causing compiler errors when users pass in integer literals directly to StatusCodeAssertions.expecting().

When this happens, compilation fails because there are two ambiguous expecting() methods: one which takes Integer objects and one which takes int primitives. The initial tests didn't catch this because in StatusCodeAssertionsTest I had wrapped some repeated code into a method testAssertion() which takes int primitives: this would resolve the ambiguity for the compiler.

However, we can't expect users to do what I did in the test; most of the time, users will supply literals directly into the method. We have to resolve this issue by removing one of the ambiguous expecting() methods.

P.S.: We have to resolve this before we can release Bastion version 1.0 for sure because this will require public API changes.

We have been invited to demonstrate Bastion at the Malta QA Professionals Meetup during the first quarter of 2017. This is a great opportunity to get other people involved with Bastion.

We need:

• A 30 minute presentation.
• A live demonstration of Bastion in a IDE.
• A volunteer who wants to conduct the presentation.

The presentation must not be longer than 30 minutes and the target audience are other developers and QA engineers. Anyone who wants to give the presentation is invited to reply to this issue so that we can help them with preparing the presentation.

Add a new request type, FileRequest, which takes a file or resource and sends it to the remote server. This request type should try to set the correct content type depending on the file chosen (some files will even have the MIME type embedded within them).

Right now, Bastion has too many dependencies which are not necessarily used by the average user. In particular, I can see two dependencies which we can start with:

• JUnit 4
• JMustache

These libraries should be optional dependencies. The user should include them in their own POM file if they are going to use these features in Bastion.

We need to implement a FormDataRequest object, similar to GeneralRequest that also provides methods which will allow its users to add form-data attributes to the request's body. The request will initially have content-type type application\x-www-form-urlencoded but this can be overridden by the user using the overrideContentType() method.

One of our tests, GoogleBooksApiTest, uses the actual Google API endpoint for testing. Unfortunately, this has been found to change. We need to replicate the responses using our own embedded server and point the GoogleBooksApiTest to our embedded test server.

When loading a request/assertions from a file, we should use a template engine such as Handlebars or Mustache to process the file as a template and replace any template variables with values passed in from a test.

This allows us to pass in variables to files from test code. Implementation-wise, even if we specify the request from a string, the string would be a template which is processed by our Template library.

The request/assertions object would then have methods, `.addTemplateValue()' which will replace all the template variables in the body with the values given by the user.

Bastion users have reported that bind() does not work very intuitively. In particular, the following request fails (assuming the server returns content-type application/json):

String data = Bastion.request("Create Sushi Request", new CreateSushiRequest())
                .bind(String.class)
                .withAssertions((statusCode, response, model) -> {
                    assertThat(statusCode).isEqualTo(201);
                }).call().getModel();

A user would expect that all the HTTP response entity data will be available as a String. Right now though, since the content-type is application/json, Bastion creates a single ObjectNode model and cannot return a String.

Here's my idea (looking for feedback):

Bastion will register a number of ResponseDecoders (as it does now). Each ResponseDecoder may return a view object (they could return nothing). These views will be constructed by parsing the HTTP response entity data into various objects. For example, the following response decoders:

• ObjectNodeResponseDecoder - Returns an ObjectNode if the HTTP response content-type is application/json.
• JsonObjectMapperResponseDecoder - If the content-type is JSON, takes the object type inside the DecodingHints object and attempts to parse the HTTP response into an instance of that object type.
• YamlObjectMapperResponseDecoder - Same as above, but for YAML.
• ProtobufObjectMapperResponseDecoder - Same as above, but for Protobuf binary data.
• MapResponseDecoder - Provides a map representation if the response is a collection of key/value pairs (eg. Form URL encoded data).
• ImageResponseDecoder - If the content-type is an image, this will return an Image object
• StringResponseDecoder - Provides a String view of the HTTP response entity regardless of the content-type.
• ... and so on.

Now, bind() will assert whether a view of the given type exists in the Response. This view will be provided to the Assertions when the assertions are executed. Furthermore, after the call() method, the user may:

• Call getModel() to retrieve the view specified in bind() earlier.
• Call getView(), providing the view object type which the user wants.
• Call getResponse(), which returns the HTTP response representation. The user may then call getModel() and getView() on the response object itself which will behave just like the above two methods.

All this allows us to extend for new future views simply by implementing more ResponseDecoders and it also frees us from having to decide (and guess) which single ResponseDecoder the user wants us to execute to get the bound model.

Replace the fluent builder with a more declarative and manageable model of annotated classes. Each class represents an ApiSuite and the annotated methods in the class represent the different API requests in that suite.

This should explain how users can create and run an API suite.

BastionRunner, our JUnit Test Runner has turned out to be quite the letdown. A JUnit Runner needs to prepare a test complete Description to show in the IDE's GUI before the tests start to run. We wanted to show each Bastion call as it happened in the IDE's GUI but it appears we cannot do it.

Also, poor documentation means that tests running using BastionRunner are causing bugs such as #49. I believe we should just get rid of BastionRunner for version 1.0 of Bastion since it's not worth the effort of maintaining for now.

What does everyone else think?

The order in which methods are expected to be called on the Bastion builder are as follows:

Bastion.request(...)
       .bind(...)          
       .withAssertions(...)
       .thenDo(...)        
       .call()
       .getModel()

The problem is that the actual BastionBuilderImpl object does not enforce this order. A user can keep a reference to the object returned by Bastion.request() and call bind() twice in a row. We could figure out ways how to handle this, or we could save us all the trouble of supporting this and just restrict the user from calling it more than once.

Notice that a fix for this issue must also enforce that the user calls the above methods in the order listed. A user cannot first provide an Assertions<String> object using withAssertions(), for example, and then choose to call the bind() method with a different object.

The methods exposed by Bastion.globals() need improved documentation in their JavaDocs. In particular, JavaDocs are missing for public methods in Configuration and GlobalRequestAttributes.

We need to explain the following:

• What globals are.
• What the various attributes mean (headers, query parameters, route parameters, etc.). To avoid duplication, we should link to the JavaDocs in HttpRequest.
• What the default values for the globals are.
• That the user should typically clear any globals set, using Bastion.globals().clear(). We should explain this in the method JavaDocs for Bastion.globals().

We should implement an Assertions object that will validate whether a JSON response conforms to a given schema. This is, in reality a weaker version of the JsonResponseAssertions because a JSON schema does not check for actual values in the return JSON response.

Despite this, such a type of JsonSchemaAssertions could prove useful if we don't know what the expected values returned by the system under test are. It also helps to verify that any input JSON assertion conforms to a more general schema in the first place.

We need to add a factory method, fromObject() or fromModel(), which takes any Java object instance, serializes it using a JSON serializer (eg. Jackson) and use the serialized object as the request body or expected body (for assertions).

We need to implement a new type of Assertions object which will simply take the expected status code as follows:

StatusCodeAssertions.of(200)

The example above will assert that the received response's status code is equal to 200 OK.

We should add support for a configurable timeout for separate Bastion requests. Each request will have a timeout. If the timeout elapses but the remote server (ie. the system under test) has not replied yet then Bastion considers the test as failed.

Once https://github.com/KPull/Bastion/issues/12 is done, this timeout can also be configured globally for all Bastion requests.