• Concept of "Generated test cases" is confusing and unclear for the user. Do not generate test cases in a separate test suite in the WebApp
• Hide "technical" test steps (which control or manipulate the test run execution sequence) in default LoD
• Fix incorrect order of test suites
• "Called test cases" appear in the wrong test suite
• Fix incorrect encoding of URL in generated requests

This enhancement requires the implementation of a XQuery module that can be loaded by BaseX-based ETSs. The module should provide XPath functions for determining all FeatureTypes in an XML document by reading and analysing the associated schema.

1. A schema-element() function is required https://www.w3.org/TR/xpath20/#id-schema-element-test . Note: for performance reasons this is not implemented by BaseX itself.
2. The function call etf:feature-members( rootNodeOrSchemaLocation ) should return a sequence of all Feature member nodes in the Feature Collection.
3. The function call etf:feature-type-names( rootNodeOrSchemaLocation ) should return a sequence of strings with the names of all Feature Type names.

See also: http://docs.basex.org/wiki/Java_Bindings

Currently processed values are not fully escaped in SoapUI requests. Example: outputformat=application/gml+xml;%20version=3.2 where application%2Fgml%2Bxml%3B%20version%3D3.2 would be correct.

in Status View

See design report 8.2

Test driver specific part. See design report 8.1.4.

Using Safari on Mac the downloaded HTML file has no .html extension. Double click does not open the report.

At the moment most model items are hold in memory

Reported by @thijsbrentjens Geonovum/etf-test-projects-inspire#48

Reported from E.L.F. on WFS tests: when using HTTP authentication, the credentials (intended for a service) are also used when getting metadata records. This may go wrong on some CSW implementations that also support authentication on metadata (like GeoNetwork) .

Proposal: only use credentials on the service requests, not on any metadata test.

Two approaches possible:

1. change the method updateCredentials in the Groovy script de.interactive_instruments.xtf.Util. We should have some kind of filter that the request is NOT part of the service, but to a metadata document (a nasty one when doing Atom tests: determine if the request is for metadata or for a service feed)
2. override/remove any Authorization header in the metadata get requests. This may have disadvantages if the metadata is also behind some authentication mechanism, but I think this is more rare

• Bugfix in etf-sui-owsgtl, will be integrated in test driver etf-suitd 1.5.5 (or the SoapUI plugin etf-sui-ae 1.3.4)

See design report 8.1.4

• Design manual Test Step and document in Design Report
• Implement in core and data storage schema
• Check and implement mapping of manual test steps in SoapUI

For BaseX based tests there is no real mapping required and it should be quite simple to implement.

It seems that Schematron validation is not exectued if XML validation fails. Is this by design?

Is there a way to force Schematron validation to always run, also if XML validation fails?

• Merge files from ETS repo

Currently the start button in the simplified workflow is only visible after opening the test project details. If possible, add the start button directly to the headline to make the workflow more intitive.

See https://github.com/opengeospatial/teamengine

• Start multiple test runs
• Rest based
• Performance, support for pagination (depens on #27)

• etf-core contains Dtos without jaxb annotations and an API used by clients
• Add an API for remote repositories to etf-core
• etf-spi is the API intended to be extended and implemented by test drivers and the datasource
• etf-bsxds implements etf-spi and provides a JAXB mapping for the Dtos
• etf-bsxds provides datastore XML-schema

Ask -when the user wants to download a Test Report- if the attachments shall be packed with the test report into a zip file.

It is also important to test test cases which do not pass. Therefore, it might be necessary to add a mockup server in front of the INSPIRE services which can return for certain requests, responses that will fail the test cases.

Currently a log4j JSON appender is implemented in ii-common-utils which is used for sending the progress log to the client. Remove this dependency.

See Design report 8.3

• Central store for credentials. Each credential is associated with an URL. Implement getCredentialsFor(URL)
• Support wildcards in URL
• add to etf-core and etf-spi

This will consume memory massively, however ensures that every test run is isolated from other test runs.

Prerequisite for JSON API and Schema #27

Invalid references to other schema documents must be reported in etf-sui-ae.

Change

• SUI testdriver
• SUI plugin

to support the new etf-core 1 model.

It's unclear to the user why a test run "hangs" in the state "finalizing" after the run has been cancelled.

Currently a test run in the "finalizing" state, waits for being garbage collected (and for result persistence) by the test run manager. States: cancelling -> canceled -> finalizing -> (X).

Maybe a simpler, clearer state transition is required, from cancelling -> canceled (O).

If I try to cancel a test run (using the "Cancel" button), this has no effect and the test run is still listed:

The test run is stalled and not executing anything anymore as far as I can see (for a brief moment "Error loading page" is shown in the middle of the log window, too):

Description

From a user we got reports that their metadata documents can not be uploaded / used in the validator at: http://validatie.geostandaarden.nl/etf-webapp/testprojects?testdomain=Metadata

An example file is attached. fabd3dfc-1436-4920-933f-924c1c3c1d2f.xml.txt

My analysis: the XML prolog is missing in this document:

<?xml version="1.0" encoding="UTF-8"?>

The XML prolog is optional as far as I know, or am I wrong? In that case, the file should be processed.

(Environment: CentOS, Java 8, Tomcat 8)

Steps to Reproduce

1. got to http://validatie.geostandaarden.nl/etf-webapp/testprojects?testdomain=Metadata
2. Choose one of the validators
3. upload file
4. start test

Expected behavior: file is processed.

Actual behavior: Error message: De geüploade file kan niet worden gebruikt: text/plainis not supported

Building the etf-webapp war-file (after resolution of #36) fails. I get a compilation error:

./gradlew build war
ETF-BDA: Applying ii BDA gradle script 1.0.23 to the project
ETF-BDA: Authenticating will be done as user [email protected]
:compileJava

FAILURE: Build failed with an exception.

* What went wrong:
Could not resolve all dependencies for configuration ':compileClasspath'.
> Could not find xercesImpl.jar (xerces:xercesImpl:2.11.0.beta).
  Searched in the following locations:
      http://services.interactive-instruments.de/etfdev-af/ext-releases-local/xerces/xercesImpl/2.11.0.beta/xercesImpl-2.11.0.beta.jar

I checked and the manual download fails too. Is the jar-file corrupt or not accessible?

Operating systems and browser

• Linux (Ubuntu 16.04 based)
• JDK 1.8.0_91 (OpenJDK), 64 bit

Steps to Reproduce

1. go to the root dir of the src code
2. execute the command: ./gradlew build war

The build fails with the above message.

Support repository items that are stored in a repository. ATS git repo support will be implemented as part of this work. Other repository items will be CC (git repo) and test drivers (mvn repo in later releases).

Depends on:
#21

HTTP requests steps that possess parameterized key value pairs are not natively encoded by SoapUI.

• check if we can register some kind of HTTP request post processing or otherwise
• implement a function that needs to be called explicitly before these request are send

References
Geonovum/etf-test-projects-inspire/pull/68
HTML URL Encoding Reference
[SoapUI API: extractParams() ](http://www.soapui.org/apidocs/com/eviware/soapui/impl/rest/support/RestUtils.html#extractParams%28java.lang.String, com.eviware.soapui.impl.rest.support.RestParamsPropertyHolder, boolean%29)
ETS-SEL: generateStandaloneHttpRequest()

Starting test run on Windows results in "Illegal character in opaque part":

2016-06-01 08:15:54.361 [http-apr-8080-exec-8] ERROR d.i.e.w.c.GlobalDefaultExceptionHandler - Last action: http://server:8080/etf-webapp/testruns/start
de.interactive_instruments.exceptions.StoreException: Initialization Failed. Illegal character in opaque part at index 2: D:\Programme\Apache Software Foundation\Tomcat 8.0\webapps\etf-webapp\WEB-INF\etf\ds\appendices3cca79c5-9fb3-441f-b37c-789864e8b9ae/appendices/
    at de.interactive_instruments.etf.dal.basex.dao.BasexTestReportDao.createFactory(BasexTestReportDao.java:147) ~[etf-bsxds-1.0.7.1.jar:1.0.7.1]
    at de.interactive_instruments.etf.dal.basex.dao.BasexTestReportDao.create(BasexTestReportDao.java:165) ~[etf-bsxds-1.0.7.1.jar:1.0.7.1]
    at de.interactive_instruments.etf.webapp.controller.TestReportController.createReport(TestReportController.java:110) ~[classes/:1.0.7.3]
    at de.interactive_instruments.etf.webapp.controller.TestRunController.start(TestRunController.java:347) ~[classes/:1.0.7.3]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_45]
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_45]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_45]
    at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_45]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:222)

For a Schematron test we need XSLT2 support. I added the attached file to use, but this resulted in the error:

Fail: This implementation of ISO Schematron does not work with
schemas using the "xslt2" query language.

Do I need to change something in the test?

schematron.sch.txt

• variable instead of endpoint is shown in "Request" field
• 0,00 MB is shown as reference size
• HTTP-Response link missing: information that a response is discarded if the test is successful

Depends on #40

http://www.slf4j.org/legacy.html

See Geonovum/etf-test-projects-inspire#60 (comment)

• Add example BaseX ETS to etf-bsx-integration-test-resources for integration tests
• An open point is still which complex test data could be used that do not contain licensed/sensitive information
• Also add test data that could be used for testing etf-gmlgeox

Setup a public demo instance which is updated by the CI.

Depends on:
#7

Publish definition to SwaggerHub:

• https://swaggerhub.com/api/etf/etf/v2
• https://swaggerhub.com/domains/etf/etf-domain/v2

TestObject wird immer neu indexiert

In the test report, I cannot really see this. At the top it says:
“Test run on 28.01.2016 - 09:04 with test project additional-rules-…”

17-02-2016 11:53:34.584 TestRunTask initialized
17-02-2016 11:53:34.593 Starting TestRunTask.6c7c8b3b-d57f-4e2f-a272-9c99fa804a5d at 2016-02-17T11:53:36.092Z
17-02-2016 11:53:34.594 Using test object "temporary-f43a0940-d475-4dad-bb05-39833de7a919"
17-02-2016 11:53:36.094 Setting state to CREATED
17-02-2016 11:53:36.095 User parameters accepted
bla
17-02-2016 11:53:45.006 Added 1 files
17-02-2016 11:53:45.008 Changed state from CREATED to INITIALIZING
17-02-2016 11:53:45.008 Changed state from INITIALIZING to INITIALIZED

Using version 1.0.2 of etf-gmlgeox, the tests for geomety validation fail to run.

I'm using this (sample) test project: https://github.com/Geonovum/etf-test-projects-nl/tree/master/src/data/GML3.2%20SF2%20en%202D-Geometrie

Log details:

[14:48:04,968]  Compiling test script
[14:48:05,007]  Starting xquery tests
[14:48:05,008]  Features: 0.3 ms
[14:48:05,010]  Statistics: 0.96 ms
[14:48:05,060]  Test Other.Geometry.01: 49.9 ms
[14:48:05,153]  Handling unexpected exception in TestRunTask.3e25c398-5d6e-4c58-b649-b243d839ff2b
[14:48:05,153]  Changed state from INITIALIZED to RUNNING
[14:48:05,156]  Testrun failed with an error: Stopped at ., 4/31:
[bxerr:BASX0000] No binding for namespace prefix imgeo

Stack Trace:
- ., 58/60
- ., 37/19
- ., 488/24
[14:48:05,156]  Duration: 0sec
[14:48:05,156]  TestRun finished
[14:48:05,156]  Changed state from RUNNING to FAILED
[14:48:05,156]  Releasing resources

Often it will be necessary to include a legal notice, information about data privacy or other disclaimers on each page of the web application, typically via links in the footer.

Example: "Testrun 1, Bit-Prüm"

Solution: escape special chars like german umlauts and comma, etc.