Currently most of them are lowered to strings by the modeler so we don't actually test anything. Headers swagger is done better from this perspective.

it's quite python specific, so should have it in python repo

#234

Seen it in https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/specification/search/data-plane/Microsoft.Azure.Search.Service/stable/2019-05-06/searchservice.json

• Azure/autorest.modelerfour#314
• Azure/autorest.python#704

Storage team uses this on metadata parameters.

  "x-ms-meta": {
    "type": "string",
    "x-ms-client-name": "Metadata",
    "x-ms-header-collection-prefix": "x-ms-meta-"
  },

This should generate a dictionary, with string for key/value types. When serialized, each key is prefixed with the value x-ms-meta-.

There appears to be a divergence of swagger2 definition of "collectionFormat" and our codegen/testserver implementation.

swagger2 on collectionFormat
https://swagger.io/docs/specification/2-0/describing-parameters/

csv -> foo,bar,baz
multi -> foo=value&foo=another_value

So "multi" seems should be "foo=value&foo=another_value".

our testserver
https://github.com/Azure/autorest.testserver/blob/master/swagger/url-multi-collectionFormat.json#L86-L101
https://github.com/Azure/autorest.testserver/blob/master/legacy/routes/queries.js#L159-L162

It seems to be "csv" instead of "mult", which is "foo,bar,baz".

I've checked java and python generator implementation, both seems to be doing "csv" on "multi".

Checked the with appplatform service, they actually requires the multiple instance of query paramter.

Spec https://github.com/Azure/azure-rest-api-specs/blob/master/specification/appplatform/resource-manager/Microsoft.AppPlatform/preview/2019-05-01-preview/appplatform.json#L1745-L1756
expects to take in "version=v1&version=v2"

We need to add { recursive: true } parameter to https://github.com/Azure/autorest.testserver/blob/master/legacy/app.js#L59

Error: EEXIST: file already exists, mkdir 'D:\a\1\s\autorest.csharp\node_modules\@microsoft.azure\autorest.testserver\TestResults'
    at Object.mkdirSync (fs.js:934:3)
    at Object.<anonymous> (D:\a\1\s\autorest.csharp\node_modules\@microsoft.azure\autorest.testserver\legacy\app.js:59:6)
    at Module._compile (internal/modules/cjs/loader.js:1118:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1138:10)
    at Module.load (internal/modules/cjs/loader.js:982:32)
    at Function.Module._load (internal/modules/cjs/loader.js:875:14)
    at Module.require (internal/modules/cjs/loader.js:1022:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (D:\a\1\s\autorest.csharp\node_modules\@microsoft.azure\autorest.testserver\legacy\startup\www.js:7:11)
    at Module._compile (internal/modules/cjs/loader.js:1118:30) {
  errno: -4075,
  syscall: 'mkdir',
  code: 'EEXIST',
  path: 'D:\\a\\1\\s\\autorest.csharp\\node_modules\\@microsoft.azure\\autorest.testserver\\TestResults'
}

[3:39 PM] Vlad Joanovic
the issue is this: client calls a CRP list VMSS. sometimes this call returns 200 with empty result with a continuation token. this happens in some cases when a particular CRP region returns a 429.

When calling coverage, it doesn't have these scenarios. I can't tell why they aren't there, since they are defined in the paging.js.

This resets coverage but not optionalCoverage.

Related: Azure/autorest.csharp#786
Related: Azure/autorest.csharp#792

We had logic to only process a single response as an 'initial' response and another response as a 'final' response. So, max number of processed responses for LRO was 2. In the PR related, we allowed all responses to be processed so that we had a complete list of status codes. You can see the example from Compute here: https://github.com/Azure/azure-rest-api-specs/blob/945ce9037b678447f0a3322a1ba669856d0b9468/specification/compute/resource-manager/Microsoft.Compute/stable/2019-12-01/compute.json#L3666-L3675

Azure/azure-sdk-for-python#14111

For scenario like this:
https://github.com/Azure/azure-rest-api-specs/blob/2e9fadf018fa40555ca5b2bc826313bdc0752cdc/specification/resources/resource-manager/Microsoft.Resources/stable/2019-10-01/resources.json#L3984

Here:

That's explictly the WRONG behavior in the Swagger spec:
https://swagger.io/docs/specification/describing-parameters#mistakes

Common Mistakes
Using default to specify a sample value. This is not intended use of default and can lead to unexpected behavior in some Swagger tools. Use the example or examples keyword for this purpose instead. See Adding Examples.

It's unclear which tests belongi n which sections (vanilla vs azure), what these sectiosn mean (i.e. what flags are inherent to these sections), and it's also unclear which flags each test require.

Step 0 will be having a wiki with the flags we want each test to be generated with.

Going forward, we might also want to change our naming of these test sections (i.e., with just azure, it's not clear what tests should have flag azure-arm and which ones don't need it).

These 4 tests in LRO have an issue where the final GET returns 202 (restarts polling).

• LRODeleteAsyncRetrySucceeded
• LRODeleteAsyncNoRetrySucceeded
• LRODeleteAsyncRetryFailed
• LRODeleteAsyncRetryCanceled

To fix this, the suggestion is:

• a test polling on Location, where Location is a different kind of URL, poll on 202, ends on 200, and return that
• a test polling on Location, where Location is a different kind of URL, poll on 202, ends on 204, and return that
• a test polling on AAO, but Location was also there, polling on AAO until done, final GET on Location and return that

Got this issue in Python, would be nice to have it in testserver:
Azure/autorest.python#113

Some LRO cases set Location header to somethingBadWhichShouldNotBeUsed. It causes failure on Java SDK, as azure-core-management tries to parse the URL and throw exception on this (which seems not an unreasonable result, as that URL is indeed invalid), even though SDK likely not going to use the URL because Azure-AsyncOperation would take precedence.

E.g.
https://github.com/Azure/autorest.testserver/blob/master/legacy/routes/lros.js#L240

Javadoc on URL
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/net/URL.html#%3Cinit%3E(java.lang.String)
It throws checked exception MalformedURLException on constructor, regardless of whether the URL is used or not.

I would like to know whether this is intended, i.e., does these cases really intend to verify that SDK should never parse the Location header, or this is just a side-effect of the placeholder string.

If it is not intended, could we replace it with something looks like URL, e.g. http://somethingBadWhichShouldNotBeUsed

Because why not? :)

The intent of the path/test is to be able to provide a value for api-version as a parameter to the REST call. However, this value never becomes a parameter. Why? Because it is modeled as (what we call in M4) a constant:

{
  "name": "api-version",
  "in": "query",
  "description": "This should appear as a method parameter, use value '2.0'",
  "type": "string",
  "required": true,
  "enum": [ "2.0" ],
  "x-ms-api-version": false
}

Since it is a non-string enum with a single value, this is converted to a constant in M4. Most languages handle constants by directly injecting them into their path creation logic, and not allow the user to provide them, since they cannot have any other value. For this test to work as intended, you'd have to add at least 1 more value to the enum (such as "3.0") so that this becomes a proper SealedChoice parameter instead.

This situation applies to:

• apiVersionLocal_getMethodLocalValid
• apiVersionLocal_getPathLocalValid
• apiVersionLocal_getSwaggerLocalValid

NonStringEnumsPostFloat test expects float to be formatted as 200.0 which seem to be verifying python specific formatting as 200 is as valid as 200.0 when serialized in JSON.

Can we change a test to verify the test to expect 406.2 or something that requires data type to be a float.

There are tests for:

• int (int32)
• long (int64)
• float (single)
• double

But no tests for:

• short (int16)
• decimal

As surprising as it might seems, I didn't find any tests that does LRO and PATCH together

In Location header for instance

m4 needs to be able to handle text/plain and application/json first

cc @lmazuel @daviwil

• check out cucumber
• if decide to use cucumber, use cucumber to verify the shape of each language's generated code. else, add routes for users to hit
• add test for base 64 constant (original issue here)

See this example in search: https://github.com/Azure/azure-rest-api-specs/blob/master/specification/search/data-plane/Azure.Search/preview/2019-05-06-preview/searchservice.json#L6787

Idea is to make use of a markdown definition file to a have readable and easy way to define apis.
Sample: https://gist.github.com/timotheeguerin/4842e0cb1d052362783f6671ed65239b

I think the most powerfull thing with this is the ability to have the body in a markdow code block and be flexible to return any content type(That is not too large) instead of having to escape everything(if using some json definition). So much more readable, easy to write and maintainable

For apis that can't just be a simple for this url pattern/body return this response. The system should support using some simple overlay on top of express. I don't believe many apis will really need this.

Initial version goal (PR in progress #245)

• Parse the markdown
• Register the mock routes + start server
• Basic command line(--debug, which folder to run, etc.)
• Raw request body match

Next:

• Start converting the test
• Add templating(So response can return content of the request)
• Body deep equal
• Tests

Nice to have:

• Add an automatic coverage(Make sure for all the swagger files there is a mock api, we can then have CI to enforce)
• Hot reload of the markdown file(Start the mock server and if a file gets updated it reload the routes)

For custombaseuri GetEmpty an accountName param is specified but is unused by the endpoint.
For morecustombaseuri GetEmpty there are several unused params, like vault, secret, etc.

To help with the migration having a way to proxy to the old system will help greatly

https://github.com/Azure/autorest.testserver/blob/master/legacy/routes/lros.js
Should be Succeeded instead of Succeded. Not all of them but several scenario names spell it wrong.

This should be a failure test case when the operationName doesn't match any operation in the swagger.

In a previous commit

allows year 10000 to be sent but now in master year 10000 will fail and return 500.

Enums are common.

Modelerfour 4.15.416 has now added the ability for non-query parameters (specifically header parameters) to be marked as api-version parameters using the x-ms-api-version extension. We should add a test case for this in one of the testserver specs so that language generators can make sure they handle it appropriately.

See https://github.com/Azure/autorest.modelerfour/issues/314

Currently we have only 2

See https://github.com/Azure/autorest.modelerfour/issues/233

Related to https://github.com/Azure/autorest.modelerfour/issues/162 and https://github.com/Azure/autorest.modelerfour/issues/163