mirsaeedi / dotnet-apim Goto Github PK

source file- Apim.DevOps.Toolkit.Core.ArmTemplates.ResourceCreators.GlobalPolicyResourceCreator, Line -20
condition:-

if (deploymentDefinition.Policy != null)
			{
				return Array.Empty<ArmTemplateResource>();
			}

probably should be like this

if (deploymentDefinition.Policy == null)
			{
				return Array.Empty<ArmTemplateResource>();
			}

definition file

version: 0.0.1
apimServiceName: apim-xxxxxxxxxxxx
policy: api-management/policies/global.xml

Variables with an underscore in the name are not supported.

Example: my_variable

Current behavior: Dupplicated Key Exception (when at least two variables end with the same suffix like "api_name" and "operation_name").

System.AggregateException: One or more errors occurred. (An item with the same key has already been added. Key: $(name))
    ---> System.ArgumentException: An item with the same key has already been added. Key: $(name)
      at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
      at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
      at System.Linq.Enumerable.ToDictionary[TSource,TKey](IEnumerable`1 source, Func`2 keySelector, IEqualityComparer`1 comparer)
      at System.Linq.Enumerable.ToDictionary[TSource,TKey](IEnumerable`1 source, Func`2 keySelector)
      at Apim.DevOps.Toolkit.Core.Variables.VariableCollection..ctor(IEnumerable`1 variables) in C:\Users\6104652\source\repos\dotnet-apim\src\apimtemplate\Core\Variables\VariableCollection.cs:line 19
      at Apim.DevOps.Toolkit.Core.Variables.VariableReplacer.GetFromString(String variables) in C:\Users\6104652\source\repos\dotnet-apim\src\apimtemplate\Core\Variables\VariableReplacer.cs:line 60
      at Apim.DevOps.Toolkit.CommandLine.Commands.CreateCommand.LoadGlobalVariables(CommandLineOption option)
      at Apim.DevOps.Toolkit.CommandLine.Commands.CreateCommand.ProcessAsync(CommandLineOption option) in C:\Users\6104652\source\repos\dotnet-apim\src\apimtemplate\CommandLine\Commands\CreateCommand.cs:line 27
      --- End of inner exception stack trace ---
      at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
      at System.Threading.Tasks.Task.Wait()
      at Apim.DevOps.Toolkit.Program.ProcessCommand(CommandLineOption option, IMapper mapper) in C:\Users\6104652\source\repos\dotnet-apim\src\apimtemplate\Program.cs:line 44

Hello Ehsan, Team,

I hope that everything is going great. Thanks for your effort on this very needed feature, so I have a suggestion/question.

• Is there a way, that we can set up a provided (self-hosted) API Developer Portal for each API using the tool ?

Thanks for any info regarding this one, have a great day ahead.
Alan

Making the discovery automatic would would eliminate some of the manual work that is required to be done when adding support for new entities.

Below is the config file. Existing api version is 1.5. Post new deployment 1.5 is replaced with new deployment. If i want to preserve both version or revision what should be the configuration.
apiVersionSets:
- id: $(title)
name: $(title)
displayName: $(title)
description: $(title)
versioningScheme: Query
versionQueryName: api-v-q
versionHeaderName: api-v-h

apis:
- name: $(title)
displayName: $(title)
type: http
description: $(desc)
serviceUrl: $(backend) # global variable
openApiSpec: D:/a/1/s/$(title)/Config/Swagger/openapi.json # global variable
policy: D:/a/1/s/$(title)/Config/Policies/policiesapi.xml
path: $(title)
subscriptionRequired: true
isCurrent: true
apiversion: $(version)
apiVersionSetId: $(title)

Hi,

Is it possible to use ARM parameters in the config file? For our backend we make use of Azure Functions with function keys. Function keys are secret and are not allowed to be stored in our repository (e.g. config files). That's why we prefer to use parameters in our ARM deployment. Is this already existing functionality? If so, can you please explain how that works? If not, is there an alternative way to change parameters / variables in config files over different environments?

Thanks in advance.

Previously, swagger import would overwrite the api display name with the title. However, this is no longer the case as per this comment. I've verified this as well. Modify dotnet-apim tool to not download the swagger. This also prevents hitting character limits if the swagger content is large.
@mirsaeedi

As mentioned in the section "Maintaining Large YAML Definitions By Splitting Them" we have do so and in our pipeline we have 2 commands:

dotnet-apim -c ".\Products-Config.yml" -o "$(Build.ArtifactStagingDirectory)/Dev" -f ".\EnvironmentConfigurations\config-dev.yml" -p "dev-"
dotnet-apim -c ".\APIM-Subscriptions-Config.yml" -o "$(Build.ArtifactStagingDirectory)/Dev" -f ".\EnvironmentConfigurations\config-dev.yml" -p "dev-"

The first one creates a Product that then the second one ( Subscriptions ) will use to set permissions, still the 2 .yml are not aggregated into one single dev-apim_deploy.template.json file, what it seems to happen is that the last overrides the first one. How should we run this so that the Products-Config.yml and the APIM-Subscriptions-Config.yml are aggregated into a single deployment definitions.

@mirsaeedi One more haha...
We have a need to reference the Service Fabric client certificate using clientCertificateId. This property is available in API version 2021-08-01.

The value in the generated ARM template would be a resourceId, i.e.

"serviceFabricCluster": {
    "clientCertificateId": "[resourceId('Microsoft.ApiManagement/service/certificates', parameters('ApimServiceName'), '[clientcertId]')]",
     ...
}

I have working code I can submit a PR for, but we'd need to update the API schema version. Lemme know your thoughts :)

The ARM schema has graphql-link spec format now. It would be nice if we could support this to allow importing GraphQL APIs.
As far as implementation, was thinking we could just add another property in the yaml file to specify graphqlspec then set the format in the ARM template to graphql-link.

@mirsaeedi Lemme know your thoughts. I can help with implementation as always. 😃

Below YAML definition currently only supports basic creation of product as part of the api deployment and assigns the "administrators" group by default.

apis:

• name: ApiD1
  displayName: API D v1
  serviceUrl: $(ApiServiceURL)
  openApiSpec: $(apimBasePath)\OpenApi.json
  path: $(prefix)/D/api
  apiVersion: v1
  apiVersionSetId: ApiVersionSetD
  apiRevision: 1
  protocols: https
  subscriptionRequired: true
  isCurrent: true
  products: ProductD1

products:

• name: ProductD1
  displayName: Product D1
  description: Product for D1
  subscriptionRequired: true
  approvalRequired: true
  subscriptionsLimit: 1
  state: published

When using the dotnet-apim tool for generating the ARM templates for creating your API in APIM, the tool allows you to specify tags.

However, our ARM template deployment fails with the message: 'Tag not found. (Code: ResourceNotFound)'

It seems that the tag needs to exist on the APIM instance first in order for the deployment of the ARM template to work, as mentioned in this GIT hub issue as well: MicrosoftDocs/azure-docs#21817

Currently we work around this issue by having an azure CLI command for creating the tags, but could the code also generate an ARM template for the creation of the tags on the APIM instance, e.g.

{
    "type": "Microsoft.ApiManagement/service/tags",
    "apiVersion": "2022-09-01-preview",
    "name": "[concat(parameters('service_apim_name'), parameters('tag_name')]",
    "dependsOn": [
        "[resourceId('Microsoft.ApiManagement/service', parameters('service_apim_name'))]"
    ],
    "properties": {
        "displayName": "parameters('tag_name')"
    }
}

version: 0.0.1
apimServiceName: myAPIMService

apis:
    - name: myApi
      type: http
      _**description: myFirstApi**.  --> (not updated when deployed to APIM)_
      serviceUrl: http://myApiBackendUrl.com

noticed in the generated ARM template the way it constructs the "value" under resources it doesn't include the "Description" when passing the whole open API spec and when i manually added the "Description" it worked.

Could you please add this property so that the tool automatically adds it when the ARM template is generated?

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "ApimServiceName": {
      "type": "string",
      "metadata": {
        "description": "Name of the API Management"
      }
    }
  },
  "variables": {},
  "resources": [
    {
      "properties": {
        "value": "{ }", ---> this doesn't have the description property set
        "format": "openapi+json",
        "Description": "myFirstApi",
        "type": "http",
        "apiRevision": "1",
        "isCurrent": true,
        "subscriptionRequired": false,
        "displayName": "myFirstAPI",
        "serviceUrl": "https://myapp-webapp.azurewebsites.net",
        "path": "find"
      },

Hi

Thanks for the tool.

How do you link the logger for application insights to an api like the yaml provides from MS https://github.com/Azure/azure-api-management-devops-resource-kit/blob/main/src/README.md#APIDiagnosticConfiguration

If I try and add it, I receive an exception

2022-07-05T01:43:42.4477879Z System.AggregateException: One or more errors occurred. ((Line: 6, Col: 5, Idx: 164) - (Line: 6, Col: 5, Idx: 164): Exception during deserialization) 2022-07-05T01:43:42.4480896Z ---> YamlDotNet.Core.YamlException: (Line: 6, Col: 5, Idx: 164) - (Line: 6, Col: 5, Idx: 164): Exception during deserialization 2022-07-05T01:43:42.4514802Z ---> System.Runtime.Serialization.SerializationException: Property 'diagnostic' not found on type 'Apim.DevOps.Toolkit.Core.DeploymentDefinitions.Entities.ApiDeploymentDefinition'. 2022-07-05T01:43:42.4520876Z at YamlDotNet.Serialization.TypeInspectors.TypeInspectorSkeleton.GetProperty(Type type, Object container, String name, Boolean ignoreUnmatched) 2022-07-05T01:43:42.4526489Z at YamlDotNet.Serialization.NodeDeserializers.ObjectNodeDeserializer.YamlDotNet.Serialization.INodeDeserializer.Deserialize(IParser parser, Type expectedType, Func3 nestedObjectDeserializer, Object& value)
2022-07-05T01:43:42.4528339Z `

How can we specify oauth server name and configuration in yml

ARM deployment fails with first attempt because Backend cannot be created due to lack of referenced NamedValue.
Folliwng Backend Configuration requires NamedValue 'func' to be created, In other case Backend deployment fails

backends ARM "dependsOn": should contain NamedValues

"type": "Microsoft.ApiManagement/service/backends",
        "apiVersion": "2019-12-01",
        "name": "[concat(parameters('ApimServiceName'), '/apim-name')]",
        "dependsOn": [],

In order to create a revision, one must append ;rev=n to the API id otherwise the revision is not created. See docs and code in azure-apim-devops-kit
.

My yml file:

The ;rev=2 is not appended to the API name:

What I'd expect

I tried to append the revision to the API name in variableString, but since it contains a semi-colon, it didn't work.

We need to check if apiRevision is specified and isCurrent is false, if so, append the suffix to the API id. This is the behavior in azure-apim-devops-kit.

See also answer from Microsoft support:

Specifying a yaml file as the value for openApiSpec results in the following exception:

System.AggregateException: One or more errors occurred. (Unexpected character encountered while parsing value: o. Path '', line 0, position 0.) ---> Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: o. Path '', line 0, position 0.
   at Newtonsoft.Json.JsonTextReader.ParseValue()
   at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Common.OpenAPISpecReader.GetOpenApiVersion() in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Core\Infrastructure\OpenAPISpecReader.cs:line 20
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Common.OpenAPISpecReader.GetOpenApiFormat() in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Core\Infrastructure\OpenAPISpecReader.cs:line 43
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.ApiTemplateCreator.CreateApiTemplateResourceAsync(ApiDeploymentDefinition api, Boolean isInitial) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\ApimEntities\Api\APITemplateCreator.cs:line 118
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.ApiTemplateCreator.CreateApiTemplateAsync(ApiDeploymentDefinition api, Boolean isInitial) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\ApimEntities\Api\APITemplateCreator.cs:line 53
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.ApiTemplateCreator.CreateApiTemplatesAsync(ApiDeploymentDefinition api) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\ApimEntities\Api\APITemplateCreator.cs:line 41

It appears that the GetOpenApiVersion method, which only supports json, is getting called even for yaml files.

Shouldn't the variables defined in apis be parameters in the deploy template and their values defined in the parameters file, the way apimServiceName is parameterized? It seems the api variable values are hard-coded into the deploy template.

Unable to create operation policies successfully. Consider the following:

Template

apis:
    ....
    policy: .\api-policy.xml
    operations:
      operation1:
        policy: .\operation1-policy.xml

Expected Result
Policy from operation1-policy.xml is used when creating the arm for the operation1 policy

Actual Result
The policy from api-policy.xml is used to create the operation1 policy.

Currently if a link is provided for the openApiSpec, the contents are downloaded so that we can workaround the issue of swagger title overwriting the API display name, which is a bug on Azure's side (see same issue on CLI). However, an easier workaround is to set up a to update the display name from a child template. See below:

First import swagger as usual:

Secondly, update display name. Here a nested template is used because ARM templates won't let you have Microsoft.ApiManagement/service/apis declared twice in the same deployment. Notice this second deployment does not include the swagger.

This also avoids ARM template character limit issues that I've encountered if my swagger file is too large.

How do you extract a running environment using this tool?

Template:

subscriptions:
  - name: MySubscription
    displayName: "Subscription for My Api"
    scope: /apis/my-api

Expected Result:
Creates arm

Actual Result
SubscriptionDeploymentDefinition.DependentUser() throws ArgumentNullException

dotnet-apim refers to Microsoft.AspNetCore.All which is EOL as part of 2.2 runtime.

According to the doc, NamedValue added in version 2019-12-01 of API.

Currently, by default the tool adds only one parameter "ApimServiceName". I need to add more parameters so I can use release pipeline to override the parameters for each environment. I am adding AppInsight logging, but I have multiple environments (dev01, dev02, qa01, qa02 etc). And I do not want to use variables options, because I do not want to store the InstrumentationKey within the project as a text value. If it is in release pipeline, I can add the key for each scope (environment) and secure it. Is there a way I can add custom parameters when ArmTemplates are generated?

Please add support to use dotnet 3.1 and above when using dotnet-apim.

Can't install in runtime 2.1 and 2.2 is not supported anymore
https://docs.microsoft.com/en-us/dotnet/core/install/linux

error NU1202: Package Apim.DevOps.Toolkit 1.1.5 is not compatible with netcoreapp2.1 (.NETCoreApp,Version=v2.1) / any. Package Apim.DevOps.Toolkit 1.1.5 supports: netcoreapp2.2 (.NETCoreApp,Version=v2.2) / any
The tool package could not be restored.
Tool 'apim.devops.toolkit' failed to install. This failure may have been caused by:

* You are attempting to install a preview release and did not use the --version option to specify the version.
* A package by this name was found, but it was not a .NET Core tool.
* The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
* You mistyped the name of the tool.

Template YAML File

version: 0.0.1
apimServiceName: apim-nm-playground

backends:
  - title: "fa-playground"
    description: "Azure Function App for Testing"
    url: $(functionUrl)
    resourceId: $(functionResourceId)
    protocol: http
    credentials:
      header:
        x-functions-key: 
          - "{{fa-function-app-key}}"

linked: false
outputLocation: C:\Playground\ApimTest
linkedTemplatesBaseUrl : https://foo.bar

Error

System.AggregateException: One or more errors occurred. (API configuration is required) ---> System.ArgumentException: API configuration is required
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.ConfigurationValidator.ValidateAPIs(DeploymentDefinition creatorConfig) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Core\Configuration\ConfigurationValidator.cs:line 135
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.ConfigurationValidator.Validate(DeploymentDefinition creatorConfig) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Core\Configuration\ConfigurationValidator.cs:line 16
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Common.FileReader.GetCreatorConfig(String yamlContent) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Core\Infrastructure\FileReader.cs:line 159
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Common.FileReader.GetCreatorConfigFromYaml(String configFilePath) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Core\Infrastructure\FileReader.cs:line 39
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.CreateCommand.GetCreatorConfig(CommandLineOption option) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\CommandLine\Commands\CreateCommand.cs:line 31
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.CreateCommand.Process(CommandLineOption option) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\CommandLine\Commands\CreateCommand.cs:line 16
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Program.ProcessCommand(CommandLineOption option) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Program.cs:line 39
---> (Inner Exception #0) System.ArgumentException: API configuration is required
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.ConfigurationValidator.ValidateAPIs(DeploymentDefinition creatorConfig) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Core\Configuration\ConfigurationValidator.cs:line 135
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.ConfigurationValidator.Validate(DeploymentDefinition creatorConfig) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Core\Configuration\ConfigurationValidator.cs:line 16
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Common.FileReader.GetCreatorConfig(String yamlContent) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Core\Infrastructure\FileReader.cs:line 159
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Common.FileReader.GetCreatorConfigFromYaml(String configFilePath) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\Core\Infrastructure\FileReader.cs:line 39
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.CreateCommand.GetCreatorConfig(CommandLineOption option) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\CommandLine\Commands\CreateCommand.cs:line 31
   at Microsoft.Azure.Management.ApiManagement.ArmTemplates.Create.CreateCommand.Process(CommandLineOption option) in C:\Users\u6104652\source\repos\azure-api-management-devops-resource-kit\src\apimtemplate\CommandLine\Commands\CreateCommand.cs:line 16<---

First of all, I love this tool! It would be nice if we didn't have to specify the products array in the yml file for the product-api assignments to be added to the output ARM template. This is useful because we manage products as separate arm templates from the APIs. This is the behavior in the apim-azure-devops-toolkit, as in this section of code, it only adds dependsOn on products/apis resource and not products. Besides, we're already adding dependsOn product in the API resource, i.e. products is added as a dependency in ApiDeploymentDefinition. Therefore, if one decides to add the products array in their yml, it should still work.

Currently, if one doesn't include products in the yml, deployment fails with this error: "The resource 'Microsoft.ApiManagement/service/products/***' is not defined in the template"

@mirsaeedi I'd be happy to submit a PR for this.

Thanks.

Hello @mirsaeedi. No rush on this one:
APIM provides self-hosted gateway, and associating APIs with particular gateway(s). We are interested in using this feature in the near future, I am looking into it and can submit a PR when I have one ready for one of the upcoming releases.

Here's the ARM schema definition for gateway APIs

From Azure Portal:

Similar to API or Product level subscriptions, would it be possible to restrict an endpoint by subscription? I know we can do it by using policies and header.

Not sure if there is a way to do this now but it would be great to be able to define named values as secrets that require their values to be passed in at deployment time via an arm securestring parameter. This would allow the value to be passed in securely and not visible anywhere.

Making the discovery automatic would would eliminate some of the manual work that is required to be done when adding support for new entities.