can you post the debug output for this please (include -d yes)

from py-az2tf.

py-az2tf aa681787$ ./az2tf.sh -s x -p yes -d yes

Subscription ID = x
Azure Resource Group Filter =
Terraform Resource Type Filter =
Get Subscription Policies & RBAC = yes
Extract Key Vault Secrets to .tf files (insecure) = no
Fast Forward = no
Validate Only = no
Debug = no

python2.7 ../../scripts/az2tf.py -s x -p yes -d yes 2>&1 | tee -a import.log
Policies & RBAC=yes
Debug=True
Get Access Token from CLI
Subscription: x
Found subscription x proceeding ...

azurerm_policy_definition 413

{
"type": "Microsoft.Authorization/policyDefinitions",
"properties": {
"displayName": "Enforce audit of MarketPlace license agreement actions",
"description": "Testing for MarketPlace Policy, this version will only audit when a hit is made",
"parameters": {
"listOfResourceTypesAllowed": {
"type": "Array",
"metadata": {
"displayName": "Allowed resource types",
"description": "The list of resource types that can be deployed.",
"strongType": "resourceTypes"
}
}
},
"policyRule": {
"then": {
"effect": "audit"
},
"if": {
"not": {
"field": "type",
"in": "[parameters('listOfResourceTypesAllowed')]"
}
}
},
"policyType": "Custom",
"mode": "All",
"metadata": {
"category": "Our Standards",
"createdOn": "2019-08-06T21:15:32.5017898Z",
"updatedOn": "2019-08-06T21:16:08.8523877Z",
"createdBy": "x",
"updatedBy": "x"
}
},
"name": "0x",
"id": "/providers/Microsoft.Management/managementGroups/Our-ETS/providers/Microsoft.Authorization/policyDefinitions/01acc02a-5d7d-478b-8f6e-f6e0a4207695"
}

File generated by py-az2tf see: https://github.com/andyt530/py-az2tf

resource azurerm_policy_definition policydefinitions__01acc02a-5d7d-478b-8f6e-f6e0a4207695 {
name = "01acc02a-5d7d-478b-8f6e-f6e0a4207695"
display_name = "Enforce audit of MarketPlace license agreement actions"
policy_type = "Custom"
mode = "All"
description = "Testing for MarketPlace Policy, this version will only audit when a hit is made"
metadata = jsonencode(
{"category": "Our Standards", "createdOn": "2019-08-06T21:15:32.5017898Z", "updatedOn": "2019-08-06T21:16:08.8523877Z", "createdBy": "x", "updatedBy": "x"})
policy_rule = jsonencode(
{"then": {"effect": "audit"}, "if": {"not": {"field": "type", "in": "[parameters('listOfResourceTypesAllowed')]"}}})
parameters = jsonencode(
{"listOfResourceTypesAllowed": {"type": "Array", "metadata": {"displayName": "Allowed resource types", "description": "The list of resource types that can be deployed.", "strongType": "resourceTypes"}}})
}

{
"type": "Microsoft.Authorization/policyDefinitions",
"properties": {
"displayName": "Enforce Resource Group default Tag value",
"description": "Apply tag and its default value to Resource Group. Applies a required tag and its assigned default value to a Resource Group if it is not specified by the user during creation time. Useful for when third party automation creates Resource Groups without the standard Our Tags, and our existing Policy that requires Tags and values denies that automation. This Policy will replace that version.",
"parameters": {
"tagValue": {
"type": "String",
"metadata": {
"displayName": "tagValue",
"description": "Default value of the tag when not explicitly set, such as 0900/1681"
}
},
"tagName": {
"type": "String",
"metadata": {
"displayName": "tagName",
"description": "Name of the tag, such as our-costcenter"
}
}
},
"policyRule": {
"then": {
"effect": "append",
"details": [
{
"field": "[concat('tags[', parameters('tagName'), ']')]",
"value": "[parameters('tagValue')]"
}
]
},
"if": {
"allOf": [
{
"field": "type",
"equals": "Microsoft.Resources/subscriptions/resourceGroups"
},
{
"field": "[concat('tags[',parameters('tagName'), ']')]",
"exists": false
}
]
}
},
"policyType": "Custom",
"mode": "All",
"metadata": {
"category": "Our Standards",
"createdOn": "2019-07-31T00:58:54.5837529Z",
"updatedOn": "2019-07-31T00:59:20.4940899Z",
"createdBy": "x",
"updatedBy": "x"
}
},
"name": "17e6a9ef-80b0-41b5-8307-2c56ac91052e",
"id": "/providers/Microsoft.Management/managementGroups/Our-ETS/providers/Microsoft.Authorization/policyDefinitions/17e6a9ef-80b0-41b5-8307-2c56ac91052e"
}
Traceback (most recent call last):
File "../../scripts/az2tf.py", line 272, in
azurerm_policy_definition.azurerm_policy_definition(crf,cde,crg,headers,requests,sub,json,az2tfmess)
File "/Users/aa681787/apps/azure/terraform/py-az2tf/py-az2tf/scripts/azurerm_policy_definition.py", line 78, in azurerm_policy_definition
prules=str(ast.literal_eval(json.dumps(azr[i]["properties"]["policyRule"])))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 80, in literal_eval
return _convert(node_or_string)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 63, in _convert
in zip(node.keys, node.values))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 62, in
return dict((_convert(k), _convert(v)) for k, v
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 63, in _convert
in zip(node.keys, node.values))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 62, in
return dict((_convert(k), _convert(v)) for k, v
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 60, in _convert
return list(map(_convert, node.elts))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 63, in _convert
in zip(node.keys, node.values))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 62, in
return dict((_convert(k), _convert(v)) for k, v
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 79, in _convert
raise ValueError('malformed string')
ValueError: malformed string
raise ValueError('malformed string')
ValueError: malformed string
Error in az2tf.py

NOTE: Seems to be choking on one of our custom Policy definitions. I appreciate you looking at this!

from py-az2tf.

I was hoping you could shed some light on how I might have someone fix this. I have access to a couple of Python developers if they know where to look. Any input on that?
Regards.

from py-az2tf.

Hi,

So it's all in that one line: prules=str(ast.literal_eval(json.dumps(azr[i]["properties"]["policyRule"])))

azr[i]["properties"]["policyRule"] is going to be the JSON returned from resources.azure.com for this policy rule

The code is attempting to get a matching string definition for what Terraform is expecting to see

In this case something is up - often this can be odd characters (control characters) hiding in the text.

It's worth retrying this with the python3 version now posted as it handles character encoding in a cleaner way.

Failing that I'd start unwinding that line to see if it still looks valid at each step
print(json.dumps(azr[i]["properties"]["policyRule"]) etc.

Also perhaps - cerate a stub .tf file for the policy and do a
terraform import azurerm_policy_definition policydefinitions.01acc02a-5d7d-478b-8f6e-f6e0a4207695
terraform state show azurerm_policy_definition policydefinitions.01acc02a-5d7d-478b-8f6e-f6e0a4207695

to help us see what terraform wants exactly.

from py-az2tf.

Just managed to reproduce your case here - and posted fixes

Let me know if all your issues are resolved.

from py-az2tf.