samuelfisher / terraformplugindotnet Goto Github PK

View Code? Open in Web Editor NEW

44.0 44.0 14.0 115 KB

Write Terraform providers in C#.

License: MIT License

C# 99.68% HCL 0.32%

csharp terraform terraform-provider

terraformplugindotnet's People

Contributors

Stargazers

Watchers

Forkers

ayende infbase mallpress edwinf bridgecrew-perf4 r-bennett alanblythe oskarm93 thomasbleijendaal hartl3y94 fjsnogueira arsanov mvh-rxt scionofdesign

terraformplugindotnet's Issues

Error when trying to create a Type of Object or map or list in Terraform

Hello,

I want to create a terraform resource that have an attribute as object type.
In C#, I create an IDictionary<string, object> but when execute the Terraform plan I got this error below:
System.NotSupportedException: Unable to convert System.Collections.Generic.KeyValuePair`2[[System.String, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Object, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]] to Terraform type.

Any idea how to create an object in terraform using the C# plugin ?
ex.:
resource "demo_player" "player" {
name = ""
player = {
name = ""
streams = ""
}
}

THank you

KeyAttribute has to match property name

First of all, thank you for creating this project. I can finally write terraform providers without learning cumbersome Go!

Found a head-scratcher while creating a new provider. Hoped this can be clarified. I thought KeyAttribute would allow you to give different names to your properties compared to what C# property is called, e.g. key attribute = role_id, but property name is RoleId.
Turns out if KeyAttribute does not match property name (case insensitive) then ReadAsync will not have that property populated. Is this an issue with serialization or expected?

Repro steps:

Take sample provider in debug mode.
Change KeyAttribute from "path" to "full_path".

    [Key("full_path")]
    [Description("Path to the file.")]
    [Required]
    public string Path { get; set; }

Create a sample configuration

terraform {
  required_providers {
    sampleprovider = {
      source  = "example.com/example/sampleprovider"
      version = "1.0.0"
    }
  }
}

provider "sampleprovider" {}

resource "sampleprovider_file" "test" {
  full_path = "c:\\temp\\test_file123.txt"
  content = "abc"
}

Plan and Apply a new resource.

PS> $env:TF_REATTACH_PROVIDERS='{"example.com/example/sampleprovider":{"Protocol":"grpc","Pid":27700,"Test":true,"Addr":{"Network":"tcp","String":"127.0.0.1:5344"}}}'
PS> terraform plan -out tfplan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # sampleprovider_file.test will be created
  + resource "sampleprovider_file" "test" {
      + content   = "abc"
      + full_path = "c:\\temp\\test_file123.txt"
      + id        = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "tfplan"

PS> terraform apply tfplan
sampleprovider_file.test: Creating...
sampleprovider_file.test: Creation complete after 0s [id=6b52fd3f-fc83-4085-868e-729011fb6271]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Plan again, so ReadAsync is called.

PS> terraform plan -out tfplan
sampleprovider_file.test: Refreshing state... [id=6b52fd3f-fc83-4085-868e-729011fb6271]
╷
│ Error: Plugin error
│
│   with sampleprovider_file.test,
│   on main.tf line 12, in resource "sampleprovider_file" "test":
│   12: resource "sampleprovider_file" "test" {
│
│ The plugin returned an unexpected error from plugin.(*GRPCProvider).ReadResource: rpc error: code = Unknown desc = Exception was thrown by handler.
╵

Property "full_path" will not be populated.

Getting this to work in Windows

Hey Samuel, thought I'd give you some feedback on getting the provider to work in 'production' mode on Windows when Terraform calls it directly. It would be nice if you could add this as Windows notes to the docs. Here's the things I needed to do:

Initially, I wasn't getting logs or anything, so I added this to Main(). When Terraform launches the .exe I get an 'attach debugger' prompt and I can just run it through Visual Studio

    Debugger.Launch();   // Comment this in to be able to debug the provider when Terraform calls it

When debugging the app startup, you have no access to the output, so it's best to use a file based logger as you do, but assert control over where the output is

      var exeDirectory = Path.GetDirectoryName(Process.GetCurrentProcess().MainModule.FileName);        
      string logFile = appSettings.GetValue<string>("Serilog:WriteTo:0:Args:path");
      Log.Logger = new LoggerConfiguration()
                .ReadFrom.Configuration(appSettings)
                .WriteTo.File(Path.Combine(exeDirectory, logFile))
                .CreateLogger();

When using appsettings.json make sure you get this from the apps folder, as you have no idea what the current folder might be:

            var exeDirectory = Path.GetDirectoryName(Process.GetCurrentProcess().MainModule.FileName);
            var appSettings = new ConfigurationBuilder()
                .SetBasePath(exeDirectory)
                .AddJsonFile("appsettings.json", optional: false)
                .Build();

In-memory self-signed certificates don't work on Windows, so I've expanded on the certificate helper below. See this discussion: dotnet/runtime#23749

The startup looks like:

    var exeDirectory = Path.GetDirectoryName(Process.GetCurrentProcess().MainModule.FileName);
    string certFileStem = Path.Combine(exeDirectory, "tfcert");
    Cert = CertificateGenerator.GenerateSelfSignedCertificate(certFileStem, "CN=127.0.0.1", "CN=root ca", 
    CertificateGenerator.GeneratePrivateKey());

And the helper class:


    public static class CertificateGenerator
    {
        private const int KeyStrength = 2048;

        public static X509Certificate2 GenerateSelfSignedCertificate(string fileStem, string subjectName, string issuerName, AsymmetricKeyParameter issuerPrivKey)
        {
            // This convoluted process of creating a certificate, saving it and reimporting it is required on
            // Windows because of a bug in Windows SSL handling, where in-memory certificates are not handled
            // correctly: https://github.com/dotnet/runtime/issues/23749
            // The presenting error is: "No credentials are available in the security package"
            //
            // "After discussions with the SCHANNEL team, it has been confirmed that this won't work in the
            // current versions of Windows due to SCHANNEL's cross-process architecture with LSASS.EXE.
            // The in-memory TLS client certificate private key is not marshaled between SCHANNEL and LSASS.
            // That is why SEC_E_NO_CREDENTIALS is returned from SCHANNEL AcquireCredentialHandle() call."

            var inMemorycert = CreateSelfSignedCertificate(subjectName, issuerName, issuerPrivKey);
            SaveCertificateToPemAndKeyFiles(inMemorycert, fileStem);
            var importedCert = CreateFromPublicPrivateKey($"{fileStem}.pem", $"{fileStem}.key");
            File.Delete($"{fileStem}.pem");
            File.Delete($"{fileStem}.key");

            return new X509Certificate2(importedCert.Export(X509ContentType.Pkcs12));
        }

        public static AsymmetricKeyParameter GeneratePrivateKey()
        {
            var randomGenerator = new CryptoApiRandomGenerator();
            var random = new SecureRandom(randomGenerator);

            // Generate Key
            var keyGenerationParameters = new KeyGenerationParameters(random, KeyStrength);
            var keyPairGenerator = new RsaKeyPairGenerator();
            keyPairGenerator.Init(keyGenerationParameters);
            return keyPairGenerator.GenerateKeyPair().Private;
        }

        public static X509Certificate2 CreateSelfSignedCertificate(string subjectName, string issuerName, AsymmetricKeyParameter issuerPrivKey)
        {
            var randomGenerator = new CryptoApiRandomGenerator();
            var random = new SecureRandom(randomGenerator);
            var certificateGenerator = new X509V3CertificateGenerator();

            // Serial Number
            var serialNumber = BigIntegers.CreateRandomInRange(BigInteger.One, BigInteger.ValueOf(long.MaxValue), random);
            certificateGenerator.SetSerialNumber(serialNumber);

            // Issuer and SN
            var subjectDN = new X509Name(subjectName);
            var issuerDN = new X509Name(issuerName);
            certificateGenerator.SetIssuerDN(issuerDN);
            certificateGenerator.SetSubjectDN(subjectDN);

            // SAN
            var subjectAltName = new GeneralNames(new GeneralName(GeneralName.DnsName, "localhost"));
            certificateGenerator.AddExtension(X509Extensions.SubjectAlternativeName, false, subjectAltName);

            // Validity
            var notBefore = DateTime.UtcNow.Date;
            var notAfter = notBefore.AddYears(2);
            certificateGenerator.SetNotBefore(notBefore);
            certificateGenerator.SetNotAfter(notAfter);

            // Public Key
            var keyGenerationParameters = new KeyGenerationParameters(random, KeyStrength);
            var keyPairGenerator = new RsaKeyPairGenerator();
            keyPairGenerator.Init(keyGenerationParameters);
            var subjectKeyPair = keyPairGenerator.GenerateKeyPair();
            certificateGenerator.SetPublicKey(subjectKeyPair.Public);

            // Sign certificate
            var signatureFactory = new Asn1SignatureFactory("SHA256WithRSA", issuerPrivKey, random);
            var certificate = certificateGenerator.Generate(signatureFactory);
            var x509 = new X509Certificate2(certificate.GetEncoded(), (string)null, X509KeyStorageFlags.Exportable);

            // Private key
            var privateKeyInfo = PrivateKeyInfoFactory.CreatePrivateKeyInfo(subjectKeyPair.Private);

            var seq = (Asn1Sequence)Asn1Object.FromByteArray(privateKeyInfo.ParsePrivateKey().GetDerEncoded());
            if (seq.Count != 9)
            {
                throw new PemException("Invalid RSA private key");
            }

            var rsa = RsaPrivateKeyStructure.GetInstance(seq);
            var rsaparams = new RsaPrivateCrtKeyParameters(rsa.Modulus, rsa.PublicExponent, rsa.PrivateExponent, rsa.Prime1, rsa.Prime2, rsa.Exponent1, rsa.Exponent2, rsa.Coefficient);

            var parms = DotNetUtilities.ToRSAParameters(rsaparams);
            var rsa1 = RSA.Create();
            rsa1.ImportParameters(parms);
            return x509.CopyWithPrivateKey(rsa1);
            
        }

        public static void SaveCertificateToPemAndKeyFiles(X509Certificate2 cert, string fileStem)
        {
            byte[] certificateBytes = cert.RawData;
            string certificatePem = CreatePemText("CERTIFICATE", certificateBytes);
            File.WriteAllText($"{fileStem}.pem", certificatePem);

            AsymmetricAlgorithm key = cert.GetRSAPrivateKey();
            //byte[] pubKeyBytes = key.ExportSubjectPublicKeyInfo();
            byte[] privKeyBytes = key.ExportPkcs8PrivateKey();
            //string pubKeyPem = CreatePemText("PUBLIC KEY", pubKeyBytes);
            string privKeyPem = CreatePemText("PRIVATE KEY", privKeyBytes);
            File.WriteAllText($"{fileStem}.key", privKeyPem);
        }

        public static string CreatePemText(string entityType, byte[] bytes)
        {
            StringBuilder builder = new StringBuilder();
            builder.AppendLine($"-----BEGIN {entityType}-----");
            builder.AppendLine(Convert.ToBase64String(bytes, Base64FormattingOptions.InsertLineBreaks));
            builder.AppendLine($"-----END {entityType}-----");

            return builder.ToString();
        }

        public static X509Certificate2 CreateFromPublicPrivateKey(string publicCert = "certs/public.pem", string privateCert = "certs/private.pem")
        {
            byte[] publicPemBytes = File.ReadAllBytes(publicCert);
            using var publicX509 = new X509Certificate2(publicPemBytes);
            var privateKeyText = File.ReadAllText(privateCert);
            var privateKeyBlocks = privateKeyText.Split("-", StringSplitOptions.RemoveEmptyEntries);
            var privateKeyBytes = Convert.FromBase64String(privateKeyBlocks[1]);

            using RSA rsa = RSA.Create();
            if (privateKeyBlocks[0] == "BEGIN PRIVATE KEY")
            {
                rsa.ImportPkcs8PrivateKey(privateKeyBytes, out _);
            }
            else if (privateKeyBlocks[0] == "BEGIN RSA PRIVATE KEY")
            {
                rsa.ImportRSAPrivateKey(privateKeyBytes, out _);
            }
            X509Certificate2 keyPair = publicX509.CopyWithPrivateKey(rsa);
            return keyPair;
        }
    }

PseudoDynamic.Terraform.Toolset

Hey, as the "Discussions"-tab is not yet enabled, I hope it is okay to contact you through an issue. 🙂

I wanted to ask you, whether you are interested in developing a more mature Terraform Plugin SDK. I started here where I have already implemented a Provider Schema API represented by POCO classes and attributes to skip the need to first understand the MessagePack protocol and Plugin protocol in detail before you are able to write schemas. By introducing a separate layer on top of MessagePack it also enables you to provide custom types, for example ITerraformValue, that allows you to get the information whether the int value is unknown or not, similiar to github.com/hashicorp/terraform-plugin-go/types and their "ITerraformValue<>"-representation for every type, for example types.Bool or types.String.

If you are interested you can contact me easily with the email in my GitHub profile. 🙂

GetProviderSchema Error

I'm trying to use this and copied sampleProvider but its giving me error whenever i run the terraform plan/apply please see below:

[21:14:03 INF] Request starting HTTP/2 POST http://unused/tfplugin5.Provider/GetSchema application/grpc -
[21:14:03 INF] Request starting HTTP/2 POST http://unused/plugin.GRPCBroker/StartStream application/grpc -
[21:14:03 INF] Request starting HTTP/2 POST http://unused/plugin.GRPCStdio/StreamStdio application/grpc -
[21:14:03 INF] Executing endpoint 'gRPC - /tfplugin5.Provider/GetSchema'
[21:14:03 INF] Executing endpoint 'gRPC - Unimplemented service'
[21:14:03 INF] Executing endpoint 'gRPC - Unimplemented service'
[21:14:03 INF] Service 'plugin.GRPCStdio' is unimplemented.
[21:14:03 INF] Service 'plugin.GRPCBroker' is unimplemented.
[21:14:03 INF] Executed endpoint 'gRPC - Unimplemented service'
[21:14:03 INF] Executed endpoint 'gRPC - Unimplemented service'
[21:14:04 INF] Request finished HTTP/2 POST http://unused/plugin.GRPCBroker/StartStream application/grpc - - 200 0 application/grpc 59.8361ms
[21:14:04 INF] Request finished HTTP/2 POST http://unused/plugin.GRPCStdio/StreamStdio application/grpc - - 200 0 application/grpc 59.8344ms
[21:14:04 ERR] Error when executing service method 'GetSchema'.
System.NullReferenceException: Object reference not set to an instance of an object.
   at TerraformPluginDotNet.Schemas.SchemaBuilder.BuildSchema(Type type)
   at TerraformPluginDotNet.ResourceProvider.ResourceRegistry..ctor(ISchemaBuilder schemaBulder, IEnumerable`1 registrations)
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Span`1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.GetService(IServiceProvider sp, Type type, Type requiredBy, Boolean isDefaultParameterRequired)
   at lambda_method1(Closure , IServiceProvider , Object[] )
   at Grpc.AspNetCore.Server.Internal.DefaultGrpcServiceActivator`1.Create(IServiceProvider serviceProvider)
   at Grpc.Shared.Server.UnaryServerMethodInvoker`3.Invoke(HttpContext httpContext, ServerCallContext serverCallContext, TRequest request)
--- End of stack trace from previous location ---
   at Grpc.AspNetCore.Server.Internal.CallHandlers.UnaryServerCallHandler`3.HandleCallAsyncCore(HttpContext httpContext, HttpContextServerCallContext serverCallContext)
   at Grpc.AspNetCore.Server.Internal.CallHandlers.ServerCallHandlerBase`3.<HandleCallAsync>g__AwaitHandleCall|8_0(HttpContextServerCallContext serverCallContext, Method`2 method, Task handleCall)
[21:14:04 INF] Executed endpoint 'gRPC - /tfplugin5.Provider/GetSchema'

Create NuGet package

This is a very cool proof-of-concept. Have you considered packaging the re-usable pieces of this up into a NuGet package? (i.e. the TerraformPluginDotNet project piece)

Inconsistencies with computed values

I've added a computed value other than Id to a resource for the first time, and I'm having trouble with it.

When the resource is first created, everything works as expected. But then on a subsequent apply, it tries to re-create the resource.

My ReadAsync and PlanAsync methods are straight passthroughs when I get this behaviour.

If I update ReadAsync to also populate the computed value, then an initial apply followed by a plan works fine (the plan now reports no changes needed). But add in a subsequent second apply, and the apply fails with

When expanding the plan for <resource>.<name> to
include new values learned so far during apply, provider
"<provider>" produced an invalid new value for
.<computed_field>: was known, but now unknown.

This is a bug in the provider, which should be reported in the provider's
own issue tracker.

If I update PlanAsync to also populate the computed values when the Id is non-null, I still get the same error.

Any ideas on how to proceed?

Debugging in Visual Studio

Hey Samuel, this is an awesome head start in creating a provider thanks. I'm also extending it to cater for data sources. It would be useful to debug the behaviour in Visual Studio. Do you have any hints on this? Googling is just bringing up the standard Go based debugging. Thanks!

Can't parse `public_dns` as list<string>

Hi Samuel,

I was wondering if you have any clue how to handle dynamic attributes, and in general all of the non-primitive types.

I have attached three files as shown below.

The main issue is within the GetTerraformType which parse the types.

With that being said I was able to catch list<string> which was inserted by a simple text string without any interpolation whatsoever. For instance in the .tf file machines = ["omer", "omer ,"etc"].

resource "aws_instance" "ubuntu_machine" {
  ami                         = "ami-09e67e426f25ce0d7"
  instance_type               = "t2.micro"
  key_name                    = "raven"
  associate_public_ip_address = true

  vpc_security_group_ids = [
    aws_security_group.ravendb_access.id
  ]
  

  tags = {
    Name = "RavenDB Instance"
  }
}


resource "ravendb_cluster" "my_cluster" {
    depends_on = [aws_instance.ubuntu_machine]
    
    machines = [flatten(aws_instance.ubuntu_machine.public_dns)]
   
}

output "ravendb_cluster_ubuntu_machine_id" {
  value = aws_instance.ubuntu_machine.public_dns
}

namespace SampleProvider
{
    [MessagePackObject]
    public class ResourceRavenCluster
    {
        [Key("id")]
        [Computed]
        [Description("Unique ID for this resource.")]
        [MessagePackFormatter(typeof(ComputedValueFormatter))]
        public string Id { get; set; }
        
        [Key("machines")]
        [Description("Machine Public Ips")]
        public dynamic Machines { get; set; }    
    }

        private static string GetTerraformType(Type t)
        {
            if (t == typeof(string))
            {
                return "\"string\"";
            }
            
            if (t == typeof(int))
            {
                return "\"number\"";
            }
            
            if (t == typeof(List<string>))
            {
                 return "[\"list\",\"string\"]";
            }
            
            if (t == typeof(List<object>))
            {
                return "dynamic";
            }
            if (t == typeof(List<object>))
            {
                return "[\"list\",\"dynamic\"]";
            }

            throw new NotSupportedException();
        }
    }

samuelfisher / terraformplugindotnet Goto Github PK

terraformplugindotnet's People

Contributors

Stargazers

Watchers

Forkers

terraformplugindotnet's Issues

Error when trying to create a Type of Object or map or list in Terraform

KeyAttribute has to match property name

Getting this to work in Windows

PseudoDynamic.Terraform.Toolset

GetProviderSchema Error

Create NuGet package

Inconsistencies with computed values

Debugging in Visual Studio

Can't parse `public_dns` as list<string>

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent