microsoft / application-insights-istio-adapter Goto Github PK

Application Insights adapter for Istio's Mixer which enables Application Insights telemetry collection for applications within Istio-enabled Kubernetes clusters.

License: MIT License

C# 99.13% Dockerfile 0.06% Batchfile 0.81%

application-insights-istio-adapter's Introduction

This solution is no longer under development. If your workload is running in Azure - check out what Azure AKS team has to offer in terms of monitoring applications natively (with no reliance on any service mesh)

Application Insights adapter for Istio Mixer

Application Insights adapter for Istio Mixer is an adapter designed to collect Application Insights telemetry in Istio-enabled Kubernetes clusters, including AKS clusters.

Please see SETUP.md for details.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Data/Telemetry

This project collects usage data and sends it to Microsoft to help improve our products and services. Read Microsoft's privacy statement to learn more. To turn the collection off, please see SETUP.md for instructions to uninstall the product.

application-insights-istio-adapter's People

Contributors

Stargazers

Watchers

Forkers

bnookala matthewtaylor1995 bhaskers-blu-org2 taffywrinkle claudiusgonzo musicman21 pbeohar test-mass-forker-org-1 jrbancel

application-insights-istio-adapter's Issues

Azure application insights for AKS not working

Followed Microsoft link to install Istio on AKS cluster
https://docs.microsoft.com/en-us/azure/aks/servicemesh-istio-install?pivots=client-operating-system-macos
Injected Istio envoy proxies to my application pods using istio-injection=enabled label to my namespaces
https://docs.microsoft.com/en-us/azure/aks/servicemesh-istio-scenario-routing?pivots=client-operating-system-macos
Installed and configured Azure Application Insights adapter with my application insights instrumentation key
https://docs.microsoft.com/en-us/azure/azure-monitor/app/kubernetes

But I am unable to see data in the application insights. I see below from application insights adapter.

2019/12/30 10:46:36.446|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2019/12/30 10:47:36.445|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2019/12/30 10:48:36.447|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2019/12/30 10:49:36.448|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2019/12/30 10:50:36.448|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2019/12/30 10:51:36.445|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2019/12/30 10:52:36.449|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2019/12/30 10:53:36.450|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]

Is it possible to view the body of a request ?

Thank you for this excellent tool, it works amazing.
However, it is logging everything but the body of a request.
What should i do in order to view it?

Not able to run multiple instance of Application-Insights-Istio-Adapter in kubernetes.

Hello Guys,
I am trying to run multiple instances of Application-Insights-Istio-Adapter in the same cluster and different namespaces. For the first instance of Application-Insights-Istio-Adapter, It's working fine and sending logs to Application Insight-1. But in the second instance, it's not sending the data to Application Insight-2. There is no error in the mixer and app insight adapter. But seen some info:-

2019-11-26T06:00:07.952673Z info scheme "" not registered, fallback to default scheme
2019-11-26T06:00:07.952702Z info ccResolverWrapper: sending update to cc: {[{application-insights-istio-mixer-adapter-service-test.test-telemetary.svc.cluster.local:6789 0 }] }
2019-11-26T06:00:07.952735Z info grpcAdapter Connected to: application-insights-istio-mixer-adapter-service-test.test-telemetary.svc.cluster.local:6789
2019-11-26T06:00:27.953469Z info grpc: addrConn.createTransport failed to connect to {application-insights-istio-mixer-adapter-service-test.test-telemetary.svc.cluster.local:6789 0 }. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.144.177:6789: i/o timeout". Reconnecting...
2019-11-26T06:00:07.953187Z info base.baseBalancer: got new ClientConn state: {{[{application-insights-istio-mixer-adapter-service-test.test-telemetary.svc.cluster.local:6789 0 }] } }

Azure Application Insights for AKS not working

Does this Setup allow the AKS Pod request logs to appear in the application insights ?
My use case is to get the pods request logs of a third party service in the application insights 'request' table.

I enabled istio-injection for my application pods using istio-injection=enabled label to my namespaces
kubectl label namespace <my-app-namespace> istio-injection=enabled

I have followed every step mentioned in the SETUP.md file but it doesn't seem to work.

I am unable to see data in the application insights. I see below from application insights adapter.

2022/04/14 16:17:06.656|INFO|We will listen for Istio's Mixer data on 0.0.0.0:6789  
2022/04/14 16:17:06.836|INFO|The library is running  
2022/04/14 16:17:06.846|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2022/04/14 16:18:06.850|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2022/04/14 16:19:06.853|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2022/04/14 16:20:06.856|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]  
2022/04/14 16:21:06.856|INFO|Istio's Mixer input: [ConnectionCount: 0, RequestsReceived: 0, InstancesSucceeded: 0, InstancesFailed: 0]

When I try troubleshooting the steps mentioned, In Step 7, I get the following error -

$ kubectl get pods -n istio-system -l "istio=mixer,app=telemetry"
No resources found in istio-system namespace.

Do we need to have istio-mixer enabled/ deployed externally ? Am I missing any configuration ?

Traces incoming to ingressgateway are not present in Azure App Insights

I have a working configuration of insights adapter. In the application map I see the istio-ingressgateway only as a source of requests to microservices.
I cannot see the traces of requests from Internet to istio-ingressgateway.

I made sure that appinsights/monitoring.enabled=true is set for the pod and it is. Troubleshooting page says that I should make sure that the istio-proxy sidecar is there.
Of course the istio-ingressgateway does not have the sidecar as it is the envoy proxy itself for the Internet.

Here is part of the pod description:

kubectl describe pod -n istio-system istio-ingressgateway-f5c4b6c68-4mt72
Name:           istio-ingressgateway-f5c4b6c68-4mt72
Namespace:      istio-system
Priority:       0
Node:           aks-agentpool-77752666-vmss000000/10.210.64.4
Start Time:     Wed, 29 Jan 2020 12:37:09 +0100
Labels:         app=istio-ingressgateway
                appinsights/monitoring.enabled=true
                chart=gateways
                heritage=Tiller
                istio=ingressgateway
                pod-template-hash=f5c4b6c68
                release=istio
Annotations:    sidecar.istio.io/inject: false
Status:         Running
IP:             10.210.64.11
IPs:            <none>
Controlled By:  ReplicaSet/istio-ingressgateway-f5c4b6c68

How to make sure that the traces from istio-ingressgatway are routed to Azure App Insights?

Ability to support istio beyond mixer ( > istio 1.7)

https://istio.io/docs/tasks/observability/mixer/metrics/collecting-metrics/

"Mixer is deprecated. The functionality provided by Mixer is being moved into the Envoy proxies. Use of Mixer with Istio will only be supported through the 1.7 release of Istio."

New default installations of istio with i.e. 1.5.1 do not work because mixer is not enabled anymore. Adapter should reflect the new architecture.

unstable tests

i was running the unit tests in the project and the following seem unstable , sometimes they pass sometimes they don't

public async Task HostTests_RunsLibrary()

Test Name:	HostTests_RunsLibrary
Test FullName:	Microsoft.IstioMixerPlugin.LibraryTest.Library.HostTests.HostTests_RunsLibrary
Test Source:	D:\src\Application-Insights-Istio-Adapter\src\LibraryTest\Library\HostTests.cs : line 18
Test Outcome:	Failed
Test Duration:	0:00:02.3174692

Result StackTrace:	
at Microsoft.IstioMixerPlugin.LibraryTest.GrpcWriter.Write(HandleTraceSpanRequest request) in D:\src\Application-Insights-Istio-Adapter\src\LibraryTest\Library\GrpcWriter.cs:line 39
   at Microsoft.IstioMixerPlugin.LibraryTest.Library.HostTests.HostTests_RunsLibrary() in D:\src\Application-Insights-Istio-Adapter\src\LibraryTest\Library\HostTests.cs:line 52
Result Message:	
Test method Microsoft.IstioMixerPlugin.LibraryTest.Library.HostTests.HostTests_RunsLibrary threw exception: 
System.InvalidOperationException: Error sending a message via gRpc. Grpc.Core.RpcException: Status(StatusCode=Unavailable, Detail="Connect Failed")
   at Microsoft.IstioMixerPlugin.LibraryTest.GrpcWriter.Write(HandleTraceSpanRequest request) in D:\src\Application-Insights-Istio-Adapter\src\LibraryTest\Library\GrpcWriter.cs:line 35
Result StandardOutput:	
2019/05/09 17:22:37.076|INFO|Starting the library...  
2019/05/09 17:22:37.079|INFO|Loaded configuration. 
<?xml version="1.0" encoding="utf-8" ?>
<Configuration>
    <Host>0.0.0.0</Host>
    <Port>49526</Port>
    <InstrumentationKey>ikey1</InstrumentationKey>
    <LiveMetricsStreamAuthenticationApiKey></LiveMetricsStreamAuthenticationApiKey>
    <Watchlist>
        <Namespaces>default</Namespaces>
    </Watchlist>
    <AdaptiveSampling Enabled="true">
      <MaxEventsPerSecond>%ISTIO_MIXER_PLUGIN_AI_ADAPTIVE_SAMPLING_EVENTS_LIMIT%</MaxEventsPerSecond>
      <!--Telemetry items other than events are counted together-->
      <MaxOtherItemsPerSecond>%ISTIO_MIXER_PLUGIN_AI_ADAPTIVE_SAMPLING_LIMIT%</MaxOtherItemsPerSecond>
    </AdaptiveSampling>
</Configuration>
  
2019/05/09 17:22:37.241|INFO|We will listen for Istio's Mixer data on 0.0.0.0:49526  
2019/05/09 17:22:37.330|ERROR|Could not start the gRPC Istio's Mixer channel. System.InvalidOperationException: Could not initialize the gRPC server. System.IO.IOException: Failed to bind port "0.0.0.0:49526"
   at Grpc.Core.Server.CheckPortsBoundSuccessfully() in T:\src\github\grpc\src\csharp\Grpc.Core\Server.cs:line 313
   at Grpc.Core.Server.Start() in T:\src\github\grpc\src\csharp\Grpc.Core\Server.cs:line 155
   at Microsoft.IstioMixerPlugin.Library.Inputs.GrpcInput.GrpcInput.Start() in D:\src\Application-Insights-Istio-Adapter\src\Library\Inputs\GrpcInput\GrpcInput.cs:line 59
   at Microsoft.IstioMixerPlugin.Library.Inputs.GrpcInput.GrpcInput.Start() in D:\src\Application-Insights-Istio-Adapter\src\Library\Inputs\GrpcInput\GrpcInput.cs:line 65
   at Microsoft.IstioMixerPlugin.Library.Library.Run() in D:\src\Application-Insights-Istio-Adapter\src\Library\Library.cs:line 132  
2019/05/09 17:22:37.330|ERROR|Unexpected error at start-up of the library. System.InvalidOperationException: Could not start the gRPC Istio's Mixer channel. System.InvalidOperationException: Could not initialize the gRPC server. System.IO.IOException: Failed to bind port "0.0.0.0:49526"
   at Grpc.Core.Server.CheckPortsBoundSuccessfully() in T:\src\github\grpc\src\csharp\Grpc.Core\Server.cs:line 313
   at Grpc.Core.Server.Start() in T:\src\github\grpc\src\csharp\Grpc.Core\Server.cs:line 155
   at Microsoft.IstioMixerPlugin.Library.Inputs.GrpcInput.GrpcInput.Start() in D:\src\Application-Insights-Istio-Adapter\src\Library\Inputs\GrpcInput\GrpcInput.cs:line 59
   at Microsoft.IstioMixerPlugin.Library.Inputs.GrpcInput.GrpcInput.Start() in D:\src\Application-Insights-Istio-Adapter\src\Library\Inputs\GrpcInput\GrpcInput.cs:line 65
   at Microsoft.IstioMixerPlugin.Library.Library.Run() in D:\src\Application-Insights-Istio-Adapter\src\Library\Library.cs:line 132 ---> System.InvalidOperationException: Could not initialize the gRPC server. System.IO.IOException: Failed to bind port "0.0.0.0:49526"
   at Grpc.Core.Server.CheckPortsBoundSuccessfully() in T:\src\github\grpc\src\csharp\Grpc.Core\Server.cs:line 313
   at Grpc.Core.Server.Start() in T:\src\github\grpc\src\csharp\Grpc.Core\Server.cs:line 155
   at Microsoft.IstioMixerPlugin.Library.Inputs.GrpcInput.GrpcInput.Start() in D:\src\Application-Insights-Istio-Adapter\src\Library\Inputs\GrpcInput\GrpcInput.cs:line 59
   at Microsoft.IstioMixerPlugin.Library.Inputs.GrpcInput.GrpcInput.Start() in D:\src\Application-Insights-Istio-Adapter\src\Library\Inputs\GrpcInput\GrpcInput.cs:line 65
   at Microsoft.IstioMixerPlugin.Library.Library.Run() in D:\src\Application-Insights-Istio-Adapter\src\Library\Library.cs:line 132
   --- End of inner exception stack trace ---
   at Microsoft.IstioMixerPlugin.Library.Library.Run() in D:\src\Application-Insights-Istio-Adapter\src\Library\Library.cs:line 139
   at Microsoft.IstioMixerPlugin.Library.Host.StartNewLibrary(String configuration, TelemetryClient telemetryClient) in D:\src\Application-Insights-Istio-Adapter\src\Library\Host.cs:line 95  
2019/05/09 17:22:37.330|ERROR|The library has failed to start. We'll restart it in a while. System.InvalidOperationException: Could not start the gRPC Istio's Mixer channel. System.InvalidOperationException: Could not initialize the gRPC server. System.IO.IOException: Failed to bind port "0.0.0.0:49526"
   at Grpc.Core.Server.CheckPortsBoundSuccessfully() in T:\src\github\grpc\src\csharp\Grpc.Core\Server.cs:line 313
   at Grpc.Core.Server.Start() in T:\src\github\grpc\src\csharp\Grpc.Core\Server.cs:line 155
   at Microsoft.IstioMixerPlugin.Library.Inputs.GrpcInput.GrpcInput.Start() in D:\src\Application-Insights-Istio-Adapter\src\Library\Inputs\GrpcInput\GrpcInput.cs:line 59
   at Microsoft.IstioMixerPlugin.Library.Inputs.GrpcInput.GrpcInput.Start() in D:\src\Application-Insights-Istio-Adapter\src\Library\Inputs\GrpcInput\GrpcInput.cs:line 65
   at Microsoft.IstioMixerPlugin.Library.Library.Run() in D:\src\Application-Insights-Istio-Adapter\src\Library\Library.cs:line 132 ---> System.InvalidOperationException: Could not initialize the gRPC server. System.IO.IOException: Failed to bind port "0.0.0.0:49526"
   at Grpc.Core.Server.CheckPortsBoundSuccessfully() in T:\src\github\grpc\src\csharp\Grpc.Core\Server.cs:line 313
   at Grpc.Core.Server.Start() in T:\src\github\grpc\src\csharp\Grpc.Core\Server.cs:line 155
   at Microsoft.IstioMixerPlugin.Library.Inputs.GrpcInput.GrpcInput.Start() in D:\src\Application-Insights-Istio-Adapter\src\Library\Inputs\GrpcInput\GrpcInput.cs:line 59
   at Microsoft.IstioMixerPlugin.Library.Inputs.GrpcInput.GrpcInput.Start() in D:\src\Application-Insights-Istio-Adapter\src\Library\Inputs\GrpcInput\GrpcInput.cs:line 65
   at Microsoft.IstioMixerPlugin.Library.Library.Run() in D:\src\Application-Insights-Istio-Adapter\src\Library\Library.cs:line 132
   --- End of inner exception stack trace ---
   at Microsoft.IstioMixerPlugin.Library.Library.Run() in D:\src\Application-Insights-Istio-Adapter\src\Library\Library.cs:line 139
   at Microsoft.IstioMixerPlugin.Library.Host.StartNewLibrary(String configuration, TelemetryClient telemetryClient) in D:\src\Application-Insights-Istio-Adapter\src\Library\Host.cs:line 95
   at Microsoft.IstioMixerPlugin.Library.Host.<>c__DisplayClass6_0.<Run>b__0() in D:\src\Application-Insights-Istio-Adapter\src\Library\Host.cs:line 57

public void LibraryTests_LibraryCleansUpInputWhileStopping()

public void DiagnosticsTests_LogsMessageToFile()


Test Name:	DiagnosticsTests_LogsMessageToFile
Test FullName:	Microsoft.IstioMixerPlugin.LibraryTest.DiagnosticsTests.DiagnosticsTests_LogsMessageToFile
Test Source:	D:\src\Application-Insights-Istio-Adapter\src\LibraryTest\Common\DiagnosticsTest.cs : line 15
Test Outcome:	Failed
Test Duration:	0:00:01.0059612

Result StackTrace:	at Microsoft.IstioMixerPlugin.LibraryTest.DiagnosticsTests.DiagnosticsTests_LogsMessageToFile() in D:\src\Application-Insights-Istio-Adapter\src\LibraryTest\Common\DiagnosticsTest.cs:line 34
Result Message:	Assert.IsTrue failed.
Result StandardOutput:	2019/05/09 17:27:22.392|INFO|Info message here 92c3a770-0b35-4b71-b399-0079014ad57e

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request