microsoft / applicationinsights-kubernetes Goto Github PK
View Code? Open in Web Editor NEWEnrich the telemetry data for .NET applications running inside containers that are managed by Kubernetes.
License: Other
Enrich the telemetry data for .NET applications running inside containers that are managed by Kubernetes.
License: Other
EnableK8s() method assumes IServiceCollection exists, which is not a good assumption for .NET Core app.
All list entities can be merged using generics.
EntityList : IEnumerable should do.
In the http client, we accepts parameter like GetAsync serialize the object into EntityList, return it as IEnumerable.
Unit tests.
It requires >=2.4.0-beta1, verify if we really need anything as new. Fall back to maybe 2.3.0?
Will be awesome.....
So at the outset, I have only basic understanding of appinsights's data model. However, I'm currently working on porting a relatively mid sized codebase to .net core with the aim to run it on k8s. Also, I've just landed on your repo from google and a quick read through isn't very informative - however, it did bring up a question.
The question I have is that would it be possible to explore running this as a sidecar/daemonset in k8s - so that the actual code doesn't have to deal with keeping a buffer/flush it's log etc. Even better if this can also take an eventflow config and be responsible for pushing AI telemetry reliably.
I integrated application insights kubernetes plugin in my ASP.NET core web api project and deployed it on a kubernetes, but there was no metrics/stats on app-insights dashboard related to kubernetes. Then I enabled self diagnostics for ApplicationInsights.Kubernetes (link) and checked the logs and found that it throws an exception and then fails to start:
PS C:\WINDOWS\system32> kubectl logs todoappcore-1886517599-52jfh
info: K8sEnvInitializer[0]
ApplicationInsights.Kubernetes.Version:1.0.0-beta1
crit: Microsoft.ApplicationInsights.Kubernetes.K8sEnvironment[0]
System.InvalidCastException: Can't figure out docker id. Input: 11:blkio:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
10:net_cls:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
9:pids:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
8:memory:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
7:freezer:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
6:cpuset:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
5:devices:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
4:hugetlb:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
3:cpu,cpuacct:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
2:perf_event:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
1:name=systemd:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
. Pattern: cpu.+docker/(.*)$
at Microsoft.ApplicationInsights.Kubernetes.KubeHttpClientSettingsProvider.FetchContainerId(String pathToCGroup)
at Microsoft.ApplicationInsights.Kubernetes.KubeHttpClientSettingsProvider..ctor(ILoggerFactory loggerFactory, String pathToToken, String pathToCert, String pathToNamespace, String pathToCGroup, String
kubernetesServiceHost, String kubernetesServicePort)
at Microsoft.ApplicationInsights.Kubernetes.K8sEnvironment.<CreateAsync>d__11.MoveNext()
fail: K8sEnvInitializer[0]
Application Insights Kubernetes failed to start.
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[3]
Hosting starting
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[4]
Hosting started
Hosting environment: Production
Content root path: /app
Now listening on: http://+:80
Application started. Press Ctrl+C to shut down.
I have tried on a minikube cluster (kubernetes 1.7.0) and also tried on kubernetes cluster 1.6.6 on Azure Container Service as well.
This helps determine which version of the package is used and helps self-diagnostics.
The sign instruction is using the old assembly name.
For now, only replica set is supported as the reference owner. Add support for other types for backward compatibility.
Because it holds an instance of KubeHttpClient, which implements IDisposable.
When Telemetry initializer setting the same value to the same property worth a trace but not a warning. It would just be much too noisy.
In the pull request #60 , @nizarq think EnableK8s() is not a good name for API because K8s is not as official as Kubernetes.
Anyone think it bothers or not? Please suggest a better name if you agree. Let's make decision before it goes public - it will be way easier we change it for now.
Suggestion from @nizarq: EnableApplicationInsightsFoKubernetes()
When logging is enabled to TraceLevel, developer can see telemetry details manipulated by KubernetesTelemetryInitializer.
After the refactor, K8sEnvornment is injected while IK8sEnvironment is required.
When fail gracefully, there still should be trace to figure out what happened
Microsoft.ASPNETCore has a dependency on .NETStandard 1.3. We shall align with the lowest possible.
I am using https://github.com/dotnet-architecture/HealthChecks.
When an HttpGet Readiness Probe is configured, either Application Insights Kubernetes fails to start, or the pod never gets to Ready.
Steps to reproduce:
add this to deployment.yaml:
readinessProbe:
httpGet:
path: /hc
port: 80
scheme: HTTP
initialDelaySeconds: 130 #(10 seconds more than the default timeout)
periodSeconds: 10
failureThreshold: 10
timeoutSeconds: 10
This will cause the pod to be ready after 130 seconds, but the logs show:
fail: Microsoft.ApplicationInsights.Kubernetes.K8sEnvironment[0]
Kubernetes info is not available within given time of 120000 ms.
fail: K8sEnvInitializer[0]
Application Insights Kubernetes failed to start.
When the probe is defined like this:
readinessProbe:
httpGet:
path: /hc
port: 80
scheme: HTTP
initialDelaySeconds: 10 #(start probing after 10 seconds))
periodSeconds: 10
failureThreshold: 10
timeoutSeconds: 10
the pod never reaches ready state and gives an error:
Readiness probe failed: Get http://xx.xxx.x.122:80/hc: dial tcp xx.xxx.x.122:80: getsockopt: connection refused
The logs show that the application never initialises further than this message:
trce: Microsoft.ApplicationInsights.Kubernetes.KubeHttpClientSettingsProvider[0]
System.Security.Cryptography.X509Certificates.X509Chain
trce: Microsoft.ApplicationInsights.Kubernetes.KubeHttpClientSettingsProvider[0]
RemoteCertificateChainErrors
trce: Microsoft.ApplicationInsights.Kubernetes.KubeHttpClientSettingsProvider[0]
ServerCert:
[Subject]
CN=apiserver[Issuer]
CN=acsk8sca[Serial Number]
XXXXXXXXXXXXX[Not Before]
10/26/2017 00:00:00[Not After]
10/26/2019 00:00:00[Thumbprint]
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXdbug: Microsoft.ApplicationInsights.Kubernetes.KubeHttpClientSettingsProvider[0]
Issuer validation passed: CN=acsk8sca
dbug: Microsoft.ApplicationInsights.Kubernetes.KubeHttpClientSettingsProvider[0]
Server certificate validate date verification passed.
dbug: Microsoft.ApplicationInsights.Kubernetes.KubeHttpClientSettingsProvider[0]
Server certification custom validation successed.
A possible solution might be to wrap the initialisation in a Task.Run(), as is done for retrieving the version info in KubernetesModule.EnableKubernetes
AI.K8s uses http requests to fetch K8s properties. These http requests are also recorded by AI sdk automatically, which usually is a noise to end user.
Investigate to see if we can provide an option to allow user to opt in these requests when only necessary.
After AI.NETCore SDK 2.1-beta2, we should have dependency collector enabled by default. Verify that. If that is the case, update the wiki to remove the step of enabling it.
Refer issue 401 for the change.
I am trying to configure ApplicationInsights Kubernetes plugin in my ASP.NET Core 2.0 application. I have followed the instructions and deployed the container image to kubernetes cluster (Azure Kubernetes Service with kubernetes 1.8.1 as well as on minikube with kubernetes 1.8.0). But I am unable to get any kubernetes related stats in the app-insights portal. I am trying to enable self diagnostics for ApplicationInsights.Kubernetes using the instructions here but its giving me error on line:
.ConfigureLogging(f => f.AddConsole(LogLevel.Trace))
The error is:
Error CS1503 Argument 2: cannot convert from 'Microsoft.Extensions.Logging.LogLevel' to 'System.Action<Microsoft.Extensions.Logging.Console.ConsoleLoggerOptions>'
It seems like the .net core 2.0 packages does not have these functions to add console. I have added below lines but still no kubernetes logs and no stats on app insight portal.
.ConfigureLogging(logging => logging.SetMinimumLevel(LogLevel.Trace))
What could be the issue?
Project / License / Image and so on and so forth for the assemblies and nuget packages
In dashboard, in default CLI, they are all names than ids.
When the data exists on the telemetry object already, is the last hit win or the first hit win? Refer to other telemetry initializers and fix when necessary.
It is expose as info, which should be exposed as debug.
The scenario of AI.K8s only works when AI.DependencyCollector 2.4 or above is there. Since AI.DependencyCollector 2.4 targets minimum of .netstandard 1.6, it doesn't make sense for AI.K8s to target framework lower than 1.6.
Besides, at this point, .netstandard 2.0 is available and 1.6 of .netstnadard shall become the stable version of .netstandard 1.x.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<!-- This variable for the package version is redundant -->
<CIPackageVersion Condition=" '$(CIPackageVersion)' == ''">1.0.0-dev</CIPackageVersion>
<!-- ... -->
<Version>$(CIPackageVersion)</Version>
Risk of MITM attack.
Although .NET Standard 2.0 project shall be able to reference .NET Standard 1.3 implementations, it is beneficial to multi-targeting both .NET Standard 1.3 & .NET Standard 2.0 to make use of the new sets of APIs. This need to be done prior to supporting the new configuration settings in ASP.NET Core 2.0 applications.
During the incubation, I have put my alias in the codebase. We'll need to remove them.
Something related to code analysis. Need to sort out a bit.
aikubequery looks more like a codename. Let's give the project a new name like: ApplicationInsights.Kubernetes.
Verify the building system is still OK with the naming change.
Doesn't make too much sense.
The referenced FxCop analyzer is out of date and is causing a lot of warnings. (1000+).
Bump up version to provide better code analysis experience.
Switch public class / interfaces to internals when not used by other projects.
This will easy the restore process.
From
if (loggerFactory != null)
{
logger = loggerFactory.CreateLogger<K8sEnvironment>();
}
to
logger = loggerFactory?.CreateLogger<K8sEnvironment>();
We are currently reference the NetstandardLibrary 1.6.1. See if we can lower the criteria.
Hello,
latest release on a nuget was in May 2017. As I can see we have some changes. Maybe it's time to release some beta2 version for dev who already invest sometime and tested this package in their apps earlier?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.