Windows containers with .NET 4.8 about blue-agent HOT 19 CLOSED

Pod IsRunning......

Azure DevOps

I'm going to separate a small project to test the Build ...

from blue-agent.

It's a scenario for legacy systems, but the solution is currently running on:
-Kubernetes
-Node (Windows)
-Containers (Windows)

The pipeline of this legacy system is on "one machine", does not scale and has poor performance

The idea is to use AzureDevOps, this solution to compile using Kubernetes with containers (windows).

from blue-agent.

I understand better your use case.

I started to develop a Windows based container. Work is done in the branch feat/win-container. There are two flavors, Server 2019 and Server 2022.

Specify this configuration to require Pods to run on Windows nodes:

# values.yaml
nodeSelector:
  "kubernetes.io/os": windows

Can you test one of these builds and give me a feedback?

from blue-agent.

I noticed .NET Framework Runtime is not packaged by default on the Windows containers. As .NET is required by the Azure Pipelines Agent, this is surely related. Fix pushed: 9046977.

Image: ghcr.io/clemlesne/azure-pipelines-agent:win-ltsc2022-4.16.1-7.9046977.

from blue-agent.

Can you test the Helm chart from a2a9b00 to deploy the service?

from blue-agent.

I didn't have much time to dedicate, but:

• MsBuild worked and compiled the project

However, I had problems using some things from the newer C# (something related to the msbuild version) and I stopped at the buildkitd and buildctl build part...

I wanted something that was very similar to how it is done in linux.

Maybe tomorrow I can work more on this pipeline.

There is a scenario where I can build\publish on windows. Then I use another linux agent to get these files and using the buildkit to generate a docker windows image.

I still don't know what the best strategy will be

Thank you so much for your effort and dedication

from blue-agent.

I squashed my commits and added zstd in the Windows build.

I'm merging the branch, I propose you to re-open an issue for future needs, to simplify my workload. Thank you for participating in the project!

Be warning, I'm releasing this in the v5.0.0, it contains some breaking changes in the Helm values (not that much I reassure you).

Build containers in the agent:

I updated the documentation for container build in the agent. This is not supported at the time by any component I'm aware of, sadly.

What I can advise you is:

• to set up a dedicated Windows Server VM only for container build
• to use a SaaS container build, like Azure Container Registry Tasks

from blue-agent.

What is your use-case requiring to use ASP.NET 7.x on Linux or .NET 4.8 on Windows?

Note that ASP.NET 7.x is not LTS, ASP.NET 6.x is.

from blue-agent.

Yes, I understand.

My question was to try to create an option so that whoever is using it can choose whether to use NET 7 (STS), NET 8 (Preview) or even NET Framework 4.8 with windows container,

from blue-agent.

I understand better. I arbitrarily chose to install the LTS non SDK version of ASNP.NET. Because :

• LTS is better supported by Microsoft than STS
• Non-SDK is lighter once included in a container, knowing that everyone won't use it for building purposes

I advise you to hard-code the framework version you want to use, in your pipeline. With this setup, the developer controls its environment, not the platform. If they decide to upgrade, they update the pipeline, if not, not. This is under the responsibility of the developer.

The framework can be installed on the fly with UseDotNet@2:

- task: UseDotNet@2
  inputs:
    packageType: sdk
    version: 7.0.5

Would this answer your first question?

Then for Windows container, can you describe your use case?

from blue-agent.

In the first test, there was a problem in Status ContainerCreating:

Normal
Successful
AttachVolume
attachdetach-controller AttachVolume.Attach succeeded to volume "pvc-51587cd6-b362-420e-abd3-76c06b5f6433"

Warning
FailedMount
kubelet MountVolume.SetUp failed for volume "kube-api-access-n59p5": chown c: \var\lib\kubelet\pods\05795027-b07d-49fd-891b-db589ae09aa9\volumes\k ││ kubernetes.io~projected \kube-api-access-n59p5..2023_04_26_22_02_22.3962106217\token: not supported by Windows

Tomorrow I will continue the tests, thanks for the support.

from blue-agent.

Seems to come from an incompatibility with the default securityContext configuration: aws/aws-node-termination-handler#595 (comment).

Can you test this configuration:

# values.yaml
nodeSelector:
  kubernetes.io/os: windows

image:
  flavor: win-ltsc2022

securityContext:
  # Windows containers doesn't support "runAsUser" command, it makes the pod fail to start.
  # See: https://github.com/aws/aws-node-termination-handler/issues/595#issuecomment-1054179123
  runAsUser: null
  windowsOptions:
    runAsUserName: ContainerAdministrator

from blue-agent.

That worked, but now another problem:

Normal
Pulled
Container image "ghcr.io/clemlesne/azure-pipelines-agent:win-ltsc2022-4.16.1-5.11171b7" already present on machine

Warning Failed
Error: failed to create containerd container: rootpath on mountPath C:\Windows\TEMP\ctd-volume1747161066\74, volume \app-root\ ││ azp-work: readlink C:\Windows\TEMP\ctd-volume1747161066\74: The system cannot find the path specified.

Kubernetes version: 1.26

I'm reading these two possible solutions:
containerd/containerd#5067
containerd/containerd#6300

from blue-agent.

• Fix pushed: bd1fa92
• Image: ghcr.io/clemlesne/azure-pipelines-agent:win-ltsc2022-4.16.1-6.bd1fa92

from blue-agent.

Failed to pull image "ghcr.io/clemlesne/azure-pipelines-agent:win-ltsc2022-4.16.1-6.bd1fa92": rpc error: code = Unknown desc ││ = failed to pull and unpack image "ghcr.io/clemlesne/azure-pipelines-agent:win-ltsc2022-4.16.1-6.bd1fa92"

failed to extract layer sha256:1c3e131b426750f22e3542b89e73a111f9586c300d8858d97e940aba07753e39: ││ link /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/172/fs/Files/Program Files/Common Files/System/ado/en-US/msader15.dll.mui /var/lib/containerd/io.containerd.snapshotter.v1.overlayf ││ s/snapshots/172/fs/Files/Program Files (x86)/Common Files/System/ado/en-US/msader15.dll.mui: no such file or directory: unknown

from blue-agent.

Which version of Windows Server are you using for your host?

from blue-agent.

Node group setup : AMI WINDOWS_CORE_2022_x86_64

from blue-agent.

With the variable isWindows=true

Status: CreaterContainerError

Warning Failed 88s kubelet Error: failed to create containerd container: rootpath on mountPath C:\Windows\TEMP\ctd-volume3851909698\300, volume \app-ro ││ ot\azp-work: readlink C:\Windows\TEMP\ctd-volume3851909698\300: The system cannot find the path specified.

Warning Failed 63s (x2 over 75s) kubelet (combined from similar events): Error: secret "azp-agent-azure-pipelines-agent" not found ││ Normal Pulled 48s (x11 over 3m2s) kubelet Container image "ghcr.io/clemlesne/azure-pipelines-agent:win-ltsc2022-4.16.1-6.bd1fa92" already present on machine ││

Warning Failed 48s kubelet Error: secret "azp-agent-azure-pipelines-agent" not found

from blue-agent.

I got this error again, I'm checking if I forgot something or if it could be some cache:

Failed
Error: failed to create containerd container: rootpath on mountPath C:\Windows\TEMP\ctd-volume1824593586\392,
volume \app-root\ ││
azp-work: readlink C:\Windows\TEMP\ctd-volume1824593586\392: The system cannot find the path specified.

from blue-agent.