Add E2E tests using CLI from storage account about azure-functions-powershell-worker HOT 15 CLOSED

Yes. Please see https://github.com/Azure/azure-functions-java-worker/blob/dev/appveyor.yml for reference

from azure-functions-powershell-worker.

cc: @soninaren
@tylerl0706 / @daxian-dbw - Let me know if you have any questions

from azure-functions-powershell-worker.

@pragnagopa We don't use func CLI in our CI.
But we should write some E2E tests for PowerShell functions with the CLI (not sure how nodejs worker does E2E testing) -- is that what you are asking for in this issue?

from azure-functions-powershell-worker.

How do you run E2E tests?

from azure-functions-powershell-worker.

I don't know 😄 that's why I was asking if that's what you want for this issue. We are not using func CLI in our CI today.

from azure-functions-powershell-worker.

Also, please see https://github.com/Azure/azure-functions-java-worker/tree/dev/endtoendtests/Azure.Functions.Java.Tests.E2E if you can just reuse that project. Ideally, I would maintain one E2E test project that can be reused across all the language workers.

from azure-functions-powershell-worker.

Great, I will take a look at the project. A quick glance tells me it's using dotnet xunit runner, which has been deprecated recently (https://xunit.github.io/releases/2.4). You possibly will need to update it to use dotnet test.

from azure-functions-powershell-worker.

cc: @paulbatum

Thanks I will look into it. Here is the flow E2E tests

• Add E2E tests that cover supported triggers and bindings
• Build powershell worker
• Download CLI from the URL above. CLI will be updated before every functions runtime release
• Copy powershell worker to CLI\workers directory
• Use CLI to install extensions
• Start functions host via CLI func host start
• Run tests against the localhost.

you can add these either as a nightly runs or run them as part of every commit/PR. Next steps would be

• Deploy to Azure
• Run E2E tests using URL for the deployed APP

As long as CI follows this pattern we are OK. If maintaining a single E2E test project makes sense, I will refactor the tests later.

from azure-functions-powershell-worker.

Something weird is happening with the downloaded Azure.Functions.Cli.zip on macOS... I can't quite figure it out.

Invoke-RestMethod "https://functionsclibuilds.blob.core.windows.net/builds/2/latest/Azure.Functions.Cli.osx-x64.zip" -OutFile Azure.Functions.Cli.zip

Expand-Archive ./Azure.Functions.Cli.zip

./Azure.Functions.Cli/func

This opens the binary as a text file for me in PowerShell:

but in bash, it works once... but now I'm just getting a bunch of:

bash: ./func: Permission denied

It.. shouldn't require sudo...?

from azure-functions-powershell-worker.

chmod -x ./func was needed 🤦‍♂️

from azure-functions-powershell-worker.

What's the proper scoping of this work item? What do we need to achieve for preview?

from azure-functions-powershell-worker.

We have these now for HttpTrigger. We can probably close this. Although I don't know if the expectation is having tests for every trigger... And how we'd do that (how do we trigger a blob trigger without Azure?)

@pragnagopa any thoughts?

from azure-functions-powershell-worker.

Let's sync with them on this next week.

from azure-functions-powershell-worker.

Apologies I missed last weeks ping on this issue.

Yes. We do need tests for the supported triggers. Please see Azure.Functions.Java.Tests.E2E for a sample E2E test project.

This is a requirement for any release. Without test coverage, there is a huge risk of regressions both ways

• Host breaking powershell functions
• Changes in powershell worker not compatible with host changes.

@TylerLeonhardt already added coverage for HttpTriggers. When you add support for rest of the triggers, this work item is part of development

• Add Unit tests
• Add E2E tests

from azure-functions-powershell-worker.

Talked to @paulbatum about this in the sync this week - he said that we can probably get some help setting these E2E tests up (creating the resources in Azure, supplying the subscription).

Ideally, we should be able to grab the E2E tests from Java or Node and run them with a PowerShell Function App.

cc @fabiocav

from azure-functions-powershell-worker.