Comments (14)
Thanks @KirkMunro for sharing your experiences. For the last two points, it's doable by loading a module to a separate AssemblyLoadContext
. Both code and data (statics) from a module can be isolated in that way.
The idea is to add a new flag -Isolated
to Import-Module
to explicitly make powershell load a module to a separate assembly context. The tricky part is how the type resolution in powershell should change accordingly. I will write up an RFC on this after PSConf Asia, probably in late November.
from azure-functions-powershell-worker.
Allow the user to declare assembly dependencies. (what UX?)
Can you give an example of this?
I updated this bullet to give an example.
from azure-functions-powershell-worker.
func
CLI as a global tool makes sense but is orthogonal to this and I suspect many PowerShell scripters won't have dotnetcli installed.
From a packaging standpoint, I think it would be interesting to have a generic way to package a script and dependencies so that ITPros can easily distribute them in their environment even outside of AzF. This can simply be a zip with a well known extension and layout (including assemblies).
I like the idea of requiring the user to package Az module rather than having it in our worker image to resolve the versioning problem.
from azure-functions-powershell-worker.
I like these thoughts. I have a few follow ups.
Allow the user to declare assembly dependencies. (what UX?)
Can you give an example of this?
I don't like having AzureRM.NetCore bundle with user's Function App
I liked our previous idea of having a Module path at the worker level and a module path a the user level.
$env:PSModule = 'UsersFunctionApp/Modules;workers/powershell/Modules'
If we go this route, then #Requires -Modules
makes sense too because it could load the "correct" version if multiple are present.
This lets us ship things in the worker while allowing the user to use another version of a module since importing resolves left to right in the module path.
Lambda tool is a .NET Global Tool "Amazon.Lambda.Tools". Maybe func CLI should consider to be a Global Tool too, given it's a .NET Core application.
This should probably be an issue in the core tools repo since it's about the func
CLI as a whole.
https://github.com/azure/azure-functions-core-tools
from azure-functions-powershell-worker.
I like the idea of requiring the user to package Az module rather than having it in our worker image to resolve the versioning problem.
If we go through with this, then we can conditionally call Connect-AzAccount
which will speed up start up for function apps that don't need to interact with Azure. That said, then we need to tread carefully so that the user knows how to package their app correctly.
from azure-functions-powershell-worker.
As a user, I am partial to @tylerl0706's plan of having some common base modules like Az in the worker image but "further right" in the PSModule path so we can optionally supply a newer one. It would be the most user friendly and performant without sacrificing ability to bring more "bleeding edge" modules if you need them for certain new features/functionality.
from azure-functions-powershell-worker.
Yeah that would play nice with #76. 😊
from azure-functions-powershell-worker.
A few general thoughts related to this. I've been using a C# Azure Function App that invokes PowerShell for quite a while, and here are the things I've had to deal with along with my principal issues with Azure Functions and PowerShell:
-
Updating PSModulePath (related to #76). My function app injects an entry into index 1 in PSModulePath so that I can override any built-in modules (AzureRM in particular) with the latest version that I upload into my function app.
-
Dealing with caching (related to PowerShell/PowerShell#7703). Caching is a huge problem for PowerShell in Azure Function Apps that needs attention. The lack of guidance and poor design of some modules, combined with Azure Function Apps not using process-level isolation, makes for a difficult environment to work with. Technically runspace-level isolation is all that is needed, but that's not even guaranteed with some modules (AzureAD for example). Depending on the setup, if you're working with multiple tenants in some cloud provider or online service, you need to be able to ensure that PowerShell commands don't impact other running instances of the Azure Functions.
-
Supporting SxS versions of binary modules. I'm not sure what is possible here, but this is another issue. If you are running PowerShell through Azure Function App endpoints where different invocations require different versions of the same binary module(s), since everything is in a single process, what can you do? I would love if you could import different versions of the same binary module(s) in different invocations of an Azure Function App endpoint. Process-level isolation gives you this for free, but otherwise, is this even possible in a single process through some modern app domain magic?
For the last two points I don't know what option there is other than looking at Lambda as a good alternative since they have process-level isolation. I've proposed the idea of a multi-process option instead of using multi-threaded workers to the Azure Function team but they don't seem interested in that approach.
from azure-functions-powershell-worker.
@daxian-dbw That's great to hear.
In addition to that work, on my second point, above, I think it would also be well worth the time for the PowerShell team to implement runspace-local storage, and that probably merits another RFC. The idea for runspace-local storage is straightforward: provide a thread-safe storage model for PowerShell modules that allows for runspace isolation of cached data in memory. If PowerShell had such a model in place, coupled with design documentation pushing binary module authors to use runspace-local storage for cached information (and, if it's not asking too much also coupled with a PowerShell Module Analyzer or FxCop style rules for Binary modules that red flag the use of static properties for memory caches, but now I'm really dreaming in color), then we wouldn't have to push module authors post-release to fix their in-memory caching logic so that it's designed properly for PowerShell.
from azure-functions-powershell-worker.
... implement runspace-local storage ... pushing binary module authors to use runspace-local storage for cached information
Binary module authors can already do this today, using ThreadStatic
or ThreadLocal
object. A ThreadStatic
member is stored in the thread local storage, and is isolated for each thread. Runspace
itself is also stored in a ThreadStatic
member Runspace.DefaultRunspace
. So when the same binary module is loaded in another Runspace
running in a different pipeline thread, the ThreadStatic
members of the module won't carry the existing cache.
from azure-functions-powershell-worker.
I know each runspace has its own thread, but isn't it possible to have multiple threads using a single runspace? In which case ThreadStatic/ThreadLocal wouldn't be appropriate, correct?
from azure-functions-powershell-worker.
@KirkMunro I get your point. It's more about the PowerShell API. With the same Runspace
, $ps.Invoke()
will start a new thread every time it's called. ThreadStatic
and ThreadLocal
would mean no caching for every run in this scenario. This makes the Runspace-local interesting for binary modules.
from azure-functions-powershell-worker.
Next steps to generate work items out of the discussion here.
from azure-functions-powershell-worker.
I think we got some good info out of this. We can continue to discuss here or in other issues but I think the original issue can be closed.
from azure-functions-powershell-worker.
Related Issues (20)
- Upgrade PowerShell SDK to latest version
- Update PowerShell 7.0 worker to latest System.Data.SqlClient HOT 1
- Known regression: Durable Functions Activity input fails to deserialize when the input is an array
- [Linux Consumption] [Portal] Portal shows Linux Consumption as an option for Powershell 7.4 it is not supported HOT 3
- [Linux Elastic Premium] Kudu SSH appears to fail for powershell7.4 HOT 3
- Linux (Consumption) not supported in PowerShell 7.4 yet HOT 1
- Feature Request: Recursive Directory Searching for function.json HOT 1
- Az-Accounts: Issue re-emerging.. HOT 4
- Support OpenTelemetry in the PowerShell worker
- Profile.ps1 not instrumented HOT 1
- Diagram of module interaction w/ worker HOT 1
- Update PowerShell SDK for 7.4 to 7.4.2
- Update PowerShell SDK for 7.2 to 7.2.19 HOT 1
- Document OpenTelemetry support HOT 1
- Workaround for issue with hosting Durable Functions app on preview Flex Consumption plan
- Convert pipelines to 1ES
- Fix OTel test concurrency issue
- Backport 1ES pipelines to all release branches HOT 1
- Add back E2E tests for 1ES official pipeline
- Fix failing public pipeline
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from azure-functions-powershell-worker.