Comments (4)
Great! Thank you very much for your feedback @liucheng-ms!
from promptflow.
Hi @giannisha
Thank you for bringing up this issue related to streaming behavior in PromptFlow. I appreciate your attention to detail and your desire to improve the system.
Upon reviewing your scenario, it seems that there might be some confusion regarding the streaming functionality within PromptFlow. To help clarify, I'd like to direct you to our comprehensive documentation on streaming mode here: Enable streaming mode in Azure Machine Learning.
As per the current design of PromptFlow, streaming is only supported by the output of the final node in a given flow. The term "last node" specifically refers to a node whose output is not utilized by any subsequent nodes in the workflow.
In the scenario you've described, the output of the llm node is being passed on to a cache Python node. This interconnection implies that the llm node is not the last node in your flow, hence it does not satisfy the criteria for streaming its output.
To achieve the behavior you expect, you would need to adjust your flow so that the streaming
node is the final node, thus allowing its output to be streamed directly to your clients. Any subsequent actions, such as caching, would then need to be managed independently outside of the PromptFlow sequence.
If you have any further questions or need additional clarification, feel free to reach out. Feedback is highly appreciated.
from promptflow.
Hi @liucheng-ms
Thank you very much for your detailed response.
Do you think this scenario, to have nodes detached from the main flow, would be good to be supported in the future?
For similar cases, or other cases where you want to fire something and forget it (cache, logs etc.).
from promptflow.
Hi @giannisha ,
I understand your request for a feature that allows nodes to be detached from the main flow, enabling actions like caching or logging to be executed independently. Your suggestion for “fire and forget” functionality could indeed be beneficial for certain use cases.
Currently, we do not have immediate plans to support this feature within PromptFlow’s scope. However, we recognize the potential value it could offer. We will consider the need for such a feature and evaluate whether it aligns with our design principles and priorities.
Thank you for your feedback, and please continue to share your thoughts and ideas with us.
from promptflow.
Related Issues (20)
- [BUG] Ensuring system metrics is not none before checking length (Trace collector) HOT 4
- [BUG] Inconsistent order for dataframe rows when loading run details via SDK HOT 5
- [BUG] pf flow build stopped working HOT 1
- Failed to check if list[str] is a custom strong type: issubclass() arg 1 must be a class HOT 1
- List index out of range error on Open Model LLM Node HOT 2
- [BUG] "(TypeError) expected string or buffer" when running a batch run a flow with langchain python node HOT 3
- [Feature Request] Support setting dimensions in embedding tool. HOT 5
- [BUG] Flow link is broken HOT 3
- [BUG] FutureWarning from fillna call HOT 5
- [BUG]: Basic chat flow raises an error: Failed to serialize inputs or output for flow run because of cannot pickle 'generator' object. HOT 2
- [Feature Request]Support for tool_calls HOT 2
- [BUG] [VSCode Extension] HOT 2
- No versions of azure-cli is compatible with promptflow 1.8.0 with azure HOT 6
- [Feature Request] Enhance Testing with Serverless Computing Options HOT 1
- [BUG] - pf.flow.test failed with PythonLoadError: Failed to load python module from file ... (NameError) name 'Document' is not defined HOT 5
- [Document] Add promptflow endpoint api document HOT 1
- [Feature Request] Allow configuration of log format HOT 9
- [BUG] [VSCode Extension] VectorDB Index Lookup doesn't take the config.json variables into consideration HOT 5
- [Feature Request] Support for recently available Azure OpenAI embedding models & dimensions HOT 1
- [BUG] Unable to make azure openai connection HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from promptflow.