Comments (14)
Thank you for your contribution request. I have a few questions:
- Could you please provide more details about the "Unify" you mentioned? Do you have any documentation or a website that we can refer to for more information?
- When you mention "integration" are you referring to contributing a flow example? If so, please refer to this folder and ensure that your example follows a similar structure and content.
from promptflow.
Hi @0mza987,
Thanks for your questions. Here's some clarification:
-
Unify (https://unify.ai/docs/index.html) is an open-source framework for unified access to various LLMs across providers, offering optimization for quality, speed, and cost-efficiency.
-
I'm proposing either a community integration or a package to integrate Unify's capabilities with Promptflow, not a specific flow example. This could involve custom components utilizing Unify's features in Promptflow workflows.
Which approach - community integration or package - would better align with Promptflow's architecture and plans?
Let me know if you need any further information.
from promptflow.
@KatoStevenMubiru Thanks for the clarification.
Could you please provide a pseudo-code example to demonstrate how users would utilize Promptflow along with the integrated Unify features? This would help us better understand the targeted user scenario.
My initial thought is that a community integration (which, if I understand correctly, involves adding Unify to Promptflow's dependencies) may not be feasible unless it addresses a significant user need and Unify is proved to be the most suitable tool for this purpose. At this stage, developing a new package might be a more practical solution.
from promptflow.
Thank you for your feedback and questions, @0mza987. I understand your concerns about community integration.
Let me provide a pseudo-code example that demonstrates how users could leverage Promptflow with integrated Unify features, addressing significant user needs:
from promptflow import flow, tool
from unify import Unify
class UnifyTool:
def __init__(self):
self.unify_client = Unify(api_key="YOUR_API_KEY")
@tool
def optimize_llm(self, prompt: str, constraints: dict):
optimal_model = self.unify_client.select_optimal_model(constraints)
response = self.unify_client.generate(prompt, model=optimal_model)
return response
@tool
def benchmark_models(self, prompt_set: list, models: list):
results = self.unify_client.benchmark(prompt_set, models)
return results
@flow
def adaptive_translation_flow(text: str, target_language: str):
unify_tool = UnifyTool()
# Step 1: Determine text complexity
complexity_prompt = f"Analyze the complexity of this text: {text}"
complexity_result = unify_tool.optimize_llm(
complexity_prompt,
constraints={"max_latency": 1000, "min_quality": 0.7}
)
# Step 2: Choose appropriate model based on complexity
if "high complexity" in complexity_result.lower():
translation_constraints = {"min_quality": 0.9, "max_cost": 0.05}
else:
translation_constraints = {"min_quality": 0.7, "max_cost": 0.02}
# Step 3: Perform translation
translation_prompt = f"Translate the following text to {target_language}: {text}"
translation = unify_tool.optimize_llm(translation_prompt, constraints=translation_constraints)
# Step 4: Quality check
quality_check_prompt = f"Evaluate the quality of this translation: Original: {text}, Translation: {translation}"
quality_score = unify_tool.optimize_llm(
quality_check_prompt,
constraints={"max_latency": 1500, "min_quality": 0.8}
)
return {
"original_text": text,
"translated_text": translation,
"quality_score": quality_score
}
# Usage
result = adaptive_translation_flow("Hello, how are you?", "French")
print(result)
# Benchmarking
benchmark_results = UnifyTool().benchmark_models(
["Translate 'Hello' to Spanish", "Translate 'Goodbye' to German"],
["gpt-3.5-turbo", "gpt-4", "claude-2"]
)
print(benchmark_results)
This example demonstrates how Unify could be integrated into Promptflow to address significant user needs:
-
Adaptive Model Selection: The flow dynamically selects the most appropriate LLM based on text complexity and user-defined constraints (latency, quality, cost).
-
Multi-step Workflows: It showcases a multi-step translation process, leveraging different models for complexity analysis, translation, and quality checking.
-
Performance Optimization: By using Unify's
optimize_llm
tool, the flow ensures each step uses the best-performing model within given constraints. -
Benchmarking: Users can easily benchmark multiple models for specific tasks, helping them make informed decisions about which models to use in their flows.
-
Cost Management: The flow demonstrates how to adjust model selection based on task requirements, potentially reducing costs for simpler tasks.
-
Quality Assurance: It includes a built-in quality check step, ensuring the output meets the required standards.
Benefits for Promptflow users:
- Improved performance and cost-efficiency through intelligent model selection.
- Easier management of complex, multi-step LLM workflows.
- Built-in benchmarking capabilities for continuous improvement.
- Flexibility to adjust model
from promptflow.
I'm happy to provide additional examples showcasing other Unify features that could benefit Promptflow users. Some areas we could explore further include:
- Optimizing large-scale batch processing
- Improving performance with caching for repetitive tasks
- Using analytics for workflow insights and optimization
- Integrating fine-tuning management
If any of these or other aspects interest the Promptflow team, I can prepare more detailed pseudo-code examples. Our aim is to demonstrate how Unify can enhance Promptflow's capabilities and address user needs.
from promptflow.
Thank you for the detailed explanation and code example. Based on the sample code, it seems that the Unify features can be seamlessly integrated into our flex flow. Please review the following guidelines and examples and see if it can meet your needs:
- Develop a flow โ Prompt flow documentation
- Getting started with flex flow โ Prompt flow documentation
- promptflow/examples/flex-flows/basic/flex-flow-quickstart.ipynb at main ยท microsoft/promptflow
from promptflow.
Hi @0mza987 ,
Following suggestions from my superiors and our previous discussions, we propose adding a unify.py
module to the tools directory of Promptflow.
This file will integrate Unifyโs functionalities, enabling Promptflow to access a diverse range of LLMs and optimize model selection based on latency, quality, and cost criteria. Our goal is to enhance the flexibility and efficiency of Promptflow through this integration.
Looking forward to your thoughts and any additional requirements you might have.
from promptflow.
Related Issues (20)
- [Feature Request] pass addition inputs to target function when using the evaluate method HOT 2
- [Feature Request] Have option to return trace as part of out put of a PromptFlow when deployed as endpoint HOT 11
- [BUG] Running evaluate from promptflow.evals.evaluate and setting trace.destination="none" or "local" causes evaluations to not be available HOT 1
- [BUG] prompt eval method does not take credential and creates PFClient without passing in credential HOT 1
- [BUG] Tracing Contextvar reset HOT 6
- Failed to load trace[BUG] HOT 3
- [BUG] Flow is not enabled HOT 1
- [BUG]evaluator keeps failing with promptflow-eval 0.3.1 but works with 0.3.0 HOT 3
- [BUG] Race condition with global state in process pool HOT 2
- [FeatureAsk] Instantiating a model config for Azure OAI using AAD instead of `api_key` HOT 11
- [Feature Request] HOT 1
- Deployed Endpoint App Insights Not Capturing Full Data [BUG] HOT 1
- Deployed Endpoint - Return Request Id as response so can easily find associated App Insights Data [Feature Request] HOT 1
- [Feature Request] Add token metrics to tracing for Non-OpenAI models HOT 1
- [BUG] promptflow-evals pypi error - ImportError: cannot import name 'serialize' from 'promptflow._utils.dataclass_serializer'
- [BUG] examples/flex-flows/chat-minimal missing flow.flex.yaml HOT 1
- [BUG] Different evaluation flow behavior locally and in Azure HOT 1
- [Feature Request] Support external Tracing Tools (e.g. langfuse) HOT 2
- [BUG] eval API errors out with UnexpectedError: Unexpected error occurred while executing the batch run. Error: (TypeError) cannot pickle '_thread.RLock' object. HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from promptflow.