[Contribution Request] Integrate Unify into Promptflow about promptflow HOT 14 OPEN

Hi @KatoStevenMubiru

Thank you for your contribution request. I have a few questions:

1. Could you please provide more details about the "Unify" you mentioned? Do you have any documentation or a website that we can refer to for more information?
2. When you mention "integration" are you referring to contributing a flow example? If so, please refer to this folder and ensure that your example follows a similar structure and content.

from promptflow.

Hi @0mza987,

Thanks for your questions. Here's some clarification:

1. Unify (https://unify.ai/docs/index.html) is an open-source framework for unified access to various LLMs across providers, offering optimization for quality, speed, and cost-efficiency.

2. I'm proposing either a community integration or a package to integrate Unify's capabilities with Promptflow, not a specific flow example. This could involve custom components utilizing Unify's features in Promptflow workflows.

Which approach - community integration or package - would better align with Promptflow's architecture and plans?

Let me know if you need any further information.

from promptflow.

@KatoStevenMubiru Thanks for the clarification.

Could you please provide a pseudo-code example to demonstrate how users would utilize Promptflow along with the integrated Unify features? This would help us better understand the targeted user scenario.

My initial thought is that a community integration (which, if I understand correctly, involves adding Unify to Promptflow's dependencies) may not be feasible unless it addresses a significant user need and Unify is proved to be the most suitable tool for this purpose. At this stage, developing a new package might be a more practical solution.

cc @eniac871 @wangchao1230

from promptflow.

Thank you for your feedback and questions, @0mza987. I understand your concerns about community integration.
Let me provide a pseudo-code example that demonstrates how users could leverage Promptflow with integrated Unify features, addressing significant user needs:

from promptflow import flow, tool
from unify import Unify

class UnifyTool:
    def __init__(self):
        self.unify_client = Unify(api_key="YOUR_API_KEY")

    @tool
    def optimize_llm(self, prompt: str, constraints: dict):
        optimal_model = self.unify_client.select_optimal_model(constraints)
        response = self.unify_client.generate(prompt, model=optimal_model)
        return response

    @tool
    def benchmark_models(self, prompt_set: list, models: list):
        results = self.unify_client.benchmark(prompt_set, models)
        return results

@flow
def adaptive_translation_flow(text: str, target_language: str):
    unify_tool = UnifyTool()
    
    # Step 1: Determine text complexity
    complexity_prompt = f"Analyze the complexity of this text: {text}"
    complexity_result = unify_tool.optimize_llm(
        complexity_prompt,
        constraints={"max_latency": 1000, "min_quality": 0.7}
    )
    
    # Step 2: Choose appropriate model based on complexity
    if "high complexity" in complexity_result.lower():
        translation_constraints = {"min_quality": 0.9, "max_cost": 0.05}
    else:
        translation_constraints = {"min_quality": 0.7, "max_cost": 0.02}
    
    # Step 3: Perform translation
    translation_prompt = f"Translate the following text to {target_language}: {text}"
    translation = unify_tool.optimize_llm(translation_prompt, constraints=translation_constraints)
    
    # Step 4: Quality check
    quality_check_prompt = f"Evaluate the quality of this translation: Original: {text}, Translation: {translation}"
    quality_score = unify_tool.optimize_llm(
        quality_check_prompt,
        constraints={"max_latency": 1500, "min_quality": 0.8}
    )
    
    return {
        "original_text": text,
        "translated_text": translation,
        "quality_score": quality_score
    }

# Usage
result = adaptive_translation_flow("Hello, how are you?", "French")
print(result)

# Benchmarking
benchmark_results = UnifyTool().benchmark_models(
    ["Translate 'Hello' to Spanish", "Translate 'Goodbye' to German"],
    ["gpt-3.5-turbo", "gpt-4", "claude-2"]
)
print(benchmark_results)

This example demonstrates how Unify could be integrated into Promptflow to address significant user needs:

1. Adaptive Model Selection: The flow dynamically selects the most appropriate LLM based on text complexity and user-defined constraints (latency, quality, cost).

2. Multi-step Workflows: It showcases a multi-step translation process, leveraging different models for complexity analysis, translation, and quality checking.

3. Performance Optimization: By using Unify's optimize_llm tool, the flow ensures each step uses the best-performing model within given constraints.

4. Benchmarking: Users can easily benchmark multiple models for specific tasks, helping them make informed decisions about which models to use in their flows.

5. Cost Management: The flow demonstrates how to adjust model selection based on task requirements, potentially reducing costs for simpler tasks.

6. Quality Assurance: It includes a built-in quality check step, ensuring the output meets the required standards.

Benefits for Promptflow users:

1. Improved performance and cost-efficiency through intelligent model selection.
2. Easier management of complex, multi-step LLM workflows.
3. Built-in benchmarking capabilities for continuous improvement.
4. Flexibility to adjust model

from promptflow.

I'm happy to provide additional examples showcasing other Unify features that could benefit Promptflow users. Some areas we could explore further include:

1. Optimizing large-scale batch processing
2. Improving performance with caching for repetitive tasks
3. Using analytics for workflow insights and optimization
4. Integrating fine-tuning management

If any of these or other aspects interest the Promptflow team, I can prepare more detailed pseudo-code examples. Our aim is to demonstrate how Unify can enhance Promptflow's capabilities and address user needs.

from promptflow.

Hi @KatoStevenMubiru,

Thank you for the detailed explanation and code example. Based on the sample code, it seems that the Unify features can be seamlessly integrated into our flex flow. Please review the following guidelines and examples and see if it can meet your needs:

• Develop a flow — Prompt flow documentation
• Getting started with flex flow — Prompt flow documentation
• promptflow/examples/flex-flows/basic/flex-flow-quickstart.ipynb at main · microsoft/promptflow

from promptflow.

Hi @0mza987 ,

Following suggestions from my superiors and our previous discussions, we propose adding a unify.py module to the tools directory of Promptflow.
This file will integrate Unify’s functionalities, enabling Promptflow to access a diverse range of LLMs and optimize model selection based on latency, quality, and cost criteria. Our goal is to enhance the flexibility and efficiency of Promptflow through this integration.

Looking forward to your thoughts and any additional requirements you might have.

from promptflow.