Code Monkey home page Code Monkey logo

llmops-gha-demo's Introduction

Project

Read more about how to get started with LLMOps on Microsoft Offical Docs

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

llmops-gha-demo's People

Contributors

abeomor avatar lordlinus avatar microsoftopensource avatar setuc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

llmops-gha-demo's Issues

PF runtime does not allow pf execution from service principal

I am unable to submit job using service principal to a compute instance owned by another user. When trying to create CI assigned to SP it doesn't recognize the SP as a valid user.

Error in GHA:
Create run failed with FlowRequestException: Calling submit_bulk_run failed with request id: 2b6ba259-cc15-4370-9d24-3998b387735b
Status code: 403
Reason: Flow Runtime operation failed with Forbidden: You are not authorized to invoke this runtime as the r
Error message: (UserError) Flow Runtime operation failed with Forbidden: You are not authorized to invoke this runtime as the runtime compute instance jacwang1 is assigned to 352ccb24-a507-464f-9f4e-d4c09a60d6c7.
Code: UserError
Message: Flow Runtime operation failed with Forbidden: You are not authorized to invoke this runtime as the runtime compute instance jacwang1 is assigned to 352ccb24-a507-464f-9f4e-d4c09a60d6c7.

Error running "run promptflow" step of login-and-run-evalpf

Forked repo, updated runtime names, and ran action. Error in the promptflow step.
image

log:
Class WorkspaceHubOperations: This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.11.4/x64/bin/pfazure", line 8, in
sys.exit(main())
^^^^^^
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/_cli/_pf_azure/entry.py", line 82, in main
entry(command_args)
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/_cli/_pf_azure/entry.py", line 65, in entry
raise ex
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/_cli/_pf_azure/entry.py", line 55, in entry
dispatch_run_commands(args)
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/_cli/_pf_azure/_run.py", line 361, in dispatch_run_commands
create_run(
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/_cli/_utils.py", line 344, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/_cli/_pf/_run.py", line 629, in create_run
run = create_func(run=run, stream=stream)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/azure/operations/_run_operations.py", line 181, in create_or_update
rest_obj = self._resolve_dependencies_in_parallel(run=run, runtime=kwargs.get("runtime"), reset=reset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/azure/operations/_run_operations.py", line 713, in _resolve_dependencies_in_parallel
task_results = [task.result() for task in tasks]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/azure/operations/_run_operations.py", line 713, in
task_results = [task.result() for task in tasks]
^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/azure/operations/_run_operations.py", line 539, in _resolve_flow
self._flow_operations._resolve_arm_id_or_upload_dependencies(flow=flow, ignore_tools_json=True)
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/azure/operations/_flow_opearations.py", line 241, in _resolve_arm_id_or_upload_dependencies
self._try_resolve_code_for_flow(flow=flow, ops=ops, ignore_tools_json=ignore_tools_json)
File "/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/promptflow/azure/operations/_flow_opearations.py", line 280, in _try_resolve_code_for_flow
upload_paths += list(traverse_directory(root, files, source_path, prefix, ignore_file=ignore_file))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: traverse_directory() takes 2 positional arguments but 4 positional arguments (and 1 keyword-only argument) were given
Error: Process completed with exit code 1.

Unable to complete execution of Run promptflow

I am following the documentation here: https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-end-to-end-llmops-with-prompt-flow?view=azureml-api-2 and have worked through some initial issues around credentials/secrets and updated variable names in both run.yml and run_evaluation.yml. However, I am stuck with the attached error in the code that appears to run run.yml - and not sure where or how to proceed to debug the issue. If there is a log generated with more details as to where or what the problem is, I was not able to find it.

I reviewed the README.md under the .promptflow folder and am not sure if that could help or where the steps should be executed. If you have any suggestions for how to troubleshoot the issue, please let me know.

Thank you~
Screenshot for LLMOps

Missing permissions for endpoint to the ml workspace during Deploy Action

Looks like the endpoint needs to be granted permissions before running "Invoke Model" step in Deployment action:

image

Error:
Run az ml online-endpoint invoke --name web-classification-0e1890 --request-file promptflow/deployment/sample-request.json -g *** -w ***
Class WorkspaceHubOperations: This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.
ERROR: (UserError) Access denied to list workspace secret due to invalid authentication. Please assign RBAC role 'AzureML Data Scientist' to the endpoint for current workspace, and wait for a few minutes to make sure the new role takes effect. More details can be found in https://aka.ms/pf-runtime.
Code: UserError
Message: Access denied to list workspace secret due to invalid authentication. Please assign RBAC role 'AzureML Data Scientist' to the endpoint for current workspace, and wait for a few minutes to make sure the new role takes effect. More details can be found in https://aka.ms/pf-runtime.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.