Comments (6)
Regarding
AttributeError: 'RemoteFunction' object has no attribute '__qualname__'
The task decorator could be updated to not require that. I'm not sure it'll get us far this this case though.
from prefect-ray.
Perhaps this link can help:
https://docs.ray.io/en/latest/ray-core/tasks/using-ray-with-gpus.html
Something like:
@task
@ray.remote(num_gpus=1) # or maybe reverse the decorators
def use_gpu():
print("ray.get_gpu_ids(): {}".format(ray.get_gpu_ids()))
print("CUDA_VISIBLE_DEVICES: {}".format(os.environ["CUDA_VISIBLE_DEVICES"]))
Let me know if that works!
from prefect-ray.
Thanks a lot for your response.
We tried mixing the ray and prefect decorators like you also suggested, but we did not have any success yet.
Therefore we are a bit uncertain how to realize the resource requirements.
Depending on the order of the two decorators the following errors are received:
Traceback (most recent call last):
File "/home/christian/Documents/temp/prefect-example/ray-anyscale-stress-test/time-wait-flow.py", line 18, in <module>
def wait_time(time_in_secs):
File "/home/christian/Documents/temp/prefect-example/venv/lib/python3.9/site-packages/prefect/tasks.py", line 778, in task
Task(
File "/home/christian/Documents/temp/prefect-example/venv/lib/python3.9/site-packages/prefect/context.py", line 163, in __register_init__
__init__(__self__, *args, **kwargs)
File "/home/christian/Documents/temp/prefect-example/venv/lib/python3.9/site-packages/prefect/tasks.py", line 136, in __init__
self.name = name or self.fn.__name__
AttributeError: 'RemoteFunction' object has no attribute '__name__'
Traceback (most recent call last):
File "/home/christian/Documents/temp/prefect-example/ray-anyscale-stress-test/time-wait-flow.py", line 18, in <module>
def wait_time(time_in_secs):
File "/home/christian/Documents/temp/prefect-example/venv/lib/python3.9/site-packages/ray/_private/worker.py", line 2603, in _make_remote
raise TypeError(
TypeError: The @ray.remote decorator must be applied to either a function or a class.
from prefect-ray.
Can you pass in a name to the task to test? @task(name="task_name")
from prefect-ray.
Okay that didn't seem to work either:
import ray
from prefect import task
@task(name="abc")
@ray.remote(num_gpus=1) # or maybe reverse the decorators
def use_gpu():
print("ray.get_gpu_ids(): {}".format(ray.get_gpu_ids()))
print("CUDA_VISIBLE_DEVICES: {}".format(os.environ["CUDA_VISIBLE_DEVICES"]))
AttributeError: 'RemoteFunction' object has no attribute '__qualname__'
from prefect-ray.
With Dask, we expect users to use Dask's config context manager. We could do something similar here...
@flow
def foo():
with ray_remote_settings(num_gpus=1):
my_task()
my_other_task()
I believe we'd need to implement this ourselves. It'd be pretty easy though. We'd then pass the remote settings to the remote
invocation in the task runner?
from prefect-ray.
Related Issues (20)
- `RayTaskRunner` crashes with `pydantic>=2.0.0` HOT 3
- Python 3.10 support
- `timeout_seconds` in `task` doesn't work with `RayTaskRunner` HOT 2
- First class results error using remote ray HOT 1
- Flow crashes after multiple hanged tasks with "Prefect Task run TASK_RUN_ID already finished" HOT 5
- Ensure that logs from Ray workers are propagated to the Prefect backend HOT 3
- Recognize Ray Cluster defined via env var RAY_ADDRESS HOT 2
- Ray is stuck at the older version
- Circular Import on Deployment (again) HOT 1
- AWS EC2 Spot Interrupt fails entire flow HOT 5
- Flows with many tasks unable to complete HOT 6
- Flow code executes before dependencies present on Ray cluster
- Prefect-Ray spins up workers before the inputs for the worker tasks are ready / before the task's dependencies are complete HOT 6
- Propagating prefect task names to ray tasks instead of it showing up as _run_prefect_task in the Ray dashboard HOT 6
- Prefect Logger does not work with native Ray Functions HOT 1
- how to set resources on @task HOT 2
- Return object references from tasks?
- prefect worker start with resources ?
- Specifying resources for task causes flow to fail/crash
- Error in test suite
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from prefect-ray.