Comments (7)
Q: Did you set object_store_memory to a float?
A: Yes, as you can see from the the reproduction script:
@serve.deployment(ray_actor_options={"object_store_memory": 20 * 1024 * 1024})
I think this line of code is the reason that cause the bug, which convert the value of object_store_memory
to float, I think change it from object_store_memory: float
to object_store_memory: int
will fix this bug.
from ray.
I think this bug is caused by this line of code, after I change the value from _counting_option("object_store_memory", False)
to _resource_option("object_store_memory")
the error fixed.
from ray.
@aRyBernAlTEglOTRO did you set object_store_memory
to a float
? Or did you set it to an int
and then see this error?
It looks like this option is expected to be an int
instead of a float
(code link). If Serve is converting it to a float
, then that sounds like a bug.
from ray.
It turns out object_store_memory
is not supported in Ray Core (see issue). I'm reopening this issue until we decide how to handle it in Ray Serve.
from ray.
@aRyBernAlTEglOTRO We're planning to remove object_store_memory
in Ray Serve. The option is not supported in Ray Core, so setting it is a no-op in Serve. Could you explain your use case for object_store_memory
, so we can see if there's another way to accomplish it?
from ray.
@shrekris-anyscale Thank you for your effort. The reason that I use object_store_memory
is due to that I met some memory leak issue when I used Ray Serve. Therefore I searched around for some parameters which can control the memory usage of deployment, at that time I encontered the object_store_memory
, but even after I set this parameter, the memory leak issue still exist.
from ray.
Can you file a separate issue for the memory leak? Do you know if it's happening in the Ray Serve code, or the application code?
from ray.
Related Issues (20)
- CI test linux://rllib:learning_tests_multi_agent_cartpole_ppo_multi_gpu is flaky HOT 1
- CI test linux://rllib:learning_tests_multi_agent_pendulum_ppo_multi_gpu is flaky HOT 1
- CI test linux://rllib:learning_tests_pendulum_ppo_multi_gpu is flaky HOT 1
- CI test linux://rllib:learning_tests_cartpole_impala_multi_gpu is flaky HOT 1
- [Core] `NodeId` and `NodeIp` tags don't match the expected instance HOT 1
- [Data] Support async callable classes in map_batches()
- [Ray Dashboard] The format of the GPU profile file downloaded from the dashboard is incorrect
- [Ray Core] Get the list of failed remote tasks
- [RLlib] - `Algorithm.add_module` does not use the `module_state` argument. HOT 1
- [autoscaler][gcp] wrong values for scheduling in example gcp cluster yaml files
- [<Ray component: RLlib] module 'numpy' has no attribute 'product'
- [Serve] does not work with tracing
- Test linux://python/ray/tests:test_channel is flaky
- [Data] Additional metrics for tracking block times
- [Ray Core] Actor On Finish Function
- [Ray Core]: Cannot find gpu on Jetson AGX Orin HOT 2
- [Dashboard] Jobs page breaks when multiple drivers associated with same job submission id
- [in-progress] Readers across different nodes - ADAG Developer Preview - Test Coverage HOT 1
- [Data] Support read Hudi table as a DataSource
- [core][experimental] Support binding kwargs to DAG nodes in accelerated DAG
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ray.