anyscale / academy Goto Github PK
View Code? Open in Web Editor NEWRay tutorials from Anyscale
Home Page: https://anyscale.com
License: Apache License 2.0
Ray tutorials from Anyscale
Home Page: https://anyscale.com
License: Apache License 2.0
It shows last year's Ray Summit.
The very first example that Tune course runs is defined like this:
analysis = tune.run(
"PPO", # Use proximal policy optimization to train
stop={"episode_reward_mean": 400}, # Stopping criteria, when average reward over the episodes
# of training equals 400 out of a maximum possible 500 score.
config={
"env": "CartPole-v1", # Tune can associate this string with the environment.
"num_gpus": 0, # If you have GPUs, go for it!
"num_workers": 3, # Number of Ray workers to use; Use one LESS than
# the number of cores you wan to use (or omit this argument)!
"model": { # The NN model we'll optimize.
'fcnet_hiddens': [ # "Fully-connected network with N hidden layers".
tune.grid_search([20, 40]), # Try these four values for layer one.
tune.grid_search([20, 40]) # Try these four values for layer one.
]
},
"eager": False, # Flag for TensorFlow; don't use eager evaluation.
},
verbose=1
)
This failed for me, saying that Tensorflow doesn't recognize eager
parameter. So I've removed it, and it failed again, this time there was some TF/Numpy conversion issue. Then I found this issue , and it confirms that eager
should not be used anymore. Following the conversation there, I got the following working code:
analysis = tune.run(
"PPO", # Use proximal policy optimization to train
stop={"episode_reward_mean": 400}, # Stopping criteria, when average reward over the episodes
# of training equals 400 out of a maximum possible 500 score.
config={
"env": "CartPole-v1", # Tune can associate this string with the environment.
"num_gpus": 0, # If you have GPUs, go for it!
"num_workers": 3, # Number of Ray workers to use; Use one LESS than
# the number of cores you wan to use (or omit this argument)!
"model": { # The NN model we'll optimize.
'fcnet_hiddens': [ # "Fully-connected network with N hidden layers".
tune.grid_search([20, 40]), # Try these four values for layer one.
tune.grid_search([20, 40]) # Try these four values for layer one.
]
},
"framework": "tfe",
#"eager": False, # Flag for TensorFlow; don't use eager evaluation.
},
verbose=1
)
Not sure if this is the proper fix, so just an issue here and not a PR. But hopefully that helps the next person going through the Tune course.
Although the grids of Games of Life should be randomly independent, there is a weird repeating pattern every six games, which becomes evident after they have evolved a while. It's not obvious they start out identical.
Hello
I spotted small typos in the ray-crash-course.
The naming and the link of item 03 are incorrect.
academy/ray-crash-course/00-Ray-Crash-Course-Overview.ipynb
Lines 66 to 70 in cb8938d
"| 03 | [Ray Objects](03-Ray-Objects.ipynb)
in Example 2: Adding two np arrays
section. I assume this should be return np.add(arr1, arr2)
academy/ray-crash-course/01-Ray-Tasks.ipynb
Lines 286 to 287 in cb8938d
For Tune's PopulationBasedTraining, Ray doc says checkpoint is needed (to load and save weights etc), but academy pbt example given doesn't have it. Why?
I have a gather Actor which takes all outputs of previous Actor:
@ray.remote
class GatherAll:
def __init__(self) -> None:
pass
# def infer(self, hands, kpts):
# if hands.shape[0] > kpts.shape[0]:
# out = hands.copy()
# out[:kpts.shape[0], :] += kpts[..., :4]
# return out
# else:
# out = kpts.copy()[..., :4]
# out[:hands.shape[0], :] += hands
# return out
def infer(self, datas):
hands = datas['a']
kpts = datas['b']
if hands.shape[0] > kpts.shape[0]:
out = hands.copy()
out[:kpts.shape[0], :] += kpts[..., :4]
return out
else:
out = kpts.copy()[..., :4]
out[:hands.shape[0], :] += hands
return out
the uncommented parts not work, which I call it in dict:
# out = G.infer.remote(hands, kpts)
out = G.infer.remote({'a': hands, 'b': kpts})
Get error:
AttributeError: 'ray._raylet.ObjectRef' object has no attribute 'shape'
Why using separated is OK, but dictionary is not ? Very confused here.
Using Tune's pbt, result analysis.dataframe() returned by tune.run doesn't contain experiment_tag that differentiate the parameters used. Instead all parameters are shown the same values as bootstrap.
In ray-rllib/recsys/01-Ressys, i found there may be a problem in calculate the distance between user's ratings to cluster's centers.
It is in the step function in env class JokeRec:
scaled_diff = abs(c[item] - rating) / 2.0
The shape for c (which is centers[i]) is 1* 24983, stands for the features in i cluster. However, item is is randomly chosen from the cluster, and the range is [0, 99]. The rest [100, 24983] in the center[i] cannot be searched. Is c[item] - rating a correct way to calculate that distance?
Hello,
The argument num_return_vals is no longer valid in ray functions but tutorials still use it. I suggest updating them accordingly.
Cheers.
In the README, we have the command
tools/cleanup.sh | while read x; do rm -rf $x; done
This seems dangerous in case the user runs it from the wrong location or something. Can we get rid of this?
Set locale like locale.setlocale(locale.LC_ALL, locale.getlocale())
here fails using python3.6 from the pip installation.
Example:
import locale
import sys
print(locale.getlocale())
print(sys.version)
locale.setlocale(locale.LC_ALL, locale.getlocale())
Returns:
('en_US', 'UTF-8')
3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0]
---------------------------------------------------------------------------
Error Traceback (most recent call last)
<ipython-input-18-e1d930bcc3dd> in <module>
4 print(locale.getlocale())
5 print(sys.version)
----> 6 locale.setlocale(locale.LC_ALL, locale.getlocale())
/code/venvs/venv/lib/python3.6/locale.py in setlocale(category, locale)
596 # convert to string
597 locale = normalize(_build_localename(locale))
--> 598 return _setlocale(category, locale)
599
600 def resetlocale(category=LC_ALL):
Error: unsupported locale setting
Resolved from the docs suggestion to set like locale.setlocale(locale.LC_ALL, '')
, but not sure if that has the same behavior. Could just require users not to use python3.6 in the README.
Thanks for the excellent tutorial! The live plots are not updating (Mac OS). Any ideas on how to get them working or how to debug it?
I tried to run
obj_ref_arr1 = read_array.remote("data/file_1.txt") print(f"array 1: {obj_ref_arr1}")
and received
raise IOError("%s not found." % path) OSError: data/file_1.txt not found.
Hi, and thanks for the course! Very nice material to work through.
The issue is about the very beginning of the first notebook in the crash course. Here is what the first section of the notebook outputs on my laptop:
At least on my laptop this is unreadable, black letters on deep dark blue background. Is this machine specific? Perhaps I didn't properly install something?
Hi, these are all great resources. Thank you very much for putting this together.
There is one link that might have been moved:
https://github.com/anyscale/academy/blob/master/rllib/00-Overview-Ray-RLlib.ipynb
The above link gives back 404-page not found
Here:
| A developer or data scientist interested in Reinforcement Learning | Ray RLlib |
Hello, Anyscale.
Ray is a popular library in Korea, too. So I will contribute by translating Korean users so that they can easily study through this tutorial. Where should I put the translated notebook file?
Thank you!
I am very sure this is not the right place to ask this question, but still,
can anyone tell me whether RAY CLUSTER works in HA or not?
What I mean is, what is the ray head node is down, whether the next leader selection happens ?
I get an AssertionError
when calling run_simulations(dmaps)
in the 01-Ray-Tasks.ipynb:
I am running locally from a conda env (followed exactly the Setup section of the README). The simulation seems to run though unaffected by the exception...
Here is the output of the traceback including a related warnings:
WARNING:param.dynamic_operation: Callable raised "AssertionError('New option id 2951106455928858 does not match any option trees in Store.custom_options.')".
Invoked as dynamic_operation(counter=0)
WARNING:param.dynamic_operation: Callable raised "AssertionError('New option id 2951106455928858 does not match any option trees in Store.custom_options.')".
Invoked as dynamic_operation(counter=0)
Traceback (most recent call last):
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/plotting/util.py", line 275, in get_plot_frame
return map_obj[key]
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 1341, in __getitem__
val = self._execute_callback(*tuple_key)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 1110, in _execute_callback
retval = self.callback(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 714, in __call__
ret = self.callable(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 1042, in dynamic_operation
key, obj = resolve(key, kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 1031, in resolve
return key, map_obj[key]
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 1341, in __getitem__
val = self._execute_callback(*tuple_key)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 1110, in _execute_callback
retval = self.callback(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 714, in __call__
ret = self.callable(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 1043, in dynamic_operation
return apply(obj, *key, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 1035, in apply
processed = self._process(element, key, kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 1019, in _process
return self.p.operation(element, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 625, in <lambda>
dmap = Dynamic(obj, operation=lambda obj, **dynkwargs: obj.opts(*args, **kwargs),
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 37, in pipelined_call
return __call__(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 573, in __call__
return self._dispatch_opts( *args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 577, in _dispatch_opts
return self._base_opts(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 656, in _base_opts
return self._obj.options(*new_args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/dimension.py", line 1307, in options
obj = obj.opts._dispatch_opts(expanded, backend=backend, clone=clone)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 577, in _dispatch_opts
return self._base_opts(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 653, in _base_opts
return opts.apply_groups(self._obj, **dict(kwargs, **new_kwargs))
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 244, in apply_groups
obj = cls._apply_groups_to_backend(obj, backend_opts, backend, clone)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 169, in _apply_groups_to_backend
return StoreOptions.set_options(obj_handle, options, backend=backend)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/options.py", line 1843, in set_options
applied = cls.propagate_ids(obj, match_id, new_id, compositor_applied+list(spec.keys()), backend=backend)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/options.py", line 1511, in propagate_ids
% new_id)
AssertionError: New option id 2951106455928858 does not match any option trees in Store.custom_options.
WARNING:param.dynamic_operation: Callable raised "AssertionError('New option id 4286036670735367693583064832671926448157 does not match any option trees in Store.custom_options.')".
Invoked as dynamic_operation(counter=0)
WARNING:param.dynamic_operation: Callable raised "AssertionError('New option id 4286036670735367693583064832671926448157 does not match any option trees in Store.custom_options.')".
Invoked as dynamic_operation(counter=0)
Traceback (most recent call last):
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/plotting/util.py", line 275, in get_plot_frame
return map_obj[key]
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 1341, in __getitem__
val = self._execute_callback(*tuple_key)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 1110, in _execute_callback
retval = self.callback(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 714, in __call__
ret = self.callable(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 1042, in dynamic_operation
key, obj = resolve(key, kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 1031, in resolve
return key, map_obj[key]
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 1341, in __getitem__
val = self._execute_callback(*tuple_key)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 1110, in _execute_callback
retval = self.callback(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/spaces.py", line 714, in __call__
ret = self.callable(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 1043, in dynamic_operation
return apply(obj, *key, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 1035, in apply
processed = self._process(element, key, kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 1019, in _process
return self.p.operation(element, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 625, in <lambda>
dmap = Dynamic(obj, operation=lambda obj, **dynkwargs: obj.opts(*args, **kwargs),
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 37, in pipelined_call
return __call__(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 573, in __call__
return self._dispatch_opts( *args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 577, in _dispatch_opts
return self._base_opts(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 656, in _base_opts
return self._obj.options(*new_args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/dimension.py", line 1307, in options
obj = obj.opts._dispatch_opts(expanded, backend=backend, clone=clone)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 577, in _dispatch_opts
return self._base_opts(*args, **kwargs)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/accessors.py", line 653, in _base_opts
return opts.apply_groups(self._obj, **dict(kwargs, **new_kwargs))
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 244, in apply_groups
obj = cls._apply_groups_to_backend(obj, backend_opts, backend, clone)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/util/__init__.py", line 169, in _apply_groups_to_backend
return StoreOptions.set_options(obj_handle, options, backend=backend)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/options.py", line 1843, in set_options
applied = cls.propagate_ids(obj, match_id, new_id, compositor_applied+list(spec.keys()), backend=backend)
File "/Users/matus/miniconda3/envs/anyscale-academy/lib/python3.7/site-packages/holoviews/core/options.py", line 1511, in propagate_ids
% new_id)
AssertionError: New option id 4286036670735367693583064832671926448157 does not match any option trees in Store.custom_options.
In part @ray.remote()
,the argument named num_return_vals
is different from that in the Ray Docs.
When I execute the sample code given in notebook:
@ray.remote(num_return_vals=3)
def tuple3(one, two, three):
return (one, two, three)
...
The following error will appear:
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-9-05124ba20d56> in <module>
----> 1 @ray.remote(num_return_vals=3)
2 def tuple3(one, two, three):
3 return (one, two, three)
4
5 x_ref, y_ref, z_ref = tuple3.remote("a", 1, 2.2)
~/opt/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/worker.py in remote(*args, **kwargs)
1926 "max_task_retries",
1927 "max_retries",
-> 1928 ], error_string
1929
1930 num_cpus = kwargs["num_cpus"] if "num_cpus" in kwargs else None
AssertionError: The @ray.remote decorator must be applied either with no arguments and no parentheses, for example '@ray.remote', or it must be applied using some of the arguments 'num_returns', 'num_cpus', 'num_gpus', 'memory', 'object_store_memory', 'resources', 'max_calls', or 'max_restarts', like '@ray.remote(num_returns=2, resources={"CustomResource": 1})'.
I checked the documentation Ray v2.0.0.dev0 and the correct parameters should be as follows:
@ray.remote(num_returns=3)
...
When I run first cell:
from task_lesson_util import make_dmaps, run_simulations, stop_simulations
from pi_calc import str_large_n
It appears following error:
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-1-9c851064aaf9> in <module>
----> 1 from task_lesson_util import make_dmaps, run_simulations, stop_simulations
2 from pi_calc import str_large_n
~/code/ray/academy/ray-crash-course/task_lesson_util.py in <module>
15 from bokeh.plotting import figure, output_file, show
16
---> 17 from pi_calc import MonteCarloPi, compute_pi_for
18
19
~/code/ray/academy/ray-crash-course/pi_calc.py in <module>
104
105 @ray.remote
--> 106 class RayMonteCarloPi(MonteCarloPi):
107 @ray.method(num_returns=5)
108 def sample(self, num_samples):
~/code/ray/academy/ray-crash-course/pi_calc.py in RayMonteCarloPi()
105 @ray.remote
106 class RayMonteCarloPi(MonteCarloPi):
--> 107 @ray.method(num_returns=5)
108 def sample(self, num_samples):
109 return super().sample(num_samples)
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/actor.py in method(*args, **kwargs)
37 assert len(args) == 0
38 assert len(kwargs) == 1
---> 39 assert "num_return_vals" in kwargs
40 num_return_vals = kwargs["num_return_vals"]
41
AssertionError:
What's the problem? Could you please help!
I follow the suggestion ,
config["framework"] = "torch"
config["num_gpus"] = 0.001 # can't work
config["num_gpus_per_worker"] = (1 - 0.001) / 4
but I get the error,
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-33-c1627876193e> in <module>
----> 1 agent = ppo.PPOTrainer(config, env=SELECT_ENV)
2
3 results = []
4 episode_data = []
5 episode_json = []
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py in __init__(self, config, env, logger_creator)
121
122 def __init__(self, config=None, env=None, logger_creator=None):
--> 123 Trainer.__init__(self, config, env, logger_creator)
124
125 def _init(self, config: TrainerConfigDict,
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/agents/trainer.py in __init__(self, config, env, logger_creator)
546 logger_creator = default_logger_creator
547
--> 548 super().__init__(config, logger_creator)
549
550 @classmethod
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/tune/trainable.py in __init__(self, config, logger_creator)
96
97 start_time = time.time()
---> 98 self.setup(copy.deepcopy(self.config))
99 setup_time = time.time() - start_time
100 if setup_time > SETUP_TIME_THRESHOLD:
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/agents/trainer.py in setup(self, config)
707
708 with get_scope():
--> 709 self._init(self.config, self.env_creator)
710
711 # Evaluation setup.
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py in _init(self, config, env_creator)
153 policy_class=self._policy_class,
154 config=config,
--> 155 num_workers=self.config["num_workers"])
156 self.execution_plan = execution_plan
157 self.train_exec_impl = execution_plan(self.workers, config)
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/agents/trainer.py in _make_workers(self, env_creator, validate_env, policy_class, config, num_workers)
795 trainer_config=config,
796 num_workers=num_workers,
--> 797 logdir=self.logdir)
798
799 @DeveloperAPI
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/evaluation/worker_set.py in __init__(self, env_creator, validate_env, policy_class, trainer_config, num_workers, logdir, _setup)
98 num_workers=num_workers,
99 config=self._local_config,
--> 100 spaces=spaces,
101 )
102
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/evaluation/worker_set.py in _make_worker(self, cls, env_creator, validate_env, policy_cls, worker_index, num_workers, config, spaces)
405 fake_sampler=config["fake_sampler"],
406 extra_python_environs=extra_python_environs,
--> 407 spaces=spaces,
408 )
409
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py in __init__(self, env_creator, validate_env, policy_spec, policy_mapping_fn, policies_to_train, tf_session_creator, rollout_fragment_length, count_steps_by, batch_mode, episode_horizon, preprocessor_pref, sample_async, compress_observations, num_envs, observation_fn, observation_filter, clip_rewards, clip_actions, env_config, model_config, policy_config, worker_index, num_workers, record_env, log_dir, log_level, callbacks, input_creator, input_evaluation, output_creator, remote_worker_envs, remote_env_batch_wait_ms, soft_horizon, no_done_at_end, seed, extra_python_environs, fake_sampler, spaces, policy, monitor_path)
535 else:
536 self.policy_map, self.preprocessors = self._build_policy_map(
--> 537 policy_dict, policy_config)
538
539 # Update Policy's view requirements from Model, only if Policy directly
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py in _build_policy_map(self, policy_dict, policy_config)
1194 # non-tf.
1195 else:
-> 1196 policy_map[name] = cls(obs_space, act_space, merged_conf)
1197
1198 if self.worker_index == 0:
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/policy/policy_template.py in __init__(self, obs_space, action_space, config)
265 action_distribution_fn=action_distribution_fn,
266 max_seq_len=config["model"]["max_seq_len"],
--> 267 get_batch_divisibility_req=get_batch_divisibility_req,
268 )
269
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/rllib/policy/torch_policy.py in __init__(self, observation_space, action_space, config, model, loss, action_distribution_class, action_sampler_fn, action_distribution_fn, max_seq_len, get_batch_divisibility_req)
153 for i, id_ in enumerate(gpu_ids) if i < config["num_gpus"]
154 ]
--> 155 self.device = self.devices[0]
156 ids = [
157 id_ for i, id_ in enumerate(gpu_ids) if i < config["num_gpus"]
IndexError: list index out of range
And after I debug, i found that ray/rllib/policy/torch_policy.py#L150 ray.get_gpu_ids()
will return a empty list, how can i use gpu for training?
Running the function ('with joblib.parallel_backend('ray'):') causes following error.
error message :
/usr/local/lib/python3.6/dist-packages/ray/util/multiprocessing/pool.py in _convert_to_ray_batched_calls_if_needed(self, func)
621 if isinstance(func, BatchedCalls):
622 func = RayBatchedCalls(func.items, (func._backend, func._n_jobs),
--> 623 func._reducer_callback, func._pickle_cache)
624 # go through all the items and replace args and kwargs with
625 # ObjectRefs, caching them in registries
AttributeError: 'BatchedCalls' object has no attribute '_reducer_callback'
While the exercises are mostly intended to promote exploration, it would be good provide solutions to discuss what can be learned from the explorations. Follow the example of other solutions notebooks...
Hey there,
Just having issues with Cell 9 of the 03-Simple-Multi-Armed-Bandit notebook. Everything else runs fine except until that point. In particular, I have ray intialized correctly and the following
ray.init(address='auto', ignore_reinit_error=True)
generates:
{'node_ip_address': '192.168.1.105', 'raylet_ip_address': '192.168.1.105', 'redis_address': '192.168.1.105:6379', 'object_store_address': '/tmp/ray/session_2020-07-18_20-05-29_369775_4270/sockets/plasma_store', 'raylet_socket_name': '/tmp/ray/session_2020-07-18_20-05-29_369775_4270/sockets/raylet', 'webui_url': 'localhost:8265', 'session_dir': '/tmp/ray/session_2020-07-18_20-05-29_369775_4270'}
The error occurs as follows:
start_time = time.time()
analysis = ray.tune.run("contrib/LinUCB", config=config, stop=stop, progress_reporter=JupyterNotebookReporter(overwrite=False), # This is the default, actually. verbose=2, # Change to 0 or 1 to reduce the output. ray_auto_init=False, # Don't allow Tune to initialize Ray. )
Generates:
`== Status ==
Memory usage on this node: 7.5/15.6 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/4 CPUs, 0/2 GPUs, 0.0/8.06 GiB heap, 0.0/2.78 GiB objects
Result logdir: /home/paul/ray_results/contrib/LinUCB
Number of trials: 1 (1 ERROR)
Trial name status loc
contrib_LinUCB_SimpleContextualBandit_65d05_00000 ERROR
Number of errored trials: 1
Trial name # failures error file
contrib_LinUCB_SimpleContextualBandit_65d05_00000 1 /home/paul/ray_results/contrib/LinUCB/contrib_LinUCB_SimpleContextualBandit_0_2020-07-19_00-46-00_376v7hp/error.txt
TuneError Traceback (most recent call last)
in
4 progress_reporter=JupyterNotebookReporter(overwrite=False), # This is the default, actually.
5 verbose=2, # Change to 0 or 1 to reduce the output.
----> 6 ray_auto_init=False, # Don't allow Tune to initialize Ray.
7 )
~/anaconda3/envs/anyscale-academy/lib/python3.7/site-packages/ray/tune/tune.py in run(run_or_experiment, name, stop, config, resources_per_trial, num_samples, local_dir, upload_dir, trial_name_creator, loggers, sync_to_cloud, sync_to_driver, checkpoint_freq, checkpoint_at_end, sync_on_checkpoint, keep_checkpoints_num, checkpoint_score_attr, global_checkpoint_period, export_formats, max_failures, fail_fast, restore, search_alg, scheduler, with_server, server_port, verbose, progress_reporter, resume, queue_trials, reuse_actors, trial_executor, raise_on_failed_trial, return_trials, ray_auto_init)
347 if incomplete_trials:
348 if raise_on_failed_trial:
--> 349 raise TuneError("Trials did not complete", incomplete_trials)
350 else:
351 logger.error("Trials did not complete: %s", incomplete_trials)
TuneError: ('Trials did not complete', [contrib_LinUCB_SimpleContextualBandit_65d05_00000])
`
The current Docker image support only runs on the Anyscale platform. All the pieces are in place for local use, but properly configuring the image for Jupyter Lab execution, including exposing the correct ports, needs to be done.
centers dimension is like [clusters, users], c dimension will become [users], why do we get c[item]?
For example, if we have cluster =7, users=2000, items=101, shouldn't we get c[user] instead of c[item]?
class JokeRec (gym.Env):
def step(self):
...
# update the coords history: agent observes its distance to each cluster "evolve"
for i in range(len(self.coords)):
c = self.centers[i]
scaled_diff = abs(c[item] - rating) / 2.0
self.coords[i] += scaled_diff ** 2.0
Hi,
Can you kindly update the environment.yml to reflect the latest ray version?
Currently, it installs ray at '0.8.7' when using conda env create -f environment.yml
while ray is at release 1.2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.