Comments (4)
Also, the use of torch.tensor must be prohibited to convert numpy array to torch tensor since it is less efficient and break memory sharing on cpu.
from tianshou.
Also, the use of torch.tensor must be prohibited to convert numpy array to torch tensor since it is less efficient and break memory sharing on cpu.
For most of the scenarios, the agent's action contains only a few elements that could be considered as negligible.
Currently, the replay buffer is stored as np.ndarray. If you want to prohibit the conversion, the underlying data structure should change to torch.tensor. But I think it is not a good approach since the memory in GPU is far less than RAM, and some basic operations (e.g. compute the returns) are more efficient in the CPU side.
from tianshou.
I'm not saying to use only torch.tensor, but convert them using torch.from_numpy
. Sorry for lack of clarity :/
from tianshou.
@Trinkle23897 The PR should be ready right now. I try to do the minimal modifications to fully support Batch from/to numpy/pytorch.
from tianshou.
Related Issues (20)
- Chinese document pages return 404 HOT 4
- data recording and saving method HOT 4
- Typing annotations of step from MyTestEnv is incompatible with its current subclass gym.Env because it can generate non-scalar rewards.
- How to monitor the episode/epoch return/length in Tianshou? HOT 1
- Replicating results in collect random operations through seed setting HOT 2
- Batch: deprecate setattr HOT 1
- Batch: don't create new objects on getitem HOT 8
- Batch: only allow entries with the same length HOT 3
- Batch: don't just set 0 when elements have None entries HOT 8
- Batch: don't just strip off empty entries when creating batches HOT 5
- Buffer: fix discrepancy in slicing order HOT 2
- Better interfaces and names for Actor, Critic, Net and other classes
- Reduce duplication between examples/atari/atari_network and examples/vizdoom/network HOT 1
- Fix docstring in BranchingNet
- Re-examine the need of utils.net.common.DataParallelNet
- Re-examine the whole state story for RNNs
- Don't pass envpool envs where vectorenvs are needed
- Missing Link HOT 5
- AttributeError: 'PPOPolicy' object has no attribute 'set_eps' HOT 2
- /test/continuous/test_ppo.py TypeError on torch.distributions HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tianshou.