Comments (2)
PyTorch's torch.utils.data.random_split
returns torch.utils.data.Subset
which is not an instance of torchdata.Dataset
, hence it has no map
functionality.
What one could do instead is:
dataset = torchvision.datasets.ImageFolder("./root")
total_num = len(dataset)
train_num = int(0.7 * total_num)
val_num = int(0.2 * total_num)
test_num = total_num - train_num - val_num
train_dataset, val_dataset, test_dataset = [
td.datasets.WrapDataset(dataset)
for dataset in torch.utils.data.random_split(
dataset, (train_num, val_num, test_num)
)
]
train_dataset = train_dataset.map(train_transform)
from torchdatasets.
the docs show a different way of using this - why? Docs example doesnt work as random_split returns Subset.
I have also not been able to get the mapped torchvision transform example in the docs to work. simply nothing happens. I think this needs an update, or is deprecated with new torch versions?
from torchdatasets.
Related Issues (16)
- pip install doesn't work in Google Colab HOT 3
- Pickle support for Storage will be removed in torch 1.5 HOT 2
- HDF5 Support HOT 1
- Concatenate datasets with different map function HOT 1
- Benchmark
- metaclass conflict HOT 6
- load N samples in memory (queue) and train on it on GPU. In the meanwhile, load another N samples into queue HOT 2
- Support stratified subsampler HOT 1
- Multiple concatenation with logical or operator yields nested concatenation
- Document gotcha when using DataLoader with workers
- Metaclass issue with Python 3.7/3.8 HOT 2
- TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases HOT 2
- Apparent mismatch between official pip version `0.2.0` and GitHub tagged version of `0.2.0` HOT 1
- python3.6 order (MRO) for bases type, GenericMeta, _DataPipeMeta HOT 1
- support for pytorch 1.3.0 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from torchdatasets.