dotnet / torchsharpexamples Goto Github PK
View Code? Open in Web Editor NEWRepository for TorchSharp examples and tutorials.
License: MIT License
Repository for TorchSharp examples and tutorials.
License: MIT License
D:\project
│ README.md
│
└───libtorch-cpu
│
└───libtorch-cuda-11.3
│
└───TorchSharp
│
└───TorchSharp.Notebooks
│ │ tutorial1.ipynb
│ │ tutorial2.ipynb
Within e.g. tutorial2.ipynb, how to define i# to load e.g
TorchSharp-cpu
orTorchSharp-cuda-windows
from the local (git clone) TorchSharp folder?
Instead of loading them from the PC default user's nuget folder
#r "nuget: TorchSharp-cpu"
#r "nuget: TorchSharp-cuda-windows"
Hi 👋
A few tutorials make use of the str
method.
It appears a recent change to TorchSharp breaks optional argument resolution.
These two tutorials make use of str
https://github.com/dotnet/TorchSharpExamples/blob/main/tutorials/FSharp/tutorial2.ipynb
https://github.com/dotnet/TorchSharpExamples/blob/main/tutorials/CSharp/tutorial2.ipynb
I've also opened an issue in the TorchSharp repo.
dotnet/TorchSharp#628
Originally posted in dotnet/TorchSharp by @jimquittenton:
The naming scheme for layers are different in the ResNet example model found in this repo and the ResNet models found in TorchVision, which prevents a model saved from Python from being loaded in TorchSharp using this example code.
Original post:
Hi,
I'm new to TorchSharp and am having trouble loading a python trained ResNet18 model. I've been following this article: https://github.com/dotnet/TorchSharp/blob/main/docfx/articles/saveload.md and have exported my python model using the 'save_state_dict' function in this script: https://github.com/dotnet/TorchSharp/blob/main/src/Python/exportsd.py .
In TorchSharp I have copied the ResNet model from https://github.com/dotnet/TorchSharpExamples/blob/main/src/CSharp/Models/ResNet.cs and then call the following:
int numClasses = 3;
ResNet myModel = ResNet.ResNet18(numClasses);
myModel.to(DeviceType.CPU);
myModel.load(mPath);
The load() line throws an exception with message Mismatched module state names: the target modules does not have a submodule or buffer named 'conv1.weight'.
If I examine the state_dict from 'myModel' prior to load(), it contains entries like:
{[layers.conv2d-first.weight, {TorchSharp.Modules.Parameter}]}
{[layers.bnrm2d-first.weight, {TorchSharp.Modules.Parameter}]}
{[layers.bnrm2d-first.bias, {TorchSharp.Modules.Parameter}]}
{[layers.bnrm2d-first.running_mean, {TorchSharp.torch.Tensor}]}
{[layers.bnrm2d-first.running_var, {TorchSharp.torch.Tensor}]}
{[layers.bnrm2d-first.num_batches_tracked, {TorchSharp.torch.Tensor}]}
{[layers.blck-64-0.layers.blck-64-0-conv2d-1.weight, {TorchSharp.Modules.Parameter}]}
{[layers.blck-64-0.layers.blck-64-0-bnrm2d-1.weight, {TorchSharp.Modules.Parameter}]}
{[layers.blck-64-0.layers.blck-64-0-bnrm2d-1.bias, {TorchSharp.Modules.Parameter}]}
whereas the corresponding entries prior to saving from python are:
conv1.weight torch.Size([64, 3, 7, 7])
bn1.weight torch.Size([64])
bn1.bias torch.Size([64])
bn1.running_mean torch.Size([64])
bn1.running_var torch.Size([64])
bn1.num_batches_tracked torch.Size([])
layer1.0.conv1.weight torch.Size([64, 64, 3, 3])
layer1.0.bn1.weight torch.Size([64])
layer1.0.bn1.bias torch.Size([64])
I tried amending the ResNet.cs code to reflect the python names, but could not get them to exactly match.
I also tried calling load() with strict=false myModel.load(mPath, false);. This seemed to get past the Mismatched names exception, but throws another exception with message Too many bytes in what should have been a 7 bit encoded Int32.
I've been struggling with this for a couple of days now so would really appreciate any help you guys could offer.
Thanks
Jim
looks like accidental output run file from @NiklasGustafsson
Hi @NiklasGustafsson, I would like to contribute to TensorSharp as we are looking at using it to replace our CNTK usage in our full end-to-end machine learning pipelines written in C#. I have been looking at this example repo in that regard, which is a great starting place. I've mainly work with image models so I've been looking at the CIFAR10 example. I understand that these examples have been created quickly and that they are bare-bones, I would like to improve them. :)
For example, I have a few issues with the Reader
s like CIFAR10Reader
and how they both randomize data by pre-defining randomized batches, which is not normally how you would do this, you create unique random batches for each epoch. Similarly, an epoch would usually (nothing is standardized here and you really can do anything you'd like so this is just IMHO) be defined by iterating our the samples of the dataset once, not by adding transforms after and hence multiplying by that like:
public IEnumerable<(Tensor, Tensor)> Data()
{
for (var i = 0; i < data.Count; i++) {
yield return (data[i], labels[i]);
foreach (var tfrm in _transforms) {
yield return (tfrm.forward(data[i]), labels[i]);
}
}
}
Also you wouldn't "transform" or augment the data if it is test data, of course you can then just not set the transforms.
Anyway, I was thinking my first contribution could be to refactor the readers to and implement concepts similar to pytorch DataSet
and DataLoader
. I have worked with this API but am not an expert, nor I am necessarily a fan of the python APIs, but it seems you'd like to have TorchSharp be similar to pytorch, so basing it on that makes sense. Would that be of any interest?
Before doing this I would very much like to migrate this example repo to .NET 6 and C# 10 too and follow standard C# code guidelines and use modern language features, to really make the examples shine with regards to C#. Since performance is my passion I'd also like the examples to at least minimally try to be efficient about what happens, even in cases where it does not matter so much.
Just an example below, I would replace the below with a proper Fischer-Yates shuffle, that is easy to implement.
Enumerable.Range(0, count).OrderBy(c => rnd.Next()).ToArray();
Reproducibility is important too, so all random stuff should be seeded.
Sorry, I am sure you know all this, but I wanted to at least ask whether such changes are of interest first? If you guys agree with them?
To recap I propose:
And we can take it from there.
https://github.com/dotnet/TorchSharpExamples/blob/main/tutorials/CSharp/tutorial2.ipynb
torch.zeros(4,4).print(style);
torch.ones(4,4).print(style);
(1,24): error CS1503: Argument 2: cannot convert from 'TorchSharp.TensorStringStyle' to 'string'
(2,23): error CS1503: Argument 2: cannot convert from 'TorchSharp.TensorStringStyle' to 'string'
Error: compilation error
Hello, I am trying to translate this project into Chinese while studying. How can I contribute the results? Or create a new repository?
The community submitted PR for a tutorial on SQuADv2.0 could be related to squad-2-0 and question-answer
This is part of TorchText. Is there a need to start organizing the TorchSharpExamples around folder structures that align with TorchText?
Making HiddenLayer available to .NET notebook [mid to long term vision]?
A lightweight library for neural network graphs and training metrics for ML.NET, TorchSharp and perhaps Tensorflow.NET for .NET notebook
This is an exploratory feedback/discussion to democratize torchsharp tutorials/examples through github pages ( most likely via Blazor WebAssembly)
Discussion towards possible path towards TorchSharp.Native.WebAssembly nuget
Is there interest to port
torch-summary to TorchSharp.Summary
from torchsummary import summary
model = ConvNet()
summary(model, (1, 28, 28))
So in TorchSharp notebook, we get:
==========================================================================================
Layer (type:depth-idx) Output Shape Param #
==========================================================================================
├─Conv2d: 1-1 [-1, 10, 24, 24] 260
├─Conv2d: 1-2 [-1, 20, 8, 8] 5,020
├─Dropout2d: 1-3 [-1, 20, 8, 8] --
├─Linear: 1-4 [-1, 50] 16,050
├─Linear: 1-5 [-1, 10] 510
==========================================================================================
Total params: 21,840
Trainable params: 21,840
Non-trainable params: 0
==========================================================================================
Input size (MB): 0.00
Forward/backward pass size (MB): 0.05
Params size (MB): 0.08
Estimated Total Size (MB): 0.14
==========================================================================================
The resnet class https://github.com/dotnet/TorchSharpExamples/blob/main/src/CSharp/Models/ResNet.cs in all the initializers has a "int numClasses," variable, but it is passed on to the main constructer as a hard wired "10" and the numClasses variable isn't actually used.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.