tnbar / tednet Goto Github PK

Considering your expertise, I was wondering if you have any plans to share or upload an API for FLOPs calculation of a Tensor Decomposition Networks on GitHub. Such an API would be incredibly valuable for the community, including researchers and practitioners working on optimizing deep learning models.
If you have already developed such an API or have plans to do so, I would greatly appreciate it if you could share the repository link or provide any details about how others can access and use your FLOPs calculation API.

Hi, it is a good work that summarizes many decomposition methods including tensor train.
I just read the whole code and API document, but I don't find any implementation about decomposing a specific value tensor into TT format directly. I guess currently, the implementation is to set the data format into TT/CP/ or other formats from the beginning and re-train the network from the scratch. Or do I miss some details?

Hi,

Are CP, BTT, Tucker-2, Tensor Train (TT) and Tensor Ring (TR) implemented?

Hi

I am trying to implement your MNIST TR classifier:

# Import Necessary Pytorch Modules
import torch
import torch.nn as nn
from torch import Tensor
from tednet.tnn import tensor_ring as tr


# A Simple MNIST Classifier based on Tensor Ring.
class TRClassifier (nn.Module) :
    def init (self):
        super (TRClassifier, self).init()

        # Define a Tensor Ring Convolutional Layer
        self.trcnn = tr.TRConv2D ([1] , [4, 5] , [ 6, 6, 6, 6], 3)
        # Define a Tensor Ring Fully−Connected Layer
        self.trfc = tr.TRLinear ([20, 26, 26],[10], [6, 6, 6, 6])

    def forward (self, inputs: Tensor ) -> Tensor :
        # Call TRConv2D to process inputs
        out = self.trcnn (inputs)
        out = torch.relu (out)
        out = out.view (inputs.size (0), -1)

        # Call TRLinear to classify the features
        out = self.trfc (out)
        return out

First, the syntax for passing inputs and outputs is not clear in any of your tutorials. I tried the following without the output yet:

TRCls = TRClassifier( X_train)
TRCls

and I am getting the error: TypeError: init() takes 1 positional argument but 2 were given

Second, I would like to do a TR decomposition of a ndarray, such as the TT decomposition syntax elsewhere:

from tensorly.contrib.decomposition import tensor_train_cross
factors = tensor_train_cross(T1, rank)
reconstruction_t = np.round(tt_to_tensor(factors), decimals=10)
reconstruction_t
TT_RMSE = math.sqrt(np.square(np.subtract(T1,reconstruction_t)).mean() )
print ("TT   for rank " + str(rank) + " RMSE = ", TT_RMSE)

or

import t3f
a_tt = t3f.to_tt_tensor(T1, max_tt_rank=2)
reconstruction_t = t3f.full(a_tt)
T3f_TT_RMSE = math.sqrt(np.square(np.subtract(T1,reconstruction_t)).mean() )
print ("T3F TT   for rank " + str(rank) + " RMSE = ", T3f_TT_RMSE)

I could not find in your APIs something similar to this. Can you please advise, if you have a TR decomposition or any other Python package provided one?

Third, it is not clear to me from the tutorials, how a linear layer takes an input shape, and an output shape, and then a specific rank for what? If you provide an example with some explanations, this will be clearer.

rank=[2, 2]
model = tr.TRLinear(T1.shape, [2, 2], ranks=rank)
tn_type = "tr"
model.tn_info["type"] = tn_type

thank you,

Manal