I execute python main.py under the examples/ folder, and get the error:

% python main.py --dataset_path criteo/train.txt
Traceback (most recent call last):
  File "main.py", line 189, in <module>
    main(args.dataset_name,
  File "main.py", line 151, in main
    dataset = get_dataset(dataset_name, dataset_path)
  File "main.py", line 33, in get_dataset
    return CriteoDataset(path)
  File "/Users/erlebach/opt/anaconda3/envs/torch/lib/python3.8/site-packages/torchfm/dataset/criteo.py", line 40, in __init__
    self.__build_cache(dataset_path, cache_path)
  File "/Users/erlebach/opt/anaconda3/envs/torch/lib/python3.8/site-packages/torchfm/dataset/criteo.py", line 56, in __build_cache
    feat_mapper, defaults = self.__get_feat_mapper(path)
  File "/Users/erlebach/opt/anaconda3/envs/torch/lib/python3.8/site-packages/torchfm/dataset/criteo.py", line 70, in __get_feat_mapper
    with open(path) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'criteo/train.txt'

Could you please help me understand how to make this example work? Would it be possible for you to provide a single example that works out of the box, or provide an example of command line arguments that will work? Thanks.

np.int and np.long are deprecated since numpy 1.20.0. It should be replaced by int (or np.int32) and np.int64.

After you train an instance of a model using PyTorch, how do you use that model to get the top 10 rated items, i.e., how do you get a model prediction.

can it be used for 1TB Criteo data on local computer
so it needs to read and process data by parts: sliding window?

When I run the main.py, I get the following error:

Traceback (most recent call last):
File "main.py", line 16, in
from torchfm.model.hofm import HighOrderFactorizationMachineModel
ModuleNotFoundError: No module named 'torchfm.model.hofm'

class FactorizationMachine(torch.nn.Module):

    def __init__(self, reduce_sum=True):
        super().__init__()
        self.reduce_sum = reduce_sum

    def forward(self, x):
        """
        :param x: Float tensor of size ``(batch_size, num_fields, embed_dim)``
        """
        square_of_sum = torch.sum(x, dim=1) ** 2
        sum_of_square = torch.sum(x ** 2, dim=1)
        ix = square_of_sum - sum_of_square
        if self.reduce_sum:
            ix = torch.sum(ix, dim=1, keepdim=True)
        return 0.5 * ix

What happened to v parameters?

Hi,

In layer.py, class FeaturesEmbedding, one finds the line

self.offsets = np.array(0, *np.cumsum(field_dims)[:-1]), dtype=np.long)

My code generates an error. I replaced np.array by torch.tensor and np.cumsum by torch.cumsum , and the code works.

I believe the code as it is has a bug, which was fixed with my change. Could you please confirm this? I am not sure how the code can work as is. thanks.

你好，你对于Class MultiLayerPerceptron里的forward function中的x的注释错误
原注释：:param x: Float tensor of size (batch_size, num_fields, embed_dim)
应改为： :param x: Float tensor of size (batch_size, embed_dim)

args = parser.parse_args()
change to ：
args = parser.parse_args(args=[])

In FeaturesLinear

def __init__(self, field_dims, output_dim=1):
    super().__init__()
    self.fc = torch.nn.Embedding(sum(field_dims), output_dim)
    self.bias = torch.nn.Parameter(torch.zeros((output_dim,)))
    self.offsets = np.array((0, *np.cumsum(field_dims)[:-1]), dtype=np.long)

def forward(self, x):
    """
    :param x: Long tensor of size ``(batch_size, num_fields)``
    """
    x = x + x.new_tensor(self.offsets).unsqueeze(0)
    return torch.sum(self.fc(x), dim=1) + self.bias

The model only assign weight value for each field as embedding, but it didn't consider the data value here.
For example, if we has a data as [[1,0]], the model will work embedding("1")+embedding("0"), but in real, we need work embedding("1")*1+embedding("0")*0.

What reason the model delete the effect of data value?

field_dims 这个有啥用，有没有详细的案例？

Traceback (most recent call last): File "C:\Users\Faaiz-PC\Downloads\Avazu\main.py", line 189, in <module> main(args.dataset_name, File "C:\Users\Faaiz-PC\Downloads\Avazu\main.py", line 151, in main dataset = get_dataset(dataset_name, dataset_path) File "C:\Users\Faaiz-PC\Downloads\Avazu\main.py", line 35, in get_dataset return AvazuDataset(path) File "C:\Users\Faaiz-PC\anaconda3\envs\fnfm\lib\site-packages\torchfm\dataset\avazu.py", line 39, in __init__ self.field_dims = np.frombuffer(txn.get(b'field_dims'), dtype=np.uint32) TypeError: a bytes-like object is required, not 'NoneType'

I am trying to Trying to run fnfm on avazu dataset. this is the error I am getting. Can you help me fix this?

Dear DaLao:
what's the function of "self.offsets" ？

self.offsets = np.array((0, *np.cumsum(field_dims)[:-1]), dtype=np.long)
def forward(self, x):
x = x + x.new_tensor(self.offsets).unsqueeze(0)

论文中每一次multi-head attention后都要做residual

Paper said :

The first hidden layer is fully connected with the product layer. The inputs to it consist of linear
signals lz and quadratic signals lp. With respect to lz and lp inputs, separately, the formulation of l1 is: l1 = relu(lz + lp + b1),

But in your code, the linear signals and quadratic signals are concated :

embed_x = self.embedding(x)
cross_term = self.pn(embed_x)
x = torch.cat([embed_x.view(-1, self.embed_output_dim), cross_term], dim=1)
x = self.mlp(x)
return torch.sigmoid(x.squeeze(1))

In FM and HOFM papers, there is a hyper parameter k which helps to control the size of latent factor. Just wondering where is the k in the code? Is that same as field_dims?

1.In the forward process, the dimension of input X in all models are (batch_size, field_size), here can I treat all these field size as feature size?

2. in fnfm.py, you forgot define self.embed_layer in forward function methed. please check.

when you forward(), the last layer of cin is not split half, while the fc_input_dim still adds cross_layer_size//=2.

How to add non-categorical features such as human age to deepfm model? It seems the deepfm implementation only supports categorical feature?

Hi,
When nfm is used as model, the return value (batch_size, batch_size) does not have the right shape.
This will cause a error when calculating the loss.

Hi,
It threw out this error RuntimeError: CUDA error: device-side assert triggered
when running ffm model, can you please take a look at it and fix it? Thank you!

Is it necessary to set the arg padding_idx of torch.nn.embbedding, as all features start from 1.
I think it's better to avoid influence of 0.

I noticed that for the deemFM model, the implemented mlp_dim is (16, 16). However in the original deepFM paper, the network structure specified is 400-400-400 (screenshot below). May I ask why is the network structure different?

import torch.nn.functional as F
代码中如果需要使用F.dropout,需要加个training=self.training
F.dropout(attn_scores, p=0.1)改成F.dropout(attn_scores, p=0.1,training=self.training)
这样model.train()可以使用dropout，model.eval()不使用dropout。

Hi, congrats on your great work!
I was wondering why you add a offset before embedding?

As I am looking at the implementation of xDeepFM, I didn't find the corresponding code in original tensorflow implementation.

HI, thanks for the amazing library.
I do not quite understand how the data should be fed. If i create a feature vector composed by real values, i receive an error because the embedding function can handle only Long tensors, i.e., integers.
I would suggest that you add some lines to understand how data should be prepared, as the documentation lack of this content.

Thanks and best regards

self.fc = torch.nn.Embedding(sum(field_dims)+1, output_dim)
https://github.com/rixwew/pytorch-fm/blob/master/torchfm/layer.py#L10

I'm wondering whether this is a writing misktake

why the field use label encode? not one-hot encode?

It will ignore the first data when the header is not None.

关于CIN的结构，是否在卷积层前将二三维度调换一下？

In the forward process, the dimension of input X in all models are (batch_size, field_size), here can I treat all these field size as feature size?

RuntimeError Traceback (most recent call last)
in
21 args.weight_decay,
22 args.device,
---> 23 args.save_dir)

in main(dataset_name, dataset_path, model_name, epoch, learning_rate, batch_size, weight_decay, device, save_dir)
23 early_stopper = EarlyStopper(num_trials=2, save_path=f'{save_dir}/{model_name}.pt')
24 for epoch_i in range(epoch):
---> 25 train(model, optimizer, train_data_loader, criterion, device)
26 auc = test(model, valid_data_loader, device)
27 print('epoch:', epoch_i, 'validation: auc:', auc)

in train(model, optimizer, data_loader, criterion, device, log_interval)
5 for i, (fields, target) in enumerate(tk0):
6 fields, target = fields.to(device), target.to(device)
----> 7 y = model(fields)
8 loss = criterion(y, target.float())
9 model.zero_grad()

H:\Anaconda\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),

~\Desktop\量化炒股\pytorch-fm-master\torchfm\model\fm.py in forward(self, x)
22 :param x: Long tensor of size (batch_size, num_fields)
23 """
---> 24 x = self.linear(x) + self.fm(self.embedding(x))
25 return torch.sigmoid(x.squeeze(1))

H:\Anaconda\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),

~\Desktop\量化炒股\pytorch-fm-master\torchfm\layer.py in forward(self, x)
17 """
18 x = x + x.new_tensor(self.offsets).unsqueeze(0)
---> 19 return torch.sum(self.fc(x), dim=1) + self.bias
20
21

H:\Anaconda\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),

H:\Anaconda\lib\site-packages\torch\nn\modules\sparse.py in forward(self, input)
124 return F.embedding(
125 input, self.weight, self.padding_idx, self.max_norm,
--> 126 self.norm_type, self.scale_grad_by_freq, self.sparse)
127
128 def extra_repr(self) -> str:

H:\Anaconda\lib\site-packages\torch\nn\functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
1812 # remove once script supports set_grad_enabled
1813 no_grad_embedding_renorm(weight, input, max_norm, norm_type)
-> 1814 return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
1815
1816

RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.IntTensor instead (while checking arguments for embedding)

in rendle's paper, the optimized parameter is b w and v, why use emdedding vector here?

Hi, I googled this error but didn't help solve this error.

Traceback (most recent call last):
  File "/root/pytorchfm/examples/main.py", line 157, in <module>
    args.save_dir)
  File "/root/pytorchfm/examples/main.py", line 127, in main
    train(model, optimizer, train_data_loader, criterion, device)
  File "/root/pytorchfm/examples/main.py", line 81, in train
    y = model(fields)
  File "/root/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/pytorchfm/torchfm/model/xdfm.py", line 27, in forward
    x = self.linear(x) + self.cin(embed_x) + self.mlp(embed_x.view(-1, self.embed_output_dim))
  File "/root/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/pytorchfm/torchfm/layer.py", line 232, in forward
    x = F.relu(self.conv_layers[i](x))
  File "/root/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 943, in relu
    result = torch.relu(input)
RuntimeError: CUDA error: device-side assert triggered

Can you take a look at it? Thank you!

can you share link to similar repo but plain python code
at least for simple models?

The original link of dataset Criteo is no longer available. Here I found a new one:

https://www.kaggle.com/datasets/mrkmakr/criteo-dataset

按照论文，FNN模型（即 FactorizationSupportedNeuralNetworkModel）应该是利用了FM模型pre-traing后的隐向量作为MLP的输入，但是我在您的FactorizationSupportedNeuralNetworkModel的code中发现并没有实现这一点。而且FNN model（即FactorizationSupportedNeuralNetworkModel） 的实现方式与您写的Wide&Deep model 的Deep部分的实现方式一模一样，这应该是不对。
希望大佬可以探讨一下，或许是我理解不到位，或者是代码实现有问题，谢谢！（抱歉，这部分拿英文复述我怕自己说得不明确，因此用中文）