glanceable-io / ordinal-log-loss Goto Github PK

20.0 1.0 3.0 82 KB

Repository of the COLING 2022 paper : Ordinal Log-Loss - A simple log-based loss function for ordinal text classification.

Home Page: https://www.glanceable.io

Python 100.00%

deep-learning machine-learning nlp ordinal-classification pytorch text-classification natural-language-processing coling2022

ordinal-log-loss's People

Contributors

Stargazers

Watchers

Forkers

beomi kasakh

ordinal-log-loss's Issues

"requires_grad=True" in loss function

Hello!
I've read the articles on ordinal log loss and very impressed with the idea of distances.
I tried to use OLL in my study, and I found some code which is not familiar to me.

def compute_loss(self, model, inputs, return_outputs=False):
    num_classes = model.module.num_labels
    dist_matrix = model.module.dist_matrix
    labels = inputs["labels"]
    outputs = model(**inputs)
    logits = outputs.logits
    probas = F.softmax(logits,dim=1)
    true_labels = [num_classes*[labels[k].item()] for k in range(len(labels))]
    label_ids = len(labels)*[[k for k in range(num_classes)]]
    distances = [[float(dist_matrix[true_labels[j][i]][label_ids[j][i]]) for i in range(num_classes)] for j in range(len(labels))]
    distances_tensor = torch.tensor(distances,device='cuda:0', requires_grad=True)
    err = -torch.log(1-probas)*abs(distances_tensor)**2
    loss = torch.sum(err,axis=1).mean()
    return (loss, outputs) if return_outputs else loss

In this code, the following line:

distances_tensor = torch.tensor(distances,device='cuda:0', requires_grad=True)

According to my best knowledge, distances_tensor itself did not change but just "given" by label information. So I think distances_tensor does not need to have "requires_grad=True". Even after removing those code, loss.backward() seems to work properly.

Is there any reason distances_tensor have "requires_grad=True"?

Thanks in advance.

Could you give an example of the parameters in loss function?

Hi,

Thanks for the code you shared.
I would like to copy the code to my project, but I encountered some mistakes.
Could you give an example of the parameters below? (such as logits, probas, true labels, dist_mat, distances)

Thanks in advance.

class OLL1Trainer(Trainer):
    def compute_loss(self, model, inputs, return_outputs=False):
        num_classes = model.module.num_labels
        dist_matrix = model.module.dist_matrix
        labels = inputs["labels"]
        outputs = model(**inputs)
        logits = outputs.logits
        probas = F.softmax(logits,dim=1)
        true_labels = [num_classes*[labels[k].item()] for k in range(len(labels))]
        label_ids = len(labels)*[[k for k in range(num_classes)]]
        distances = [[float(dist_matrix[true_labels[j][i]][label_ids[j][i]]) for i in range(num_classes)] for j in range(len(labels))]
        distances_tensor = torch.tensor(distances,device='cuda:0', requires_grad=True)
        err = -torch.log(1-probas)*distances_tensor
        loss = torch.sum(err,axis=1).mean()
        return (loss, outputs) if return_outputs else loss

glanceable-io / ordinal-log-loss Goto Github PK

ordinal-log-loss's People

Contributors

Stargazers

Watchers

Forkers

ordinal-log-loss's Issues

"requires_grad=True" in loss function

Could you give an example of the parameters in loss function?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent