Hi @karlhigley, I just noted this oddity while playing with your implementation. Unfortunately, I didn't have time to further investigate the issue.

Observed behaviour:

import torch
from ranking_metrics_torch.cumulative_gain import ndcg_at

true = torch.tensor([[10, 0, 0, 1, 5]])
scores = torch.tensor([[0.1, 0.2, 0.3, 4, 70]])
ks = torch.tensor([3])
ndcg_at(ks, scores=scores, labels=true)

Returns
tensor([[1.]], dtype=torch.float64)

However adding another k to evaluate returns the correct ndcgs:

ks = torch.tensor([3, 5])
ndcg_at(ks, scores=scores, labels=true)

Returns:
tensor([[0.4124, 0.6957]], dtype=torch.float64)

Hi @karlhigley ,

while working on my own implementation of ndcg and checking it against sklearn's and yours, I noticed that your ndcg implementation produces different scores than sklearn. I added a simple unit test in this PR to demonstrate: #2

I am new to hypothesis and couldn't check why all your tests pass nevertheless. Maybe there is no bug and I am missing something?
The dcg scores are fine by the way (see first added test in PR). Looks to me like a problem with the normalizing gains but I don't understand the tie-aware implementation yet.