Code Monkey home page Code Monkey logo

Comments (2)

PistonY avatar PistonY commented on June 4, 2024

I’ll check it out later.

from torch-toolbox.

PistonY avatar PistonY commented on June 4, 2024

Hi @pinouchon ,I read the paper again, and it didn't say that evo_norm could work with 1d conv layer, so I may not provid 1d version of this.
But I could tell you how to change it to suit for this. You only need to re-write instance_std and group_std like this.

def instance_std_1d(x, eps=1e-5):
    var = torch.var(x, dim=(2, ), keepdim=True)
    std = torch.sqrt(var + eps)
    return std

def group_std_1d(x: torch.Tensor, groups=32, eps=1e-5):
    n, c, l = x.size()
    x = torch.reshape(x, (n, groups, c // groups, l))
    var = torch.var(x, dim=(2, 3), keepdim=True)
    std = torch.sqrt(var + eps)
    return torch.reshape(std, (n, c, h, w))

And the training var should be

var = torch.var(x, dim=(0, 2), keepdim=True)

So if you test evo_norms could work with 1d conv, you could make a pr to me.
Good luck.

from torch-toolbox.

Related Issues (11)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.