The adamw-and-sgdw from shaoanlu

adamw-and-sgdw's Introduction

👋Hello, I'm Shao-An!

Working as a Control Software Engineer.
Grew up in Taiwan, residing in the vibrant city of Tokyo, Japan.
Interested in Control Systems, Optimization, and Deep Generative Models.
Currently engaged in the exploration of Software Architecture within Robotic Systems.

Recent Activity

🎉 Merged PR #1 in shaoanlu/CBF_QP_safety_filter
💪 Opened PR #1 in shaoanlu/CBF_QP_safety_filter
💪 Opened PR #234 in qpsolvers/qpsolvers
🎉 Merged PR #6 in shaoanlu/qpsolvers
💪 Opened PR #6 in shaoanlu/qpsolvers

Projects

Control and Optimization
- Diffusion policy for Quadrotor Control: Implements cutting-edge diffusion models for imitation learning in quadrotor control.
- Safety Filter for Collision Avoidance: Employs Control Barrier Functions (CBF) as safety filters for collision avoidance.
- Motion Planning with MPPI: Integrates Model Predictive Control and CBF in a sampling-based approach for motion planning.
Deep Learning and Computer Vision
- Few-Shot Face Translation: Features Generative Adversarial Networks trained on the VGGFace2 dataset for face-translation with few-shot examples.
- Face Swapping GAN: Applies Generative Adversarial Networks for face-swapping in images.
- Facial Analysis Toolbox: Offers a collection of deep learning models for facial analysis in Keras.

adamw-and-sgdw's People

Contributors

Stargazers

Watchers

adamw-and-sgdw's Issues

Is w_norm a hyperparameter?

Thank you for your implementation.
What do you mean that " (w is the normalized one)'" in your sentence "w_norm: designed weight decay factor (w is the normalized one)"
In your example, you chose w_norm as 0.0025. I think it is a hyperparameter. Am I right?

TypeError: init() missing 1 required positional argument: 'name'

Hi while working East model, I have the following issue with adamw.py

Error I am getting is:
**super(AdamW, self).init(kwargs)
TypeError: init() missing 1 required positional argument: 'name'

Code is as below:
class AdamW(Optimizer):
"""Adam optimizer.
Default parameters follow those provided in the original paper.
# Arguments
lr: float >= 0. Learning rate.
beta_1: float, 0 < beta < 1. Generally close to 1.
beta_2: float, 0 < beta < 1. Generally close to 1.
epsilon: float >= 0. Fuzz factor.
decay: float >= 0. Learning rate decay over each update.
weight_decay: float >= 0. Decoupled weight decay over each update.
# References
- Adam - A Method for Stochastic Optimization
- Optimization for Deep Learning Highlights in 2017
- Fixing Weight Decay Regularization in Adam
"""

def __init__(self,lr=0.001, beta_1=0.9, beta_2=0.999, weight_decay=1e-4, epsilon=1e-8, decay=0., **kwargs):
    super(AdamW, self).__init__(**kwargs)
    with K.name_scope(self.__class__.__name__):
        self.iterations = K.variable(0, dtype='int64', name='iterations')
        self.lr = K.variable(lr, name='lr')
        self.beta_1 = K.variable(beta_1, name='beta_1')
        self.beta_2 = K.variable(beta_2, name='beta_2')
        self.decay = K.variable(decay, name='decay')
        self.wd = K.variable(weight_decay, name='weight_decay') # decoupled weight decay (2/4)
    self.epsilon = epsilon

Recommend Projects