Light

statmixedml / dgbm Goto Github PK

Distributional Gradient Boosting Machines

License: Apache License 2.0

distributional-regression xgboost lightgbm normalizing-flow probabilistic-modeling uncertainty-quantification quantile-estimation machine-learning gradient-boosting decision-trees

dgbm's Introduction

DGBM: Distributional Gradient Boosting Machines

We present a unified probabilistic gradient boosting framework for regression tasks that models and predicts the entire conditional distribution of a univariate response variable as a function of covariates. Our likelihood-based approach allows us to either model all conditional moments of a parametric distribution, or to approximate the conditional cumulative distribution function via Normalizing Flows. As underlying computational backbones, our framework is based on XGBoost and LightGBM. Modelling and predicting the entire conditional distribution greatly enhances existing tree-based gradient boosting implementations, as it allows to create probabilistic forecasts from which prediction intervals and quantiles of interest can be derived.

News

🚧 Repo is under construction

Reference Paper

dgbm's People

Contributors

Stargazers

Watchers

Forkers

vishalbelsare

dgbm's Issues

Improve runtime efficiency of TensorFlow gradient_tape computation of gradients and hessians.

Summary

As we state in our paper, DGBM is not yet competitive in terms of runtime, constantly exceeding those of other approaches by several orders of magnitude. From experiments we have conducted so far it appears that the automatic derivation of the hessian poses a significant computational bottleneck. The reason is that when computing the hessian (i.e., gradient of a gradient) of a loss function with vector input (i.e., multi-parameter optimization), instead of returning the element-wise hessian, the second gradient appears to return the row-wise sum of the hessian. For a more detailed discussion, we refer to tensorflow/tensorflow#29064.

Ask to the community

As we show in this notebook, we can circumvent the problem of incorrect hessians. However, these implementations are not as efficient as the original nested gradient_tape version.

I am reaching out to the TensorFlow community asking for help and guidance on how to further improve the computational efficiency of gradients and hessians.

Still Active?

Hi, this is a good idea. Are you still actively pursuing this?

Thanks

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.