zafarali / better-sampling Goto Github PK
View Code? Open in Web Editor NEWinvestigating sampling
investigating sampling
We probably want to use set_grad_enabled
from the 0.4 release: http://pytorch.org/docs/stable/autograd.html?highlight=grad#torch.autograd.set_grad_enabled
From rw-bugfixing
:
*********************************************
Sampler: ISSampler
Start Estimate: 1.80106, Variance: 0.0213128, Prop Success: 1 ESS: 409.786
KL(true|est)=0.00403086, KL(obs|est)=0.00392562
*********************************************
*********************************************
Sampler: ABCSampler
Start Estimate: 1.23824, Variance: 10.6266, Prop Success: 0.319
KL(true|est)=0.0253446, KL(obs|est)=0.0261764
*********************************************
*********************************************
Sampler: MCSampler
Start Estimate: 1.47959, Variance: 9.19856, Prop Success: 0.392
KL(true|est)=0.00221258, KL(obs|est)=0.0022405
*********************************************
*********************************************
Sampler: RVISampler
Start Estimate: 1.33802, Variance: 0.0218989, Prop Success: 0.909 ESS: 443.872
KL(true|est)=0.00589845, KL(obs|est)=0.00582546
*********************************************
From batch-RVI
:
*********************************************
Sampler: ISSampler
Start Estimate: 1.80106, Variance: 0.0213128, Prop Success: 1 ESS: 409.786
KL(true|est)=0.00403086, KL(obs|est)=0.00392562
*********************************************
*********************************************
Sampler: ABCSampler
Start Estimate: 1.23824, Variance: 10.6266, Prop Success: 0.319
KL(true|est)=0.0253446, KL(obs|est)=0.0261764
*********************************************
*********************************************
Sampler: MCSampler
Start Estimate: 1.26426, Variance: 9.13137, Prop Success: 0.333
KL(true|est)=0.0116782, KL(obs|est)=0.011906
*********************************************
*********************************************
Sampler: RVISampler
Start Estimate: 1.59196, Variance: 0.0232969, Prop Success: 0.374 ESS: 364.28
KL(true|est)=0.0106005, KL(obs|est)=0.0105296
*********************************************
From mergebrv-rwb
:
*********************************************
Sampler: ISSampler
Start Estimate: 1.80106, Variance: 0.0213128, Prop Success: 1 ESS: 409.786
KL(true|est)=0.00403086, KL(obs|est)=0.00392562
*********************************************
*********************************************
Sampler: ABCSampler
Start Estimate: 1.23824, Variance: 10.6266, Prop Success: 0.319
KL(true|est)=0.0253446, KL(obs|est)=0.0261764
*********************************************
*********************************************
Sampler: MCSampler
Start Estimate: 1.47959, Variance: 9.19856, Prop Success: 0.392
KL(true|est)=0.00221258, KL(obs|est)=0.0022405
*********************************************
*********************************************
Sampler: RVISampler
Start Estimate: 1.59196, Variance: 0.0232969, Prop Success: 0.374 ESS: 364.28
KL(true|est)=0.0106005, KL(obs|est)=0.0105296
*********************************************
I am noticing a discrepancy between branches rw-bugfixing
and batch-RVI
Right now we have an opportunity to learn from multiple samples at the same time. This naturally would leverage GPUs. We should think about adding support for this.
See A.4
https://drive.google.com/file/d/1foEpoVJ7tsiqVGUoehZA93ZXcZp4Bykb/view?usp=sharing
However, now it will become a sparse reward problem
To reduce the variance in the gradient updates we might be able to take advantage of using the GAE:
https://danieltakeshi.github.io/2017/04/02/notes-on-the-generalized-advantage-estimation-paper/
This would require us to switch to a neural network baseline.
Schulman, John, et al. "Gradient estimation using stochastic computation graphs." Advances in Neural Information Processing Systems. 2015.
Use a learned value function as a baseline. Does performance improve? do the training curves become less noisy?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.