Comments (19)
Ah, just saw https://github.com/blei-lab/edward/pull/557/files / https://github.com/blei-lab/edward/blob/master/examples/probabilistic_matrix_factorization.py from @siddharth-agrawal
Still happy to make a notebook using the netflix data in / notebooks
from edward.
I was browsing the Stan-users mailing list at 3am in the morning. Someone implemented a Bayesian neural network in Stan(!). There are two different implementations available: [1], [2a, 2b], discussion here
We should include one of these as a model example, and also implement the same model in TensorFlow (which is very easy).
from edward.
@dustinvtran I want to contribute to Edward and feel that working on this issue will help me more as well. Do you have any suggestions as I go forward? I'm using this guide for the dev workflow.
from edward.
perfect! i think a good way to start is maybe to select one of the models you're interested in above and then to implement a working version of it using some data and edward's inference.
from edward.
@dustinvtran I'm thinking of starting with 'Poisson process' next. Do you have any references that might help me? I'm not familiar with many variational inference models.
from edward.
@siddharth-agrawal: Cool. You can try building off #294, which I think @diengadji was working on but is no longer(?). It defines a Cox process for spatial data.
You can look into using the same example or apply a simpler version (e.g., Poisson vs Cox, i.i.d. data instead). I also recommend using a real data set, perhaps one typically applied with Poisson processes. For a more vanilla Poisson process, I also recommend point estimation via MAP instead of VI.
from edward.
@dustinvtran After I play a bit more with Edward and look through the code in more depth, I'd like to have a go at implementing a markov random field. Aware of anyone working on undirected models?
from edward.
Undirected is difficult. We haven't focused on it, although there is limited support such as for undirected models that can be conditionally specified (see https://github.com/mariru/exponential_family_embeddings).
We don't really know how to expose an undirected model's graph structure in the computational graph. But that would be the first step.
from edward.
Please use gibbs sampling as inference method
from edward.
@dustinvtran I'd be happy to take a stab at PMF using the original netflix ratings. Is anyone else on this already?
from edward.
from edward.
I've been trying to get vanilla PCF working on the larger movie lens dataset (20 million ratings, ~20k movies ~130k users) without much luck.
Would greatly appreciate any tips!
from edward.
@patrickeganfoley i looked your notebook. a couple suggestions:
-
start simple: use a MAP point estimate rather than going full variational. under coordinate ascent setting, for Gaussian MF, MAP point estimate matches variational mean, and you won't get much from the (under-estimated) approximating variances anyway.
-
start simple 2: use a global regularization rather than per-user/item, especially when fitting only on observed ratings -- the data is actually very sparse and probably cannot afford giving you reasonable estimate on a per-user/item basis.
(Updated)
- (slightly) more complex model:
K = 2
is too simplistic. I would at least start withK = 5
.
from edward.
Thanks @dawenl ! Will try MAP now.
on 2 - I'm not sure if I completely understand. I am pretty sure I am only setting up 4 regularization terms (user offset, movie offset, user vec, movie vec) which doesn't seem to be too much to ask of 20m ratings.
from edward.
OK, I should have looked more closely. I thought sigma_user_betas
and sigma_movie_betas
are set up on a per-user/item basis.
from edward.
@dawenl thank you for your help! (discussing here #682 so as to not block discussion of other examples)
from edward.
@dustinvtran I would like to start working on Bayesian word embedding model. Do you have any references on how to implement a variational inference model ? I saw the LSTM language model in Edward examples. Thanks.
from edward.
@iliemihai I know @mariru was extending https://github.com/mariru/exponential_family_embeddings to work on probabilistic embedding models with variational inference. In particular, normal priors over embedding and context vectors, and normal variational distributions to approximate their posteriors. It could be useful to start from there. (alternatively, if you want just Bayesian RNNs, you can take the LSTM language model and replace MAP)
from edward.
@iliemihai check out the branch feature/edward in the https://github.com/mariru/exponential_family_embeddings repo. It uses edward's MAP inference
from edward.
Related Issues (20)
- Link to reference broken in examples/pp_dirichlet_process.py
- Didn't support latest tensorflow 1.13 HOT 1
- Support for decision trees in Edward
- Fail to load celegans data from observations
- Optimizing ELBO with respect to variational model only
- i am not able to import edward library HOT 3
- Should pip install edward>=2.0.0 just use github.com/google/edward2? HOT 3
- casual inference for recommendation
- why can't I run probabilistic_matrix_factorization.py
- Had trouble in importing Edward
- installing edward in anaconda HOT 1
- edward import error HOT 4
- flag error HOT 1
- How this happen? HOT 3
- If the history of a specific question is one...(in bayesian IRT)
- Compositions of inference
- Attribution error for TensorFlow when running the PPCA example HOT 1
- Compatibility with TF 2.0+ HOT 3
- ed.set_seed doesn't work HOT 1
- Cannot import Edward model in GoogleColab HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from edward.