Comments (17)
Feel free to contribute @sdwfrost, these are useful extensions to RWM. A couple of comments:
- It may be the case that we want to add several proposals other than the Normal or truncated Normal. Maybe we could approach this in a systematic way by adding
Proposal
andRWMProposal
as abstract types, then define specific types for each proposal distribution. Alternatively we would use only a wrapper function calledproposal()
to point to the user-chosen proposal mechanism. If the RWM definition changes drastically, we can define it then as a separate typeRWMx()
, which is sth that @fredo-dedup has already started doing on the basis of the employed RWM tuning methods. - The integer random walk sampler would probably be a separate RWM type for example.
- Adding Gibbs sampling capabilities is certainly a great addition, though it will take quite a lot of work so feel free to contribute. BUGS/WinBUGS is the reference point when it comes to simulation using Gibbs algorithms, so we may want to draw some ideas from there.
from klara.jl.
Thanks for your participation @sdwfrost !
- Another solution that would change less existing types : extend the scale parameter to accept not only scalars but also distributions, i.e. a vector
[ Normal(0,1) , Cauchy(0,1), ...]
- Following that idea, a DiscreteDistribution would signal to the sampler that RWM should use an integer random walk. But that may not be such a good idea, that information is best associated with the model definition, not the sampler's
- Another sampler field describing a partition of the parameter vector ? Using the PMap type ?
from klara.jl.
- It is true that by allowing
scale
to accept distributions the code may be more compact, yet at the same time it won't be conceptually clear. In general, the distribution is naturally part of the proposal mechanism, not so directly implicated to scaling, so it may be more transparent not to fuse these two different aspects of the MCMC samplers... - I agree that integer/continuous variable definition is part of the model definition, not the sampler's.
- That's probably a good idea, not clear in my mind yet.
from klara.jl.
(3). has some conceptual similarity with sequential MC : run in succession different samplers on the same model; but with only a subset of parameters active on each sampler. It may be a good idea to follow that idea (generalize seq MC).
from klara.jl.
Yes, that makes more sense, and it is probably in your coding realm, since you have done the seqMC coding :)
from klara.jl.
@scidom - for Metropolis-within-Gibbs, I was really thinking more of attaching different samplers to different parameters of a model, then having a sampler that would go through each component of the model and update them separately.
How about 'IRWM' for an integer random walk sampler as a separate type?
For truncation, it would be nice to have a macro to truncate a distribution from Distributions.jl, in a similar way to what gen.trun does in the R library gamlss.tr; then, having a mechanism to specify the proposal distribution would be the same regardless of whether the distribution was truncated or not, whether a random walk sampler or an independence sampler, etc. was used.
from klara.jl.
@sdwfrost, assuming different samplers for different parameters is sth that we haven't considered before and it is important to update the package to include such functionality; please feel free to contribute any time as it would be great help. @fredo-dedup, what is your opinion on this?
IRWM
sounds good to me.
Yes, I agree with your suggestion about having a macro for truncating distributions - I think this should become part of the Distributions
package, and then we can invoke it from MCMC
. Would you like to file an issue in Distributions
about truncation and see what the rest of people think about it?
from klara.jl.
There was a truncate macro in Distributions, which we replaced with a Truncated type.
-- John
On Aug 27, 2013, at 9:27 AM, Theodore Papamarkou [email protected] wrote:
@sdwfrost, assuming different samplers for different parameters is sth that we haven't considered before and it is important to update the package to include such functionality; please feel free to contribute any time as it would be great help. @fredo-dedup, what is your opinion on this?
IRWM sounds good to me.
Yes, I agree with your suggestion about having a macro for truncating distributions - I think this should become part of the Distributions package, and then we can invoke it from MCMC. Would you like to file an issue in Distributions about truncation and see what the rest of people think about it?
—
Reply to this email directly or view it on GitHub.
from klara.jl.
Regarding different samplers for each parameter, this is what WinBUGS and JAGS both do. The BUGS book has a nice description of how this is handled: sampler algorithms go through nodes in the graphical model looking for subgraphs to which they apply.
-- John
On Aug 27, 2013, at 9:27 AM, Theodore Papamarkou [email protected] wrote:
@sdwfrost, assuming different samplers for different parameters is sth that we haven't considered before and it is important to update the package to include such functionality; please feel free to contribute any time as it would be great help. @fredo-dedup, what is your opinion on this?
IRWM sounds good to me.
Yes, I agree with your suggestion about having a macro for truncating distributions - I think this should become part of the Distributions package, and then we can invoke it from MCMC. Would you like to file an issue in Distributions about truncation and see what the rest of people think about it?
—
Reply to this email directly or view it on GitHub.
from klara.jl.
That's a very useful reference, didn't know about this book (it came out last year as it seems). If MCMC
is going to allow different samplers for each parameter, I think we should do it the BUGS way (ie using graphical models). By the way, is there any Julia code for graphical models?
from klara.jl.
I agree. I think we'll want to have a proper graphical model representation. I spent some time at Hacker School on this, but it needs a lot more work.
I'll write something up later this week so that others can see if anything I've done would be useful.
-- John
On Aug 27, 2013, at 10:05 AM, Theodore Papamarkou [email protected] wrote:
That's a very useful reference, didn't know about this book (it came out last year as it seems). If MCMC is going to allow different samplers for each parameter, I think we should do it the BUGS way (ie using graphical models). By the way, is there any Julia code for graphical models?
—
Reply to this email directly or view it on GitHub.
from klara.jl.
Great, keep us posted about the graphical model coding.
from klara.jl.
Is there somebody working on extending RWM along the lines mentioned in the subject line of this issue? I am very interested to work on it. Especially the block-wise RWM. @johnmyleswhite I know of your package dealing with graphical models. Can you point me to the BUGS book where it is explained how they deal with component-wise RWM?
from klara.jl.
Chapter 12 of this book is a good start: http://www.amazon.com/The-BUGS-Book-Introduction-Statistical/dp/1584888490
from klara.jl.
You're most welcome to work on this @gragusa and we will happily integrate your contributions !
Do you have already a plan ? Are you thinking of an automated choice of sampler depending on the kind of parameters (like BUGS) or user supplied information ?
from klara.jl.
@fredo-dedup thank you. I think I should have some free time in the next couple of week and i will start working on it. I am leaning toward user supplied information since the problem I am thinking are non-standard enough that any automatic choice will result (generally) in poor results. But I have to carefully think this through though. We will see.
from klara.jl.
The new graph-based model is now ready to accommodate Gibbs and Metropolis-within-Gibbs relatively easily. I will only need to set up a Gibbs-related job over the next few weeks (which has been logged already as an issue). As for the truncated normal, we haven't added this functionality yet. I will close this issue and will split it into sub-issues to track it better.
from klara.jl.
Related Issues (20)
- Decide whether to add univariate AM or drop AM sampler HOT 1
- Possible DSL sitting atop of existing functionality
- Drop Basic prefix in various type names
- Add some simple examples from OpenBUGS, volume 1
- MethodError: Cannot `convert` an object of type Array{Float64,1} to an object of type Float64 HOT 3
- Add option to bound parameters HOT 3
- Deprecation warnings for v0.6 HOT 2
- World age issue due to code generation inside an inner constructor HOT 2
- Issue related to where keyword and parametric types HOT 7
- How to replace consume and produce with Channels HOT 1
- Implement AMWG sampler
- MCMC in GaussianProcesses.jl HOT 24
- Eliminate code generation to avoid world age issues in Julia v0.6
- HMC with reverse-mode autodiff HOT 1
- ForwardDiff 0.4 0.6 HOT 4
- Is there any guidance for custom sample space HOT 2
- Monitoring Metropolis-Hastings Internal Quantities
- Support Julia 1.0 HOT 5
- Gibbs sampling from a posterior distribution kernel
- Package compatibility caps
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from klara.jl.