gmm_a does not seem to work properly

[haohaiziround]

For the gmm_a model with and without symmetry, the inference seems weird for the mu2 (the mu on the right). mu1 converges fine and the posterior is around -2, but the mu2 spares quite far away, from 0 to 10! Need to find out the reason.

[haohaiziround]

Can see the results by first running run_gmm_1d_a_ordered and then diagnose_gmm_1d_a_ordered.

[bayesianbrad]

If you plot the graph model, you see that the compiler is not plotting the conditional nodes. But, it is compiling them. It could potentially be to do with how the compiler is extracting the model. Or, it could be not enough mixing on the inference side.

[bayesianbrad]

The other problem is, it is only observing the same 3 y variables -2, -2.5 , -1.7 that makes it incredibly difficult to predict something that it does not observe

[haohaiziround]

The second comment makes sense. Basically, it means we think there are 2 clusters from prior, but it is really hard to distinguish from the observation with only -2, -2.5, -1.7. So the two modes might be both around -2.

[haohaiziround]

I have not noticed the first one. Will check it out.

[bayesianbrad]

Is it meant to select different data points each run?

[haohaiziround]

The if statement in get-ordered-mu maintains the order of mus, which ensures within this run, the smaller mu is always the cluster 0.

[haohaiziround]

The other interesting result is that, when I run pymc3 with NUTS + Metropolis-within-gibbs, HMC + Metropolis-within-gibbs, Metropolis + Metropolis-within-gibbs , even with the symmetric version, all algorithms seldom mix the modes (but might have a longer tail) with 2000 samples.

[bayesianbrad]

Are they sampling different data, or keeping the same 3 observed values?

[haohaiziround]

What do you mean by 'sampling'?
We sample 2 mus and 10 zs and observe 10 data/ys.

[haohaiziround]

The previous result might because sometimes we have modified N in the model to 3 and run the inference based only on 3 data. Let me try to re-run it with 10 data and update here.

[bayesianbrad]

Well, by sampling, I mean when you set N=3 you simply will not get an accurate model as you only see the same 3 y's, unless you sample from your observed values at random, so that the model in the inference observes the more of the system.

[bayesianbrad]

Also, I just ran it for all 10. I put the burnin at 20000 and then created 2000 more samples, to represent the true stationary distribution. With this I get a mu1 = 2.3 and mu2 = -2.3