What to do when `delta_ELBO` never converges

[sayadennis]

Hello!

Just wanted to get some advice on what we can do when the delta_ELBO never converges. I have been running SpatialDE for over 2 weeks now with 40 CPUs, and it looks like it's having problems converging. The tail of the outlog looks something like this:

iter 0, ELBO: -1.41e+13
iter 1, ELBO: -7.04e+12, delta_ELBO: 7.03e+12
iter 2, ELBO: -7.04e+12, delta_ELBO: 3.49e+07
iter 3, ELBO: -7.04e+12, delta_ELBO: 4.99e+05
iter 4, ELBO: -7.04e+12, delta_ELBO: 3.77e+04
iter 5, ELBO: -7.04e+12, delta_ELBO: 3.93e+04
iter 6, ELBO: -7.04e+12, delta_ELBO: 2.96e+04
iter 7, ELBO: -7.04e+12, delta_ELBO: 1.15e+04
iter 8, ELBO: -7.04e+12, delta_ELBO: 4.46e+03
iter 9, ELBO: -7.04e+12, delta_ELBO: 1.42e+03
iter 10, ELBO: -7.04e+12, delta_ELBO: 7.42e+02
iter 11, ELBO: -7.04e+12, delta_ELBO: 5.60e+02
iter 12, ELBO: -7.04e+12, delta_ELBO: 3.48e+02
iter 13, ELBO: -7.04e+12, delta_ELBO: 2.23e+02
iter 14, ELBO: -7.04e+12, delta_ELBO: 1.26e+01
iter 15, ELBO: -7.04e+12, delta_ELBO: 1.26e+02
iter 16, ELBO: -7.04e+12, delta_ELBO: 3.07e+01
iter 17, ELBO: -7.04e+12, delta_ELBO: 1.75e+02
iter 18, ELBO: -7.04e+12, delta_ELBO: 8.01e+01
iter 19, ELBO: -7.04e+12, delta_ELBO: 5.48e+01
iter 20, ELBO: -7.04e+12, delta_ELBO: 9.78e+00
iter 21, ELBO: -7.04e+12, delta_ELBO: 5.32e+01
iter 22, ELBO: -7.04e+12, delta_ELBO: 5.01e+01
iter 23, ELBO: -7.04e+12, delta_ELBO: 9.11e+01
iter 24, ELBO: -7.04e+12, delta_ELBO: 3.23e+01
iter 25, ELBO: -7.04e+12, delta_ELBO: 2.03e+01
iter 26, ELBO: -7.04e+12, delta_ELBO: 7.81e+01
iter 27, ELBO: -7.04e+12, delta_ELBO: 2.55e+01
iter 28, ELBO: -7.04e+12, delta_ELBO: 3.17e+01
iter 29, ELBO: -7.04e+12, delta_ELBO: 1.96e+01
iter 30, ELBO: -7.04e+12, delta_ELBO: 1.29e+01
iter 31, ELBO: -7.04e+12, delta_ELBO: 8.89e+00
iter 32, ELBO: -7.04e+12, delta_ELBO: 6.46e+00

Could I get any advice on how to handle this situation? Any input is appreciated!

[ViHammer]

Using StereoSeq data I chose a larger bin size. That reduced the dataset size and the model finally converged.