Negative KL loss

[ManalHamzaoui]

When I tested the Poincaré version on MNIST with latent_dim> 32 I had a negative KL loss, is this normal?

[xinyue96]

I also encounter negative KL loss when I just run the synthetic experiment following the instruction:
python3 pvae/main.py --model tree --manifold PoincareBall --latent-dim 2 --hidden-dim 200 --prior-std 1.7 --c 1.2 --data-size 50 --data-params 6 2 1 1 5 5 --dec Wrapped --enc Wrapped --prior RiemannianNormal --posterior RiemannianNormal --epochs 1000 --save-freq 1000 --lr 1e-3 --batch-size 64 --iwae-samples 5000

[saniazahan]

Hi also got negative KL loss with the synthetic experiment after epoch 10. Did you find any solution?

[omiethescientist]

I am also getting negative loss. I do not know why this is the case at low dimensions.

[isacostamaia]

Hi, did any of you manage to fix that? I am also encountering the same issue and didn't manage to reproduce the figure of the Synthetic Dataset's embeddings in the latent space (their positions on the disk do not match their hierarchy). Does anyone have a hint on what could be the source of the problem? Thanks.
@saniazahan @xinyue96 @omiethescientist @ManalHamzaoui