Skip to content

Conversation

@stduhpf
Copy link
Contributor

@stduhpf stduhpf commented Nov 28, 2025

Seems like the proj is harder to train for this new VAE, reminds me of SDXL.

p

@stduhpf
Copy link
Contributor Author

stduhpf commented Nov 28, 2025

Not working, let me look into it. I think the latents are patchified by 2x2 patches or something

Edit: fixed

@stduhpf stduhpf marked this pull request as draft November 28, 2025 21:23
@stduhpf stduhpf marked this pull request as ready for review November 28, 2025 23:53
@leejet
Copy link
Owner

leejet commented Nov 29, 2025

Perhaps placing the logic in preview_latent_video could make the code cleaner.

@stduhpf
Copy link
Contributor Author

stduhpf commented Dec 1, 2025

@leejet is it good now or should I actually move the calls to unpatchify_latents and repatchify_latents to be contained within preview_latent_video?

@leejet
Copy link
Owner

leejet commented Dec 1, 2025

I think the best approach is to move unpatchify_latents logic into preview_latent_video, which would eliminate the need to call repatchify_latents.

@stduhpf
Copy link
Contributor Author

stduhpf commented Dec 1, 2025

I think the best approach is to move unpatchify_latents logic into preview_latent_video, which would eliminate the need to call repatchify_latents.

I will look into that soon. Maybe there's a way to avoid allcoating a temp buffer too by doing so, that would be great

@stduhpf stduhpf changed the base branch from flux2 to master December 2, 2025 22:27
uint32_t dim = latents->ne[ggml_n_dims(latents) - 1];

if (preview_mode == PREVIEW_PROJ) {
const float(*latent_rgb_proj)[channel] = nullptr;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really related to the PR, but: maybe move the proj matrix selection to a virtual VAE object member, to avoid the decision tree here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, that would probably be cleaner. Maybe in a follow-up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants