-
Notifications
You must be signed in to change notification settings - Fork 457
feat: add Flux.2 VAE proj matrix for previews #1017
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
Not working, let me look into it. I think the latents are patchified by 2x2 patches or something Edit: fixed |
|
Perhaps placing the logic in preview_latent_video could make the code cleaner. |
|
@leejet is it good now or should I actually move the calls to |
|
I think the best approach is to move unpatchify_latents logic into preview_latent_video, which would eliminate the need to call repatchify_latents. |
I will look into that soon. Maybe there's a way to avoid allcoating a temp buffer too by doing so, that would be great |
| uint32_t dim = latents->ne[ggml_n_dims(latents) - 1]; | ||
|
|
||
| if (preview_mode == PREVIEW_PROJ) { | ||
| const float(*latent_rgb_proj)[channel] = nullptr; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not really related to the PR, but: maybe move the proj matrix selection to a virtual VAE object member, to avoid the decision tree here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed, that would probably be cleaner. Maybe in a follow-up PR.
Seems like the proj is harder to train for this new VAE, reminds me of SDXL.