Cross self attention switch#251

Merged

coolkp merged 27 commits intomainfrom

cross_self_attention_switch

Nov 12, 2025

Commits on Oct 10, 2025

skip flash block sizes setting for cross attention.

entrpn
authored and
coolkp
committed
change sharding based on cross/self attention.

entrpn
authored and
coolkp
committed
update sharding rules for attn.

entrpn
authored and
coolkp
committed
lint.

entrpn
authored and
coolkp
committed
ring attention rules are added at front if not present to shard sequence on fsdp axis
coolkp
committed
test fix
coolkp
committed

Commits on Oct 20, 2025

Add dense padded attention kernel and use unsafe rng key for generation
coolkp
committed

Commits on Oct 22, 2025

Commits on Nov 10, 2025

Flag for using segment ids and masking padding tokens in attention
coolkp
committed

Commits on Nov 11, 2025

Commits on Nov 12, 2025