Saya memberi anotasi kernel perhatian kilat tinygrad untuk memastikan saya memahaminya. secara otomatis menghasilkan ini GENERIK cukup keren!
the tiny corp
the tiny corp3 Agu, 07.29
Finally, real flash attention. With the new indexing, it's completely automatic and generic, see TestFuse.test_flash_attention. ENDRANGE is STORE+LOAD, and you end a range if the children of a UOp mismatch on that axis.
74,29K