Dynamic shapes — the per-request sequence length is unknown at compile time. Paged KV cache — attention reads/writes a block-paged buffer addressed indirectly through a block table. This wave delivers ...
Since Github doesn't provide a great way for you to learn about new releases and features, don't just star the repo, join the mailing list. dsq will likely work on other platforms that Go is ported to ...