Mi450

AMD's MI450 Gluon-based attention decode kernel achieves up to ~85% of peak HBM bandwidth (about 16–17 TB/s of a 20 TB/s peak) using a four-stage pipeline, TDM data paths, and Split-k across KV partitions, backed by detailed hardware features and open-source references.

Kog claims real-time LLM inference on standard datacenter GPUs can reach about 3,000 tokens/s per request on a 2B model by co-designing a monokernel runtime, GPU code, and a Laneformer architecture, with scalability toward frontier MoEs as memory bandwidth grows.