NVIDIA的GB200 NVL72超级计算机在DeepSeek V2上实现2.7倍更快的推理
In collaboration with NVIDIA, researchers from SGLang have published early benchmarks of the GB200 (Grace Blackwell) NVL72 system, showing up to a 2.7× increase in LLM inference throughput...
SGLang与NVIDIA合作发布了GB200 NVL72系统的初步基准测试，LLM推理吞吐量提升2.7倍。优化措施包括FP8矩阵乘法和加速注意力内核，增强了多GPU性能。该系统适用于大规模AI训练和推理，未来将进一步优化预填充延迟和内存利用率。
