Intel Hits Back At AMD's Data Center AI Performance Claims: Says 5th Gen Xeon Faster Than AMD EPYC Turin Using Proper Optimizations
Intel has followed up on AMD's data center AI performance claims for its 5th Gen EPYC Turin CPUs, saying that the 5th Gen Xeon is faster with precision optimization.
AMD showed its 5th Gen EPYC Turin CPUs with Zen 5 Cores beating 5th Gen Xeon chips in AI benchmarks, but Intel says 5th Gen EPYC gains an edge against the right optimizations when used. goes
At Computex 2024, AMD officially announced it. The 5th Gen EPYC CPU family is codenamed Torn which uses the latest Zen 5 core architecture. The company put up some big numbers against the Intel 5th Gen Xeon family, codenamed Emerald Rapids, especially in AI throughput workloads, but Intel has now clarified that those benchmarks are without proper optimization for its Xeon family. were made, and with their implementation, even the 5th Gen. The Xeons should easily surpass the 5th Gen EPYC Turin in AI performance.
The benchmark specifically highlighted by Intel is the Llama2-7B chatbot which is based around INT4 inference throughput and was performed at 50ms latency. AMD showed that its 5th Gen EPYC CPUs offer performance of up to 671 tokens per second with 128 cores in a 2S (dual socket) configuration, while Intel's 5th Gen Xeon Platinum 8592+ chips run in a single dual socket with 64 cores. have been. The configuration only produced 125 tokens/s output. This was a huge 5.4x advantage for the AMD EPYC Turin CPUs.
Intel says the benchmarks were done without the right software suite for 5th Gen Xeon SKUs, and AMD doesn't provide any details regarding the Intel configuration in its footnotes.
The blue team has now conducted their own performance benchmarks on the same AI workload and the results appear to be radically different.
Using the Intel Extension for PyTorch (P99 Latency), 5th Gen Emerald Rapids Xeon CPUs can deliver 5.4x better performance than AMD's performance. The 686 token/s output exceeds the performance of AMD's 5th Gen EPYC Zen 5 CPUs. This is thanks not only to Intel's software optimizations for Llama2, but also the AI hardware accelerators in the 5th Gen Emerald Rapids family that provide a nice boost.
Intel doesn't stop there, the company also reports that in two other workloads, translation and summarization, the company saw a 1.2x to 2.3x increase in performance compared to the results AMD used in its Computex 2024 presentation.
Furthermore, the more important aspect is that these benchmarks are heavily dependent on memory and the next-generation EPYC Turin family will have a 12-channel DDR5 interface. While Intel's current-generation Xeon “Emerald Rapids” CPUs reach 8 DDR5 memory channels, Next generation Granite Rapids “Xeon 6700P/6900P” CPUs Not only will it come with the same “Up To” 12-channel memory interface, but it will also feature the same 128 core count with P-Cores and 288 E-Cores with the Xeon 6900E family arriving in early 2025. Granite Rapids Xeon 6700P CPUs will be available in Q3 2024 with 86 P-Cores.
It seems that all the big firms currently involved in the AI business are very serious not only about their performance claims, but also about what others are showing. NVIDIA and AMD were engaged in a similar battle a few months ago. When the Red Team announced their MI300X figures against the Hopper lineup and It was back and forth from there..
Similarly, in terms of AI PC stuff, the companies AMD, Intel, Qualcomm, and NVIDIA are engaged in a very tough battle where AI TOPs are the talk of the town. With the growing demand for AI in the tech sector, all these tech firms are trying to get more customers to use their products and ship more units to meet the growing need for capable hardware.


