First, the author admits that the Gigabit ethernet link was the bottleneck for the Xeon system, capping it at a mere 15% CPU utilization. However, he goes on to use published TDP numbers as the system power draw. TDP numbers are only approached under the most demanding of loads and 100% CPU usage, which this clearly was not. At bare minimum, the author needs some sort of power measurement device to make a reasonable comparison.
Second, serving static web pages is not a difficult task. The Xeon system is overkill for such a task, so of course it will be less energy efficient. In the real world, the extra capacity on this server could be used to perform more difficult tasks or run other processes.
Third, if we assume the Xeon server would scale linearly without being bottlenecked at 15% CPU usage by the Gigabit ethernet link, then it would serving approximately 46,300 pages per second, or almost 8.5X that of the ARM server. Take into account that the actual power consumed in the real-world will be less than the TDP, and the efficiency gap between the ARM server becomes very narrow. Even if the Xeon TDP numbers are accurate, the new margin is still less than 2X.
Finally, the Total Cost of Ownership calculations in the conclusion are based (as the author admits) on the flawed benchmark numbers. If they can only achieve a 77% TCO reduction by completing handicapping the Xeon system, then the ARM system may not be that advantageous. Especially when you consider that without the bottleneck or with a more demanding workload, you would might need as many as 8 times as many ARM servers to replace the performance of the Xeon server, which isn't going to help real-world TCO.
I'm a big fan of the ARM platform, and I'm very excited to see ARM servers enter the marketplace. However, false benchmarks like these aren't going to help anyone. ARM servers will have certainly have their place for a lot of different workloads, but to suggest that they are 15X more energy efficient and have a 77% lower TCO based on these numbers is disingenuous.
First, the author admits that the Gigabit ethernet link was the bottleneck for the Xeon system, capping it at a mere 15% CPU utilization. However, he goes on to use published TDP numbers as the system power draw. TDP numbers are only approached under the most demanding of loads and 100% CPU usage, which this clearly was not. At bare minimum, the author needs some sort of power measurement device to make a reasonable comparison.
Second, serving static web pages is not a difficult task. The Xeon system is overkill for such a task, so of course it will be less energy efficient. In the real world, the extra capacity on this server could be used to perform more difficult tasks or run other processes.
Third, if we assume the Xeon server would scale linearly without being bottlenecked at 15% CPU usage by the Gigabit ethernet link, then it would serving approximately 46,300 pages per second, or almost 8.5X that of the ARM server. Take into account that the actual power consumed in the real-world will be less than the TDP, and the efficiency gap between the ARM server becomes very narrow. Even if the Xeon TDP numbers are accurate, the new margin is still less than 2X.
Finally, the Total Cost of Ownership calculations in the conclusion are based (as the author admits) on the flawed benchmark numbers. If they can only achieve a 77% TCO reduction by completing handicapping the Xeon system, then the ARM system may not be that advantageous. Especially when you consider that without the bottleneck or with a more demanding workload, you would might need as many as 8 times as many ARM servers to replace the performance of the Xeon server, which isn't going to help real-world TCO.
I'm a big fan of the ARM platform, and I'm very excited to see ARM servers enter the marketplace. However, false benchmarks like these aren't going to help anyone. ARM servers will have certainly have their place for a lot of different workloads, but to suggest that they are 15X more energy efficient and have a 77% lower TCO based on these numbers is disingenuous.