Huawei was caught cheating on phone benchmarks (updated)
The 3DMark team has delisted phones as a result.
You'd think smartphone manufacturers would have learned their lesson after being caught cheating multiple times, but that's apparently not the case. UL Benchmarks (aka the Futuremark team) has delisted Huawei's P20 Pro, P20, Nova 3 and Honor Play from 3DMark's charts after internal testing and an AnandTech exposé showed the devices ramping up performance whenever they detected the public versions of benchmarking apps. These were not subtle differences, either, with results up to 47 percent higher than they were with private test variants Huawei couldn't catch.
When confronted, Huawei's Wang Chenglu told AnandTech that it wanted to embrace standardized benchmarks that were closer to real-world experience, and that it was fudging existing benchmark results because "others do the same testing." It was a "common practice" in China, Wang said, suggesting that Huawei would lose sales if it didn't cheat. The company also told UL that it was planning to offer a speed-boosting mode through a future update (it's not clear if this is GPU Turbo from the Honor Play and 10) so that any app could benefit from the same performance boost.
Not that the company is eliciting much sympathy. It's using the "other kids are doing it too" excuse you used in grade school -- just because it's common doesn't make it acceptable. And when Huawei is the second-largest smartphone maker on the planet, shouldn't it be setting an example instead of following the herd? If nothing else, you'd think that Huawei would realize that the long-term consequences of being discovered would outweigh the benefits. The cheating not only took Huawei's phones out of contention, it eroded trust at a time when the company is already struggling to maintain its reputation.
Update: Huawei has issued a statement both reiterating its previous points (including plans for a "Performance Mode") and saying that the two jointly delisted the phones on the 3DMark charts. The handsets will be back once "all users" of Huawei's phones have access to that higher-performance option. You can read the full response below.
"Huawei and UL (creators of 3DMark) have held comprehensive discussions on benchmarking practices this week, and have reached a positive agreement on the next steps in working together.
"In the discussion, Huawei explained that its smartphones use an artificial intelligent resource scheduling mechanism. Because different scenarios have different resource needs, the latest Huawei handsets leverage innovative technologies such as artificial intelligence to optimize resource allocation in a way so that the hardware can demonstrate its capabilities to the fullest extent, while fulfilling user demands across all scenarios.
"UL understands the intent of Huawei's approach, but is opposed to forcing the use of a 'Performance Mode' by default when a benchmarking application is detected by the device. UL rules require a device to run the benchmark as if it were any other application.
"Huawei respects consumers' right to choose what to do with their devices. Therefore, Huawei will provide users with open access to 'Performance Mode' in EMUI 9.0, so that the user can choose when to use the maximum power of their device.
"Huawei and UL have also discussed current common benchmark testing methodologies in general. UL and Huawei would like to participate in an industry movement to develop benchmarking standards that best serve the needs of manufacturers, press, and consumers.
"To prevent confusion around current benchmarking results, after discussion, UL and Huawei have temporarily delisted the benchmark scores of a range of Huawei devices, and will reinstate them after Huawei grants all users of Huawei handsets access to the Performance Mode."
