MediaTek Tried to Refute Benchmark Cheating Allegations Before They Were Even Published


Earlier this week, we covered MediaTek’s exciting new “Benchmark cheating as a service” (BCaaS) operating mode that it ships on its products. Today, we discovered that MediaTek attempted to refute the claims with an article of its own before the investigation results had even been published by Anandtech.

Pro tip: If a company publishes a response to an article in a reputable publication before that one even runs, they’re invariably attempting to get out ahead of it with their own spin.

Let’s examine MediaTek’s statement in some detail. I’ll italicize the various dishonest statements, misrepresentations, and a few flat-out lies.

Just about every modern smartphone is designed to operate differently under different circumstances, meaning that devices are optimized to adjust for power and performance depending what applications are being run. MediaTek is known for our intelligent power savings innovations, which encompass a wide range of technologies that dynamically manage a device’s computing resources to provide a sustained user experience. This means that the device will only run full throttle (which eats up battery life) when it’s absolutely necessary for a good user experience.

This is a lie. The entire reason Andrei Frumusanu detected a problem in the first place is that he noticed an enormous difference in a PCMark score. Here’s the cheating versus non-cheating results, running on an Oppo Reno3 Pro P95:

Image by Anandtech

That’s a 6781 for the non-cheating score versus 9048 for the cheating score, or a 1.33x performance improvement in an application that has absolutely nothing to do with anybody’s “good user experience.” In fact, overclocking an application like PCMark is particularly terrible, because the entire point of PCM is to create a more realistic testing environment in which real-world latencies, load times, and the like are all taken into effect. It isn’t meant to be a strictly synthetic or single-feature test the way some mobile benchmarks are.

Our chipsets deliver a big power boost when the most demanding applications are being run so people can enjoy smoother gameplay and get the most out of the latest AI applications. On the other hand, our chipsets will operate in an ultra-power savings mode when you’re doing light tasks like checking email.

MediaTek may be referring to the High Power mode option now available on some phones, but this is an entirely distinct feature from the whitelist overclocking the company is engaging in. High-performance mode is an explicit capability you enable in Android, typically after clicking through a warning about the impact it may have on your phone’s battery life and SoC lifespan. Furthermore, none of the whitelisted applications are games or applications users can “get the most out of” in the first place, unless your goal is to lie about your SoC’s performance.

I included the last sentence because this one isn’t a lie. Salting a document with a mixture of blatantly false claims and explicitly truthful ones is a common PR tactic in this writing genre.

We do find it interesting that AnandTech has called into question the benchmarking optimizations on MediaTek powered devices, when these types of configurations are widely practiced across the industry. If they were to review other devices, they would see, as we have, that our key competitor has chipsets that operate in the exact same way – what AnandTech has deemed cheating on device benchmarking tests.

The entire insinuation of this paragraph is a lie. Over the past seven years, Anandtech has investigated mobile benchmark cheating on devices manufactured by Asus, Honor, HTC, Huawei, Google, LG, Motorola, Nvidia, Samsung, and Xiaomi. (Not every company received its own story, and not every company cheats, but the links above cover reports on all of the companies in question.)

Some companies cheat more than others and some haven’t been found to cheat at all. Samsung and Huawei have both cleaned up their acts since being found to be cheating.

One of the most common ways that companies attempt to hide their own malfeasance is by claiming bias, incompetence, or both on the part of the investigator. Here, MediaTek is flatly declaring that Anandtech failed to perform due diligence and implying that this failure is why the site has published a poor article. They aren’t bad people, exactly — they just don’t know the ins and outs of the mobile industry, or how companies actually build products. The best part about a phrase like “We do find it interesting,” is that MediaTek doesn’t have to make any specific accusation that can be refuted. “We do find it interesting” means “We want you to believe our implication that Anandtech targeted us for invalid reasons, but you get to make up your own explanation as to why that is.” Maybe you’ll pick racism, or jingoism, or professional incompetence. The point is, MediaTek wants you to be thinking about it.

I realize that the ins and outs of corporate communication aren’t particularly interesting to a lot of people. That’s also part of the point. MediaTek knows that most people will never do the research to find out if its counter-claims are true or not. It knows that there are plenty of people who are predisposed to see the media as immediately in the wrong, and who will take the company’s side. It knows that raising the idea of discrimination will strike a chord with its own customer base in a way that emphasizes how an American website is picking on a Chinese company at a time of increased tensions between the US and China.

I think it’s important to show how companies attempt to manipulate the conversation around their own products by mixing lies and honest statements. In this case, MediaTek’s pre-published response to an article it hadn’t even read, combined with a close reading of the text itself, illustrates just how much of this blog post is composed of false claims.

Now Read:

Leave a Reply

Your email address will not be published. Required fields are marked *