Futuremark, maker of the 3DMark cross-platform benchmarking app, announced it removed from its scoreboard several Samsung and HTC devices “suspected of breaking [its] rules.”
The devices in questions are the Galaxy Note 3 (Exynos and Snapdragon versions), Galaxy Note 10.1 2014 (Exynos and Snapdragon versions), the HTC One, and the HTC One Mini.
Established in 1997 in Finland, Futuremark is one of the most respected benchmarking software developers, having built a reputation for itself with benchmarking tools for gaming PCs. Earlier this year, the company released 3DMark for Android, iOS, and Windows, as one of the few benchmarks that allows for comparisons between devices on various platforms.
Futuremark did not specify the reasons it delisted the HTC and Samsung devices, but hinted at the practice of detecting the presence of benchmarks and artificially increasing CPU and GPU performance to obtain top scores.
People rely on Futuremark benchmarks to produce accurate and unbiased results. That's why we have clear rules for hardware manufacturers and software developers that specify how a platform can interact with our benchmark software. In simple terms, a device must run our benchmarks without modification as if they were any other application.
In fact, the devices that Futuremark delisted are exactly the same devices that AnandTech accused of cheating 3DMark in a report from October. Samsung, HTC, Asus, and LG were all found to be rigging some benchmarks by AnandTech.
What’s the consequence of this delisting? The devices will appear unranked, and without scores, at the bottom of the 3DMark scoreboard.
In the bigger picture, this kind of public shaming could make manufacturers think twice before tampering with benchmarks. Futuremark is the first benchmark maker to delist devices due to suspicions of rule breaking, but others may follow suit.
Android Match
Post a Comment