23 January 2017
Apple's new MacBook Pro series for 2016 was a notable improvement over past iterations, and included an innovative Touch Bar that was the subject of much talk. But despite all the goods of the new MacBook, its battery performance remains shrouded in controversy. Consumer Reports, which previously tested the battery and released results that showed inconsistent performance, has been working with Apple ever since and has decided to re-run the tests again.
The inconsistent battery life caused the new MacBook Pro to lose out on a recommendation by Consumer Reports, which is the first time such a thing has happened for a MacBook. Consumer Reports is a well-known nonprofit organization that provides unbiased product ratings and reviews. Following reports of inconsistent battery performance from the new MacBook, Consumer Reports did an experiment of their own to establish this. And their findings were less than satisfactory.
For the battery test, we download a series of 10 web pages sequentially, starting with the battery fully charged, and ending when the laptop shuts down. The web pages are stored on a server in our lab, and transmitted over a WiFi network set up specifically for this purpose. We conduct our battery tests using the computer’s default browser—Safari, in the case of the MacBook Pro laptops.
During the tests, we set each laptop screen to remain on. We use an external meter to set the display brightness to 100 nits—a typical level you might use indoors or out. And, we turn off any automatic brightness adjustment in the laptop’s settings.
The test was done on three MacBook Pro variants; the 13 inch Touch Bar model, 13 inch model without the Touch Bar, and a 15 inch model with Touch Bar. The test showed a battery timing of 16 hours on the Touch Bar model, with the same giving 4 hours on a re-run of the same test. The same happened with the model without the Touch Bar, giving a battery timing of 19.5 hours and 4.5 hours in subsequent tests.
After the results were revealed, Apple disputed them. But Consumer Reports refused to run the tests again and stood by the results:
"We don’t believe re-running the tests are warranted for several reasons. First, as we point out in our original article, experiencing very high battery life on MacBooks is not unusual for us – in fact we had a model in our comparative tests that got 19 hours. Second, we confirmed our brightness with three different meters, so we feel confident in our findings using this equipment. Finally, we monitor our tests very closely. There is an entry logged every minute, so we know from these entries that the app worked correctly."
Now however, after working with Apple on the battery issue, Consumer Reports has agreed to run the tests again. It is now being said that the issue resided on the software level, and has been fixed by a macOS beta update (10.12.3) which was released only yesterday. Regarding this collaboration, Apple said:
“We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage. [Consumer Reports’] use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab. After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life…. This is the best pro notebook we’ve ever made, we respect Consumer Reports and we’re glad they decided to revisit their findings on the MacBook Pro.”
CR is now running the tests again and will release its report soon, which should indeed confirm that the issue was not hardware-related afterall.