Consumer Reports Explains How It Manages New Vehicle Testing Bias

In a podcast episode this week, Consumer Reports reveals what makes its reporting different from most of its peers.

In an unusual move, Consumer Reports has addressed what makes its vehicle test reports, reviews, and commentary different from its peers like Torque News, Car and Driver, Car Talk, and other automotive publications. The primary focus of its discussion was how Consumer Reports rents or buys its test vehicles instead of using borrowed media test vehicles. We’ll break down what Consumer Reports says, and then engage in a little fact-checking of our own to see how the company’s statements jive with your author’s decade of experience in automotive reporting.

What Is Consumer Reports?
Consumer Reports is a trusted source of information for many Americans. The nonprofit consumer group has been in business for 86 years. Unlike many automotive publications, Consumer Reports is not vehicle-centric. It reviews, rates, recommends and ranks everything from fertilizer to investment funds. Consumer Reports also advocates (verb). It takes a position on certain issues, and then it strives to convince readers to follow its beliefs. All of these things make the group very different from its peers in the automotive arena.

The video above contains the discussion we refer to in this story.

How Consumer Reports Vehicle Tests Differ From Other Publications
If you are not familiar with Consumer Reports, let us give you a quick overview what the Cars team at Consumer Reports does, and how vehicle consumers use the company’s vast amount of data and opinions.

First, Consumer Reports gathers a lot of owner data using questionnaires. It then distills this data down into reliability and satisfaction ratings based on the data - not its employees’ opinions. Consumer Reports also creates some fun lists where it then simply analyzes the results of the data. These lists are like “10 Best Cars,” and “10 Worst Cars.” That sort of thing. These are not just opinions of Consumer Reports staffers but rely mainly on what owners of the vehicles say in scientific surveys or what analytical testing uncovers.

Second, Consumer Reports provides “Test data.” The group has its own closed course, so it can do things like 0-60 MPH testing, brake distance testing, and other such analytical testing. It then reports that data in its reviews, or as part of its overall vehicle information packages subscribers and magazine buyers have access to. Consumer Reports was able to improve the braking of the Tesla Model 3 this way. Along with Car and Driver, the folks at Consumer Reports pointed out that the Model 3’s braking had issues. Distances for stopping were way too long. Based on Consumer Reports reporting, Tesla investigated the problem and fixed the brakes. All Tesla Model 3 owners benefitted.

Third, Consumer Reports drives cars that it buys or rents (more on that in a minute) and provides expert opinion. We say expert because the Cars group at Consumer Reports has multiple engineers and technicians on staff. Among them are its VP of Advocacy David Friedman (engineer), the Cars groups leader, Jennifer Stockburger (engineer), Senior Director of Auto Testing Jake Fisher (engineer), and Steven Elek (data analysts). We know many of the other contributors to Consumer Reports as experts who have training from performance driving schools and who attend vehicle testing and information programs. We have seen them at track events, and they walk the walk, as the saying goes. We should also add that the group is always willing to help a colleague trying to report on a reliability topic. They even have an outreach team member who handles PR.

What Makes Consumer Reports Different According to the Podcast This Week
In its podcast, team members explain that Consumer Reports buys the cars it tests from dealers or manufacturer’s retail outlets. They pay for the vehicles, so they are not beholden to the manufacturers to say nice things about them if those nice things are not warranted. However, there are hundreds of vehicles for sale in America each model year, and within those hundreds of models often up to a dozen trims. Obviously, Consumer Reports cannot buy them all. Further compounding the challenge is that there are often waiting lists months long to buy a new model. Some, like the Ford Maverick, are not even available for order.

To allow its team access to the most current technology and newest models, Consumer Reports says that it “Rents” cars from the manufacturers. Instead of borrowing a media fleet vehicle at no charge, it demands that the manufacturer accept compensation. The cars it rents are sometimes the same ones you see in images at Torque News. By renting instead of borrowing, Consumer Reports takes an added step to insulate itself from bias. This makes sense since Consumer Reports does actual testing. Not just reviews and opinion stories.

The rented cars (which are media test vehicles) are not used for the reliability or analytical testing Consumer Reports performs according to podcast member Mike Quincy. Instead, the purchased vehicles are used.

The team also says that it is rare an automaker will send anything but a fully-loaded top trim to a media fleet. While true, base models are included in the media fleet. Torque News had a chance to try out the $23K base Ford Maverick last month. We have reviewed lower than top trim media fleet vehicles from VW, GM, Honda, and Toyota. Honda placed a wide variety of Civics in the press fleet over the past year including Sports, Si cars, and Hatchback trims. The primary reason top trims are usually sent to media fleets by automakers is that they contain all the possible options. If the option is not included, it cannot be reviewed or evaluated. One risk automakers run by only providing top trims is that the prices reported are always the highest possible.

The Consumer Reports podcast team also touched on media member attendance of vehicle launch events, saying they are sometimes in top-tier hotels in places like Hawaii. Having attended events in Northern NH, rural Maine, and Minneapolis, I can attest to the fact that the events are held all over the U.S., and in some cases, lodging is not provided. Consumer Reports writer Keith Barry points out that prior to joining the CR team, he drove back and forth to such events in places like New Jersey.

Related Story: A Second New Toyota Tacoma Recall - Was Consumer Reports Right About Poor Reliability?

Look closely at the Consumer Reports “Worst” cars it lists by category, and you will find models from big-name brands like Toyota, Lincoln, Cadillac, Buick, Hyundai, and other brands. Those brands also make lists by Consumer Reports of the “Best” cars in a given category. There is a reason that Consumer Reports Autos is so respected by so many. The group tries hard to insulate itself as best it can from bias and is not afraid to talk about its methods. Tell us in the comments below if you feel Consumer Reports' efforts result in bias-free automotive content.

Author note: The author of this story has never worked for Consumer Reports in any capacity.

John Goreham is a long-time New England Motor Press Association member and recovering engineer. John's interest in EVs goes back to 1990, when he designed the thermal control system for an EV battery as part of an academic team. After earning his mechanical engineering degree, John completed a marketing program at Northeastern University and worked with automotive component manufacturers, in the semiconductor industry, and in biotech. In addition to Torque News, John's work has appeared in print in dozens of American news outlets and he provides reviews to many vehicle shopping sites. You can follow John on TikTok @ToknCars, on Twitter, and view his credentials at Linkedin

Re-Publication. If you wish to re-use this content, please contact Torque News for terms and conditions.