With a unique lifelong career in consumer-advocacy organizations, the Digital Lab team interviewed Consumer Reports’ VP of research, testing and insights, Liam McCormack, about the role of standards with the consumer movement, current challenges and future opportunities for industry standards and comparative ratings. One particular example we talked about illustrated the strong culture at CR to protect the rights, interests and needs of consumers by integrating stakeholders across industries.
“We knew there was a problem with furniture tip-overs injuring or killing children,” explained McCormack, who shared an impactful initiative in CR’s history. To spark industry change, CR presented scientific data from their testing labs to a standards committee. Beyond highlighting statistics and numbers, CR elevated voices of parents who had experienced loss due to these furniture tip-overs to speak for themselves. “The combination of rigorous testing and research, journalism, and advocacy — and feedback directly from consumers helped us convince the industry that something needed to be done,” he said. CR is now pursuing a strong, mandatory stability standard and will continue, as we always do with priority consumer issues, to hold the industry accountable.
At Consumer Reports, McCormack oversees over 60+ product testing laboratories and 140 product test engineers, researchers and safety experts who work on consumer testing, ratings and reviews.
What is the role of creating scores and standards in facilitating the consumer movement to protect and advocate for consumer interests?
Laying the baseline for performance. Standards and comparative evaluations are geared to amplify and provide help to ensure consumer protections as products and services evolve. In our conversation, McCormack articulates how standards are critically important both as a marketplace protection and as input for developing CR protocols — they lay a baseline performance level of safety for products for sale and use in the US. “In whatever product category [we investigate], we often identify basic issues or gaps in safety, rights or performance,” he explains. “While standards are important, it is critical that they are kept up to date in terms of new product developments and how consumers actually use products.”
The power of comparative reviews. Yet, setting a minimum threshold for manufacturers should not be just checking a box to appear minimally viable — the organization focuses on the impact comparative ratings can have on the market. Take car seats, for example. “If you want to buy a product that exceeds the minimum standards and offers higher protection, our ratings can provide that,” McCormack says. “For car seats in particular, we use a scale of good, better, best — you can identify one that provides your child with the greatest margin of safety over and above the legal standard.”
A testing process to engage manufacturers in raising the standard. Scores and standards also impact how manufacturers and companies think about designing and testing their products and services. “There’s always a certain amount of skepticism to begin with,” he shared with us. After years of testing these products, “many manufacturers now go to our labs and use our test methodologies” in order to better understand how their product performs before launching and deploying millions of car seats in the market.
What are the key challenges that CR faces in planning, executing, and deploying standards and ratings?
Exploring consumer insights. McCormack highlighted that a crucial element is understanding how consumers actually use things at home and how products and services may positively or negatively impact their lives. “Your weakest link is your most vulnerable point,” he said. “Perhaps the weakest link might be your coffee maker that is connected to the internet,” he highlighted. “The manufacturer may not have a lot of expertise and may not be committed to keeping their software updated as new vulnerabilities are being identified.” He explained that the team is always looking to expand product categories and what we test.
Uncharted terrain with creating digital standards for devices and services. We also discussed the challenges of the Digital Standard, a set of standard guidelines to guide the future design of consumer software, digital platforms and services. Standards may not exist at all for new products or services, especially in articulating how data should be used or managed. In practice, “when [Consumer Reports] applies the Digital Standard, we have to boil it down to what a technician is doing to evaluate a broad range of products, which [will likely] need to be done on a category by category basis,” he outlined. Employing standards such as privacy or security may greatly differ from evaluating a videoconferencing service to a reproductive health app tracker.
Creating more transparency in industry practices. One way the testing team focuses on transparency is by analyzing products through a Document Review. During this, CR researchers comb through privacy policies, terms of service, and other publicly available documentation that will better clarify how companies use, manage, and share consumer information. CR also shares it’s digital standard-based protocols broadly so that manufacturers are clear about the benchmark that their products are being evaluated against. “The more you make a standard a living, breathing thing where a lot of people are looking, referring, commenting on it — the better,” McCormack explained. “Our standards are created from a consumer perspective and in an ideal world manufacturers will refer to that and will perform better.”
What will Consumer Reports need to do in the future to continue raising the standard for the industry?
“It behooves CR to develop methods more quickly,” McCormack explained. This allows the organization to cover more types of products and services — ”a role we must fill to realize new audiences.” He shared how the team is always looking for opportunities for engineers to be on standards committees so they can learn what is going on in other industries and report back.
Quickly evolving topics where there may not be current in-house expertise requires CR to find external experts to work with. “Just keeping on top of technical developments is clearly critical and we realize we can’t do it all ourselves, McCormack refined. “When we enter new spaces, it has to be done the right way.” For topics like privacy, security, algorithmic bias and disparate impact on marginalized populations, we need to work with other experts, stakeholders, and audiences, to come up with CR’s scores, ratings and recommendations.
In this one conversation, we talked about countless anecdotes, the highs and lows and progress in CR’s history. Looking back at the many years fighting for consumers through standards, I asked McCormack what has kept him involved in this work. “CR really was the first game in town in 1936 when it began comparing products. Although ratings and reviews are now ubiquitous, our rigorous scientific methods remain a unique benchmark in the marketplace.
Seeing products and services improve for all consumers based on our data and evaluations is what keeps me motivated.”
If you would like to get more involved with helping to shape The Digital Standard, please email digitallab@cr.consumer.org.
