Editor’s note: this is a guest post from students in the Master of Human-Computer Interaction (MHCI) program at Carnegie Mellon University, who have been working with CR to imagine and design AI agents that help solve common consumer problems. Read on to learn about their design process and check out other installments in the series.
In our previous blog post, we, five graduate students at the Human-Computer Interaction Institute at Carnegie Mellon University, explored several rounds of user testing aimed at optimizing AI interactions through multi-modal communication channels. This included asynchronous and synchronous setups, evaluating AI-operated service calls, and enhancing error repair mechanisms to strengthen user trust.
This week, we’re excited to unveil the culmination of our extensive research, testing, and prototyping: FairPlay, your AI advocate for all things customer service. How does FairPlay champion your rights as a consumer? Let’s take a look at how Alex, a hard-working millennial, utilizes FairPlay to uncover and counter the obscured terms often hidden by internet service providers. FairPlay ensures that you don’t get played; you get fair! Check out the video below:
Hate Calling Customer Service Agents? FairPlay AI Has Your Back
Imagine a world where the dread of customer service calls becomes a thing of the past. With FairPlay’s revolutionary agentic AI, this would be a reality. This advanced system would act autonomously on your behalf, skillfully handling customer service issues with minimal input from you. Simply choose the company, describe your problem, and FairPlay would take over—seamlessly managing calls behind the scenes. Whether it’s negotiating an overcharged bill or addressing poor services, FairPlay’s AI would operate with autonomy and efficiency while providing user control. Curious about what goes on behind the scenes? Here is a sneak peek:
Keeping You Informed Every Step Along the Way
We recognize the importance of user control when it comes to innovative technologies like agentic Voice AI. That’s why we would integrate robust, fail-proof features into FairPlay AI to keep users well-informed and in command at every step. Users would receive timely push notifications before, during, and after each call. These updates would allow users to verify account details, make crucial time-sensitive decisions—such as accepting promotional offers—and review a comprehensive summary of the call. Through these push notifications, we aim to build trust and transparency with our end users.
Policy-based Arguments at Your Fingertips
During our exploratory research, we discovered that while some individuals prefer to handle customer service calls themselves—citing concerns that agentic AI may not fully match the advocacy level of a human representative—they still seek guidance to effectively assert their rights. In response, we’ve introduced pre-call company policy cards, along with tips and tailored talking points specific to the customer service issue at hand. Prior to their call, users would be able to describe their issue to our system, and FairPlay AI would generate relevant policy arguments and talking points. Additionally, during the call, users would have access to all necessary account information, policy cards, and talking points, ensuring they are well-prepared to advocate for themselves effectively.
Accessing the Most Optimal Contact Channel for Any Company
Each company has its preferred methods of contact, whether through a phone call, live text chat, or email. For instance, resolving an issue with Uber Eats might take three hours over the phone compared to just ten minutes using their live chat feature in the app. FairPlay AI would leverage this knowledge of behind-the-scenes shortcuts and recommend the fastest and easiest way to contact a specific company based on aggregated company data. This would enable users to bypass lengthy queues and address their issues quickly and effortlessly, ensuring a hassle-free experience.
Decoding the Core Technology of FairPlay’s Agentic AI
At the heart of FairPlay’s agentic AI would lie a sophisticated Multi-agent Large Language Model (LLM) system. Here’s how we imagine it would work : When interacting with a customer service representative, anything said is converted into text—what’s termed a “literal utterance.” The system’s first agent, the Intent Mapper, identifies and maps this utterance to a specific intent. This intent is then relayed to the second agent, the Policy Matcher, which retrieves relevant company policies to support the user’s argument.
The process doesn’t stop there. The third agent, the Response Generator, crafts a reply based on these policies, while the fourth agent, the Placeholder Filler, integrates the user’s personally identifiable information (PII) like name, address, and purchase details into the response. Importantly, our system is designed to ensure that the LLMs cannot access sensitive user information, maintaining strict privacy standards.
To further refine the accuracy of the system’s responses, an additional Discriminator Agent is employed. This agent evaluates the effectiveness of our responses based on the customer service representative’s subsequent utterances. Feedback from this evaluation is then used to update the other agents, continuously improving our system’s accuracy and reliability. Here is a graph illustrating the complex interrelationships within our proposed multi-agent system:
Ensuring Data Transparency and User Control
At the core of FairPlay’s user experience would be a strong commitment to data transparency. Throughout the onboarding process, we’d guide users through how FairPlay utilizes their data and precisely what data we can access. Users would be granted the flexibility to allow or deny access to their information, empowering them to control their privacy effectively. Behind the scenes, we would be committed to ensuring that users’ private data is exchanged minimally, maintaining the highest standards of data integrity and security.
While FairPlay AI would not require access to necessary PII and purchase details to provide case-specific recommendations based on company policies, we would prioritize user consent. This sensitive information would be shared with businesses strictly under the user’s explicit permission, reinforcing our dedication to maintaining trust and transparency in every interaction.
Navigating the Legal Landscape of AI and Phone Call Recordings
You may wonder, is it legal to record a phone conversation without the consent of the other party, or to have an agentic AI make calls on your behalf?
In the U.S., the legality varies by state. As illustrated in the map below, states colored blue are considered single-party consent states, meaning you can legally record a conversation if you are one of the participants without needing the other party’s consent. In these environments, FairPlay AI would operate within the confines of single-party consent states.
Regarding the legality of AI acting on one’s behalf, the Uniform Electronic Transactions Act of 2000 provides that electronic agents may form contracts on behalf of individuals without their direct oversight of the agent’s actions or the agreement’s terms. This legislation permits FairPlay’s agentic AI to negotiate autonomously with companies, even if the user is not present.
Additionally, the “Notice and Explanation Clause” mandates that AI products must inform and clearly explain their functions and actions to users. These provisions collectively offer a strong legal foundation for FairPlay and potentially set a precedent for other agentic AI applications to operate autonomously on behalf of users.
Co-creating Value for Both Consumers and Businesses
Consumer Reports (CR) has always been committed to fostering a fair and just marketplace for all. FairPlay would not only deliver value to consumers but also offer compelling benefits to businesses. Why would businesses choose to participate in the FairPlay ecosystem? The answer lies in the mutual advantages that arise from this collaboration.
FairPlay would function as a centralized platform dedicated to customer service, amassing a wealth of anonymized customer data on a variety of issues across industries. This would allow businesses to glean valuable insights from these anonymized customer experience (CX) datasets. By leveraging this data, businesses could enhance their customer service strategies and potentially reduce their operational costs.
The benefits for businesses are clear: improved customer service leads to greater consumer loyalty and retention. This, in turn, boosts business reputation and drives higher revenue. The symbiotic relationship that would exist between consumers and businesses in the FairPlay ecosystem is illustrated in the graph below.
Final Presentation at the CR NYC Office
Recently, my team and I had the privilege of presenting our FairPlay Agentic AI concept at CR’s NYC office. The event was well-attended, and the session sparked a lively discussion, with attendees keen to understand the future dynamics of AI interactions. One question that stood out was about a not-so-distant scenario where FairPlay AI would interact not with humans but with other agentic AIs. How would FairPlay handle this? We argue that in AI-to-AI communications, FairPlay’s role becomes crucial. Equipped with extensive knowledge, FairPlay would be adept at managing complex interactions with other AIs, ensuring users’ needs are proficiently advocated.
Another interesting question from the audience concerned whether other AI startups are offering similar services on behalf of users. Indeed, there are numerous startups out there, which automate tasks from booking doctor’s appointments to drafting legal documents. We believe that the legal framework, provided by the two legislations previously discussed, underpins the operations of FairPlay and similar AI services, granting them the legal authority to operate effectively.
That’s a Wrap!
The past seven months of collaboration with CR have been an extraordinary journey. We are deeply grateful to everyone at CR for providing us with the platform and opportunity to shape the future of customer service through the power of Agentic AI. A special shoutout to Daniella Raposo, Justin Huyhn, Ben Moskowitz, Ginny Fahs, and Dan Leininger for your invaluable support and guidance throughout this process. We can’t wait to see the next chapter for FairPlay and are excited to continue helping consumers advocate for their rights in the digital age.
If you have any questions about FairPlay, reach out to us at:
- Mikayla Alsopp, malsopp194@gmail.com
- Vadeihi Chotai, cvaidehi08@gmail.com
- Peiwen Huang, pwh27design@gmail.com, peiwenhuang27.github.io
- Kai Lin, kailindesigner@gmail.com, www.kaidoescoolstuff.com
- Jonah Sklar, jonah.sklar@gmail.com
