Verizon Communications

Experiential AI for theLargest Wireless Carrierin the United States

How Verizon Communications partnered withexperts at the Institute for Experiential AI tointegrate Responsible AI into its innovation roadmap.

Delivered

• Comprehensive revision of Verizon's Responsible AI roadmap
  Charts a responsible course for AI innovation
• Full update to Verizon's AI Risk Assessment Framework
  Enables AI to be “responsible by design”
• Technical assessment of AI model fairness valuation metrics
  Empowers teams to examine fairness themselves

“We want to set the standard for what being responsible means in applied AI in a large enterprise.”

Meghna Sinha, VP of Artificial Intelligence and Data

“We are looking to do something more than avoid risk. We want to build a program that is sustainable so we can keep optimizing and building.”

Xuning Tang, Associate Director of Responsible AI

Challenge

Be the Leader in Responsible AI

The AI leaders at Verizon Communications knew they needed a framework to proactively evaluate new AI initiatives in areas such as privacy, bias, and social and environmental impact. Without being a leader in AI Ethics, they couldn’t be a leader in AI overall.

Chief Data & Analytics Officer Kalyani Sekar and Meghna Sinha, Vice President of Artificial Intelligence and Data, brought on Xuning Tang, former director of the Berkeley Research Group, to lead ResponsibleAI programs at Verizon.

Tang and his team created a three-year Responsible AI Roadmap that outlined the key areas in which they needed to invest in order for Verizon’s AI to be truly responsible by design. They also wanted experts to help them operationalize Responsible AI at an enterprise level. That’s why they reached out to the Institute for Experiential AI (EAI) at Northeastern University.

“At the onset we saw Responsible AI as a three-legged stool: Technology, Governance, and Policy,” Sinha says. “We found that most vendors and consulting organizations offered capabilities in only one of three areas. Therefore, we have set out to build our program and roadmap as modularized services that can be developed independently and connected as we operationalize them,” she continues.

“Northeastern provides a different perspective,” says Tang. “Not just an artificial intelligence and machine learning perspective and not just an applied ethics perspective. It’s a portfolio of different schools of thought. That is very important and very hard to find.”Together, Verizon and EAI got to work.

Verizon's "Three-Legged Stool" of Responsible AI

Approach

Identify Top Priorities

“Verizon created a Responsible AI roadmap built to handle not just current uses of their technology, but multiple future projects,” says Cansu Canca, Director of Responsible AI Practice. “They also saw the need to assess ethics risks and embed fairness within their innovation cycle.”

Putting principles in practice meant selecting a partner who could develop both technical and ethical approaches. Verizon focused on operationalizing ethics at an enterprise scale. They quickly began describing the work as Responsible AI instead of Ethical AI, which limited the scope of the work too much. “Responsible AI is about creating systems and processes to ensure AI is developed and used ethically and keeps at pace with the evolving AI regulations,”Sinha says.

Canca, EAI Director of Research Ricardo Baeza-Yates, Senior Data Scientist Ben Batorsky, and Postdoctoral Researcher in RAI Laura Haaber leveraged technical and ethics expertise to support operationalizing AI ethics at Verizon. Together with Sinha and Tang, they identified three critical work streams: AI Ethics Roadmap,Ethics Risk Assessment Framework, and Fairness Monitoring Methodology.

Research Current States

Verizon had already made significant progress in each of these three areas. The EAI team reviewed established processes and workflows in the context of Verizon’s top business and data initiatives in areas such as marketing and new product development.

From there, the EAI team worked with the Verizon Responsible AI team to develop a high-level Responsible AI roadmap which incorporates considerations of playbook, process, and people to turn Responsible AI into an innovation booster. The team identified which stakeholders from Verizon’s Agile Business Delivery needed to be involved in Responsible AI workflows.

“Our ambition was always about operationalized Responsible AI and therefore our scope included Principles, Playbooks, and Operational Capabilities.”

Meghna Sinha, VP of Artificial Intelligence and Data

Activate PiE Model

“Cansu [Canca] has a way to bucket tasks into people, processes, and playbooks. That is very helpful,” says Tang. “Otherwise, it’s easy to have blind spots.”

A model called the Puzzle Solving in Ethics (PiE) Model, developed by Canca during her time as founder of the AI Ethics Lab in Cambridge, MA, provided a foundation for the Responsible AI Roadmap and the Ethics Risk Assessment Framework. The PiE model identifies strategy, analysis, and training as the critical components of a successful roadmap.

“The strategy portion of the PiE Model involves putting a structure in place to use in every single innovation cycle, not just as a one-off in a project,” explains Canca. “Verizon understood that AI ethics is an ongoing effort.”

Operational Capabilities

Model Registry: a centralized location for teams to manage models according to Verizon Responsible AI practices and principles

Risk Rubric: a formalized evaluative structure used to qualify and quantify the potential effects of model deployment

Model Observability: the degree to which a model can be evaluated according to transparency, reproducibility, and similar standards

Results

At the conclusion of the project, EAI Responsible AI and the Verizon Responsible AI team had completed a comprehensive revision of Verizon’s Responsible AI Roadmap.

Canca’s team also supported a full update of Verizon’s AI Ethics Risk Assessment Framework.

“The world has witnessed rapid AI development over the last few months. However, putting AI to use is not an easy task,” explained Tang, “partially because there are a lot of unknowns and risks in AI.”

Tang continues, “But the Responsible AI program we have now enables us to explore that in a way that is safe.”

In addition, Baeza-Yates and Batorsky delivered an assessment of Verizon’s current model fairness evaluation metrics that could be used to improve Verizon’s model fairness detection methodology.

Beginning in early 2023, these three deliverables were integrated into the AI product development cycle. Tang pointed to recent growth in AI globally to demonstrate how the integration of ethics provides value to Verizon.

“We always aimed at being proactive,” says Sinha.

“Proactive means being a few steps ahead of regulations but it also means we are responsible by design,” explains Sinha. “Therefore, we want to set the standard for what being responsible means in applied AI in a large enterprise.”

Collaborating with the Institute for Experiential AI provided Verizon with the tools to become the industry leader in Responsible AI. Since Responsible AI is an ever-evolving effort, the Institute’s ability to enable Verizon’s employees–rather than simply “throw something over the wall”–proved to be a critical differentiator.

“We are looking to do something more than avoid risk. We want to build a program that is sustainable and we can keep optimizing and building,” Tang says. “By working with the Institute for Experiential AI Northeastern University, we are building our capabilities and not just acquiring a tool.”

Responsible AI