top of page
Re-imagining a trustable AI evidence package for enterprise CRM
In the competitive landscape of enterprise CRM platforms, our sales teams expressed a growing concern: a lack of trust in the AI-generated results. This hurdle was impacting their decision-making process and overall performance. The need was clear - a redesign of the AI evidence package that not only improved trust but also aligned with the sales team's workflow.
-
My role: lead designer
-
Collaborators: 1 PM, 1 researcher, 5 data scientists, 5 engineers
-
Design Time Frame: Nov 2022 - Jan 2023
-
Design Success Metrics: Scalability, Explainability, Actionability, and Implementation Viability
-
Tools: Figma, ReGraph
Introduction
WHAT IS CRM
Context & Pain Point
CURRENT USER GROUP FOR C3 CRM: INTERNAL SALESPEOPLE
Let's meet Jackie. He is a sales representative whose goal is to reach quarterly sales target by managing his deals and forecasting what deals he can close.
On a daily basis, Jackie would come to the opportunity detail page to manage and track his deal. On the top panel, while he estimates the likelihood of closing the deal for this quarter on the "Probability" field, the "C3 AI Probability" also provides a AI prediction as a reference.
On the bottom right side, the "C3 AI Insights" section provides a collection of data and insights that explain how an AI prediction or recommendation was made.
However, Jackie has been lacking trust in the CRM AI Insights because he is not sure why there is a big gap, or what contributed to the changes in AI probability over time, and he thinks the presented evidence is insufficient to justify the score change.
Unfortunately, Jackie was not alone. A researcher and I have interviewed 7 sales representatives and we concluded that they all share a similar experience.
See quotes from different users
Therefore, we initiated the project of redesigning a trustable CRM AI Evidence Package for Salespeople. I collaborated with 1 researcher, 1 PM, 5 data scientists, and 5 engineers on this project.
It is essential to build trust for AI because before we ask users to use AI, we first need to build trust between humans and AI. Only with strong user trust, we can achieve a higher usage rate and better user satisfaction, and our salespeople can sell to external customers more confidently.
Solution Approach
To solve the issue of lacking user trust in AI, I approached the challenge with 3 steps: evaluation, investigation, and action.
Solution Approach
EVALUATION - Why is the current AI evidence package not trustable?
In the first step, I want to learn why the current AI evidence package is not trustable by better understanding users' pain points.
1. Evalute Issue
To break down the lack of user trust issue, I summarized 3 main pain points from the user research.
Firstly, the presented evidence is insufficient to justify the score change. For instance, sales reps normally know how long their deal is in a certain sales stage very well; seeing information they already know causing a score change does not help them believe in the power of AI. They'd like to see supporting evidence that is novel to them.
Secondly, while we show an AI probability score and its history, users do not understand what evidence below contributes to a specific score change on the chart. It would be more helpful if the evidence could be closely associated with the score change over time.
Thirdly, users cannot drill into insight details like raw data. The lack of actionability also resulted in the lack of user trust in the AI evidence package.
To concur with those issues, I have done a number of brainstorming sessions with PMs and data scientists, and we eventually landed on three major solutions - improving the variety of data, providing a time component for the package, and enhancing the interpretability and actionability of AI evidences.
INVESTIGATION - What additional data can make it more trustable?
Now that we understand user pain points and have the solutions in mind, I then move on to discuss and understand data and viability with data scientists and the PM.
2. Investigate Data
After a number of brainstorming sessions on the helpful factors for different types of salespeople, we summarized three data types that could be integrated into the evidence package - coutinuous data, static data, and discrete data.
With the 3 types of data in mind, the challenge for me as a designer is then - how might we design a scalable pattern that can integrate any kind of data?
ACTION - How can design improve it?
At last, we are in the ACTION step, which is using design to solve the issue. There are 4 major design decisions during the process - data categorization, enrich data visualization, simplify navigation, and enhance actionability.
3. Design Decisions
1. DATA CATEGORIZATION
As we know the 3 types of data, what do they look like if we translate them to visuals directly?
They look very complex and hard to understand, which is not suitable for salespeople who are non-technical.
To solve this issue, my task is then to translate DATA language to USER language.
Instead of using data types that are complex to understand, I found that using data categories like score movers, activities, external events, and opportunity changes that are very familiar to the users is much more appropriate.
2. DO ADDITION - ENRICH DATA VISUALIZATION
To enrich data visualization, there are two main challenges - how might we design an easy-to-understand evidence package for salespeople who are non-technical? How might we design a scalable pattern that can integrate any kind of data?
After exploring 50+ potential designs, I always feel like something is missing in each design and I can hardly make a design decision...
Therefore, I developed a set of success metrics for my design to help me streamline my design process, including scalability, explainability, and implementation viability.
Then, I tested a few designs with 3 salespeople using the success metrics, which helped me narrow down my design direction.
Once I had a relatively satisfied design, I brought it to engineers to discuss implementation viability. Besides giving a rating, engineers also shared with me a few additional features supported by Kronograph. The PM and I found that using icons and a horizontal layout of visualization and evidence can be very helpful for our evidence package.
At last, we had a winner that is scalable, easy to understand, and viable to implement.
3. DO SUBTRACTION - SIMPLIFY AND IMPROVE NAVIGATION
After having an enriched data visualization, if we put the evidence package back to the opportunity page, we can spot a few issues:
1. The probabilities on the top panel and the evidence package are repetitive;
2. The association between the probabilities on the top and the evidence package is weak because there is an opportunity details section in the middle;
3. While the forecast movers should be closely associated with the probability line chart, they are now separated.
Therefore, I decided to:
1. remove repetitive probabilities on the evidence package, and enhance the probabilities on the top panel;
2. Remove the middle section to enhance the association of AI probability and evidence package;
3. Integrate the forecast movers into the probability line chart.
4. ENHANCE ACTIONABILITY
At last, I'd like to design a scalable pattern that helps enhance AI interpretability and user actionability.
I've explored several options. The first one is to make feeds expandable/collapsable. However, the expandable section has very limited space, which is not scalable for all kinds of data.
The second exploration is adding an additional side panel for double-click actions. However, it will squeeze the chart component a lot, lead to a messy information hierarchy, and it is also hard to implement as we do not have this component for section level.
At last, my final decision is to use a modal for users to drill into insight details. A modal would be scalable for all kinds of data; it provides a focused view for raw data; and it is easy to implement as an existing component.
New Experience
REDESIGNED VERSION OF C3 AI CRM EVIDENCE PACKAGE
Final Design
Solution Summary
In summary, we have redesigned the full page of opportunity detail to enhance user trust in the AI evidence package.
Soluton Summary
The new experience has solved all user pain points by improving the navigation, the variety of data, and adding in double-click actions.
After seeing the design of the new experience, our salespeople were very excited about the change, especially about seeing external information without leaving the application.
Reflection
Reflection
bottom of page