Building C3 Generative AI Search & Chat Experience for Enterprise Users from Scratch
Introducing our latest venture: the C3 Generative AI Search & Chat Experience for Enterprise Users, aimed at revolutionizing data accessibility and data transparency. Say farewell to hours wasted on disparate platforms - our intuitive interface streamlines data retrieval, empowering users to swiftly act on insights with confidence in AI-generated responses.
-
Mission: To provide a unified knowledge source that enables enterprise users with rapidly locating, retrieving, and acting on enterprise data and insights through an intuitive search and chat interface.
-
Team: 6 Data Scientists, 10 Engineers, 3 Product Managers, 2 Designers
-
Design Time Frame: January 2023 - June 2023
-
Tools: Figma
Ask critical questions efficiently -
Search to retrieve data, Use AI summary & chat to process data
User Control on Data Sources & AI Output through Document Filters
Context
2023: THE YEAR OF GENERATIVE AI
It is fair to say that 2023 was a year of Generative AI. There were so many new GenAI apps came out, benefiting people's life through creative content generation. C3 AI as a pioneer in creating AI tools for enterprise users also created a Generative AI app tailored to enterprise use cases.
So, how might we leverage Generative AI technology to benefit enterprise users?
COMMON PAIN POINTS OF ENTERPRISE USERS
Let's meet one of the target users. James is a sales manager whose goal is to effectively create a sales plan, manage sales staff, and support the sales team to achieve the sales goals. Everyday, he asks himself a few critical questions, like "Will my team hit our target this quarter?". However, can he confidently answer critical questions efficiently?
The answer is no -- his current approach is not sustainable. He has to navigate to a number of platforms to find relevant documents or product information he needed, and use a number of tools to visualize the views he wants. As a result, 60% of his time was spent on searching through product documents versus on core sales activities, leading to leaving revenue on the table.
Unfortunately, James was not alone. After interviewing 15 enterprise users from different industries, we concluded that they all face the common pain points of -
-
Spending long hours on retrieving, and processing enterprise data from different platforms
-
Lacking trust for AI-generated answers, fear hallucination response
-
Lacking control over data sources and AI output
SOLUTION: C3 GENERATIVE AI
That was why we created this new app - C3 Generative AI. It enables rapid access to information, analyses, and predictive analytics associated with and derived from enterprise and external systems
With this app that integrates all necessary data, enterprise users like James can simply ASK a question in an easy-to-use, straightforward interface. And then, they could get ANSWERS to those questions quickly, efficiently, easily, without needing to navigate disparate platforms. At last, they could actually take ACTION on those insights. Now James can access product info easily and spend more time on core sales, resulting in more revenue, better win rate, and higher productivity.
Building C3 Generative AI from Scratch
KEY DESIGN DECISIONS
To build C3 Generative AI, we went through three main design decisions - set layout, increase data transparency, and enhance user control.
DESIGN DECISION 1: SET A SCALABLE LAYOUT FOR COMMON USE CASES
To determine the effective features and the layout from a blank paper, we approached the challenge with three steps - learn common use cases, analyze scalability for layouts, and define new design components.
WHAT ARE THE COMMON USE CASES?
To understand common use cases, we first reached out to domain experts for each product. By conducting 27 interviews in total, we brainstormed a high number of use cases. Then, by reading through all of them, we realized nearly half of the use cases are related to documents. Upon discussion with PMs and data scientists, we learned that structured data like PDF, Excel, and CSV documents are much easier to analyze and implement because the specific and organized nature of structured data allows for easy manipulation and querying of that data. Therefore,we limited the scope of our first version to be document-based Generative AI.
After organizing all the FigJam Cards, we noticed that there are three main use cases for document-based GenAI:
-
Locate and navigate to different docs
-
Summarize/extract documents
-
Ask follow-up questions based on documents or summaries
Upon brainstorming with the whole team, we realized that we could achieve these use cases through the following GenAI solutions:
-
Search: retrieve relevant information based on user query, provide links to documents
-
AI Summary: provide insights in natural language based on document sources
-
Chat: allow users to ask followup questions in a given context
WHAT LAYOUT IS SCALABLE FOR ALL USE CASES?
In the beginning of the project, we have explored a variety of layouts:
Some of them are more search-centric --
Some of them are more chat-centric --
After exploring so many layouts...
All these layouts seem okay, but which one is most intuitive to enterprise users?
To find an answer, I first reached out to the domain experts again and gathered 30 potential queries users would ask from each product team. After reading through all the queries, we found that users generally follow an order of understanding "what happened", "why this happened", and "how to improve it" when they investigate issues.
Then, by re-studying where users' pain points occurred, we discovered that users normally encountered a long process of information retrieval and content generation when they investigate "what happened"; and then, they need to spend a lot of time and effort on manual analysis to understand "why this happened" and "how to improve it".
At last, to concur these problems in the workflow, we identified that users can search and get an AI summary of what's going on when they investigate "what happened", and then they can ask follow-up questions in chat to figure out "why this happened", and "how to improve it".
Therefore, we chose this search-centric design as the final layout because users can first use search results and AI summary to investigate "what happened", and then ask followup questions to learn "why this happened" and "how to improve".
DEFINING NEW COMPONENTS
As this is a new app with many novel design patterns like Search, AI Summary, and Chat, we worked with a design system designer to define those design components. We've explored different background colors, font sizes, paddings, and more to finalize on this version.
Detailed documentation of design components
Documentation of interactions
Switchable between dark and light mode
At last, we conducted UX QA with engineers and provided additional design specs to make sure the implementation is consistent with the designs.
With a scalable structure that fits users' workflow, James can now ask critical questions efficiently in a unified knowledge source without having to spend long hours on retrieving and processing data from different platforms.
DESIGN DECISION 2: INCREASE DATA TRANSPARENCY
To improve users' trust in AI-generated responses, we conducted tests and research. We learned that enterprise users care a lot about where the generated result comes from. Thus, it's essential for us to increase the data transparency of AI responses.
To create a trustable design, I first defined four design principles to help me evaluate my designs and guide me through design decisions:
-
Explainability
-
Actionability
-
Efficiency
-
Feasibility
When I struggled to decide whether I should have sources of the AI response hidden in a button or displayed directly under the response, the principles guided me that the second option is better because it's more explainable by displaying sources directly and it's more efficient for users to dig into the source document.
Moreover, when I was trying to decide the interaction pattern for showing relevant passages, I could quickly analyze that clicking to view the source document in a new tab is inefficient, and clicking to view relevant passage in a modal would be hard to implement due to lack of diagram parser at the moment. Therefore, hovering on source button to view relevant excerpt is the best option as users can view the contexts in the source document efficiently and be able to dig into the document.
Final design
Users can hover on the source button to view relevant excerpts that help generate the AI response
By providing transparency on document sources, the AI-generated answers become much more trustable for enterprise users.
DESIGN DECISION 3: ENHANCE USER CONTROL
Almost all consumer-facing GenAI apps do not have the option of filtering data sources and configuring AI outputs. However, it is eseential for enterprise users to filter documents using keywords, dates, names, and more so that they can receive more specific AI outputs efficiently.
To enhance user control through document filters, I've explored 4 main filter styles -- filter panel, filter pills next to search, auto-detect filters from search query, and a filter bar.
1. Filter Panel
The filter panel is the existing filter pattern in our design system. However, the panel would take up a large amount of space in the Search & Chat experience which is not suitable. Moreover, if there are a high number of filters, the panel would grow very long, which would cause difficulty in locating any specific filter and excessive scrolling. Therefore, I seek a new filter pattern that can take little space while being scalable.
2. Displaying filter pills next to the search bar
While this approach saves a lot of space on the interface, it is still not scalable for a high number of filters as the top bar would run out of space easily.
3. NLP Filters
This approach was inspired by AWS QuickSight Q in which the filterable keywords can be auto-detected from search queries for users to quickly adjust. However, while it is intuitive to use and quick to change filters, it is hard for data scientists and engineers to implement at the moment.
4. Filter Bar
The fourth approach is to add a filter bar below the search. Users can first select a filter category like "date created" or "document name"; then they can configure the detailed filter like the date range of when the documents were created; at last, once they apply the filters, the search results would be updated based on the filters. The filter bar approach is clean visually and is scalable for many filters. The downside is that there would be extra clicks as users need to go through 2 layers of filter selection.
To make a decision, I mapped out a diagram evaluating the effectiveness and feasibility of the four options. The filter bar option landed on the quadrant that is highly effective and highly feasible. Therefore, we proceeded with the filter bar pattern as the final choice.
Final design
Enhance User Control through Document Filters
By providing document filters, users can now control data sources of the AI outputs.
Results & Impacts
MARKET REACTION
In July 2023, we released the MVP version of C3 Generative AI that delivered a document-based search and chat experience for enterprise users. It set up a successful beginning for the GenAI product suite as the app was adopted by a high number of internal and external users. We've attracted 20+ new customers for GenAI trials and pilots.
USER FEEDBACK
We've also received a lot of positive feedback from users that C3 Generative AI has made their daily workflows much more efficient than before.
