Find entry points for Gen AI to elevate employees' intranet experience
Client
Uber
Project length
4 months
(April 2023 - July 2023)
Team size
1 UX researcher (me) + 4 UX designers
Methods used
User interviews
Expert interviews
Surveys
Usability testing
Secondary research
IMPACT
-
Guided designers in creating an AI-powered intranet that significantly improved employee satisfaction with their intranet experience, validated through usability testing.
DOMAIN CONTEXT
If you are not familiar with Gen AI, here's what you need to know before reviewing this project...
1. What is artificial intelligence (AI)?
Many technologies around us were initially built to do things humans cannot, freeing us from our limitations. For example, humans can't fly, but we've created aircraft that can travel across continents and even into space. With these technologies, we can now travel faster and explore more of the world than ever before.
​​
AI, however, is different.
As its name suggests, "artificial intelligence" is designed to simulate human intelligence in an artificial, algorithmic way. It’s not a technology built to do what humans can’t. Instead, it aims to create machines and applications that can replicate human abilities like vision, language, and thinking—and even perform these tasks better than we can.​​
2. What is generative artificial intelligence (Gen AI)?
Generative AI is a powerful type of AI that has recently captured the technology industry's attention, leading to the creation of many popular applications. This AI takes prompts and generates the requested content, such as text or images.​
​
Gen AI app examples​​
Gen AI app
Text-generated AI such as ChatGPT
​
Prompts
"Can you tell me what is artificial intelligence? In a very succinct way."
AI responses
Gen AI app
Image-generated AI such as Midjourney
​
Prompts
"Student housing blended into the natural environment."
AI responses
BACKGROUND
The era of Gen AI is arriving, with people exploring how it can enhance user experiences across various aspects of life.
Since the launch of popular generative AI apps like ChatGPT and Midjourney, millions of users have been exploring ways to enhance their lives with these tools. For example, people use text-generating AI to edit papers and articles, while others use image-generating AI to efficiently create artwork, designs, and more. Many users reported becoming more efficient at completing their tasks and more capable of handling tasks they've never done before, thanks to AI.
​
Now, people are exploring ways for generative AI to enhance user experiences across various aspects of daily life.
PROBLEM
How can we use Gen AI to enhance employees' intranet experience?
Generative AI has significant potential to enhance work life by increasing productivity and enabling people to experiment with new ideas, aligning well with many business needs.
​
Our client, Uber, challenged us to explore how generative AI could enhance employees' experiences on the intranet—a network of tech products and systems that companies use for collaboration, communication, and information sharing to support daily operations.
​
I teamed up with four designers who share a passion for exploring the future of generative AI and using its potential for the benefit of humanity to start this project.
​
Time constraint
4 months
Users
Tech company employees
My teammates
4 UX designers
My role
UX research lead
CHALLENGES
I faced four key challenges in deciding on an effective and efficient research approach for an AI design project.
Designers were new to AI design
Designers joined this project to learn more about AI design but lacked prior experience. My research needed to offer context on both users and the technology to ensure successful AI design.
Gen AI responses can be unpredictable
Unlike websites or mobile apps with predefined responses, generative AI generates responses through complex algorithms, making them difficult to predict consistently. This posed additional challenges for designers to define user flows.
People have varied mental models of AI
AI products were new and mysterious, leading people to approach them with different mental models, which made data collection challenging. It was essential to align all stakeholders on a shared understanding of AI to prevent misunderstandings when discussing AI concepts.
Tight deadline
Our team had a fixed deadline to deliver the final prototype in four months. The research approach needed to be feasible within this timeframe while delivering timely insights to effectively support the deisgn team.
MY RESEARCH APPROACH
I provided insights about both the users and AI, enabling designers to understand the problem, ideate, and continually refine the design.
Empathize & Define
​
Ideate & Prototype
​
Evaluate
Iterate
Evaluate again
User research
User interviews
User surveys
Design research
Secondary research
Expert interviews
Low-fi usability testing
Expert review
Hi-fi usability testing
Gen AI testing
User research: User interviews, user surveys, 2 rounds of usability testing
WHY did I choose these methods?
​
6 user interviews
17 user surveys
5 low-fi usability testing
5 hi-fi usability testing
An efficient method to understand user habits, pain points, and their needs and preferences for Gen AI on the intranet.
​​
​
Prioritized the most important design challenge by adding weight to user insights collected in interviews.
Evaluated how well the low-fi prototype solved the design challenge and iterated the prototype with user feedback.
Evaluated design decisions during the previous round of iteration, and continued refining the design.
HOW did these research efforts solve challenges for this project?
​
Identified the "baseline" of intranet experience
Optimized the utilization of team's design power
Aligned the design with users' mental models
To propose a new intranet enhanced by AI features, the team must first understand current user behaviors and thoughts on the intranet to identify entry points for new features.
​​
​
Users would need to adapt to new AI-driven features on the intranet. Aligning the design with their mental models facilitated a smoother transition and increased product or feature adoption.
Using user insights to guide prioritization and design decisions could channel the team's design efforts to areas that generate the greatest impact on users.
IMPACT
-
Directed the team to the design direction of enhancing data search and team collaboration with AI features, the top user tasks and the primary areas needing improvement on the intranet.
-
Created and refined two personas to consistently guide the design process:
​​
​
Persona 1. Enhance data search with AI
Persona 2. Enhance team collaboration
Design research: Secondary research, expert interviews, expert review, Gen AI testing
WHY did I choose these methods?
​
Secondary research
5 expert interviews
1 expert review
Gen AI testing
Learning about generative AI through online courses, articles and journals equipped the design team to formulate insightful questions for experts.
Talking with experienced AI designers and software engineers provided the team with a quick overview of design best practices and technical assumptions for AI.
​
Critiquing the low-fi prototype with an AI design expert highlighted design decisions that required further consideration.
A feedback from experts. Testing prompts with real AI models such as ChatGPT to observe their responses inspired designers to align the prototype with realistic outcomes.
​
​
HOW did these research efforts solve challenges for this project?
​
Filled designers' knowledge gaps on AI
Optimized the utilization of AI tech power
Guided design with best practices
Knowledge is power. Bridging the AI knowledge gaps for designers empowered them to make more informed design decisions when needed.
Hearing of best practices and past failures in AI product design enabled the team to quickly eliminate misguided design decisions and had guidance to select the most suitable option.
​​
Understanding AI's capabilities and limitations inspired designers to explore more possible design directions and encouraged them to reassess ideas that exceeded current technological constraints.
IMPACT
-
Clarified the AI technical assumptions for the design project, focusing on an AI model with capabilities and limitations aligned with those of current popular AI products.
-
Developed two user flows for Gen AI features on intranet based on common tasks across various job roles in a typical large tech company:
​​
​
User flow 1. Enhance data search with AI - Schedule personal leave
User flow 2. Enhance team collaboration - Organize a team bonding event​
RESEARCH FINDINGS AND IMPACT
Finding 1: It's all about data. Users access the intranet primarily to find and use data, and AI relies on data to operate effectively.
During the 'empathize & define' stage, user interviews, user surveys, and expert interviews highlighted that the core of designing an AI-powered intranet is data, for several reasons:
​​
-
User need is data use: Users primarily used the intranet to find and utilize company data, such as accessing HR or IT information.
-
User pain point is to improve data use: Most companies used multiple platforms to distribute different types of information, creating barriers that make it difficult for users to easily access all the data they needed.
-
AI is capable of assisting data use: AI excels at processing data and identifying patterns to assist users.
-
AI is limited by data it can access: AI’s effectiveness depends on robust data training. The type and quality of data available for AI to learn from, as well as the data AI can access from users, will significantly impact how effectively AI can assist them.
​
Searching data is the No.1 user need on intranet.
​
- Finding from 17 user surveys.
"But what data do you have in mind for the AI to be trained on? I think you need to be very cautious about it when designing an AI product."
- Quote from expert interview.
During the "evaluate" stage, two rounds of usability testing gathered insights into the types of data users prioritize when scheduling personal leave and organizing team bonding events. Testing prompts with generative AI models further refined specific user-AI interactions.
These insights led designers to introduce features including:
FEATURE 1
An all-in-one intranet platform for company data searching and community communication
FEATURE 2
In the 'schedule personal leave' user flow, AI accesses users' personal calendar and highlights events with time conflicts when searching "personal leave for next two weeks" on intranet.
"I use calendar all the time to schedule leave or other agenda. It would be very convenient if AI can help with that or somehow use those information?"
- Quote from low-fi usability testing.
​
FEATURE 3
In the 'schedule personal leave' user flow, AI summarizes the process and guides users through the steps to submit a personal leave request when expanding search results for "personal leave for next two weeks."
​
IMPACT
-
Greatly enhanced the ease of searching for company data for tasks such as taking personal leave, as shown in the final round of usability testing.
Finding 2: Users want control. They prefer to manage when AI is engaged and to review AI-generated responses before moving forward.
One key finding from usability testing was that users value having more control over AI during task completion. For example, if the goal is to create a poll for teammates to decide on event timing and location, users prefer AI to suggest options first, obtain their approval, generate the poll, and then confirm the final details, rather than generating the poll directly.
With this insight, I guided designers to:
ITERATION
In the 'organize a team bonding event' user flow, incorporate more touch points for users to confirm and edit AI-generated responses.
​
Before: AI directly generates the poll.
After: Users can check and edit AI's responses.
IMPACT
-
Significantly increased user satisfaction with the intranet AI features, as demonstrated by the comparison between the final and first rounds of usability testing results.
Finding 3: Users view AI chat and AI search as distinct 'AI' entities, even when they are powered by the same model.
Another interesting finding from usability testing was that users perceive AI based on its functions. Even if the entire platform is powered by the same AI model, users see AI search and AI chat as separate entities when they have different interaction entry points.
​
"When you say AI Patty, do you mean the chatbot or the whole platform?"
​
- Quote from low-fi usability testing.
​
This created difficulties in shifting between mental models for AI search and AI chat, causing confusion about where to initiate the AI process for various tasks.
​
For all participants starting with AI search tasks, they also went for the search bar during AI chat tasks at first.
For one participant starting with AI chat tasks, they tried to use it at first during the AI search tasks.
​
- Findings from low-fi usability testing.
​
With this insight, I inspired designers to:
ITERATION
Include additional visual cues to encourage users to open AI chat when they need assistance.
​
Before
After
IMPACT
-
Improved the task success rate for users to locate and engage with AI when needed, as confirmed by the final round of usability testing.
REFLECTION
What I did well
-
Optimized research resource usage: Effectively leveraged my personal network in the AI field to provide our design team with valuable expert insights and data resources for critiques and second opinions.
-
Clear stakeholder communication: Delivered comprehensive and clear insights about both users and AI technology to designers unfamiliar with either area.
-
Increased AI domain expertise: I got valuable experience about AI product design and the research that supports it during this project, which will undoubtedly be beneficial for my future UX work in the AI era.​​​
“Your research insights have always been crystal clear to me, and they have definitely helped us speed up discussions on design decisions.”
—— One of my designer teammate
What I would do differently
-
Advocate for important research insights: There were several user insights that I found important but were ultimately not incorporated by the designers, such as users' preferences for prompts in different AI-engaged tasks.​
For example:
​
Users tend to use different prompts for AI search and AI chat, even on same tasks. AI search prompts are more keyword-like and general, while AI chat prompts are more conversation-like and specific.
One participant in low-fi usability testing mentioned that instead of the pre-set “personal leave for next two weeks” search keywords, they would most likely just type in “personal leave” or “time-off.” They would however ask Patty “personal leave for next two weeks” in the AI chat.​​
​
Since AI relies on data, these differences would influence its responses, thereby affecting user flows, potentially
significantly. Although these insights were left out due to time constraints, I believe I should advocate more
strongly for them in team discussions.
​
-
Research limitation: To simplify communication of AI concepts during research and considering time constraints, we primarily involved employees with prior experience using AI products. However, in reality, it's likely that some employees using the AI-powered intranet would be interacting with AI features for the first time and might have different perspectives on the proposed features. If time allowed, further research should be conducted with this user group to continue refining the product.
Other projects
Revamp the roadmap of clean energy mining trucks | Company: First Mode
User research, Research operations, Stakeholder communication
Redesign a database website for the ocean science community | Client: IABP
User research, Concept testing, Usability testing