Designing a clearer, more context-aware AI career coach for students
Problem
Students weren't completing the assessment because the AI felt repetitive, confusing, and untrustworthy.
Results
20% increase in task completion and 10% faster completion with voice interactions
Timeline
6 months
My Role
Product Design Lead, UX Research
Team
2 Product Designers
2 Software Engineers
We tested with 10 students and worked with a 6-person advisory board of vocational psychologists. The students and advisors did not trust in the efficacy of the AI prompts and questions.
To improve the chatbot's output, I defined 3 principles for a trustworthy AI experience:
• Be clear. Ask one question at a time using language students naturally understand
• Be aware. Carry context forward so students feel heard
• Be consistent. Create repeatable patterns for how the AI communicates
These principles are outlined in a pattern library that defines how the chatbot should ask questions, acknowledge responses, handle uncertainty, and recover from errors.
An AI Pattern Library to define how the chatbot should interact with the user
I designed a multimodal experience supporting voice-only and voice-to-text interactions, allowing for more natural and accessible experience.
Before
AFTER
what changed?
A full screen Voice-only experience for a focused experience
Option to exit or switch to text at any time (so the student doesn't feel trapped)
The student is still able to see where they are in the assessment (on the top left)
Pause or Redo at anytime
A transcript will appear as the student speaks
The redesigned message bar:
Option to choose between the full-screen "Voice only" experience (shown above) or "Speech to text" (below)
Student is speaking
Student is finished speaking and is able to edit their transcript before pressing send
increase in task completion
faster completion time with voice interactions
The MVP is preparing for a pilot with First Gen Scholars, a nonprofit supporting first-generation and low-income students pursuing college opportunities. Hundreds of students from this community will be among the first to use the redesigned experience.
Takeaway
Getting input from stakeholders and user testing was extremely valuable in defining the proper guardrails for safety and effectiveness for this assessment.
