Emly AI Chatbot
Improving customer support with conversational AI Chatbots
Platform: Emly Labs AI Chatbot
Industry: B2B SaaS, AI
Timeline: 8 Weeks
Introduction
AI chatbots have become pivotal in enhancing user experiences across digital platforms, offering instant assistance and personalized support. However, their success largely depends on the quality of human-AI interaction. Research by Chen et al. (2023) highlights a common challenge: users often face frustration due to the chatbot’s limited understanding of complex queries, impacting overall satisfaction. Similarly, Guingrich and Graziano (2023) found that while chatbots can act as social companions, their perceived human-likeness influences user engagement and trust, underlining the importance of context-aware responses.
This UX research focuses on EmlyBots, an AI chatbot designed to streamline user interactions across various tasks. Despite its potential, EmlyBots struggles with issues like understanding user intent, handling multi-turn conversations, and providing error recovery, echoing the findings from Zheng et al. (2022) on common pain points in conversational AI. By conducting a comprehensive heuristic evaluation, we aim to identify and address these usability problems, aligning the chatbot's design with user needs.
Our objective is to bridge the gap between AI capabilities and user expectations, leveraging insights from human-chatbot interaction studies to enhance the user experience. Through iterative testing and user feedback, we seek to transform EmlyBots into a more intuitive, context-aware assistant, ultimately increasing user satisfaction and engagement by 35%. This case study documents our process, challenges, and solutions in redesigning EmlyBots, aiming to contribute valuable insights for improving conversational AI design in broader applications.
Problem statement
Despite its potential to streamline user interactions, EmlyBots faces significant usability challenges that hinder its effectiveness. Users frequently experience frustration due to the chatbot’s limited understanding of complex queries, unclear scope of capabilities, and poor handling of multi-turn conversations. Additionally, the lack of effective error recovery and concerns over data privacy exacerbate the negative user experience. These issues lead to reduced engagement, lower satisfaction, and higher bounce rates. To address this, a comprehensive heuristic evaluation is needed to uncover the core usability problems and redesign EmlyBots with a focus on intuitive, user-centered improvements, ultimately enhancing the chatbot's ability to meet user needs effectively.
Plan
Phase 1: Initial User Test Report and Heuristic Evaluation
Creating a feedback survey
General Usability and User Experience
What are your initial thoughts on the overall user experience of EmlyBots?
What aspects of EmlyBots do you find most frustrating or confusing? Why?
How would you describe the chatbot’s ability to understand user intent? What improvements can be made?
In your opinion, how well does EmlyBots handle multi-turn conversations? What specific issues did you notice?
What usability features do you believe are missing in EmlyBots?
Error Handling and Recovery
Describe a scenario where EmlyBots handled user errors effectively. How can this be enhanced further?
In cases where the chatbot fails to understand a query, how does it respond?
What alternative strategies would you recommend for error recovery?
What changes could help EmlyBots provide clearer feedback when it encounters an error or does not understand user input?
Scope and Capabilities
How clear is the scope of EmlyBots’ capabilities to users?
What suggestions do you have to better communicate what the chatbot can and cannot do?
In what ways can the chatbot’s responses be improved to better align with user expectations?
Do you believe EmlyBots has any features that are underutilized or misunderstood by users?
How can these be highlighted more effectively?
Conversational Flow and User Engagement
How would you rate the naturalness and flow of conversations with EmlyBots?
What aspects disrupt the conversation flow?
What changes would you recommend to make EmlyBots’ responses feel more human and engaging?
How can EmlyBots better maintain context during multi-turn conversations to avoid user frustration?
What features or enhancements could help increase user engagement and retention when interacting with EmlyBots?
Privacy and Trust
Do you notice any concerns related to privacy and data security when interacting with EmlyBots?
How can these be addressed effectively?
How can the chatbot better reassure users about the safety and confidentiality of their information?
Improvement and Future Enhancements
What are three key improvements you would suggest for EmlyBots to enhance its overall user experience?
Are there any design patterns or best practices from other successful chatbots that you believe could be applied to EmlyBots?
Is there any additional feedback or insights you’d like to provide that could help in the redesign of EmlyBots?
Early Implementations
High Severity Implementations
Improve Feedback and Status Indicator
Implement responsive feedback elements such as typing indicators, progress loaders, and real-time acknowledgment of user inputs.
Enhance Error Handling and Recovery Mechanisms
Design clear, informative error messages that suggest corrective actions and offer alternative options for users to proceed.
Introduce Clear User Control Options
Add intuitive navigation features such as “Back,” “Reset,” and “Help” buttons to provide users with greater control over their experience.
Standardize Conversational Tone and UI Elements
Develop a style guide for both conversational language and UI components, ensuring a consistent tone and design throughout the chatbot interactions.
Optimize Multi-turn Conversation Flow
Implement improved context management techniques, retaining conversation history and using memory-based responses for a smoother dialogue flow.
Medium Severity Implementations
Simplify and Clean Up the Interface Design
Redesign the interface with a minimalist approach, reducing unnecessary elements and simplifying visual hierarchy.
Increase Personalization and User Preferences
Incorporate user preference settings and leverage data from previous interactions to offer more tailored, personalized responses.
Enhance Privacy and Data Security Communication
Add clear, transparent data privacy messages and options for users to manage their data. Include a privacy policy link within the chatbot interface.
Integrate Advanced Help and Documentation
Add a dedicated help section with quick tips, FAQs, and direct access to live support if needed.
Introduce Efficiency Features for Power Users
Include features like quick commands, keyboard shortcuts, and the ability to bypass onboarding for returning users.
Fixing the Old UI using Posiedon DS
Fixing the High Severity pain points
Designing a better Language
As we had already implemented a unified design language across the products of Emly Labs, the Guide book I created for them saved us a lot of time. We put the framework in use for this project for the first time and it was fairly smooth.
User Testing
The redesign of EmlyBots successfully resolved the critical pain points identified during the evaluation phase. User experience improved significantly, with higher satisfaction rates, increased engagement, and a stronger sense of trust in the chatbot’s capabilities. Business performance also saw measurable benefits, including reduced support costs, higher conversion rates, and a positive shift in brand perception.
The strategic implementation of a consistent design system and advanced context management positioned EmlyBots as a more reliable, intuitive AI assistant. The project’s success demonstrates the value of a user-centered design approach, grounded in thorough research and iterative testing.
Next Steps and Future Work
While the redesign made substantial progress, there are several areas for continued improvement and future growth:
Ongoing Usability Testing: Regular testing will help refine features based on evolving user needs and feedback.
Advanced AI/ML Integration: Upgrading EmlyBots with state-of-the-art language processing will further enhance its ability to handle complex queries. Additions like, ChatGPT or Gemini AI will boost business outcome.
Multichannel Expansion: Integrating the chatbot with additional platforms like WhatsApp and Slack will broaden its reach and user engagement.
Enhanced UI AI Design System: Increasing the level of Posiedon based on AI preferences can make a decent impact on shipping out AI based product.
Gen AI Elements: Adding features like Generative AI, Predictive and Data Visulaization AI elements is on the cards.
Enhanced Privacy Controls: Offering more granular options for data management will further build user trust, aligning with evolving data privacy regulations.
These steps will help EmlyBots remain adaptive and responsive to user needs, ensuring its continued success in the competitive landscape of AI chatbots.