Building a Bridge Over the Last Mile: Solving Problems with AI and Filling in the Human Experience.
World Usability Day 2025 is a global event that brings together people across industries to ensure the services and products essential to life are easier to access and simpler to use. This year’s theme focuses on maintaining human-centered design in the development and implementation of emerging technologies, such as artificial intelligence, to solve user problems and create innovative solutions that respect privacy and individual needs.
Remembering that AI is not Human
One of the most common ways we experience AI today is through a chatbot, where usability is largely dependent on users’ own ability to chat (by voice or typed text) in addition to typical challenges of making the UI discoverable and accessible. In terms of context, the chatbot relies on the user to provide and specify their goals and knowledge. This interaction style is more tailored to task completion and linear or step-by-step investigation, such as a user trying to research reserving a rental venue, enrolling in academic courses, analyzing data, or purchasing a product.
Some experiences cannot be replaced with generated text. While AI can help support task completion, people still need to browse and explore to fulfill human needs. If you are exploring wedding venues or looking for a hiking route, an AI agent might tell you if the location is available or how long the hike will take, but it cannot browse a photo gallery for you. A website (or other platform UI) with all of its usual expected navigation and content not only supplies us with information, but can convey emotion and build trust and connection as part of that company or organization’s brand and larger story. Designers can also use colors, shapes, and typography to influence perception and guide users’ attention.
As examples, consider the websites for the Museum of History and Industry in Seattle, WA (MOHAI) or the City of Tacoma, which greet website visitors with videos on the homepage backgrounds to pull people in and show visitors what to expect in person.
This duality of tasks and browsing also means there is an increase in the overall need for usability, both for traditional websites and the chatbot or other new AI features they might host.
Users who start with a chatbot have nothing to go on but their own knowledge and context, so discovery is limited to anything the AI bot says that the user requests more information about. This limitation also impacts a basic tenet of good user experience design: recognition vs. recall (there is a lower cognitive load for users to recognize information than recall it from memory). If AI chatbots become the sole means of interaction, we lose the ability to freely browse and explore different contexts. Therefore, we must avoid a future where every website is only a logo and a chatbot.
With emerging technologies, user experience still matters.
Regardless of how something is built or how helpful a Large Language Model (LLM) might be, people still have goals to accomplish, jobs to do, and lives to live. People still need to buy food whether they’re making a shopping list with ChatGPT or a ballpoint pen. For many people, new technologies bring change and users can be understandably wary. To vault over the barrier of the combination of novelty and lack of trust (along with fear from past bad experiences with new technology), a clear and predictable user experience needs to give a good first and second impression. Emergent technologies need to be easy to use, and their applications need to be human-centered.
How can we make AI more usable? How can we make everyday use of AI more human-centered?
When using AI to accomplish goals for everyday use, we run into specific limitations, many of which are determined by limitations of context.
AI needs detailed instructions.
In the use of generative tools, a prompt is needed to instruct the AI on what the task is and, in many cases, how to do it. Prompt engineering has exploded as a skill set on its own. Much like mastering the use of search engines, it involves using the right words in the right way to give the AI the right context, instructions, and examples to generate the expected kind of output in the expected format and tone of voice.
As part of our Sitka Insights suite of tools, we have a feedback collector that can be added to a site. A user can answer a question like “Was this page helpful?”. If they say no, they can choose to add a comment on what they were looking for. One of AI’s strengths is quickly analyzing and summarizing large amounts of text. In the past, we manually categorized each comment line-by-line. Now, AI is helping us quickly find common themes across hundreds of user comments for a site, by categorizing and grouping comments together and summarizing common themes, sometimes dozens or hundreds of times faster than our previous process. We include in the instructions to the AI that we want to count the number of times something is mentioned and to list the original comments in order to gauge user impact.
AI can provide ideas, but more context is often needed.
This approach helps us find the most common issues more efficiently, but in order to solve users’ issues, we need real, actionable solutions. Asking the AI to generate ideas on possible solutions at this stage would typically result in simply flipping the meaning of the user feedback. For example, if users said they were looking to pay a ticket, the AI suggestion was to add a ticket payment form on the page. While straightforward, this does not take into account the existing website’s structure, navigation, or content.
In addition to feedback, we can supply the markup or content of the page where feedback was received, so the AI tool has the context of available information and functionality on the site for those pages. We can then ask the AI for suggestions on how to improve the site as well. This addition vastly improves the precision and relevance of the answers and recommendations generated and helps us explore opportunities for cross-linking. To follow up on the ticketing example, a suggestion might become something like “There is a ticket payment link on another page in that section. Add a prominent Pay My Ticket link to the top of this page.”
However, there is still context missing. As another example, we recently saw a large volume of user feedback that they could not find job opportunities on a page with a URL /job-board. We checked the address they were using and reached a 404 (page not found) error. The AI-generated suggestion was to add a Careers page or add job information to that page. However, the site already had job opportunities and career information linked on the /careers page with a prominent link in the main navigation.
Why were over one thousand users arriving on a non-existent page? We worked with our content team to check the analytics. The data there filled in the missing piece of the puzzle: a third-party site had a link with a bad address linking to a non-existent page. We added a redirect to the proper Careers page and were able to stop additional errors.
This kind of investigation and solution illustrates what we think of as the “last mile” problem. As humans, we can dig into mysteries, ask good questions and find the root cause of issues. AI tools, even with the right data, lack this level of intuition and wisdom.
Trust
The first step toward making AI tools more trustworthy is to recognize their limitations. Despite massive volumes of training data, AI tools can “hallucinate” (generate false or fictitious statements) and make mistakes because of the text-predicting nature of LLMs, which predict the most likely text to follow based on what has already been generated. So, how can we make day-to-day use of AI tools more accurate, and, at the same time, more trust-worthy?
Cite sources and show original data.
Much like academic research papers and essays, citing specific sources, by URL or otherwise, can go a long way to increase trust. References, claims, and conclusions with references to source materials (regardless of AI) are more trustworthy because it makes fact-checking easier and more actionable. Prompting the AI to include direct quotations or examples from analyzed sets of data can further increase credibility.
Reference data directly.
With agentic tooling, it is possible to reference data sources directly via APIs or other connections. AI agents can utilize these tools to query for precise and up-to-date structured information that is specific to you and your business, instead of relying on irrelevant or outdated training data. Finally, if you self-host the LLM, AI tools can be used securely with private or protected sources of data, such as confidential company information.
Agentic AI: Autonomous Action under Human Control
Autonomous Agentic AI systems need to be human-centered.
One way we can do this is by following Jakob Nielsen’s 10 Usability Heuristics. The most relevant of these are number one: Visibility of System Status, and numbers five and nine on Error Prevention and Error Recovery.
Agentic AI applications need transparency to ensure precision.
For a system to act autonomously, showing its status helps users understand what is happening and provide oversight. One way AI agents can show their status is to list their capabilities and plan or next step before, during and after execution. In the context of a chatbot, this is relatively simple to implement in the user interface by sending messages to the user with status updates as conditions change or as progress is made. For example, suppose an AI Agent available in a chatbot is booking a trip for a user who needs to go to New York City for a conference. Before searching for flights or looking for hotel reservations, the AI application should confirm with the user the plan for the desired travel, ask good follow-up questions, and describe how it might search for flights or hotels, but its knowledge might be limited about the conference venue. Status messages for the user during the agent’s search might include “Looking for flights” and “Found flights matching your preferences. Looking for hotels with check-in and check-out dates that match.” This added transparency helps establish trust and also gives the user more opportunities for oversight to guide the agent when needed.
Autonomous action also means putting trust in the AI system to act as expected.
When AI agents make mistakes or follow an unexpected path, users need a way to easily prevent or undo autonomous actions without cost or damage. In the example of booking a trip, suppose the only flights available on those travel dates are to Scranton, Pennsylvania. The best outcome at this point would be for the agent to pause and ask the user for guidance to either change the travel dates or change the destination. Depending on the agent’s available context, it might need more help determining whether the dates or location are more flexible. After the agent has a recommended itinerary, the user needs a chance to review it and confirm. In a hypothetical case where the agent has the user’s payment information and books a trip that needs to be modified after booking, there needs to be some allowance for tickets or hotel stays to be modified or refunded.
When combined, these heuristics help create Agentic AI applications for day-to-day use that are more transparent, precise, trustworthy, and therefore more usable and human-centered.
More Ideas
We’ve also been working on AI-powered semantic search. Users often may use different words or phrases to look for the same thing. This presents a challenge for traditional search algorithms. Based on the results we have seen in internal testing, AI semantic search using similarity scores will help more users find what they are looking for, compared to traditional keyword matching. By making search work better for the variance in human language, we are keeping search human-centered.
We can also extend semantic search to help users in real-time and meet them where they are. We can tie our semantic search service to the same feedback collector mentioned earlier. When someone adds a comment, we can send the message to the same AI-powered search engine to meet users where they are and help them find what they’re looking for.
Send us a Message
If you’d like to learn more about human-centered design at SiteCrafting or want to talk about how we can help with emerging technologies, give us a call at 253-272-2248 or send us a message at hello@sitecrafting.com.
Happy World Usability Day!
