Pieces 🌟If a user-experience/user-interface (UX/UI) project is going to involve an AI "intelligent" system, immediately ask two questions about the AI and UX.
Is the design *FOR* an intelligent-system interface? This is UX AI design.
Is the design to be done *WITH* an intelligent system? This is AI UX design.
The basic user interaction design principles apply to both. However, they are two very different design use cases and should be treated as such, even within the same project. Thus, they are discussed in separate sections of this post.
UX Design and AI
Because of the increased visibility of "intelligent" systems, it is a common myth that designing intelligent user interfaces (IUIs) is a new design problem. However, the 29th annual ACM Conference on Intelligent User Interfaces (ACM IUI) 2024 is next year. The conference announcement emphasizes discussion of "state-of-the-art advances at the intersection of Artificial Intelligence (AI) and Human-Computer Interaction (HCI)."
The 25th IUI Conference included a "reflection paper ... to analyse in detail the understanding of intelligence in the community".1 The authors extracted and manually reviewed every instance of the word "intelligence" in the entire set of IUI proceedings. Their goal was to answer "three main questions:
- What is deemed intelligent?
- How (else) is it characterized?
- What capabilities are attributed to an intelligent entity?
They concluded: "Despite the focus on intelligent UIs, it has remained elusive what exactly renders an interactive system or user interface “intelligent,” also in the fields of HCI and AI at large." That is still true.
User Experience and AI
The APA Dictionary of Psychology defines intelligence as an "individual’s ability to learn, reason, understand concepts, and acquire knowledge." (APA, Measures of Intelligence) A recent usability survey of software developers found that these capabilities in a software system can make users uncomfortable.
A recent usability study surveyed 410 software developers who include AI code generation tools in their workflow. The results describe users' opinions regarding AI-driven UX interactions. There are major usability issues caused solely by the AI aspects of an AI based UX design.
The survey found that "developers are most motivated to use AI programming assistants because they help developers reduce keystrokes, finish programming tasks quickly, and recall syntax, but resonate less with using them to help brainstorm potential solutions." In other words, AI is used for low-cognition tasks and often not trusted for involvement in tasks that could be creatively productive.
This view of the results is reinforced by the survey's negative findings. The "most important reasons why developers do not use these tools are because these tools do not output code that addresses certain functional or non-functional requirements and because developers have trouble controlling the tool to generate the desired output."
Designing the AI User Experience
There are numerous reasons why developers hesitate to use a copilot AI for high-cognition tasks such as generating code. For example, a 2022 study of 24 developers using the GitHub copilot found numerous developers had "difficulties in understanding, editing, and debugging code snippets" that were AI generated. The combination of UX and AI "did not necessarily improve the task completion time or success rate."
However, participants reported the copilot often provided a good starting point that made searching online unnecessary. Even when the AI-generated code contained an error, it guided the user into useful ways to think about the code they were writing.
On the other hand, attempting to debug the incorrect code would sometimes put users into a rabbit-hole that wasted time. The participants reported that when a long snippet of generated code contained an error, they would reject the entire snippet and start over. It was too time consuming and cognitively difficult to understand and debug code they had not written.
The study participants did not report using the GitHub copilot to debug its own code. This indicates Pieces’ ability to debug code needs to be strongly emphasized. Pieces easily debugged some UX code for Obsidian I had pasted in for analysis. The usability of generated code could be greatly increased by teaching developers how to have the AI debug its code.
Also, the usability of generated code could be increased by presenting snippets with identified parts and a comment explaining each part. Explanation of its code is a major feature of Pieces for Developers, and that increases its usability over copilots without explanations.
Understanding generated code may require knowledge of programming principles, frameworks, calls to specific APIs, and/or specific language libraries. Lastly, developers questioned whether the generated code adhered to their project’s coding style.
Some of these usability difficulties may occur because the use of LLMs by developers is a new field for usability research and design. There is a need to switch to descriptive design rather than prescriptive design because of the introduction of AI-system intelligence.
"Prescriptive" design is used for traditional systems because the design process defines the complete set of possible inputs and outputs. The design defines a limited number of possible paths available in response to a specific input.
An input outside the designed paths results in the display of an error message (or worse, a system crash). Expert users of prescriptively designed systems are accustomed to having a feeling of control, of making the system (a "thing") do what the human wants.
For example, a driver can control a car without understanding anything about engines and other functional parts controlled through the input devices. This prescriptive user-system relationship does not work well for AI user interface design. The AI system has a 'mind' of its own that is not, and cannot, be controlled by the user. It cannot be controlled even by its programmers.
Consequently, the design process must emphasize descriptive information over prescriptive commands. Ill-defined communication is inherently part of a system that allows the user access to the 'intelligence' embedded in the system. Therefore the user needs help in understanding (1) how much and when to trust the accuracy of the output of the intelligence and whether it is consistently reliable, (2) how to efficiently and effectively communicate with the intelligence, and (3) how to make valid predictions about the properties and actions of the intelligent system.
Using AI Tools for UX Design
When discussing AI and UX design, people often ask, "Will AI replace UX designers?" The answer is a resounding "No!", at least for many years in the future. The design of good user interactions is a complex creative task, especially at the stages of conceptual and logical design.
A completely AI-generated UX design strategy would be equivalent to expecting a human UX designer to start and complete the entire UX/UI system in one conversation. It cannot be done.
However, there are many ways a generative AI for UX can be helpful in AI UX projects. How AI can be used depends on the project. For example, an AI can track and analyze user data when there are relevant sources, such as a website, user forum, or another type of information available on the internet. The data patterns discovered by the AI can then be input as prompts for the AI to construct user personas.
LLMs obviously can help with writing tasks required during design. These range from drafting questions for interviews to summaries of the findings to writing long reports. When a designer is comfortable using an AI assistant, many other uses will come to mind during a UX/UI design project.
Future AI tools are being developed that can aid in creating and describing workflow design. This includes creating various types of prototypes that can be used as is or modified for usability testing. An AI tool also could be used to control the usability testing and analyze the resulting data. Then the AI tool could quickly iterate the prototype(s) for further testing.
There is a major danger when doing AI UI design. It could be too easy to get sucked into focusing on the visual aspects of the design. If the conceptual design and logical (sequential workflow) design are not well done before doing visual design, usability issues are guaranteed to occur.
AI and User Experience
In summary, it is a myth to think that using AI for UI/UX design will be possible without human intervention during the design process. Although it is helpful to use AI for UI design, the emphasis is on a human using AI in UX design.
